zhangjunphy (u/zhangjunphy)

zhangjunphy commented on LLM Daydreaming gwern.net/ai-daydreaming... · Posted by u/nanfinitum

amelius · 8 months ago

But what if the critic is just hard reality? If you ask an LLM to write a computer program, instead of criticizing it, you can run it and test it. If you ask an LLM to prove a theorem, let it write the proof in a formal logic language so it can be verified. Etcetera.

zhangjunphy · 8 months ago

I think if we can have a good enough simulation of reality, and a fast one. Something like an accelerable minecraft with real world physics. Then this idea might actually work. But the hard reality we currenly could generate efficiently and feed into LLMs usually has a narrow scope. It feels liking teaching only textbook math to a kid for several years but nothing else. The LLM mostly overoptimize in these very specific fields, but the overall performance might even be worse.

zhangjunphy commented on LLM Daydreaming gwern.net/ai-daydreaming... · Posted by u/nanfinitum

imtringued · 8 months ago

That didn't stop actor-critic from becoming one of the most popular deep RL methods.

zhangjunphy · 8 months ago

True, and the successful ones usually require an external source of information. For AlphaGo, it is the simple algorithm which decide who is the winner of a game of Go. For GAN, it is the images labled by human. In these scenarios, the critic is the medium which transforms external information into gradient which optimized the actor, but not the direct source of that information.

zhangjunphy commented on LLM Daydreaming gwern.net/ai-daydreaming... · Posted by u/nanfinitum

zhangjunphy · 8 months ago

I also hope we have something like this. But sadly, this is not going to work. The reason is this line from the article, which is so much harder that it looks:

> and a critic model filters the results for genuinely valuable ideas.

In fact, people have tryied this idea. And if you use a LLM or anything similar as the critic, the performance of the model actually degrades in this process. As the LLM tries too hard to satisfy the critic, and the critic itself is far from a good reasoner.

So the reason that we don't hear too much about this idea is not that nobody tried it. But that they tried, and it didn't work, and people are reluctant to publish about something which does not work.

zhangjunphy commented on Ask HN: Why do half of Internet users think we are living in a simulation? · Posted by u/alister

space_fountain · 2 years ago

There’s a counter argument from my physics undergrad brother that I found convincing.

We already do have a law of physics that is relevant here. We know that the information capacity of space is finite and fixed. A centimeter of space can only store so much information before it becomes a black hole. That means that to build a simulation in our universe you can only ever subdivide a fixed pie of information. That means the more relevant thing to ask is if a quantity of information is more likely to exist in the base reality or the simulated one. Because we have to assume that the base reality is not carpeted over with simulation super computers it seems safe to assume a random bit of information is more likely to be part of the base reality rather than a simulation all else being equal.

I think the idea of the universe being a simulation is just more fun

zhangjunphy · 2 years ago

It is a counter argument against that we can build a perfect simulator of our universe, but not a good one against that our universe is simulated.

In fact if I were to build a simulator, I most likely have to design a mechanism to prevent its residents from observing beyond a certain micro scale due to limited cpu/mem resources and laziness to implement all the details. Tiny black hole is a good mechanism to reduce resource consumption when simulating a fixed volume of this universe. Imagine living in the world of Minecraft, the minimal unit is a block. Trying to look inside of it yields nothing. All physically meaningful characteristics are described by its surface. In our universe this looks very much like a blackhole.

zhangjunphy commented on The diamond world takes radical steps to stop a pricing plunge bloomberg.com/news/articl... · Posted by u/toomuchtodo

zhangjunphy · 2 years ago

The market has changed drastically since some Chinese factories start mass producing synthetic diamonds around 2019.

Currently in the end market the price is $200~300 for a 1CT synthesized one, and the quality are simply better all around. The only way to distinguish natural ones is to track from the start where each is mined and processed, which is a bit absurd.

Many of these factories were associated with drill head industry and the like until they found the jewelry market is much more profitable and got the trick to mass-produce them. If they are willing to compete a bit the price might even go lower.

zhangjunphy commented on Time Doesn’t Belong to Physics – When Bergson met Einstein iai.tv/articles/time-does... · Posted by u/mellosouls

likeabbas · 2 years ago

>But modern thinkers cannot be so readily absolved

No, but they can go on a 40 year goose chase trying to reconcile the many different string theories that don't have any physical evidence to substantiate.

QFT is one of the most successful theories we've created, but GR is still our best theory regarding gravity. There's still no evidence for the graviton. We probably need something else besides QFT to describe the nature of reality

zhangjunphy · 2 years ago

QFT is too successful that experimentalists stuck for half a century with no meaningful surprise. But theoreticians got to maintain a publication streak if trying to stay in academy. Thus string theory becomes the perfect field. Most ones doing it seem to know it is a goose chase but no one dare to admit it publicly as your colleagues stripped of funding would tear you to pieces.

I feel we really should sort these papers into a category "physics in imaginary universe", so other people do not get confused.

zhangjunphy commented on LK-99: Phonon bands, Localized Flat Band Magnetism, Models and Chemical Analysis drive.google.com/file/d/1... · Posted by u/carabiner

lamontcg · 3 years ago

> There have been multiple videos of tiny LK99 specs completely levitating on single magnets, and resisting motion when pushed, and staying afloat when the magnet is inverted.

One of those has already been admitted to being outright faked:

https://www.tomshardware.com/news/lk-99-video-fraud-taken-do...

zhangjunphy · 3 years ago

Some additional context here. The author has a further statement, in which he apologizes that the previous video was misleading due to two points: 1. The sample was not LK99. 2. Both the larger piece held by the tweezers and the smaller levitating one were from the same sample. But he also states that no tricks of any kind were used nor the video was edited.

So, some other material levitating itself in seemingly room temperature?

Currently this guy is practically using his real name, with his university and professor exposed. It takes some courage to lie at this point.

link(in Chinese): https://bilibili.com/video/BV1Zh4y1r7XL

======================

Edit: People sometimes speak in convoluted ways. I feel "The sample was not LK99" could have two interpretations here: 1. It is a completely different compound. 2. It is a derivative from LK99 with a different synthesis/doping method.

zhangjunphy commented on GitHub Copilot available for JetBrains and Neovim github.com/github/copilot... · Posted by u/orph

mewse · 4 years ago

I’ve never understood the value proposition for Copilot.

In terms of difficulty, writing code is maybe on average a two out of ten.

On average, maintaining code you wrote recently is probably a three out of ten in terms of difficulty, and maintaining code somebody else wrote or code from a long time ago probably rises to around a five out of ten.

Debugging misbehaving code is probably a seven out of ten or higher.

GitHub Copilot is optimising the part of the process that was already the easiest, and makes the other parts harder because it moves you from the “I wrote this” path to the “somebody else wrote this” path.

Even during the initial write, it changes the writing process from programming (which is easy) to understanding somebody else’s code to ensure that it’s right before accepting the suggestion (which is much less easy). I just don’t understand how this is a net time/energy savings?

zhangjunphy · 4 years ago

I guess it really depends on what you are working on.

If you are writing some non-trivial algorithms or working on some projects which requires delicate handling of things, then Copilot is most likely going to mess up.

But if you are working on many of those frontend code or backend CRUDs which are usually quite repetitive. Then Copilot could be helpful.

zhangjunphy commented on Is education no longer the ‘great equalizer’? nytimes.com/2021/06/23/op... · Posted by u/gumby

zhangjunphy · 5 years ago

It is really interesting to read the comments in this thread.

From an outsider's perspective, US has been the most prosperous country in this planet. People generally have good chances to move up the class ladder. And there are a lot of active enterprises. But I had the strong feeling that this stop to be the case in recent decades.

The reason, as I see it, is the natural tendency of capitalism, people seeking to maximize their individual capital. In the past century, with the pressure of the cold war, this tendency is balanced by the government, as US needed its working class to actually make innovations and make products to gain the advantage in the competition with the Soviet Union. In fact, the working class got their best treatment during the cold war, and then it worsens in recent decades.

Now with the Soviet Union dead, and China is still too young to be an actual competitor. There is no obstacle for the capitalists to play their games. It should be evident for most Americans that the price of properties are inflating while the increase of their salary slowed. The government also got eroded, most of the people there are either capitalists or have good relationship with them. What will be their incentives to change the rules?

This is not just the problem with US, it happens also in China. And I don't see a solution to it. The lower classes are controlled tightly by the media, which is controlled by the capitalists. And the middle class is thin and divided. In fact a lot of the middle class try very hard to climb the social ladder so they can be one of the upper class. They don't want the game changed, just "modified" to benefit themselves a bit.

zhangjunphy commented on Why is Stack Overflow trying to start audio? meta.stackoverflow.com/qu... · Posted by u/iokanuon

coldpie · 7 years ago

Why are you allowing arbitrary javascript to be served to your users?

zhangjunphy · 7 years ago

Revenues are important. The users will not notice unless something happens. And when something happens they forget fast.