jbay808 (u/jbay808) - Readit News

jbay808 commented on Vision Language Models Are Biased vlmsarebiased.github.io/... · Posted by u/taesiri

jbay808 · 3 months ago

I disagree with the assertion that "VLMs don't actually see - they rely on memorized knowledge instead of visual analysis". If that were really true, there's no way they would have scored as high as 17%. I think what this shows is that they over-weight their prior knowledge, or equivalently, they don't put enough weight on the possibility that they are being given a trick question. They are clearly biased, but they do see.

But I think it's not very different from what people do. If directly asked to count how many legs a lion has, we're alert to it being a trick question so we'll actually do the work of counting, but if that image were instead just displayed in an advertisement on the side of a bus, I doubt most people would even notice that there was anything unusual about the lion. That doesn't mean that humans don't actually see, it just means that we incorporate our priors as part of visual processing.

jbay808 commented on Vision Language Models Are Biased vlmsarebiased.github.io/... · Posted by u/taesiri

energywut · 3 months ago

Are they? Did you see the picture of the chicken with three legs? Because there's no human I know who would confidently assert that chicken has two legs.

jbay808 · 3 months ago

If I were given five seconds to glance at the picture of a lion and then asked if there was anything unusual about it, I doubt I would notice that it had a fifth leg.

If I were asked to count the number of legs, I would notice right away of course, but that's mainly because it would alert me to the fact that I'm in a psychology experiment, and so the number of legs is almost certainly not the usual four. Even then, I'd still have to look twice to make sure I hadn't miscounted the first time.

jbay808 commented on New material gives copper superalloy-like strength news.lehigh.edu/new-mater... · Posted by u/gnabgib

jbay808 · 4 months ago

This might be a great alternative to beryllium copper for the spring contact element in high-current electrical connectors.

jbay808 commented on The LLMentalist Effect softwarecrisis.dev/letter... · Posted by u/zahlman

manmal · 7 months ago

Have you considered that the nature of numeric characters is just so predictable that they can be sorted without actually understanding their numerical value?

jbay808 · 7 months ago

Can you say more precisely what you mean?

jbay808 commented on The LLMentalist Effect softwarecrisis.dev/letter... · Posted by u/zahlman

dartos · 7 months ago

Of course, that’s how attention works, after all.

But by specifically avoiding certain cases, wet could verify if the model is generalizing or not.

jbay808 · 7 months ago

I mean that needing to scan the full context of tokens before the nth is inherent to the problem of sorting. Transformers do scan that input, which is good; it's not surprising that they're up to the task. But pairwise numeral correlations will not do the job.

As for avoiding certain cases, that could be done to some extent. But remember that the untrained transformer has no preconception of numbers or ordering (it doesn't use the hardware ALU or integer data type) so there has to be enough data in the training set to learn 0<1<2<3<4<5<6, etc.

jbay808 commented on The LLMentalist Effect softwarecrisis.dev/letter... · Posted by u/zahlman

dartos · 7 months ago

I don’t really understand what you’re testing for?

Language, as a problem, doesn’t have a discrete solution like the question of whether a list is sorted or not.

Seems weird to compare one to the other, unless I’m misunderstanding something.

What’s more, the entire notion of a sorted list was provided to the LLM by how you organized your training data.

I don’t know the details of your experiment, but did you note whether the lists were sorted ascended or descended?

Did you compare which kind of sorting was most common in the output and in the training set?

Your bias might have snuck in without you knowing.

jbay808 · 7 months ago

> I don’t really understand what you’re testing for?

For this hypothesis: The intelligence illusion is in the mind of the user and not in the LLM itself.

And yes, the notion was provided by the training data. It indeed had to learn that notion from the data, rather than parrot memorized lists or excerpts from the training set, because the problem space is too vast and the training set too small to brute force it.

The output lists were sorted in ascending order, the same way that I generated them for the training data. The sortedness is directly verifiable without me reading between the lines to infer something that isn't really there.

jbay808 commented on The LLMentalist Effect softwarecrisis.dev/letter... · Posted by u/zahlman

dartos · 7 months ago

Not saying I disagree with the thesis, but I don’t think this proves anything.

If every pair of digits appears sorted in the dataset, then that could still be “just” a stochastic parrot.

I’m kind of interested to see if an LLM can sort when the dataset specifically omits comparisons between certain pairs of numbers.

Also I don’t think OC was responding to commenters, but the article

jbay808 · 7 months ago

It might seem like you could sort with just pairwise correlations, but on closer analysis, you cannot. Generating the next correct token requires correctly weighing the entire context window.

jbay808 commented on The LLMentalist Effect softwarecrisis.dev/letter... · Posted by u/zahlman

jbay808 · 7 months ago

I was interested in this question so I trained NanoGPT from scratch to sort lists of random numbers. It didn't take long to succeed with arbitrary reliability, even given only an infinitesimal fraction of the space of random and sorted lists as training data. Since I can evaluate the correctness of a sort arbitrarily, I could be certain that I wasn't projecting my own beliefs onto its response, and reading more into the output than was actually there.

That settled this question for me.

jbay808 commented on Show HN: Real-time nonlinear optics simulation (JS/GLSL) github.com/westoncb/nonli... · Posted by u/westoncb

dvh · 8 months ago

Does nonlinear optics also create harmonics as with nonlinear electronics?

jbay808 · 8 months ago

Yes, this is how frequency-doubled lasers work (eg. 532 nm green laser pointers, which are generated as a harmonic from a 1064 nm Nd:YVO4 laser by a nonlinear KTP crystal).