Readit News logoReadit News
grugnog commented on Gemini 3 Pro: the frontier of vision AI   blog.google/technology/de... · Posted by u/xnx
Workaccount2 · 2 months ago
Well

It is the first model to get partial-credit on an LLM image test I have. Which is counting the legs of a dog. Specifically, a dog with 5 legs. This is a wild test, because LLMs get really pushy and insistent that the dog only has 4 legs.

In fact GPT5 wrote an edge detection script to see where "golden dog feet" met "bright green grass" to prove to me that there were only 4 legs. The script found 5, and GPT-5 then said it was a bug, and adjusted the script sensitivity so it only located 4, lol.

Anyway, Gemini 3, while still being unable to count the legs first try, did identify "male anatomy" (it's own words) also visible in the picture. The 5th leg was approximately where you could expect a well endowed dog to have a "5th leg".

That aside though, I still wouldn't call it particularly impressive.

As a note, Meta's image slicer correctly highlighted all 5 legs without a hitch. Maybe not quite a transformer, but interesting that it could properly interpret "dog leg" and ID them. Also the dog with many legs (I have a few of them) all had there extra legs added by nano-banana.

grugnog · 2 months ago
Testing on Gemini Pro thinking I was able to generate a 5 legged dog first try, which is pretty impressive. Pasting that into a new session it counted 4 at first, but when I asked it to count more carefully it came back with 5 (and a detailed description of the positioning of each limb). Interestingly it suggested the anomaly could have been caused by AI! This is a good test I think - we seem right on the threshold of it being doable.
grugnog commented on How Not to Test GPT-3   garymarcus.substack.com/p... · Posted by u/laurex
grugnog · 3 years ago
The first example doesn't make it clear that the reader was out of the room when the bookmark was moved. I altered the text to make it clearer that they were unaware of the change and it responded correctly:

You are reading a book and put a bookmark on page 132. You put down a book and leave the room. While you are out a friend secretly moves the bookmark to page 345. When you return planning to open the book what page do you expect the bookmark to be on?

Based on the information given, you would likely expect the bookmark to be on page 132, the page where you originally placed it. Since you were not aware of your friend moving the bookmark, you would assume that it is still on the page you left it on. Therefore, you would likely be surprised to find it on page 345 instead.

u/grugnog

KarmaCake day2May 29, 2013View Original