Ooh, fun, a FizzBuzzFeed quiz!
Readit NewsBjorkbat commented on Dev Compass – Programming Philosophy Quiz treeform.github.io/devcom... · Posted by u/todsacerdoti
Bjorkbat commented on Genie 3: A new frontier for world models deepmind.google/discover/... · Posted by u/bradleyg223
The goal is to eventually be able to model physics and all the various interactions accurately.
Sure, but if you're trying to get there by training a model on video games then you're likely going to wind up inadvertently creating a video game simulator rather than a physics simulator.
I don't doubt they're trying to create a world simulator model, I just think they're inadvertently creating a video game simulator model.
Bjorkbat commented on Genie 3: A new frontier for world models deepmind.google/discover/... · Posted by u/bradleyg223
Genuinely technically impressive, but I have a weird issue with calling these world simulator models. To me, they're video game simulator models.
I've only ever seen demos of these models where things happen from a first-person or 3rd-person perspective, often in the sort of context where you are controlling some sort of playable avatar. I've never seen a demo where they prompted a model to simulate a forest ecology and it simulated the complex interplay of life.
Hence, it feels like a video game simulator, or put another way, a simulator of a simulator of a world model.
Also, to drive my point further home, in one of the demos they were operating a jetski during a festival. If the jetski bumps into a small Chinese lantern, it will move the lantern. Impressive. However, when the jetski bumped into some sort of floating structure the structure itself was completely unaffected while the jetski simply stopped moving.
This is a pretty clear example of video game physics at work. In the real world, both the jetski and floating structure would be much more affected by a collision, but in the context of video game physics such an interaction makes sense.
So yeah, it's a video game simulator, not a world simulator.
Bjorkbat commented on Genie 3: A new frontier for world models deepmind.google/discover/... · Posted by u/bradleyg223
Genuinely technically impressive, but I have a weird issue with calling these world simulator models. To me, they're video game simulator models.
I've only ever seen demos of these models where things happen from a first-person or 3rd-person perspective, often in the sort of context where you are controlling some sort of playable avatar. I've never seen a demo where they prompted a model to simulate a forest ecology and it simulated the complex interplay of life.
Hence, it feels like a video game simulator, or put another way, a simulator of a simulator of a world model.
Bjorkbat commented on OpenAI claims gold-medal performance at IMO 2025 twitter.com/alexwei_/stat... · Posted by u/Davidzheng
OpenAI simply can’t be trusted on any benchmarks: https://news.ycombinator.com/item?id=42761648
Somewhat related, but I’ve been feeling as of late what can best be described as “benchmark fatigue”.
The latest models can score something like 70% on SWE-bench verified and yet it’s difficult to say what tangible impact this has on actual software development. Likewise, they absolutely crush humans at sport programming but are unreliable software engineers on their own.
What does it really mean that an LLM got gold on this year’s IMO? What if it means pretty much nothing at all besides the simple fact that this LLM is very, very good at IMO style problems?
Alright, it does look pretty charming, and I especially like that it's open-source since pretty much anyone buying a domestic robot is likely to be a tinkerer of some sort, but at the same time it reminds me of the Jibo (https://robotsguide.com/robots/jibo).
For those who don't remember (I couldn't remember the name, only the face, had to look hard for it) it was a desktop robot released in 2014 that was hyped pretty hard at the time. It didn't help that the company that launched it was founded by a fairly well-known MIT professor.
And yeah, it was a flop. The $900 price tag wasn't helping things, but neither was the fact that it didn't really do anything that an Alexa couldn't. You bought it solely because you really liked the idea of robots and thought it was cool, not at all for its value around the house.
I'm not gonna dunk on this too hard since it's probably just a fun company side-project, but I might change my tune if they get too high on hype.
> An individual author doesn't make a meaningful contribution to the training of an LLM, but a large number of popular and/or prolific numbers can.
That's a point I normally use to argue against authors being entitled to royalties on LLM outputs. An individual author's marginal contribution to an LLM is essentially nil, and could be removed from the training set with no meaningful impact on the model. It's only the accumulation of a very large amount of works that turns into a capable LLM.
Yeah, this is something I find kind of tricky. I definitely believe that AI companies should get permission from rightsholders to train on their works, but actually compensating them for their works seems pointless. To make the royalties worthwhile you'd have to raise the cost per query to an absolutely absurd level
Bjorkbat commented on François Chollet: The Arc Prize and How We Get to AGI [video] youtube.com/watch?v=5QcCe... · Posted by u/sandslash
Absolutely. Presumably there is some specific considerations or evidence that helped him evolve his opinion. I would be interested in seeing a writeup about it. With him having been a very public advocate against AGI, a writeup of his evolution seems appropriate and would be very edifying for a lot of people.
I recall it as less an evolution and more a complete tonal shift the moment o3 was evaluated on ARC-AGI. I remember on Twitter Sam made some dumb post suggesting they had beaten the benchmark internally and Francois calling him out on his vagueposting. Soon as they publicly released the scores, it was like he was all-in on reasoning.
Which I have to admit I was kind of disappointed by.
> a consideration of impact on the potential market for a rightsholder's present and future works
This is one of those mental gymnastics exercises that makes copyright law so obtuse and effectively unenforceable.
As an alternative, imagine a scriptwriter buys a textbook on orbital mechanics, while writing Gravity (2013). A large number of people watch the finished film, and learn something about orbital mechanics, therefore not needing the textbook anymore, causing a loss of revenue for the textbook author. Should the author be entitled to a percentage of Gravity's profit?
We'd be better off abolishing everything related to copyright and IP law alltogether. These laws might've made sense back in the days of the printing press but they're just nonsensical nowadays.
Personally I think a more effective analogy would be if someone used a textbook and created an online course / curriculum effective enough that colleges stop recommending the purchase of said textbook. It's honestly pretty difficult to imagine a movie having a meaningful impact on the sale of textbooks since they're required for high school / college courses.
So here's the thing, I don't think a textbook author going against a purveyor of online courseware has much of a chance, nor do I think it should have much of a chance, because it probably lacks meaningful proof that their works made a contribution to the creation of the courseware. Would I feel differently if the textbook author could prove in court that a substantial amount of their material contributed to the creation of the courseware, and when I say "prove" I mean they had receipts to prove it? I think that's where things get murky. If you can actually prove that your works made a meaningful contribution to the thing that you're competing against, then maybe you have a point. The tricky part is defining meaningful. An individual author doesn't make a meaningful contribution to the training of an LLM, but a large number of popular and/or prolific numbers can.
You bring up a good point, interpretation of fair use is difficult, but at the end of the day I really don't think we should abolish copyright and IP altogether. I think it's a good thing that creative professionals have some security in knowing that they have legal protections against having to "compete against themselves"
It’s hard to tell how total that was compared to today. Of course the amount of money involved is way higher so I’d expect it to not be as large but expanding the data set a bit could be interesting to see if there’s waves of comments or not.
It never had a public product, but people in the private beta mentioned that they did have a product, just that it wasn't particularly good. It took forever to make websites, they were often overly formulaic, the code was terrible, etc etc.
10 years later and some of those complaints still ring true