There has been some good research published on this topic of how RLHF, ie aligning to human preferences easily introduces mode collapse and bias into models. For example, with a prompt like: "Choose a random number", the base pretrained model can give relatively random answers, but after fine tuning to produce responses humans like, they become very biased towards responding with numbers like "7" or "42".
Readit Newstehsauce commented on People are just as bad as my LLMs wilsoniumite.com/2025/03/... · Posted by u/Wilsoniumite
tehsauce commented on Show HN: Beating Pokemon Red with RL and <10M Parameters drubinstein.github.io/pok... · Posted by u/drubs
tehsauce commented on Show HN: Beating Pokemon Red with RL and <10M Parameters drubinstein.github.io/pok... · Posted by u/drubs
Can't Pokemon be beaten by almost random play?
It's impossible to beat with random actions or brute force, but you can get surprisingly far. It doesn't take too long to get halfway through route 1, but even with insane compute you'll never make it even to viridian forest.
Anyone interested in watching lots of reinforcement agents playing pokemon red at once, we have a website which streams hundreds of concurrent games from multiple people’s training runs to a shared map in real time!
https://pwhiddy.github.io/pokerl-map-viz/
(works best on desktop)
tehsauce commented on Ask HN: Resources for general purpose GPU development on Apple's M* chips? · Posted by u/thinking_banana
wgpu has its own Metal backend that most people use by default (not MoltenVK).
There is also a Vulkan backend if you want to run Vulkan through MoltenVK though.
the metal backend does currently generate quite a lot of unnecessary command buffers, but in general performance seems solid.
I haven’t gone through the paper in detail yet but maybe someone can answer.
If you remove the hidden state from an rnn as they say they’ve done, what’s left? An mlp predicting from a single token?
tehsauce commented on · Posted by u/Hyper_Spire
The water consumed to produce a single hamburger is over 2000 liters, and the power likely well over 100 watt-hours.
That means gpt can write >1000 emails using the resources of feeding a single person lunch. The resource efficiency of these machines already is really quite astonishing.
Awesome article! Something slightly misleading though - the first image shows the intersection of a non-convex shape, but it isn't revealed until much later that the algorithm only works for convex shapes, not the type shown in the first image.
tehsauce commented on Grokked Transformers Are Implicit Reasoners arxiv.org/abs/2405.15071... · Posted by u/jasondavies
I just learned about grokking; reminds me of double descent, and I looked up a 2022 paper called "Unifying grokking and double descent". I'm still unclear on what the difference is. My basic understanding of double descent was that the regularization loss made the model focus on regularization after fitting the train data.
Grokking is a sudden huge jump in test accuracy with increasing training steps, well after training accuracy has fully converged. Double descent is test performance increasing, decreasing, and then finally rising again as model parameters are increased.
I didn't see it in an obvious place on your github, do you have any plans to open source the training code?