363849473754 (u/363849473754)

363849473754 commented on Mathematical Foundations of Reinforcement Learning github.com/MathFoundation... · Posted by u/ibobev

al_th · 6 months ago

This is entirely doable.

I'm absolutely not versed in RL, but I wanted to understand GRPO, the RL algorithm behind Deepseek's latest model.

I started from a very simple LLM, inspired from Andrej Karpathy's "GPT from scratch" video (https://www.youtube.com/watch?v=kCc8FmEb1nY). Then, I added onto that the GRPO algorithm, which in itself is very simple.

I made a GitHub repo if you want to try it out : https://github.com/Al-th/grpo_experiment

363849473754 · 6 months ago

GRPO project is neat. Would you be willing to do a Karpathy-style explainer, breaking down the algorithm from scratch? It’s hard to understand on its own without prior background knowledge.

363849473754 commented on How to get from high school math to cutting-edge ML/AI justinmath.com/how-to-get... · Posted by u/ahiknsr

JustinSkycak · a year ago

My pleasure! I don't plan on writing any more math textbooks.

I had fun writing them and I'm glad that they are making a positive impact, but since then I've been consumed by my work on Math Academy, which I find even more fun/impactful. (We do have a Methods of Proof course out, which is many times more scaffolded, refined, comprehensive, and generally instructionally superior to any textbook I could write independently, not to mention it's adaptive.)

So, long story short, I enjoyed writing those textbooks and am glad they're seeing the light of day, but I've moved on to a new chapter of life and don't plan on writing any more math books in the future (with the possible exception of something super niche like the math behind maximizing learning efficiency in hierarchical knowledge structures).

363849473754 · a year ago

Understood! How much is it to enroll in your “Methods of Proof” course or in Math Academy more generally? I didn’t didn’t quite understand how it works based on the FAQ, is it lecture-problem based with an interactive testing element for course placement?

363849473754 commented on How to get from high school math to cutting-edge ML/AI justinmath.com/how-to-get... · Posted by u/ahiknsr

JustinSkycak · a year ago

You may already be aware, but just in case there's some confusion, I want to clarify: if your intention is to get from high school math to cutting-edge ML/AI using the most direct / efficient / well-scaffolded path, then the resources that I'd recommend looking at are the ones that I refer to within the main body of the post, which are for the most part different from the books on that page.

(But to answer your question: the math books on that page have "correct answer" solutions where you can tell if you got it right, but not fully-worked-out solutions. Introduction to Algorithms and Machine Learning technically does not have any solutions, but most of the problems involve constructing code implementations that match up with worked examples, or that give a desired result, so you can tell if you got it correct and in many cases you can follow along with the worked example to debug your code if it's not producing the desired output.)

363849473754 · a year ago

Thank you for the response and for making these resources fully available.

Do you plan to make a proofs based book available as well?

363849473754 commented on How to get from high school math to cutting-edge ML/AI justinmath.com/how-to-get... · Posted by u/ahiknsr

downrightmike · a year ago

Looks like just time: https://www.justinmath.com/books/

363849473754 · a year ago

This is phenomenal. I currently don’t have a great internet connection, so the pdfs won’t load. Do these books have solutions?

363849473754 commented on How to get from high school math to cutting-edge ML/AI justinmath.com/how-to-get... · Posted by u/ahiknsr

363849473754 · a year ago

What is the pricing model for this? I’m interested in enrolling.

363849473754 commented on Reproducing GPT-2 in llm.c github.com/karpathy/llm.c... · Posted by u/tosh

karpathy · a year ago

Zero To Hero doesn't make it all the way to a chatbot, it stops at pretraining, and even that at a fairly small scale or character-level transformer on TinyShakespeare. I think it's a good conceptual intro but you don't get too too far as a competent chatbot. I think I should be able to improve on this soon.

363849473754 · a year ago

Thanks! So, you are considering expanding the Zero to Hero series to include building a basic GPT-2 toy chatbot? I believe you mentioned in one of the early lectures that you planned to include building a toy version of Dalle. Do you still have plans for that as well?

363849473754 commented on Reproducing GPT-2 in llm.c github.com/karpathy/llm.c... · Posted by u/tosh

karpathy · a year ago

Hi HN the main (more detailed) article is here https://github.com/karpathy/llm.c/discussions/481

Happy to answer questions!

363849473754 · a year ago

You might have covered this topic before, but I'm curious about the main performance differences between nanoGPT and llm.c. I'm planning to take your "Zero to Hero" course, and I'd like to know how capable the nanoGPT chatbot you'll build is. Is its quality comparable to GPT-2 when used as a chatbot?

363849473754 commented on A group of Motherboard folks just spun up their own new independent outlet 404media.co/welcome-to-40... · Posted by u/ohjeez

jkoebler · 2 years ago

Hey there, Jason from 404 Media here. We're humbled that someone posted this and just wanted to say I'll stick around for an hour or so before I have a few interviews for articles scheduled, if anyone has any questions/thoughts/feedback. We're very grateful for the support and thrilled to be here

363849473754 · 2 years ago

Do you plan to do any investigative pieces on AI alignment? I’d be interested in a piece that interviewed people like Paul Christino, Eliezer Yudkowsky, Chris Olah and the like. Covering opposing views from doomerism to e/acc.