krychu (u/krychu) - Readit News

krychu commented on BitNet: Inference framework for 1-bit LLMs github.com/microsoft/BitN... · Posted by u/redm

One of the things I often wonder is "what will be the minimally viable LLM" that can work from just enough information that if it googles the rest it can provide reasonable answers? I'm surprised something like Encyclopedia Britanica hasn't yet (afaik) tried to capitalize on AI by selling their data to LLMs and validating outputs for LLM companies, it would make a night and day difference in some areas I would think. Wikipedia is nice, but there's so much room for human error and bias there.

krychu · 2 days ago

Unfortunately reasoning ability depends on (or is enabled by) information intake during training. A model will know better what to search for and how to interpret it if the information was part of the training. So there is a trade off. Still I think the question is a practical one. Perhaps there are ideas to focus training on a) reasoning / conceptual modeling and b) reliance on external memory (search etc.) rather than internal memorization.

krychu commented on Hashcards: A plain-text spaced repetition system borretti.me/article/hashc... · Posted by u/thomascountz

krychu · 3 months ago

Self-plug. For anyone working in the terminal: https://github.com/krychu/lrn.

A very simple cli tool, consuming basic txt format. You can use it in a second window while waiting for your compilation to finish.

Recently I’ve been also experimenting with defining QA pairs in my note files (in a special section). I then use a custom function in emacs to extract these pairs and push to a file as well as Anki.

krychu commented on Building more with GPT-5.1-Codex-Max openai.com/index/gpt-5-1-... · Posted by u/hansonw

viking123 · 4 months ago

I think it kind of shines in this type of task. I am building my own game engine and it's very good for this type of refactoring. On some other tasks though, it clearly makes bad architectural decisions imo, like more junior developer might not get them but for instance in my game engine, it often tries to be too generalist like trying to build something akin to Unity that can do all sorts of games rather than focus on the type of game I am building it for unless I very explicitly always say it

krychu · 4 months ago

It’d be probably useful to include this very comment in your system prompt or a separate file which you ask the coding agent to read at the beginning of each session.

krychu commented on 'Attention is all you need' coauthor says he's 'sick' of transformers venturebeat.com/ai/sakana... · Posted by u/achow

samsartor · 5 months ago

I'm skeptical that we'll see a big breakthrough in the architecture itself. As sick as we all are of transformers, they are really good universal approximators. You can get some marginal gains, but how more _universal_ are you realistically going to get? I could be wrong, and I'm glad there are researchers out there looking at alternatives like graphical models, but for my money we need to look further afeild. Reconsider the auto-regressive task, cross entropy loss, even gradient descent optimization itself.

krychu · 5 months ago

BDH

krychu commented on Vibe engineering simonwillison.net/2025/Oc... · Posted by u/janpio

krychu · 5 months ago

It’s great to read in the comments about experiences of others with vibe coding. But I also feel like lots of opinions are not coming from actual experience, or “serious” attempts at vibe coding, and more from theoretical deliberations. I might be wrong.

Here are some of my own high-level experiences / thoughts:

- Perhaps contrary to popular belief I think vibe coding will bring the best software / system architects. This is due to massively shortened feedback loop between architectural idea and seeing it in action, easiness with which it can be changed, and the ability to discuss it at any moment.

- We’re not really coding anymore. This is a new role, not a role of a senior dev reviewing PRs of junior devs. Devs are just best suited (currently) to take on this new role. I came to realization that if you’re reviewing all generated code in detail you’re doing it wrong. You just shifted bottleneck by one step. You’re still coding. You should skim if the code is in line with your high-level expectation and then make LLM maintain an architecture doc and other docs that describe what and how you’re building (this is the info you should know in detail). You can do audits with another LLM whether the implementation is 100% reflecting the docs, you can chat with LLM about implementation at any moment if you ever need. But you should not know the implementation the way you know it today. The implementation became the implementation detail. The whole challenge is to let go of the old and embrace and search for efficiency in the new setup.

- Connected to the above: reading through LLM outputs is a massive fatigue. You are exhausted after the day, because you read hundreds of pages. This is a challenge to fight. You cannot unlock full potential here if you aim at reading and reviewing everything.

- Vibe coding makes you work on the problem level much more. I never liked the phrase “ideas are cheap”. And now finally I think the tides will turn, ideas are and will be king.

- Devil is in the detail, 100%. People with ability to see connections, distill key insights, communicate and articulate clearly, think clearly, are the ones to benefit.

Hope this is helpful for others.

krychu commented on Less is more: Recursive reasoning with tiny networks alexiajm.github.io/2025/0... · Posted by u/guybedo

versteegen · 5 months ago

Awesome work, thanks for writing it up! Replication is absolutely critical, as is writing down and sharing learnings.

krychu · 5 months ago

Thanks, appreciated

krychu commented on Less is more: Recursive reasoning with tiny networks alexiajm.github.io/2025/0... · Posted by u/guybedo

krychu · 5 months ago

I implemented HRM for educational purposes and got good results for path finding. But then I started to do ablation experiments and came to the same conclusions as the ARC-AGI team (the HRM architecture itself didn’t play a big role): https://github.com/krychu/hrm

This was a bit unfortunate. I think there is something in the idea of latent space reasoning.

krychu commented on So You Think You Know C? (2020) [pdf] wordsandbuttons.online/SY... · Posted by u/signa11

AnimalMuppet · 2 years ago

Worse, Reasonable Compiler 1 may give different results, depending on what flags you pass it at compile time.

krychu · 2 years ago

Your code dictates the compiler's behavior. If you grant the compiler flexibility, it's unreasonable to complain about unexpected results.