Readit Newsremilouf commented on .txt raises $11.9M to make language models programmable techcrunch.com/2024/10/17... · Posted by u/cpfiffer
The llama.cpp --grammar flag just raised $12m from European investors.
This is actually pretty funny.
remilouf commented on Show HN: Infinite Testimonials with FastHTML, Outlines, and Modal github.com/aastroza/fasth... · Posted by u/cpfiffer
So it's a bullshit generator? Hopefully this is just for demos.
That’d be a pretty inefficient way to generate bullshit at scale
remilouf commented on · Posted by u/remilouf
LLM evaluations are very sensitive to the details of the prompt's structure. This post shows how using structured generation reduces the results' variance and the ranking shifts.
my concern with grammar based sampling is that it makes the model dumber: after all, you are forcing it to say something else than what it thought would be best.
Looks like it’s quite the opposite: http://blog.dottxt.co/performance-gsm8k.html
That whole structured generation line of work looks promising. I hope someone else takes this and runs evaluations on other benchmarks. Curious to see if the results translate!