Readit NewsantoineMoPa commented on Show HN: Price Per Token – LLM API Pricing Data pricepertoken.com/... · Posted by u/alexellman
It would be fun to compare with inference providers (groq/vertex ai, etc.).
I'm making a tiny tiny LLM in rust (using candle) to teach myself AI https://github.com/antoineMoPa/rust-text-experiments
antoineMoPa commented on The FFT Strikes Back: An Efficient Alternative to Self-Attention arxiv.org/abs/2502.18394... · Posted by u/iNic
A fully connected layer has different weights for each feature (or position in input in your formulation). So the word "hello" would be treated completely differently if it were to appear in position 15 vs. 16, for example.
Attention, by contrast, would treat those two occurrences similarly, with the only difference depending on positional encoding - so you can learn generalized patterns more easily.
I think that this is the explanation I needed, thanks!
antoineMoPa commented on The FFT Strikes Back: An Efficient Alternative to Self-Attention arxiv.org/abs/2502.18394... · Posted by u/iNic
What I don't get about attention is why it would be necessary when a fully connected layer can also "attend" to all of the input. With very small datasets (think 0 - 500 tokens), I found that attention makes training longer and results worse. I guess the benefits show up with much larger datasets. Note that I'm an AI noob just doing some personal AI projects, so I'm not exactly a reference.
antoineMoPa commented on Forgejo: A self-hosted lightweight software forge forgejo.org/... · Posted by u/thunderbong
It's not clear from the landing page whether it's a git code platform / mercurial / entirely new VCS. I wish it was clearer (looking at the Readme, looks like it's indeed a git hosting platform).
I don't really care about the governance model as a user seeing this landing page for the first time, so I wonder why it's so prominent, vs telling me what the actual product is.
antoineMoPa commented on Show HN: SuperSplat – open-source 3D Gaussian Splat Editor playcanvas.com/supersplat... · Posted by u/ovenchips
Did anyone build a text-to-splat 3D generation model? Seems like it would be pretty straightforward? Should make it really easy to generate assets for video games.
EDIT: yep - https://gsgen3d.github.io/
Am I the only one that really dislikes the syntax choices here?
Nope! I'm also not convinced by it.
antoineMoPa commented on Mira Murati leaves OpenAI twitter.com/miramurati/st... · Posted by u/brianjking
I am glad most people do not talk in real life using the same style this message was written in.
To me, this looks like something chatgpt would write.
> Developers are not Idiots
I'm often distracted and AIs are idiots, so a stricter language can keep both me and AIs from doing extra dumb stuff.