Always had the curiosity to put something together with pytorch but it always seemed either a steep learning curve or there wasn't a big motivator (project, problem to solve, something in my daily routine to optimize).
Does anybody have a good starting point to learn with hands-on projects and also that could accommodate for flexattention?
IMO the PyTorch getting started tutorials are really good (https://pytorch.org/tutorials/beginner/basics/intro.html).
A classifier for handwritten digits in the MNIST dataset is generally considered the "Hello World" of neural networks. I went over it in a course, but there are countless tutorials to be found online, i.e. https://www.digitalocean.com/community/tutorials/introductio...
Once you begin to understand how to handle data and how to define layers, you can start playing around with whatever your heart desires. The rabbit hole is vast and endless :)
Also the results were not great. Are there any good embeddings api providers?
- JinaAI (https://jina.ai/embeddings/) v3 and v4 performed well in my testing. - Google's Gemini-001 model (https://ai.google.dev/gemini-api/docs/models#gemini-embeddin...).
Overall, both were surpassed by Qwen3-8b (https://huggingface.co/Qwen/Qwen3-Embedding-8B).
Note, this was specifically regarding English and Code embedding generation/retrieval, with reranking.