Readit Newsamasad commented on Replit's new AI Model now available on Hugging Face blog.replit.com/replit-co... · Posted by u/todsacerdoti
I know a lot depends on architecture and number representation, but do people have a sense for how big a compute cluster is needed to train these classes of models from 1.5B, 3B, 7B, 13B, 70B?
Didn’t Meta say they trained on 2k A100s for LLama 2?
We're on a budget :) trained on 128 H100-80GB GPUs for a week (200B tokens over 5 epochs, ie 1T tokens).
Tech talk here with timestamp: https://www.youtube.com/live/veShHxQYPzo?si=UlcU9j2kC-C4oWvj...
amasad commented on Changes to Hosting on Replit blog.replit.com/hosting-c... · Posted by u/marjipan200
i stop using replit after it started taking minutes to install crates...
It started out so awesome, professor started using it instead of eclipse, it was super easy to teach on, and group projects were made easy for pair programming.
Crates is especially challenging -- the build process is very expensive on all resources (cpu, ram, and disk), and packages are very hard (impossible?) to cache. It works super well on the Pro plan.
Perhaps there is some weirdness if you’ve signed up with GitHub. Feel free to email me and we can take a look: amjad@repl.it