Readit News logoReadit News
py4 commented on My new hobby: watching AI slowly drive Microsoft employees insane   reddit.com/r/ExperiencedD... · Posted by u/py4
WalterGR · 7 months ago
py4 · 7 months ago
Had missed it. Tnx
py4 commented on Training for one trillion parameter model backed by Intel and US govt has begun   techradar.com/pro/the-gpt... · Posted by u/goplayoutside
py4 · 2 years ago
It's not clear from the article whether it's a dense model or MoE. This matters when it comes to comparing with GPT-4 - in terms of # params - which is reported to be MoE
py4 commented on Training for one trillion parameter model backed by Intel and US govt has begun   techradar.com/pro/the-gpt... · Posted by u/goplayoutside
huijzer · 2 years ago
Karpathy in his recent video [1] agrees, but at this point scaling is a very reliable way to better accuracy.

[1]: https://youtu.be/zjkBMFhNj_g?si=eCH04466rmgBkHDA

py4 · 2 years ago
This. We have not exhausted all the techniques at our disposal yet. We do need to look for a new architecture though, but these are orthogonal

u/py4

KarmaCake day189November 2, 2021View Original