Readit News logoReadit News
willyyr commented on Mistral: Our first AI endpoints are available in early access   mistral.ai/news/la-platef... · Posted by u/georgehill
ur-whale · 2 years ago
This actually begs the question:

Does anyone know the kind of actual infrastructure something like gpt4-32k actually run on?

I mean when I actually type something in the prompt, what actually happens behind the scenes?

Is the answer computed on a single NVidia GPU?

Or is it dedicated H/W not known to the general public?

How big is that GPU?

How much RAM does it have?

Is my conversation run by a single GPU instance that is dedicated to me or is that GPU shared by multiple users?

If the latter, how many queries per seconds can a single GPU handle?

Where is that GPU?

Does it run in an Azure data center?

Is the API usage cost actually reflective of the HW cost or is it heavily subsidized?

Is a single GPU RAM size the bottleneck for how large a model can be?

Is any of that info public ?

willyyr · 2 years ago
Mark Russinovich shares some of it in this recent Ignite session: https://ignite.microsoft.com/en-US/sessions/49347847-9ae4-43... *I work at Microsoft but have nothing to do with the datacenter engineering or other insights into the details behind it.
willyyr commented on Steam turns 20 today: “We had to try a lot of different things over the years”   rockpapershotgun.com/stea... · Posted by u/haunter
willyyr · 2 years ago
My Steam account is turning 20 today at 4:58pm (CET) and I cannot help but feel a bit old for the very first time. I still remember all my friends hating it so much in the beginning because it would slow our PCs down and cost us valuable FPS in Counter-Strike. It's amazing what Valve has achieved with Steam since then. Happy Birthday Steam!
willyyr commented on OpenAI and Microsoft extend partnership   openai.com/blog/openai-an... · Posted by u/hmate9
willyyr · 3 years ago
Related MS announcement with some more details: https://blogs.microsoft.com/blog/2023/01/23/microsoftandopen...
willyyr commented on Ntfy.sh – Send push notifications to your phone via PUT/POST   ntfy.sh/... · Posted by u/tambourine_man
binwiederhier · 3 years ago
Hey cool, this is my project. Happy to answer questions.

Edit: I am also looking for a new opportunity. If you need a good Staff/Principal Engineer, check out my resume here: https://heckel.io/resume.pdf

willyyr · 3 years ago
Thank you for creating this. I was just looking for something like that to send notifications to my mobile without setting this all up and deploying my own app to the store. Using ntfy now :)
willyyr commented on Show HN: I built an internet speed analytics tool   ronaldlangeveld.com/isp-l... · Posted by u/ronaldl93
willyyr · 5 years ago
Cool project. Looks a little bit like the one i've been using as a basis: https://github.com/brennentsmith/internet-speed-logger I just combined it with a CosmosDB and put the dashboard online with the actual tester running locally of course.
willyyr commented on Pi-hole 5.0   pi-hole.net/2020/05/10/pi... · Posted by u/Iolaum
Doxin · 6 years ago
Setting up automatic updates is probably a good idea.
willyyr · 6 years ago
Can you recommend a good way to make sure the Pi and Pi-hole are always up to date? I imagine a simple cron with pihole -up is not sufficient?

u/willyyr

KarmaCake day555March 24, 2014
About
Technology Strategist at Microsoft

https://willy.me

@willyyr on Twitter

View Original