Interesting that the hardware is NVidia Blackwell, not Google TPUs. That means Google will likely have an energy efficiency and cost advantage, and keep their proprietary hardware out of other people's reach.
Getting a whole business set up to build TPU hardware for third parties (design, build, sell, support, etc.) is probably not worth it when there is overflowing demand for TPUs in their cloud already.
Businesses running their own hardware probably prefer CUDA as well for being more generally useful.
Part of the reason for this is likely due to customers preference to have CUDA available which TPUs do not support. TPU is superior for many use cases but customers like the portability of targeting CUDA
A bit thin on detail, but will this require confidential VMs with encrypted GPUs? (And I wonder how long before someone cracks SEV-SNP and TDX and pirate copies escape into the wild.)
The number of folks that have the hardware at home to run it is going to be very low and the risk of companies for leaking it is gonna make it unlikely IMHO.
I did my undergrad internship on federated learning. I was tasked with implementing in a simulator different federated algorithms, so to have a way to compare them in a meaningful way. The last that had to be implemented was FedMA. We didn't manage to do it. That algorithm is absolutely devilish. Every issue that I solved made other two issue arise, and neither my supervisors could help. The sheer idea of matching neurons in different networks might (and does) make sense, but the way the approximate costs are calculated require other 2/3 math papers that I could follow for only the first lines of the abstract. I'm happy for the time I spent in my internship there. I'm also happy it's over
The general understanding of how it works is surprisingly easy though, you can find the paper here https://arxiv.org/abs/2002.06440
That's the point of the privacy scheme. It would only be able to learn things common to multiple clients. Private data wouldn't make it through the noise.
Financial firms with significant on-prem datacenter use will love this as well. My company still stays away from the cloud -- we have 6 DCs in the building, and run everything else out of colocated racks.
I don’t think so. To my knowledge GCP has no approval for classified networks, which is by far the hardest part. Contrast with Azure OpenAI has been approved to run on government networks for over a year now.
This feels like a play for companies in highly regulated industries, GCP has a notable list of biopharma customers.
>Today at Google Cloud Next, we're thrilled to announce another significant milestone for Google Public Sector: the authorization of Google Distributed Cloud Hosted (GDC Hosted) to host Top Secret and Secret missions for the U.S. Intelligence Community, and Top Secret missions for the Department of Defense (DoD).
> Our GDC air-gapped product, which is now authorized for US Government Secret and Top Secret missions, and on which Gemini is available, provides the highest levels of security and compliance.
Banking as well, this is the kind of offering they've been looking for a while. Google just saw the demand decided to jump in while OpenAI and Anthropic probably calculated they don't have the manpower to deal with the support for this.
With a few exceptions for companies with highly secretive data, you do have to be a government agency or working in a highly regulated government-adjacent area for secured private clouds to be a requirement carved in stone and therefore worth investing a ton of extra money into though.
Curious if this was forced on Google Cloud by Sundar, or was it something that Google Cloud as an org wanted to do?
At first glance, it seems Google Cloud might lose some revenue from customers who can now deploy Gemini in-house. On the other hand, it's not a complete loss, since presumably Google Cloud is still involved in providing some underlying tech? Not to mention, some customers would never consider using off-premises setup anyway.
Absolutely many would, especially those with deep pockets. The biggest concern I'm hearing from companies adopting AI, for basically any use case, is data leaving their network. Especially (but not only) in the EU.
I don't understand how Google is willing to do this but won't sell TPUs to other days centers. It should be obvious from Nvidia's market cap that they're missing a huge opportunity.
The only reasons I can think of is they see them as their secret sauce, they don't want to support them for customers long-term, or they don't have the foundry capacity.
It's definitely #3. The GPUs have to first satisfy Google's own computing needs, and only then can they start selling them to others. Given how much training and inference the company is doing and how much demand there is internally it's very unlikely they are able to manufacture loads of extras, especially not profitably.
Readit News
https://cloud.google.com/blog/products/ai-machine-learning/r...
Businesses running their own hardware probably prefer CUDA as well for being more generally useful.
They're in limited supply. Even Google doesn't have enough for their own use.
https://bughunters.google.com/blog/5424842357473280/zen-and-...
Deleted Comment
https://federated.withgoogle.com/
The general understanding of how it works is surprisingly easy though, you can find the paper here https://arxiv.org/abs/2002.06440
This feels like a play for companies in highly regulated industries, GCP has a notable list of biopharma customers.
https://cloud.google.com/blog/topics/public-sector/google-pu...
> Our GDC air-gapped product, which is now authorized for US Government Secret and Top Secret missions, and on which Gemini is available, provides the highest levels of security and compliance.
Deleted Comment
You don't have to be a government agency to not want your company's data all over the place.
At first glance, it seems Google Cloud might lose some revenue from customers who can now deploy Gemini in-house. On the other hand, it's not a complete loss, since presumably Google Cloud is still involved in providing some underlying tech? Not to mention, some customers would never consider using off-premises setup anyway.
Deleted Comment
Deleted Comment