OpenAI models are coming to Google Cloud

Posted by armcat 22 hours ago

Counter295Comment122OpenOriginal

Comments

Comment by anthonypasq 15 hours ago

Sergey better really go into "founder mode" or else Thomas Kurian (head of GCP) will continue to eat Deepmind's lunch.

Between selling TPUs to external companies and now hosting competitor models, Gemini just doesnt have the full faith of the Alphabet capital allocators, and I can't really blame them. A mature company like google would rather just expand their extremely profitable cloud business rather than dump billions into a black hole on the off chance they achieve RSI or superintelligence or whatever.

Comment by bitpush 14 hours ago

> Our cloud business is at any given moment in time forgoing short-term revenues because we are making compute trade-offs to support frontier AI development

Sundar Pichai just this week.

https://finance.yahoo.com/technology/ai/articles/google-ceo-...

Comment by shizcakes 2 hours ago

We’ve been seeing compute allocation failures in GCP us-central1 for about a month now- that can’t be related can it? These aren’t TPUs, this is normal compute for dataflow/etc jobs.

Comment by twoodfin 25 minutes ago

Frontier AI development via agents uses plenty of CPU compute. Maybe in a lopsided way during development as opposed to training?

Comment by scarmig 12 hours ago

This can be just as well phrased as:

> Our frontier AI business is at any given moment in time forgoing short-term development because we are making compute trade-offs to support our cloud business revenues.

Comment by bitpush 9 hours ago

Exactly, the definition of tradeoffs

Infact all companies, at all times are doing tradeoffs. You and I can be doing something else right now but I'm here on HN replying to you - and that's a tradeoff.

If someone suggests -just do X then you don't have to do a tradeoff, that just sounds wrong to me.

Comment by paxy 14 hours ago

What does “capital allocators” mean here? If your implication is that the Gemini org is resource starved and a few extra dollars will put their model in the lead that is 100% incorrect. If anything it’s the bloat and bureaucracy keeping it behind.

Signing partnerships and hosting third party models costs Google nothing, and in fact brings in a ton of revenue that will eventually go back into Gemini.

Comment by skohan 14 hours ago

Infra might very well be the right play. Over the past year, it seems like the evidence is on the side that there's no "secret sauce" in AI, and models will all converge towards similar capabilities. Not much margin to be had for having the best model for a month or less until another lab leapfrogs you by 3%.

Comment by StilesCrisis 11 hours ago

I think RAM turned out to be the secret sauce all along!

Comment by oscarfr 3 hours ago

Google are now offering Claude models in the Gemini "app". https://cloud.google.com/blog/products/ai-machine-learning/w...

I think it makes sense for them to separate the application/productivity layer from the model layer. And offer models from many model providers, including OpenAI.

To stay competitive with Microsoft and other productivity suites.

Comment by mclion 2 hours ago

AI hosting currently generates revenue, so the choice is trivial. Building an LLM model is company PR, while hosting another’s model provides immediate income.

Comment by hax0ron3 15 hours ago

Google uses Gemini for its AI Overview on Google Search results, though. And it seems to me that this is an area where they really have a big incentive to run their own AI that they completely control. AI Overview sits at the top of the Google Search results first page, so when it comes to marketing it's sitting in the most coveted place on the entire Internet.

Comment by anthonypasq 14 hours ago

im talking about research compute allocation

Comment by kenhwang 15 hours ago

It's also marketing that their AI is hilariously terrible because AI Overview is the biggest posterchild for AI slop and often used as an example of how bad Google is at AI. Gemini proper isn't that bad, just the ultra cost cutting version on search is.

Comment by eecks 14 hours ago

It has gotten a lot better and it's useful for information on most normal search queries now.

Comment by bitpush 9 hours ago

Do you think most people around the world want access to ultrathink?

For a lot of folks, something low cost and low intelligence is what they want. And for a few that needs high intelligence, there's stronger models.

Google has both bases covered. It is brilliant.

Comment by sebzim4500 15 hours ago

To be fair the search version of Gemini is nowhere near as bad as it used to be. The bar is in the floor I know.

Comment by londons_explore 16 hours ago

> Enterprises with Google Cloud spend commitments will be able to draw down their GCP commitments for these purchases.

This is a fairly big move by Google... It means revenue they have already booked and reported to shareholders they now might have to give away - potentially all of it. Bean counters hate that!

Comment by eitally 11 hours ago

No. It means customers can turn booked revenues into billed revenue by consuming OAI models on GCP (pass-through contracting). OAI will get their money, too, but Google will charge the customer for actual platform consumption. This was previously missing and GCP customers who wanted ChatGPT or Codex were limited to just making API calls from their GCP projects .

Comment by paxy 16 hours ago

“Reported to shareholders” doesn’t really mean anything. There are thousands of such line items on financial reports. The ones that matter, such as recognized revenue, don’t count future bookings. It is understood that customers renege on these commitments all the time.

Google is simply telling customers - there’s a new Google Cloud service, and you can use it under your existing account.

Comment by jaynate 12 hours ago

That’s not the way it works. Google still recognizes the revenue. The cost of goods sold and gross profit are what get hit here but it’s no different than any other partner than can sell their product through the Google marketplace and take advantage of committed spend.

Comment by redwood 14 hours ago

Let's not quite right... you don't recognize revenue in a consumption business until the usage actually happened

Comment by londons_explore 14 hours ago

pretty sure that's what the "spend commitment" changes. It's a "use it or lose it" agreement. It's as good as a debt the moment it is signed, used or not.

Comment by mediaman 13 hours ago

GAAP does not permit booking contracted future revenue onto the balance sheet or income statement, because the spend is not actually an unconditional debt until the service has been rendered.

It doesn't matter if it's committed or not, it still won't be recognized on the formal financial statements.

Comment by redwood 10 hours ago

My understanding is that "use it or lose it" means you book it the last day of the contract or similar. Anyway not at the front so same point applies

Comment by MisterMunchkin 13 hours ago

I think they declare it as pipeline revenue or something instead of current year revenue

Comment by jaynate 11 hours ago

RPO remaining performance obligations (not on balance sheet) unless prepaid then its deferred revenue (balance sheet). When it’s consumed it is recognized as revenue.

Comment by open592 17 hours ago

I don't understand all the wild speculation and question here - openai is selling a shit ton of inference through third parties. these third parties want to provide inference to their customers who use gcp (and don't want to also inference through bedrock or azure ai foundary.

these are all zdr engagements and thus "enterprise" contracts.

basically it boils down to - anthropic already provides inference direct through GCP, Azure, and AWS - so to compete openai is doing the same.

Comment by felindev 4 hours ago

My work experience with trying to access AI models from GCP has been rather spotty. Gemini works great for work we need it to do, but dynamic usage limits are hitting us hard. And while we tried to use Anthropic models on there, we could not get them fully enabled, for one reason or another.

Comment by nsagent 18 hours ago

Gotta love the disclaimer at the bottom:

  Microsoft remains OpenAI’s primary cloud partner.

Comment by hobofan 17 hours ago

Things that you only say because you are contractually forced to say them.

Comment by merelydev 17 hours ago

Or don't want to crash stock price.

Comment by cmiles8 8 hours ago

Would have been more accurate to write “We are contractually obliged to say that ‘Microsoft remains our primary cloud provider,’ whatever that means.”

Comment by neuroelectron 13 hours ago

Isn't Google renting cloud compute from XAI?

Comment by v4dm 13 hours ago

Yep, and they pay them like 900 mil per mo for it, pretty crazy from the company that first brought TPUs to market

Comment by v4dm 13 hours ago

Google also has a stake in xAI... so they are financing themselves too

Comment by paxy 13 hours ago

That’s putting it lightly. Google was one of the biggest early investors in SpaceX and is currently the second largest shareholder after Musk.

Comment by fakedang 13 hours ago

AI centipede

Comment by The_Blade 13 hours ago

string theory Ouroburos

Comment by echelon 17 hours ago

OpenAI buys from Azure because of the Microsoft investment deal.

OpenAI sells and reaches enterprise customers via GCP.

Enterprises want zero data retention guarantees and they're already on AWS or GCP, so this is how OpenAI reaches those customers and makes more money.

Comment by luma 11 hours ago

Enterprise* customers are overwhelmingly more likely to primarily host on Azure than GCP, m365 runs the corporate world top to bottom. OpenAI doesn't care where the customers come from, more platforms == more reach, they have zero reason to be preferential to any single hosting platform.

* By "enterprise" here I mean "big company that is not a tech company". If most of your userbase is carrying a mac, you're probably a tech company. If you're mostly making widgets in factories, chances are high Windows and the ecosystem around it is involved.

Comment by nullpoint420 10 hours ago

I've heard this described as having a "real" or "fake" corporate job on social media.

If you have a Thinkpad, you have a real job. And if you have a MacBook, you have a fake job.

Comment by sdenton4 6 hours ago

What if I have a MacBook which is primarily used for ssh'ing into Linux machines?

Comment by asdfsa32 4 hours ago

Fake. We all know you actually just talk to an agent all day.

Comment by NewJazz 4 hours ago

You might be Chinese!

Comment by 10 hours ago

Comment by bitpush 9 hours ago

Running m365 doesn't meant their boring ERP system has to run on Azure even if MS would love them to.

These are the companies running SAP and Oracle. It can run anywhere, including GCP and AWS.

Comment by _boffin_ 15 hours ago

Microsoft has had ZDR for inference since pretty much the beginning. Just needed to fill out a form and get approved.

Comment by _diyar 14 hours ago

However, consider this:

> Enterprises with Google Cloud spend commitments will be able to draw down their GCP commitments for these purchases.

Comment by bitpush 9 hours ago

What does this mean?

Comment by manquer 9 hours ago

You can use your preexisting committed spends for GCP on Open AI, i.e. if an enterprise 10M/year contracted spends for first party GCP services, now they can also use that on OAI models hosted on GCP.

Meaning they can swap out compute/storage spends on GCP to LLM model spend or use left over budget ( which would go waste anyway ) and so on . Any OpenAI sales by GCP teams would be considered against their quota typically in this type of arrangement [2]

This is normally not done to third party[1] marketplace services, Azure already does this for OpenAI, it helps drive some enterprise spends as it is from for fungible budgets or different approving authority internally in a org when it is not a new vendor, new budget etc.

----

[1] It is not too hard to get into these types of programs i.e. it not only for OpenAI level partners mid sized startups can also be part of these deals in marketplace, the requirement usually is minimum size and the partner having to prove/show that are spending that customer's revenue on the compute etc so money is going back to them, then AWS/Azure/GCP will consider an org for this type of program.

Given that GCP is hosting these models on their cloud and therefore OpenAI is spending a % of every dollar back on GCP for every token sold like this; it wouldn't have been too hard for OpenAI to got such a program even if they were a mid sized startup.

[2] Multiples maybe involved (also used to push some first party products) 1x would 1M in OAI sales counts as 1M in their Qtr/annual targets 0.5x would mean 500k and 5x means 5M quota would be retired for 1M in sales etc.

Comment by bitpush 8 hours ago

So when cloud deals are signed, it is with a spending figure? Do you get discounts if it is a large number?

Thanks for you reply, it was very enlightening

Comment by disillusioned 6 hours ago

The way they work, you get a preferred price for various services at a substantial discount to list rate, but you commit to a guaranteed spend level for that period. The guaranteed spend is typically account-wide, but you have to pay a true-up at the end of the period if you fail to hit it.

It's a bit of calculus: will your growth increase enough such that, especially with the discount, you'll reach the minimum?

You can see these agreements littering 10-Qs, or similar, like when Snapchat filed their S-1 and revealed:

>When Snap originally filed for its IPO in 2017, it revealed a massive $2 billion commitment over five years ($400 million annually) with Google Cloud.

Comment by manquer 6 hours ago

Yes; larger discounts with more commits, you also get some price guarantees on SKUs quite valuable these days and other benefits - like say premium support, architecture guidance, access to product teams and so forth.

Self-service retail buyers of cloud have access to some of these price advantages as well, 1-3 Year reservations(even paid monthly) are 40% cheaper of the bat after all. PAYG with fluctuating usage is quite more expensive than committed consumption for cloud.

Comment by keerthiko 16 hours ago

isn't the largest portion of openAI's reach to (overwhelming cloud provider leader AWS) enterprise customers via amazon bedrock?

Comment by paxy 16 hours ago

OpenAI only launched on bedrock ~3 months ago, so I doubt it. It has been running on azure since day 1.

Comment by booi 12 hours ago

so.. yeah i guess bedrock is their largest distribution system.

Comment by 11 hours ago

Comment by 123haw 17 hours ago

Preparing for Oracle bankruptcy.

Comment by kps 16 hours ago

There's always a silver lining!

Comment by pixl97 10 hours ago

Do not anthropomorphize Larry Ellison.

Comment by esseph 16 hours ago

I could only hope...

Comment by mlmonkey 14 hours ago

Google should just spin out DeepMind as an independent company and get out of the model building business: just sell tents and shovels as they say, and leave the actual (gold) mining to others.

Comment by hax0ron3 14 hours ago

Google Search is still a large fraction of Alphabet's revenue, and Alphabet needs to be able to deliver AI results for people's searches if it wants to stay relevant in the search game.

I imagine it's much easier to have fine grained control over AI search results if you develop and own the AI yourself.

Comment by dpkirchner 14 hours ago

Last I checked their AI results were terrible, worse than anything I get from ChatGPT or Claude. The only benefit is their AI results are fast.

If I were google there's no way I'd leave their AI overviews in place as-is.

Comment by JacobAsmuth 4 hours ago

Can you give an example of a search query with a terrible AI Overview? Or even better, a class of search queries that produce terrible AI Overviews? Would really appreciate it!

Comment by hax0ron3 14 hours ago

I was thinking about them fine-tuning the results for marketing purposes. But I just did a bit of research and it seems that would probably actually be illegal. So never mind, maybe?

Comment by kccqzy 8 hours ago

Does anyone still remember OpenAI’s founding? I had thought that OpenAI didn’t want to give the weights to Google because the original reason for its existence is to prevent Google from achieving AGI.

Comment by 16 hours ago

Comment by bananzamba 17 hours ago

How are they trusting Google not to destill them

Comment by dumberquestions 17 hours ago

This is speculation, but funnily enough Gemini models show the least evidence of being distilled from OAI or Anthropic compared to everything else in the market.

Comment by BudaDude 16 hours ago

I feel this is correct. Google has probably the most data of any company in the world. For better or worse, they don't need to train on the outputs of other models.

Comment by runeblaze 12 hours ago

At a org like Google Cloud barring rogue L5/6/7s acting on their own any sniffing of distillation will get leaked. Too many chaos exist at a org size of Google to systematically do it given the contract at hand. This is a company where every spicy memo or mild gossip will risk leaks.

Comment by lrvick 17 hours ago

Probably a very strict contract, and letting OpenAI read all the session logs like they do for their own customers.

Comment by iamdelirium 17 hours ago

GCP already hosts Anthropic models so probably trusts and lawyers.

Comment by aviinuo 17 hours ago

Nvidia GPUs let you load encrypted weights onto them and to decrypt on chip so Google would need to have some sort of Nvidia management firmware attack to read from HBM while the GPU is live

Comment by zeroxfe 16 hours ago

Distillation does not require access to weights.

Comment by jayd16 16 hours ago

Isn't distillation more of a black box type affair?

Comment by binlog 17 hours ago

Doesn’t really matter what happens at the GPU level when Google can see all API traffic.

Comment by worldsavior 17 hours ago

Except google cloud doesn't use Nvidia GPUs but TPUs.

Comment by Mond_ 16 hours ago

Google has both. Afaik we don't know whether OpenAI models will run on TPUs or GPUs here.

Comment by tgma 15 hours ago

I bet it would be the NVIDIA, at least initially, not for the encryption capability per se, but for simple fact that adapting to TPU at high performance is non-trivial.

Comment by amelius 16 hours ago

First answer this: How are they trusting OpenRouter not to distill them?

Comment by skohan 14 hours ago

Open router doesn't host the models, so they wouldn't have access to the weights or reasoning traces right?

Comment by amelius 14 hours ago

They can still collect (input,output) pairs.

Comment by platinumrad 5 hours ago

OpenRouter doesn't see the encrypted reasoning traces.

Comment by amelius 2 hours ago

But they do see input and output, which gives them a treasure of training data.

Comment by binlog 17 hours ago

Contracts and lawyers are more powerful than any technical tool.

Comment by amelius 16 hours ago

Yeah like the copyright lawyers of content owners that got their works trained on?

Comment by paxy 16 hours ago

How many trillions of dollars were those content owners worth?

Comment by kridsdale1 16 hours ago

Enough. They just weren’t working in coordination.

Comment by paxy 14 hours ago

You are vastly overestimating the power of publishers. The top book publisher in the US is worth $5 billion. All publishers together are about $50 billion. The New York Times is $11 billion. Fox is about $15-25 billion. Add together every single book publisher, newspaper, magazine, TV/radio, digital publisher and really every print and broadcasting platform in the country and you’d reach about $100-$150 billion. That is roughly 3% of Google.

These companies have a ton of cultural capital, sure, but in American capitalism that means nothing.

Comment by tjwebbnorfolk 6 hours ago

I'd bet the distillation balance-of-trade goes the other way on that one

Comment by WarmWash 17 hours ago

Google Cloud couldn't give a shit about Google Deepmind, and backdooring Clouds customers only stands to hurt Cloud.

Comment by tgma 17 hours ago

America is a high-trust society.

Comment by toephu2 15 hours ago

Japan is a high-trust society. America is the opposite.

Comment by skohan 14 hours ago

The US is medium trust.

Comment by tgma 15 hours ago

Everything is relative, but fair enough.

Comment by paxy 16 hours ago

More like - America is a high-litigation society.

Comment by tgma 16 hours ago

I don't believe threat of litigation alone explains the extent of it.

The way an American company, esp a reputable one like Google, conducts itself internally is different. There will be much more internal backlash to do the wrong thing way before litigation.

Comment by apple1234 3 hours ago

Test

Comment by AznHisoka 16 hours ago

Does this mean OpenAI is going to use TPU now?

Comment by TiredOfLife 15 hours ago

And Google is renting compute from SpaceX.

Comment by paxy 13 hours ago

Google is also the second largest shareholder of SpaceX.

Comment by AndrewKemendo 18 hours ago

We need a new acronym

Google, NVIDIA, OpenAI, MSFT…

GNOM…oh wait no

The strategy that I’ve heard said out loud now by all the major technology companies is “we’re all doing it (AI) together”

In my view this was more or less the political result of the White House summit

So I think we’re seeing what is basically going to be a giant amorphous set of collaborations such that nobody gets taken down, and the whole market doesn’t collapse

If Alphabet, OAI, MSFT, AMZ…etc all agree to be interoperable then they will be acting more or less as a single body in tension.

Whether that’s good or not is a question of your worldview but I think that’s the strategy

Comment by simonw 16 hours ago

Add Anthropic and you get MANGO. MSFT, Anthropic, NVIDIA, Google, OpenAI.

Comment by AndrewKemendo 16 hours ago

Let’s go with Mango!

Comment by paxy 17 hours ago

Giant companies working together isn’t an AI-age phenomenon. Go back 10, 20, 30 years and you’ll see that despite the public perception of competitiveness all the largest tech companies (Apple, Microsoft, Google, Samsung, Amazon) buy/sell to each other, sign partnerships and have tight economic interdependencies.

Comment by AndrewKemendo 16 hours ago

Who said it’s new? :)

However if you look at the trust busting era what preceded it was this exact kind of private “cooperation” to prevent government involvement

Comment by selimthegrim 17 hours ago

This is like that SNL skit with Tim Meadows where Will Ferrell has the F on his shirt on the kids TV show and keeps getting in the inappropriate order

Comment by solyin 17 hours ago

Add in Meta (Muse app is a winner)

MOM-NG GNOMM NGMOM MGNOM MGMNO MGMON ONMGM NOMGM NOMMG

Comment by guystone 13 hours ago

[dead]

Comment by cubefox 17 hours ago

Is this the begging of the end of Gemini?

Comment by iamdelirium 17 hours ago

GCP has been hosting Anthropic models for a while.

Comment by cubefox 17 hours ago

Google owns Anthropic stock.

Comment by tancop 17 hours ago

Why would it? The point of Gemini is provide user facing services cheaper than if they were paying OpenAI, make some money on the side with API credits and position themselves as a leader when investors want to see AI everywhere.

Google Cloud is competing with AWS. Amazon has Bedrock so Google needs to keep up with them by serving both Claude and GPT, along with Gemini and open models.

Comment by dgellow 17 hours ago

No? Why would you think that’s the case? Gemini fills its own niche

Comment by cubefox 16 hours ago

Gemini and GPT fill the same niche. They are direct competitors.

Comment by lemonish97 16 hours ago

"OpenAI models will soon be available in the Gemini Agent"

They definitely need to have a better name if they're going to use 3p models

Comment by torben-friis 16 hours ago

New to Google?

Comment by game_the0ry 16 hours ago

Prediction: google cloud will pivot to being a hyperscaler versus a model + hyperscaler. Kind of like how xAI started hosting anthropic workloads with colossus cluster.

bc the alpha is not the model, the alpha is having the physical infra (chips, power, etc).

Comment by paxy 16 hours ago

A company is allowed to do two things

Comment by hackernudes 16 hours ago

But x.ai also hosts Grok? Why would google stop developing their own models? I think you're right that Google will try to make money from their infrastructure (that's basically what Google Cloud is too).

Comment by ReptileMan 15 hours ago

As much as I like Elon, if Grok was a hero shooter its concurrent steam player count will be in low hundreds.

Comment by cma 16 hours ago

>Kind of like how xAI started hosting anthropic workloads with colossus cluster.

Google has been hosting part of Anthropic's load long before that deal. They had a big degradation due to compiler bug for TPUs in an optimization they made to work around TPUs not being able sample from the logits efficiently.

Comment by barumrho 16 hours ago

Is that a long term prediction? Are you saying that physical infra will always lag demand from here on?