Moonshot’s Kimi uses 20k Nvidia chip cluster from Alibaba

From the article:

> A person familiar with Moonshot’s procurement strategy confirmed that the company does indeed have a channel for accessing Blackwell processors via Southeast Asia. They didn’t specify whether this was a rental channel — which is legal, in most cases — or direct purchases, which are a breach of US regulations. The Information reported this week that Moonshot is seeking additional Blackwell processors to train its next model.

> The 20,000 chips Moonshot accesses via Alibaba, meanwhile, are from Nvidia’s earlier generation of Hopper products, the people familiar with the agreement said.

So the 20k GPUs from Alibaba is only a lower bound on how many you need to train a model like Kimi K3.

The advantage of having more GPUs in any case is not so much that you can train bigger models, but that the turnaround time is faster, so you can run more experiments to dial in training choices. It's entirely possible that Musk has more than enough compute, but can't hire the talent to run all those experiments. (That would also explain why he has excess capacity he can rent to Google.)

> they also mentioned only having a 20K GPU cluster (unclear if NVIDIA, or Huawei).

A few quotes from the transcript:

> Our current computing capacity is approximately 20,000 H-equivalent units, most of which have just arrived within the past month or two

> Regarding the Huawei 950, Huawei currently provides us with 16,000 SIM cards

> A Huawei 950 [cluster] with 16,000 cards is equivalent to only a B-series card [cluster] with 4,000 cards.

> Struggle to build a competitive 1T model (Grok 4.5) with massively more compute (Colossus-1 100-250K GPUs, Colossus-2 500K+ GPUs).

This feels like a very American way of designing things - just throw more horse power at it, bigger is better! The rest of the world is usually a bit more resource constrained and efficient at using those resources.

See also Mustangs vs German sports cars, giant American fridges, giant American suburban McMansions vs livable cities etc etc.

From a superficial look I think the submitted title ("Moonshot built on 20k Nvidia chip cluster from Alibaba") might have been misleading. Article says "That hardware forms a substantial chunk of the overall computing capacity Moonshot uses for its Kimi models", implying that other hardware was also used.

(We've changed the title to what the article says now.)

It's not that surprising to me. Most of the innovation in Chinese models has been in efficiency gains and optimizations. K3 coming from the factory in MXFP4 weights is a pretty relevant factor. Big performance gap probably also due to Moonshot doing QAT. Throw in the fact that Musk has easier access to compute, and I think you have your answer on the disparity.

Genuine question. Is there a time factor part of that equation?

> Musk (who freely admits to distilling OpenAI's models) struggle to build a competitive 1T model (Grok 4.5) with massively more compute (Colossus-1 100-250K GPUs, Colossus-2 500K+ GPUs)

Are you sure they are using all of their compute on training? Didn't they rent out a ton to other AI companies?

Do we believe them? It seems there’s no reason with the resources OpenAI and Anthropic have they wouldn’t be incentivised to build equally efficient pipelines? I think it’s mostly the fine tuning with distillation that gives Chinese labs their big advantage. However I’m pretty sure all the main labs are poisoning the output now when they detect Chinese activity so while K3 looks frontier on tests when you use it it’s miles off Sol and Fable.

One does wonder whether there has been an expiration of the actual weights of Opus at one point.

everything about the current 'a.i' models - east vs west points the wastefulness of western labs - whether in terms of gpu clusters being ran, cost of inference.

the only way is down for the massive valuations and 'a.i' revenue projections.

> Musk (who freely admits to distilling OpenAI's models)

Source?

Yes, continual learning is currently a hot topic since this is what will allow an "AI intern or new employee" to learn on the job rather than be stuck on groundhog day, or pre-trained for every eventuality - needing to anticipate all the quirks and proprietary knowledge of every customer!

Continual learning tends to imply individualized models, else there is no data privacy (the secrets learned on the job at your company now being available to your competition), which really turns the current AI business model of a single centralized model served to everyone on it's head. If every customer has a different model that essentially means the end of batch processing with the same weights loaded into the GPU.

The direction this suggests is a move away from centrally served common models to locally served individual ones, which generally requires them to be smaller, even if some larger companies may be willing to invest in beefier hardware.

I think this is at least in part why the AI companies are trying NOT to implement true continual learning and see if they can instead finesse it by implementing continual compacted(?) memorization instead, since then it's "just" additional context that needs to be recalled and fed into every request, not weights that need feeding into the GPU. I don't think memorization is any substitute for learning, especially learning of practiced skills, but since it's far easier to implement, and non-disruptive to the cloud-based API business model, this is what we will see first.

The recent news of NVIDIA' investment in Sutskever's SSI has a tiny hint of this also, talking about SSI advising on NVIDIA's future architectural direction - apparently pushing it in a different direction than current (cloud-based, pre-trained) models. NVIDIA may be quite happy to see a move towards local models.

It begs an interesting question as well: are frontier labs stuck in the incumbent phase of the "innovator's dilemma," where there's such intense pressure to maintain the flavor/character of their widely-loved existing models, that they dare not invest their resources into radical reimagination of their approach towards architectures for smaller models? A company like Moonshot does not have this cultural limitation.

The frontier labs would be well served in carving out 20k sub-clusters and giving research teams carte blanche in building things with radically different architectures - with full permission to distill whatever they want from the flagship models. We'd expect to see more product lines that feel "different" from the flagship models if this were already being done.

Eventually, multi-modality might even get offloaded as workflows, which might allow models to do this at a fraction of the data and compute required.

I am adding multi-modality to https://github.com/guilt/TinyToT, and I see that dis-aggregating capabilities, very similar to how our own sensory organs work, seems to be paying off quite well.

It feels like we're not really bottlenecked by model performance now anyway. Most cases I find where a model is being an idiot can be remedied by rethinking the context and tooling available to that model.

If they got a lot cheaper and only a little dumber, and we got a little smarter about how we use them, they could appear to the bystander as much more useful than they are.

> It kind of confirms a hypothesis I have that the next phase of AI development will be about getting smaller

isn't that, in the end, the case with all/most technologies?

i can get a petaflop of compute capacity in a dgx spark for relatively cheap nowadays. that used to be a whole supercomputer like 20 years ago.

This is likely what Google has been doing, as it suites them best to have small fast models rather than hulking slow giants. Lower intelligence but way more ability to serve. Anthropic would probably need a datacenter the size of a small country to serve Fable on a Google search/services scale.

Getting smaller is how the last revolution in computing happened.

A VAX 11/780 was good, but an 80386 was a lot better, since the latter could run on 3 AA batteries and the former needed 6,000 watts of 3 phase.

You can only control the compute if it is not commoditized, which at the current moment, it is not, leading to industry players having 75-80% margins.

Someone will find a way to make cheaper compute, and since nothing fundamental changed there (LLM didn't change how silicon were made), that's bound to happen.

Thank you

No one is using that dataset for any serious models. It’s way too small and old.

Huawei A910 is supposed to have native support for MXFP4. The problem is that huawei doesn't have the capacity to build more GPUs right now. Those yield numbers must be really terrible.

Personally, I think 20K nvidias is a stop gap solution because they really don't have the capacity to serve their models to earn any money right now.

Weights will be fetched 4-bit from memory, but compute happens in FP8 (W4A8) or BF16 (W4A16). This is still better than fetching 8-bit weights from memory.

Both W4A8 and W4A16 schemes are supported by Hopper GPUs and commonly used to serve mxfp4 Kimi models on Hopper.

> Either they have Blackwell

From the article:

A person familiar with Moonshot’s procurement strategy confirmed that the company does indeed have a channel for accessing Blackwell processors via Southeast Asia. They didn’t specify whether this was a rental channel...

What do you mean by "it's meaningless" can you expand?

"2.8TB model, fast shipping!"

"what is the point?" There is no point, just zero-sum thinking which isn't compatible with reality.