Running Kimi K3 on MI355X at Better Performance per Dollar Than B300
wafer.ai162 points by ilreb 10 hours ago
162 points by ilreb 10 hours ago
I wish they wouldn't call them "open source models". They aren't open source. They didn't publish the training data. They didn't publish the tools they used to train the model.
They published the weights. It's an "open weight model", a term that it seems nearly everyone has agreed is appropriate. Why is this company not using it?
> They didn't publish the tools they used to train the model.
That's not totally true - for example Ziphu (Z.ai, developers of GLM) have published their Slime RL-training framework, developed together with Tsinghua University.
https://github.com/THUDM/slime
Also, if you read Moonshot's Kimi 3 report, it does give a lot of architectural detail and details on training.
https://github.com/MoonshotAI/Kimi-K3/blob/main/k3_tech_repo...
Some discussion on this by HuggingFace here:
https://www.youtube.com/watch?v=MW8-kqd2SD8
Yes, we all realize that "open weights" is more accurate than "open source", and anyways the source code would not be very interesting - it's the training data and methods that mostly define these models.
Sorry, where exactly is "data" in "source"?
I can understand not wanting to call it open source if they don't give you the algorithm and software used for training, but wanting the training data too? That's completely different
For LLMs the training data is the source of their weights.
You can‘t reproduce the LLM without the same data
My understanding was that there was a lot of non determinism in the training process because of many factors: how GPUs optimise floating point arithmetic, how the compute is distributed between the thousands of GPUs etc
I'm not sure how static training data is either (or how you'd distribute it considering its size and nevermind the legality of sharing copyrighted things).
You'd likely get a model with very similar behaviour but the weights would be different.
Please someone correct me if I'm wrong.
That’s not really true. The models have source code defining their architecture and it is open-source.
People keep trying to shoehorn OSS concepts onto model weights, but the concepts don’t fit because the weights aren’t software. They aren’t compiled code. They are learned parameters to use with a (very big) function that itself is expressed in the code.
So they’re a very valuable asset that complements the code, but they are not the code. You could use randomly initialized weights and the software will work - it will output tokens. They just won’t have useful patterns.
I don’t think OSS definitions have ever required that assets have their source included. For example, artwork is very important to a game, but nobody thinks a game is not open source if it doesn’t come with sketches and a copy of Adobe Illustrator to recreate the artwork from scratch.
Edit: The distinction I would draw between models like Deepseek and models like OLMo is whether they are open science. With a model like OLMo, they have published everything you need to replicate the training experiment. Whereas Deepseek does share a lot of knowledge, but keeps a lot proprietary too.
Your game example is a great one: If a game didn’t come with its 3D models, sound, level files, game data, or assets of any kind, but its engine was open source, I don’t think anyone would say “hey this game is open source”. They’d say the engine is, but that’s it. All of the above things are not code, but they are part of a game and yes, expected to be included in an open source game.
People refer to games with proprietary assets as open source all the time. The source code is open - the assets are not. For one example: https://news.ycombinator.com/item?id=28201265 / https://github.com/beyond-all-reason/Beyond-All-Reason/blob/...
Source as a word has a meaning that doesn't include non-source things.
Doom pioneered this model, the Doom engine was open-sourced while the assets remained proprietary. This is where projects like Freedoom exist, to create an entirely free software equivalent.
I think this misses the point I was trying to make. Both an open source game and an open weights LLM include the necessary assets. But people seem to be saying, “you need everything that was used to create the assets”. For game art, that would include sketches, prior art used as inspiration, project files that aren’t used by the game engine but were used during the design process, and so on - if you apply the same standard. But I don’t think anyone does.
Similarly to this I've never seen a project request anyone license their github PR comments under any license whatsoever even though they're undoubtedly copyrighted and used in the creation of the actual source code.
If they open sourced the training data and code but you had to train the network yourself, would that be open source?
Yes, obviously.
Even though none of us could actually afford to train it?
Some of us might be rich, or get funding, or in the future it could be useful when training is cheaper.
Open source has benefits even if you can run yourself. You can read the code for understanding/insights. Other labs could replicate/build on it.
> Some of us might be rich
I sure wish I had a few 100M of disposable income to train a frontier model.
> or in the future it could be useful when training is cheaper.
I do not think that physics will allow hardware getting that much faster. But maybe we will have different, cheaper architectures by then.
You say that but for like 2 years there was a guy on huggingface releasing quants as TheBloke. No identity no nothing except he likely had a grant or a university job that let him do it. Quants aren't the same as training but it was beyond 99% of people at the time (as you say this is)
Cant stand when you guys try and force something as impossible on the rest of us simply because you could never accomplish it.
I did not say that it is impossible. I just think that we need architectural improvements, or maybe even a fundamentally different approach to get something like Kimi K3 for cheap. The point I was trying to make was that we shouldn't just laze about and hope that hardware improvements will get us there.
(Also, I know Tom Jobbins (TheBloke), and have personally contributed to increase the adoption of GGUF, e.g. in the transformers and ktransformers libraries, so I find the personal dig quite amusing.)
If you would tell people at the start of 20th century how much energy we consume, they may not believe you or think it is wasteful.
Looks like global energy consumption has risen by an order of magnitude from 1900 to 2000: https://www.encyclopedie-energie.org/en/world-energy-consump...
Unfortunately, electricity prices did not fall by the same factor, so I fear that training a frontier model will still cause a an unsustainable dent in my monthly budget.
Prices have dropped by a factor of around 6 in real terms since 1925 for US household electricity.
https://chatgpt.com/share/6a6f3f3b-7624-83ed-a11a-248cb39728...
Yet, I couldn’t afford a PC to run the original Crysis when it came out, now the hardware isn’t an issue anymore
As an example, Grok 4 took $388M to train
Compiling source code also takes computing resources, only the scale is different. It's a very close analogy: source code with training data, and compiled binaries with model weights.
The weights are literally a binary blob.
The weights are the modifiable representation. You modify them with gradient updates.
They are a cooked stew; the ingredients have already been chopped and mixed together. You can add things, yes, but you can't inspect the ingredients.
If we start with the same ingredients and independently make our stews they're likely still going to taste a bit different due to the non deterministic nature of the process. Isn't it similar with training LLMs at scale? Do you get identical weights if you do multiple runs?
I can also open broadcom's antenna firmware in any hex editor. Why are you arguing that makes it open source?
The antenna firmware is software. Model weights are essentially learned settings, and they are not software.
Note also that software is copyrightable and model weights aren’t (in the US anyway).
Not sure why it matters whether weights are software or not.
One can equally argue that firmware is just settings for the CPU.
If weights aren't copyrightable then doesn't this argue against the idea of them being able to have the quality of open-sourceness? Meanwhile the training code could be copyrightable.