816 billion parameters, 42 billion awake at once

Meta has released Muse Spark Open, a mixture-of-experts model with open weights that accepts text, images, audio and video and returns text. The published figures are roughly 816 billion total parameters with about 42 billion active per token, a context window of up to one million tokens, and 256 experts per mixture-of-experts layer of which 8 are selected for each token. Checkpoints ship in BF16 and NVFP4.

That ratio is the point of the architecture. Only a small share of the model takes part in any one token, which is what makes a model of this size servable at all. Hugging Face describes it as the most capable US open-weights model to date, and as the mixture-of-experts design introduced with Muse Glimmer applied at a much larger scale.

Meta describes the model as coding-optimised and trained on real developer workflows such as multi-file refactors and long debugging sessions. It is built to plan, act, observe and correct itself over long tasks, including browsing the web with images. Because the other modalities are inputs rather than outputs, the same model can read a screenshot, listen to a voice note or watch a screen recording while it works on code.

The checkpoints are published as Muse-Spark-Open-816B-A42B and an NVFP4 variant, and the accompanying post covers running and fine-tuning the model. For anyone sizing hardware the two parameter counts answer different questions, and conflating them is the usual way to get the estimate wrong. The expert routing is where they diverge: 256 experts sit in each mixture-of-experts layer and 8 are selected per token, so the model that has to be stored and the model that has to be computed are different sizes by design.