Muse Spark Open: Meta's 816B multimodal mixture-of-experts, with open weights
Meta has released Muse Spark Open, an open-weight multimodal mixture-of-experts model with about 816B total parameters, about 42B active per token, and a 1M-token context. Here is what shipped, what the architecture says about running it, and where I think it fits.
Meta has released Muse Spark Open, an open-weight mixture-of-experts model that takes text, images, audio and video as input and generates text. The Hugging Face launch post gives it about 816 billion total parameters, about 42 billion active per token, and a context window of up to one million tokens. Hugging Face calls it "the most capable US open-weights model to date." I'll leave that claim to the benchmarks, but the spec sheet alone makes it one of the more consequential open releases this year.
What shipped
There are two checkpoints: Muse-Spark-Open-816B-A42B in BF16 (about 1.5 TB) and Muse-Spark-Open-816B-A42B-NVFP4, a 4-bit version for NVIDIA Blackwell that the post says loses a negligible amount of accuracy. It is the full-size sibling of Muse Glimmer, the small dense model Meta released in August, and shares its tokenizer. Day-one support covers Transformers, vLLM, SGLang, llama.cpp and MLX, along with Inference Endpoints and TRL for fine-tuning.
Meta's stated target is agentic coding: reading a codebase, multi-file refactors, long debugging sessions, and tool calls. Because the model is multimodal, the same agent can also read a screenshot, listen to a voice note, or watch a screen recording while it works.
The architecture in one picture
Each of the 62 layers pairs grouped-query attention with 256 experts, and a router picks 8 of them per token. The experts hold about 97% of the parameters. A 1.8B-parameter perception encoder handles images and video frames, and a convolutional audio encoder emits one token every 80 ms.
Why it matters for deployment
The key line in the post is that all 256 experts stay resident: you need the memory of an 816B model, and you get roughly the per-token compute of a 42B one. Sparsity saves you FLOPs. It does not save you HBM. The BF16 checkpoint runs across four nodes of eight H200s, while the NVFP4 checkpoint fits on a single B200 node. For most teams, NVFP4 is the one they will actually run.
The context window needs budgeting too. Each token adds about 250 KB of KV cache in BF16, so a full million-token context costs about 266 GB on top of the weights. And reasoning cannot be turned off: effort runs from low to xhigh, and the default is xhigh. If you serve simple queries on the default, you will pay for thinking you didn't need.
My take
I see this as a systems release as much as a model release. At this scale, what makes an open model usable is less whether the weights are downloadable and more whether a team can afford to keep 816B parameters resident and serve experts in parallel properly. Shipping NVFP4 for a single node, recipes for every major engine, and an explicit cache budget lowers that bar about as far as anyone can on day one.
For coding agents, the multimodal input matters more than it might seem. Debugging evidence often arrives as a screenshot of a stack trace or a recording of a broken UI, and today that usually goes through a separate captioning step. One model that reads the image and the repository in the same context removes a lossy hop.
Before committing, I would test three things on my own workloads: whether quality holds at the low and medium reasoning settings, how throughput behaves under expert parallelism at realistic batch sizes, and whether long-context recall holds up near the million-token mark. The weights are open, so all three can be measured instead of taken on trust.