Native Multimodality vs. Post-hoc Adaptation

Most labs train vision capabilities by adding adapters to a pre-trained text model. MiniMax argues this is suboptimal because it harms text performance and struggles to converge. Instead, they train M3 as a native multimodal model from the very first step. This approach requires careful handling of interleaved data—keeping images and videos in the training stream rather than masking them—and rigorous reward modeling to prevent the model from collapsing during training. This native integration ensures the model understands text, images, and video as a unified representation, which is critical for agentic tasks that require processing unstructured data like long videos or complex slide decks.

Efficient Long-Context via Sparse Attention

To handle million-token context windows without the prohibitive cost of standard quadratic attention, MiniMax developed the MiniMax Sparse Attention (MSA) architecture. The design uses an index branch to identify high-value information blocks, followed by a sparse attention branch that performs calculations only on those selected blocks. This architecture is designed to be scalable, allowing the team to increase both context length and model size in future iterations. The efficiency gains from this approach are significant, enabling the model to process massive inputs—such as entire books or long-form video—while remaining performant enough for real-time agentic interactions.

Research Culture and Agentic Feedback Loops

MiniMax maintains an open research culture where any team member, including interns, can propose and lead projects. This environment led to the development of their sparse attention architecture. The company uses an internal "agent harness" system where their own frontier models are used to automate research workflows, including data generation and kernel optimization. This creates a self-improving loop: the current model (M3) is actively used to accelerate the development of the next version (M3.1). The team emphasizes that community feedback from open-sourcing their models is a primary driver for identifying weaknesses and prioritizing new features like "thinking effort" and improved multimodal reasoning.