#512K-context

Huawei has open-sourced OpenPangu-2.0-Flash, a Mixture-of-Experts (MoE) large language model with 92 billion total parameters and 6 billion active parameters, supporting a 512K token context length. The release includes model weights, inference code, and training operations. This open-sourcing provides the community with a high-performance, long-context MoE model from a major tech company, potentially lowering the barrier for researchers and developers to experiment with large-scale MoE architectures. It also signals Huawei's growing involvement in the open-source AI ecosystem. The model uses an MoE architecture where only 6B of the 92B parameters are activated per token, enabling efficient inference. It achieves a 512K token context window, and a larger Pro variant (505B total, 18B active) is slated for release in July.