ERNIEFrom the builder
ERNIE 5.0: A 2.4 Trillion-Parameter Unified Multimodal Foundation Model
Baidu details ERNIE 5.0, a 2.4 trillion parameter model trained from scratch to handle understanding and generation across text, images, video and audio within one autoregressive framework. All modalities share a token space, a mixture-of-experts design routes tokens without modality-specific experts and activates only about 3 percent of parameters per token, elastic training spins off smaller sub-models directly, and context grows in stages from 8K to 128K alongside a dedicated reinforcement learning pipeline.
It is a Unified Multimodal Model trained from scratch to integrate text, images, video, and audio within a single autoregressive framework.
Saved . A shortened excerpt from the official blog; the original publisher retains authorship.