ByteDance Seed releases Cola2, a continuous-latent model that compresses meaning then generates text

Official source · auto-listedOriginal title ByteDance-Seed/Cola2

ByteDance Seed released Cola2, a continuous-latent language model that trains semantic compression with discrete text generation. A dynamic-compression VAE encodes text into compact continuous latents; a roughly 4B block-causal Diffusion Transformer (DiT) generates latent blocks; a span-causal token decoder with latent feedback turns them into tokens. Training first jointly fits the encoder and decoder, then adapts the VAE with token-side supervision while the DiT learns a detached latent distribution through Flow Matching. Inference repeatedly proposes a latent block, decodes tokens, and re-encodes committed text. The released checkpoint was trained on 346.03B source tokens and has a reported 60.10 average on eight tasks. Runs may differ (stochastic latents). Paper is arXiv:2610.31305.

Auto-listed by dalao.ai: the headline is a fixed template and the original title is shown as written. The official page is the record; its publisher keeps authorship.