MOSS-Image Technical Report

Reading guide

Problem and motivation

Closed-source image generators (e.g., GPT Image 2, Seedream 5.0 Pro) can reliably produce complex visual communication materials such as posters, infographics, and user interfaces. Open-weight models, however, still struggle with two coupled challenges:

1. Rendering: Faithfully drawing sustained, dense bilingual (English/Chinese) text and adhering strictly to 2D spatial layouts.
2. Specification: Translating brief user prompts (often fewer than 10 words) into rich, structured content plans and coordinate layouts.

Existing text benchmarks like LongTextBench average only 120 characters per target and are near saturation (96–98% coverage), failing to test page-length literary text or complex multi-element compositions.

Key ideas

• Unified Text-Conditioning Pathway: Natural language recaptions and structured bounding-box layouts (JSON) are fed through a single text conditioning pathway (via a frozen Qwen3.5-9B encoder), allowing the backbone to handle both descriptive prose and coordinate-level layout constraints natively.
• Annotate Rather Than Filter: Instead of discarding imperfect images (e.g., watermarks, compression artifacts, blur), the team retains broad visual distributions and explicitly describes these artifacts in the text captions.
• Preserving Line Breaks: Exact newline characters (\n) in annotations are retained to explicitly teach the DiT how text wraps and organizes across lines.
• Agentic Prompt Enhancement (PE): An inference-time planner (GPT-6) routes requests by medium (poster, infographic, UI), retrieves knowledge and visual references, and produces structured layout JSON specifications.
• New Benchmarks: Introduction of LongLongTextBench (LLTB) (averaging 1,339.8 characters across English and Chinese) and RealLayoutArena (RLA) (blind pairwise human evaluations across 61 complex design tasks).

Method

1. Backbone Architecture

• Model: 5B single-stream Diffusion Transformer (DiT), 42 blocks, hidden size 3072, 24 heads (128 dims/head), FFN size 8192.
• Encoders: Frozen Qwen3.5-9B text encoder and frozen FLUX.2 VAE (patch size $2\times 2$, effective stride 16 pixels).
• Feed-Forward: SiTU-GLU with smooth capping $c_{\beta }(v)=\beta \tanh (v/\beta )$: $$\mathrm{FFN}(\mathbf{x})=W_o\left[c_{{\beta }_g}(\mathbf{g})\odot \mathrm{sigmoid}(\mathbf{g})\odot c_{{\beta }_u}(\mathbf{u})\right]$$
• 4D RoPE: Query/key heads are partitioned along coordinates $\mathbf{p}=(r,t,w,h)$ representing reference index, sequence position, width, and height: $${\mathrm{RoPE}}_{4D}(\mathbf{z},\mathbf{p})=[R_r(r){\mathbf{z}}_r;R_t(t){\mathbf{z}}_t;R_w(w){\mathbf{z}}_w;R_h(h){\mathbf{z}}_h]$$ Without external reference images, $r=0$, leaving the leading 32 dimensions unrotated (partial RoPE).
• Training Loss: Packed flow-matching objective over valid latent tokens $N_i$ across batch samples: $${\mathcal{L}}_{\mathrm{FM}}=\mathbb{E}\left[\frac{1}{{\sum }_i{N}_i}\sum _i\sum _{p=1}^{N_i}\frac{1}{C}{\Vert v_{\theta }(z_{{\sigma }_i},1000{\sigma }_i,c_i{)}_p-({ϵ}_i-z_{0,i}{)}_p\Vert }_2^2\right]$$
• Optimizer: Hybrid Muon–AdamW optimizer, applying Muon to eligible 2D matrices and AdamW to vectors/embeddings, yielding ~28–33% faster convergence than AdamW alone.

2. Multi-Stage Training and Supervision

• Trained over 3 stages up to step 310,000, capping target image area at $1024^2$ pixels and text capacity at 4,096 tokens.
• Captions are sampled at short, medium, and long granularities, mixed with explicit layout annotations (scene/element or summary/items JSON schemas).

Results

• LongLongTextBench (LLTB): MOSS-Image achieves 96.3 Mean-6 coverage across 6 language-length bins, outperforming open-weight models including Qwen Image 2512 (64.9 Mean-6). Ideogram 4.0 reaches 63.5 (or 68.1 if safety refusals are excluded).
• 7Bench Layout Control: MOSS-Image reaches 0.8786 ${\mathrm{mIoU}}_{\det }$ and 0.7031 Layout AUC, outperforming Ideogram 4.0 (0.4449 ${\mathrm{mIoU}}_{\det }$, 0.2046 Layout AUC), GPT Image 2 (0.3042 ${\mathrm{mIoU}}_{\det }$), and Seedream 5.0 Pro (0.3273 ${\mathrm{mIoU}}_{\det }$).
• RealLayoutArena (RLA): In pairwise human evaluation on 61 design tasks:
  • End-to-end: MOSS-Image scores 1698.70 Elo, ranking second behind GPT Image 2 (1844.35 Elo) and ahead of Seedream 5.0 Pro (1661.16 Elo).
  • Shared Enhanced Prompts: When all generators receive the same natural-language prompt rewrite, the MOSS-Image backbone ranks first at 1807.34 Elo.
  • Prompt Strategy: JSON layout conditioning obtains 1622.87 Elo, beating natural prose layout (1506.20) and short prompts (1081.70).

Limitations and open questions

• The DiT backbone cannot natively condition on image references or perform iterative localized image editing.
• The Layout Prompt Enhancer relies on external commercial language models (GPT-6); aesthetic planning or factual inaccuracies in LLM outputs directly constrain generation quality.
• Performance on the longest Chinese text tasks (92.4% coverage) still lags slightly behind English long text (97.1%).