SenseTimeSenseTime2026/09/1750
We introduce SenseNova U1.5, a natively unified multimodal model built on an 8B-MoT architecture that understands, reasons about, and generates visual content without external vision encoders or variational autoencoders (VAEs). We improve the visual interface through a reconstruction approach that preserves spatial coherence between image patches, and scale up training with curated generation and editing data, improved task modeling, structured prompt augmentation, and training at native resolutions of up to 4K. During post-training, we separately optimize specialists for visual aesthetics, bilingual text rendering, infographic generation, and image editing, then integrate their capabilities through multi-expert on-policy distillation.
Comprehensive evaluations show that SenseNova U1.5 makes significant progress in image fidelity, text rendering, complex compositions, multi-reference image editing, and interleaved generation, while also improving instruction following and maintaining consistency in subject identity, geometric structure, and non-edited regions. Despite limited coverage of structured formats in the generation training data, the model generalizes effectively to long and complex structured visual instructions, further demonstrating that multimodal understanding can transfer to visual planning and creation.
These results show that native unified modeling offers a promising path toward end-to-end systems that integrate perception, reasoning, and creation. We will open-source the training code, including supervised fine-tuning, reinforcement learning, and on-policy distillation. Code and more information about the model:https://www.sensenova.cn/
SenseNova U1.5 Evaluation Overview
Figure 1. Examples of SenseNova U1.5 generating infographics and people.
Figure 2. Examples of SenseNova U1.5 performing image editing and multi-reference image generation.
1 Introduction
In recent years, visual generation has rapidly moved beyond traditional image synthesis to become a general-purpose medium for visual creation. It now encompasses multilingual typography and layout, information-dense design, high-resolution rendering, multi-reference image composition, fine-grained object and background editing, and interleaved generation. This expansion, however, has exposed a fundamental architectural divide: most systems perceive images through pretrained vision encoders (VEs) but generate them through variational autoencoders (VAEs). As a result, understanding and generation operate in different representation spaces—the former optimized for semantic abstraction and the latter for pixel-level fidelity. Although this separation is effective, it limits the extent to which perception, reasoning, and generation can work seamlessly together within a unified visual framework across diverse forms of visual creation.
Native unified modeling takes a different approach by learning directly from pixels and text. SenseNova U1 demonstrated the viability of this paradigm through its NEO-unify architecture, which does not rely on external encoders or VAEs and brings perception, reasoning, and pixel-space generation into a single end-to-end model. However, its deliberately lightweight visual interface reconstructs each visual token as an independent RGB image patch. While efficient, this factorized approach prevents the final stage of image formation from exchanging information across neighboring patches, making seams, texture discontinuities, and geometric inconsistencies increasingly pronounced at high resolutions.
This paper introduces SenseNova U1.5, an 8B-MoT native unified multimodal model for visual understanding, reasoning, and generation. Its first major contribution is a fundamental architectural shift: From independent image-patch prediction to spatial joint reconstruction. Rather than mapping each visual token to pixels in isolation, we project the tokens into a two-dimensional feature field and progressively reconstruct the image through spatial convolutions and Pixel Shuffle upsampling. This allows neighboring regions to exchange information before the pixels are finalized, jointly determining color, texture, and geometric structure instead of processing each patch independently. The resulting design retains the efficiency of a compact visual sequence while substantially improving spatial consistency and supporting native generation at resolutions of up to 4K.
The second major advance is a shift from joint reward optimization to a specialize first, unify later strategy. Visual creation encompasses fundamentally different capabilities, including image generation for visual aesthetics, bilingual text rendering, infographic design, and image editing. Each capability involves distinct reward signals, sampling-trajectory dynamics, and optimization challenges. Jointly optimizing these capabilities within a single policy can entangle competing objectives and dilute task-specific gains. We therefore specialize first and unify later: for each capability, we separately optimize dedicated reinforcement learning (RL) experts using tailored data, rewards, sampling strategies, and regularization methods, and then integrate their complementary strengths through multi-expert on-policy distillation. This distillation is performed along the student model」s own generation trajectories, preserving the experts」 complementary strengths while transferring their capabilities into a unified policy.
Extensive evaluations further show that a single native visual representation can serve as a shared foundation for understanding, reasoning, generation, and editing—without parallel visual processing pathways or repeated conversion between encoder features and VAE latents. Although each 32×3-pixel region is compressed into a single visual token, this compact representation still supports efficient inference while delivering strong performance in image fidelity, bilingual text layout, complex composition, multi-reference image editing, interleaved generation, instruction following, visual content preservation, and fine-grained control. More notably, despite limited reliance during training on fixed generation templates, SenseNova U1.5 generalizes to longer, compositional, and highly structured visual instructions. This suggests that the structural knowledge and planning capabilities acquired through multimodal understanding and reasoning can naturally transfer to visual creation. Taken together, these results show that native unification is more than an architectural simplification: a compact, shared visual representation can efficiently support both 「seeing」 and 「creating,」 while allowing capabilities learned through one form of visual intelligence to strengthen the other.
2 Related Work
2.1 Native Unification of Image-Generation Data in Multimodal Models
Native vision-language models (VLMs) process visual inputs directly without relying on external encoders. Models such as Fuyu-8B, EVE, Mono-InternVL, NEO, Gemma4-12B, and Inkling have steadily narrowed the performance gap with leading modular VLMs. A parallel trend has emerged in visual generation: recent studies model pixels directly, demonstrating that high-fidelity synthesis does not necessarily require highly compressed latent spaces. Building on these advances, an increasing number of native unified multimodal models are seeking to integrate understanding and generation within a single framework. Discrete native models unify multimodal learning through token-level autoregression, while continuous native approaches explore end-to-end modeling without explicit tokenizers or latent bottlenecks. Building on NEO-unify, our SenseNova-U series extends this direction toward a fully native foundation architecture, in which understanding, reasoning, and generation emerge from a shared visual foundation.
2.2 Reinforcement Learning for Diffusion Models
In language modeling, reinforcement learning from human feedback (RLHF) established a general post-training paradigm in which learned rewards guide policy optimization, while PPO enables stable updates relative to a reference policy. Notably, GRPO and DAPO improve efficiency and scalability by removing the explicit value model, introducing within-group relative advantages, and adopting improved online optimization strategies. Online reinforcement learning has also expanded into visual generation: ReFL, DDPO, and DPOK optimize diffusion models using preference-based rewards or policy gradients, while AlignProp and D3PO improve efficiency and reduce reliance on explicit reward models. More recently, Flow-GRPO and DanceGRPO have extended within-group relative optimization to diffusion and flow-matching models, improving preference alignment, compositional accuracy, text rendering, and visual quality.
For flow-matching models, online reinforcement learning must introduce stochastic exploration beyond deterministic ordinary differential equation (ODE) inference. Flow-GRPO enables this exploration through sampling trajectories based on stochastic differential equations (SDEs). Coefficients-preserving sampling (CPS) and Precise improve few-step sampling by better preserving the underlying flow dynamics and balancing exploration with distributional fidelity. Beyond sampling, GRPO-Guard mitigates reward overoptimization through controlled clipping and noise-aware gradient reweighting. Existing approaches to vision reinforcement learning also rely on capability-specific rewards, including preference models that assess perceptual quality and text-image alignment, as well as specialized rewards for typography and editing. Inspired by these advances, we use task-dependent reinforcement learning post-training that combines CPS or Precise sampling with alternating rewards or task-specific rewards to address the distinct optimization requirements of generation and editing.
2.3 On-Policy Distillation for Unified Models
Recently, on-policy distillation (OPD) has addressed the distribution mismatch problem in conventional knowledge distillation by providing students with dense supervision from teachers on trajectories generated by the students themselves. Building on this paradigm, MOPD extends OPD to integrate capabilities from multiple teachers: independently optimized domain experts supervise the student's on-policy sampling trajectories, thereby consolidating multiple reasoning capabilities into a single model. Beyond language modeling, OPD has also been applied to multimodal understanding and reasoning, with teachers supervising multimodal trajectories generated by students. Its applications have recently expanded to visual generation. Specifically, Flow-OPD, DiffusionOPD, and DanceOPD distill task-specialized generators along student-generated denoising trajectories using teacher consistency signals, transition matching, and velocity-field regression, respectively. DiffusionOPSD, meanwhile, removes the external teacher and uses differentiable reward gradients to construct a bounded self-distillation objective. Our approach is complementary to these methods: rather than forcing all tasks to use a single distillation scheme, we retain four task-specialized external experts and use hard routing to feed their supervision signals into the same native pixel-space model. This design integrates specialized capabilities while preserving each expert's task-dependent conditional inputs, guidance methods, and resolution strategies.
3 Method
3.1 Model Architecture
Near-Lossless Visual Interface SenseNova U1.5 retains the lightweight native vision interface proposed by NEO, directly transforming raw images or noised visual inputs into compact token sequences without an external vision encoder or VAE. Specifically, two convolutional projections with GELU activations perform 16× and 2× downsampling, respectively, so that each 32 × 32 image region corresponds to one visual token. Two-dimensional sinusoidal positional embeddings preserve spatial coordinates, while special tokens delimit individual visual blocks. Text is tokenized using the original language tokenizer, after which the visual and textual representations are projected into a shared hidden space and jointly processed by a unified backbone. This design preserves nearly lossless visual information while keeping sequence lengths within a range manageable for large-scale multimodal modeling.
Figure 3. Overview of SenseNova U1.5. Compared with SenseNova U1, U1.5 further improves the near-lossless vision interface at both the encoding and decoding ends. Resolution-aware noise conditioning is extended to 4096 × 4096, and the original per-image-block MLP output head is replaced with a lightweight spatial decoder that uses Pixel Shuffle and 3 × 3 convolutions. These improvements retain the compact representation in which each 32 × 32 pixel region corresponds to one visual token, while enhancing spatial continuity, high-resolution fidelity, and downstream robustness.
Table 1. SenseNova U1.5 configuration. The model uses a higher spatial compression ratio, a Pre-Buffer design, native RoPE for unified spatiotemporal encoding, and a Mixture-of-Transformers architecture for joint multimodal understanding and generation.
For generation tasks, the effective noise magnitude varies with image resolution, making resolution an important condition for the denoising process. SenseNova U1.5 explicitly introduces resolution-dependent noise-scale embeddings,and expands the reference image resolution from 2048 × 2048 to 4096 × 4096 to better support native high-resolution generation. The model normalizes and encodes the corresponding noise scale, then fuses it with the diffusion timestep representation. This enables the denoiser to account for both the current denoising stage and resolution-related noise characteristics, allowing it to adapt more stably to different resolutions and aspect ratios.
Compact image-block partitioning reduces the cost of image modeling, but independently decoding each image block with an MLP can disrupt local continuity, resulting in seams, grid artifacts, and texture discontinuities at high resolutions. To address this issue, SenseNova U1.5 replaces the original MLP output head with a lightweight spatially coupled decoder. As shown in Figure 3, the decoder first restores the visual tokens output by the backbone to a two-dimensional feature map, then progressively reconstructs a full-resolution RGB image through three stages of Pixel Shuffle upsampling. Local convolutions between stages allow adjacent regions to exchange information and jointly determine pixels near image-block boundaries, rather than predicting each block independently.
The decoder restores local spatial interactions before pixel synthesis, enabling global semantics, composition, and fine-grained structure to be jointly optimized within a single end-to-end framework. It is trained together with the backbone network under a flow-matching objective. With only a modest increase in computational overhead, it improves cross-patch consistency and local continuity, reduces boundary artifacts, and enhances the stability of high-resolution generation and downstream adaptation.
Native Mixture-of-Transformers. SenseNova U1.5 retains a native Mixture-of-Transformers (MoT) design, integrating understanding and generation within a single Transformer backbone rather than separating them into two fully independent networks. Clean image-text context and noise-conditioned visual states are interleaved in a unified sequence, allowing semantic representations, visual evidence, and generation dynamics to interact directly through shared self-attention. This design enables the generation process to continuously leverage representations formed during multimodal understanding, without introducing auxiliary fusion modules or cross-space feature transformations.
The attention patterns are structurally designed to coordinate causal language modeling with bidirectional visual interaction. Text tokens attend only to preceding context, while tokens within each clean image patch attend bidirectionally to capture spatial dependencies. Noise-conditioned generation tokens likewise interact bidirectionally within their respective image patches and attend to all preceding clean multimodal context. The reverse information path is explicitly masked, preventing clean representations from accessing randomly generated states. This asymmetric information flow allows the generation process to leverage rich semantic and visual context while preserving the integrity of the representations used for understanding and reasoning.
Crucially, architectural unification does not mean that all positions share the same parameters. Understanding and generation retain separate attention projections, normalization layers, and feed-forward modules, with tokens dynamically routed by type at each Transformer layer. Shared attention therefore serves as an interface for communication across streams, while stream-specific parameters preserve the distinct computations required for perception and synthesis. This combination of dense interaction and parameter specialization allows both capabilities to benefit from a shared representation space without forcing their different optimization objectives into the same computational pathway.
Unified training objectives. SenseNova U1.5 combines autoregressive language modeling, pixel-space flow matching, and perceptual supervision to jointly train understanding and generation capabilities within a single model.
For multimodal understanding, the model sequentially predicts subsequent text tokens based on the existing image-text context and preceding text. By increasing the probability of correct predictions, it learns semantic understanding and multimodal reasoning.
For visual generation, the model learns a continuous transformation from noise to images directly in RGB pixel space. During training, real images are mixed with Gaussian noise whose scale is adjusted to the target resolution, creating noisy states corresponding to different stages of generation. From these states, the model predicts the clean image and uses it to estimate the velocity of the generation trajectory, then minimizes the difference between the predicted and target velocities through a flow-matching loss.
Building on this foundation, the model introduces LPIPS perceptual loss to compare predicted and ground-truth images in feature space. This further constrains structural consistency, local texture, and visual coherence, complementing the supervisory signal in pixel space. The three types of loss are combined by weight and jointly optimized, allowing semantic understanding, image generation, and visual quality constraints to work together so that high-level semantics and low-level visual generation are learned jointly within a unified representation.
3.2 Training Process
SenseNova U1.5 progressively develops native multimodal capabilities through generative pretraining, unified mid-training, and unified supervised fine-tuning (Stages 1–3), as detailed in Table 2. It then undergoes capability-specific learning (Stage 4) and multi-expert same-policy distillation (Stage 5).
Table 2. Training configuration for SenseNova U1.5.
Stage 1: Generative Pretraining. Building on the pretrained understanding branch, we randomly initialize the generative branch and train it with pixel-space flow matching, conditioned on representations provided by the frozen understanding branch. SenseNova U1.5 increases the computational budget and introduces a dedicated native 4K training stage. Training initially uses text-to-image data at resolutions ranging from 256×256 to 1024×1024. During this stage, we train for 180K steps at a constant learning rate of 2×10-4 and a sequence length of 8,196. We then expand the text-to-image data to higher resolutions ranging from 512×512 to 4096×4096 to enhance the model」s native 4K generation capabilities. This stage runs for 100K steps at a constant learning rate of 1×10-4 and increases the sequence length to 20,480. In the final stage, we introduce image editing and interleaved generation tasks and continue training for 185K steps, extending the model」s generative capabilities to a range of downstream scenarios. The mixed training data consists of 60% text-to-image data, 30% image-editing data, and 10% interleaved image-text data. We use a cosine learning-rate schedule, decaying the learning rate from 1×10-4 to 2×10-5, while maintaining a sequence length of 20,480. Starting at this stage, we add LPIPS-based perceptual loss to the flow-matching objective, with a weight of 0.1, to improve the generation of fine-grained visual details and local visual coherence. Notably, this combined generative objective remains unchanged in the subsequent unified mid-training and supervised fine-tuning stages.
Stage 2: Unified Mid-Training. We jointly optimize the two branches, using shared attention to facilitate information exchange across branches while retaining task-specific representations. To balance general-purpose multimodal capabilities with diverse generative capabilities, we construct a mixed corpus consisting of 30% pure-text and multimodal understanding data, 40% text-to-image data, 20% image-editing data, and 10% interleaved image-text data. This mixture exposes the model to complementary forms of perception, synthesis, editing, and multimodal interaction during unified training. The model is trained for 80K steps with a maximum sequence length of 32,768 tokens and a constant learning rate of 2×10-5 . The weights of the understanding and generative losses are 0.1 and 1.0, respectively. This weighting helps preserve pretrained understanding capabilities while assigning greater optimization weight to the more challenging generative objective, enabling stable joint training and more effective capability integration.
Stage 3: Unified Supervised Fine-Tuning. We further fine-tuned the model on carefully curated, high-quality instruction-following data, with a task composition similar to that of Stage 2. This stage further enhanced instruction following and consolidated the capabilities acquired during early training into a unified model. Training ran for 10.5K steps using a cosine learning-rate schedule, decaying the learning rate from 2×10-5to 0 while retaining the same loss coefficients as in Stage 2 to maintain a balance between the understanding and generation objectives.
Figure 4. The post-training process for SenseNova U1.5, including multi-expert reinforcement learning and on-policy distillation.
Stage 4: Multi-Expert Reinforcement Learning. As shown in Figure 4, we trained four experts separately for aesthetics, text rendering, infographic generation, and image editing. Each expert used task-specific data, rewards, sampling, and regularization settings.
(1) Aesthetic Expert. Optimizing only for visual preferences may improve overall appearance at the expense of text legibility. Therefore, we alternated between aesthetic preference data and typography data across training epochs to train the aesthetic expert. Samples from each data partition were routed to task-specific rewards: HPSv3++ evaluated perceptual quality and prompt-image alignment, while a bilingual OCR reward based on PaddleOCR measured text fidelity. This routing avoided mixing rewards with different scales and semantics while preserving typographic accuracy during preference optimization.
For each prompt, we generated 16 candidate results using 30-step trajectories, a guidance scale of 4.0, and a timestep shift of 3. We used coefficient-preserving sampling (CPS) with η = 0.7 to reduce the limited-step artifacts associated with conventional SDE sampling while introducing stochastic exploration.
3.3 Reward Modeling
4 Data Construction
4.1 Image Generation Data
4.2 Image Editing Data
4.3 Interleaved Image–Text Data
4.4 Reinforcement Learning Training Data
5 Experiments
6 Conclusion
