After DiT.
Decode, transfer, encode.

A pipelined post-DiT system

Each decoded temporal window is spatially tiled across eight GPUs. Rank zero gathers and blends those tiles, then emits only temporally finalized frames. Video transfer and CPU encoding can begin while later VAE windows are still being decoded. Audio follows a separate branch and joins at muxing.

Figure 1. Dependency graph, not a timing diagram. Separate streams and threads permit overlap; they do not guarantee that all work is hidden. The API still returns a complete MP4, so early encoded packets are not yet client-visible video.

For the current five-second request, seven temporal windows produce 7 × 17 + 5 = 124 frames. The 28 spatial tiles per window yield 196 real tile outputs across eight ranks. The scheduler pads collective buffers where needed, but never decodes or presents padding tiles. The five-frame overlap is an internal VAE reconstruction rule; it is distinct from inter-clip motion context in a livestream.

Remove repeated work before lowering precision

The exact-op installer combines Q/K RMS normalization with RoPE, uses fused SiLU-and-multiply for the feed-forward activation, and combines scaled residual operations while preserving the qualified rounding behavior. It also persists decoder linear weights in FP16 on the unquantized path, matching the casts already performed by FP16 autocast. This avoids repeatedly rebuilding equivalent weight casts. The encoder remains FP32.

That FP16 residency change is superseded by resident packed weights when NVFP4 is enabled. The exact operators remain active. Native blocks expose fused QKV, output, FF input and FF output projections: four linears per block, or 144 total. The stock FastVideo representation instead exposes separate Q/K/V projections and has 216 linears for the equivalent 36-block decoder.

Figure 2. September 6 real-weight one-window A/B, one SM120 GPU; latent [1, 24, 7, 44, 80], tile size 256. Two warmups and ten CUDA-event samples per arm. Median 2.287 → 1.723 s (24.65% lower; 1.327×). Peak allocated memory 11.308 → 6.811 GiB. Output was bitwise equal. Stock ran first; the installer changes weights in place. This is the combined installer result, not a whole-video or eight-GPU speedup.

Table 1. Isolated operator measurements from two campaigns. Q/K norm–RoPE and residual: 50 CUDA-event samples after five warmups; shapes [4, 1797, 32, 64] and [4, 1797, 2048]. RGB and blend: September 18, seven randomized rounds after one warmup, ten calls per sample; a 17 × 704 × 1280 frame block and a 22 × 256 × 256 tile with 96-pixel overlap. The RGB baseline already caches normalization constants. These component savings are not additive E2E savings.

Batch tiles without changing the assembly contract

The paired scheduler distributes two temporal windows together. Mixed batching groups up to four independent tile decodes while preserving each tile's position in the gather schedule. Attention output projection remains at its original B1 shape and stride. On the current request, every rank makes seven batched decoder calls instead of separately invoking the decoder for its 24 or 25 tiles; output projections still run per tile. Batching is an execution change, not a blanket proof of bitwise equivalence for arbitrary GEMM shapes.

No isolated full-request speedup for pairing or B4 batching is assigned here. Their inclusion is verified by source and dispatch counters; the measured decoder A/B above has its own explicitly stated scope.

NVFP4 is a measured precision tradeoff

The native NVFP4 path replaces only the 144 transformer-block linears. Original FP32 weights are quantized once at installation and remain packed on the GPU. Each invocation dynamically quantizes its FP16 activations, executes cuBLASLt block-scaled GEMM, and returns FP16. Activation quantization and scale reduction remain in the timed decode path.

Table 2. September 16: same native DiT, full moon-teahouse prompt, seed 1101, four forwards, 1280 × 704, 362 frames, eight GPUs. One warmup and one formal HTTP-to-MP4 request per arm. The combined 529 ms difference is an observation from single samples; the extra 28 ms attributed by subtraction to early gather is not a demonstrated stable gain. Synthetic-latent interleaved VAE-only trials gave 3.087443 versus 3.086844 s, with identical raw FP32 output.

For this native NVFP4-versus-MXFP8 video, all 362 decoded frames gave mean SSIM 0.972081 and PSNR 40.057 dB; decoded audio PCM matched. These are similarity measurements against an already quantized reference, not a lossless-quality claim.

Independent precision-only quality evidence

The original FastVideo gallery also tests VAE MXFP8 and NVFP4 with DiT unchanged. Across seven prompts, VAE input latents exactly match Original and decoded audio is identical. Only the selected decoder GEMMs change. This is a separate implementation and experiment from native serving above.

Table 3. Original FastVideo baseline, 1280 × 704, 361 decoded video frames at 24 fps, seed 1101. Metrics use aligned decoded 8-bit YUV420p MP4 frames, without resizing, cropping or temporal search, and include codec effects. Final row is the arithmetic mean of seven per-prompt scores, not pooled PSNR. Similarity to Original does not establish perceptual acceptability or long-session identity stability.

Figure 3. The existing moon-teahouse precision-only clips, reused without re-encoding. These are approximately 15-second quality examples, not the five-second timing run. Browser playback synchronization is approximate. All seven prompts · Original ablation validation · Per-prompt metrics.

Move work off the serial tail

VAE gather uses a dedicated NCCL stream and FP32 payloads. Early gather sends the first four decoded tiles while the remaining three are computed; rank-zero assembly can overlap the next decode round. CUDA events and stream ownership retain buffers until the consuming work finishes. This VAE gather path is separate from the custom RDMA used inside DiT.

The audio VAE starts on an auxiliary stream before video decoding. After the waveform is ready, a separate stream transfers it to pinned CPU memory; a background thread encodes AAC. Finalization reuses the encoded packets, preserving codec settings, extradata, timestamps and mux order. This reschedules audio processing; it does not replace the audio decoder or change its precision.

Figure 4 and Table 4. September 18 scheduling A/B, one warmup plus three formal complete HTTP requests per configuration; 1280 × 704, 362 frames, seed 1101. First DiT step BF16, remaining three MXFP8/Sage; NVFP4 VAE in every arm. Dots show all formal samples, vertical marks show medians; horizontal axis is expanded. Every warmup and formal MP4 matched the corresponding original reference byte for byte.

Selected change: cached constants + early AAC. Median request latency fell 15.240 → 15.044 s, or 196 ms (1.29%). The logged audio-finalize scope fell from approximately 152–153 ms to 6–8 ms. Early first-window assembly did not demonstrate a clear additional gain and remains disabled. All three selected samples were still above 15 s.

Cached normalization constants remove per-chunk constant transfers and the repeated host-visible standard-deviation check. Its initial implementation retained the original FP32 subtract/divide operations. The current fused RGB path incorporates the same cached-constant semantics, so these savings must not be counted a second time.

Fuse the output while preserving rounding

The five-frame overlap stays in floating point until temporal blending is finished. Only finalized pixels are converted to uint8. Quantizing the overlap earlier would introduce an additional numerical change and is not part of this path.

Figure 5. September 18 full-model output-fusion A/B; all four DiT steps MXFP8/Sage, NVFP4 VAE, identical CRF18/SLICE/from_ndarray output settings. Baseline requests: 13.303 / 13.334 / 13.362 s; fused: 13.212 / 13.261 / 13.336 s. One warmup and three formal requests per arm, 362 frames at 1280 × 704. Median improvement: 73 ms (0.55%). This is a separate baseline from Figure 4.

Thirty-six standalone CUDA checks passed. Real-model warmup compared 965 FP32 blend outputs and 22 RGB chunks with zero fallback; all matched. Complete candidate MP4 files matched the contemporary baseline. The output pipeline span's median fell 2.291 → 2.210 s, but this span overlaps upstream decoding and cannot be added to the 73 ms E2E improvement.

The RGB kernel improved 2.842 → 0.193 ms per measured main chunk, a 14.69× component speedup. The full request improved by 0.55%. This difference is expected when most media production already overlaps GPU decoding.

Transfer finalized bytes; consume them in order

The chunked output path avoids assembling and transporting a full-resolution floating-point video on rank zero. GPU conversion produces RGB24 bytes before D2H. At the current 1344 × 768 resolution, 124 frames require 366.19 MiB of RGB uint8 payload, compared with 1,464.75 MiB for the same RGB pixels in FP32: four times fewer transfer bytes by dtype alone. This byte-count ratio is not a measured fourfold latency gain.

An earlier single-output fast path also removed a redundant full-clip torch.cat: one generated sample can be returned directly, while true multi-output requests retain concatenation. The chunked-MP4 route goes further by never constructing that full decoded output tensor. These are related stages in the output architecture, not two independently measured savings to add together.

CPU candidates: measured separately, not yet active

The strongest standalone CPU candidate combines x264 FRAME threads, constant frame rate and VideoFrame.from_numpy_buffer. Borrowing the existing RGB buffer removes the frame-wrapper copy. The most useful isolated comparison holds FRAME12/CFR/CRF17 fixed and changes only copy → view.

Table 5. Two existing H3 clips, each decoded to 360 RGB frames, 1280 × 704, 24 fps. One warmup and seven randomized rounds per configuration. Bulk includes frame wrapping, initialization, RGB-to-YUV conversion, encode/mux and flush/close; it excludes input loading, GPU work, D2H, audio and file I/O. Replay feeds the same pixels at 22 historical D2H-ready times, with the five-frame tail trimmed to three. It does not reproduce live GPU contention, backpressure or pinned-memory NUMA behavior.

With FRAME12 fixed, buffer views reduced teahouse consumer time 249.80 → 147.87 ms, including frame-wrapper work 115.50 → 1.72 ms. Decoded pixels matched for that copy/view comparison. The combined SLICE/CRF18 → FRAME16/CRF17 change also changes encoding parameters, so the entire combination must not be labeled lossless. Two-slot overwrite-and-reuse tests validated the borrowed RGB buffer lifetime for the tested PyAV 18.1/libx264 conversion path.

FRAME16 returns its first packet after 17 input frames in the tested CFR configuration. That fits one main VAE output chunk, but it does not mean the HTTP client receives a frame at that time. The complete-MP4 API boundary remains unchanged.

Current five-second runtime: CPU libx264, CRF18, ultrafast, automatic thread count, observed SLICE behavior, and from_ndarray. FRAME16, CRF17 and buffer views remain candidates. NVENC is disabled; standalone encoder throughput does not establish a full H3 gain at matched quality.

More pinned slots, smaller D2H chunks and GPU YUV conversion remain unadopted ideas. Current logs show no producer backpressure at millisecond precision, and the historical D2H copy cost is small. CPU colorspace-thread experiments produced only small, overlapping replay-tail differences; neither those changes nor a hardware encoder are counted as deployed improvements.

Separate actual transfer from waiting for frames

Figure 6. Existing Nsight Systems trace of one 362-frame Config A request, with time zero at the video-VAE CPU scope entry. The scope includes submission and waits; it is not continuous GPU execution. Narrow D2H spans are actual CUDA memcpy intervals. The plot is derived from recorded timestamps, not a simulated schedule. Configuration: first DiT step BF16, later MXFP8/Sage, NVFP4 VAE, cached constants and early AAC, before later output fusion. Profiled timing is diagnostic, not the current five-second latency.

The trace contains zero cudaStreamSynchronize calls inside the 22 normal output submission ranges. Early AAC runs from 12,948.704 to 13,123.102 ms after request origin and completes before video GPU work ends at 15,417.805 ms. These observations validate overlap mechanisms. Summing kernel time across streams, or treating the 695 ms consumer wait as removable copy overhead, would overstate the remaining opportunity.

Raw traces are not republished here. The sanitized timestamp series and component aggregates are included in the downloadable evidence, with source-artifact hashes.

What the published benchmark actually executes

The optimized + PR #5317-fences arm runs on eight GB202 GPUs with TP1/SP8, four DiT forwards, seed 1101 and the unchanged moon-teahouse prompt. Two warmups precede five formal requests. It generates 124 video frames at 24 fps, or 5.1667 seconds, at native 1344 × 768. The tables below are extracted from that run, not projections from the older 15-second experiments.

Table 6. Five formal samples; log-derived scopes use millisecond precision except AAC's dedicated measurements. Median output-pipeline span is 657 ms, while median complete request is 4,673.30 ms. The pipeline span begins only when the first finalized chunk is submitted: it is neither total post-DiT latency nor VAE-only latency. An additive five-second post-DiT critical-path breakdown requires a matching profile; the older trace cannot substitute for it.

Remaining work, ordered by evidence

First, qualify the CPU buffer-view candidate inside the complete five-second path, holding encoder parameters fixed to isolate the copy removal. Then evaluate FRAME/CFR settings separately, including buffering, video quality, timestamps and the final response tail. The CPU replay suggests an opportunity; it does not prove a live gain.

For larger decode gains, investigate the actual VAE compute path: activation quantization, scale reductions, activation/normalization boundaries, and B4/B3 GEMM shapes. The historical profile identifies work to study, but none of its kernel totals is a guaranteed saving. More D2H slots are lower priority until measured backpressure or transfer saturation appears.

Long-video identity drift, audio articulation and livestream viewer latency require separate evaluations. Decoder precision metrics, byte-exact output transforms and faster MP4 production do not by themselves establish those product-level properties. Twitch buffering, network transport and the interaction harness are outside this report's measured boundary.

MINIMAX_H3_VAE_DECODER_STREAM_TEMPORAL_CAT=1
MINIMAX_H3_VAE_NVENC_PIPELINE=0
VLLM_OMNI_H3_FULL_VAE_AUDIO_OVERLAP=1
VLLM_OMNI_H3_FULL_VAE_EARLY_AAC=1
VLLM_OMNI_H3_OUTPUT_FUSION=1
VLLM_OMNI_H3_VAE_BATCH4=1
VLLM_OMNI_H3_VAE_CACHED_NORMALIZE=1
VLLM_OMNI_H3_VAE_EARLY_GATHER=1
VLLM_OMNI_H3_VAE_FIRST_WINDOW_EARLY=0
VLLM_OMNI_H3_VAE_GATHER_OVERLAP=1
VLLM_OMNI_H3_VAE_INT8_CONVROT=0
VLLM_OMNI_H3_VAE_MIXED_BATCH=1
VLLM_OMNI_H3_VAE_MXFP8=0
VLLM_OMNI_H3_VAE_NVFP4=1
VLLM_OMNI_H3_VAE_PAIR_PIPELINE=1
VLLM_OMNI_MINIMAX_H3_CHUNKED_CPU_MP4_OUTPUT=1
VLLM_OMNI_MINIMAX_H3_CHUNKED_CPU_MP4_SLOTS=2
VLLM_OMNI_MINIMAX_H3_GPU_UINT8_OUTPUT=0
VLLM_OMNI_MINIMAX_H3_VAE_DECODER_TILE_SIZE=0
VLLM_OMNI_MINIMAX_H3_VAE_EXACT_OPS_SM120=1
VLLM_OMNI_MINIMAX_H3_VAE_TEMPORAL_CHUNK_PARALLEL=0
VLLM_OMNI_MINIMAX_H3_VAE_TEMPORAL_PRE_GATHER_PRUNE=0