Measured inference · September 23, 2026

FastH3 at five seconds.
Precision, communication, latency.

The original moon-teahouse prompt at 768p. Compare complete request latency and original audiovisual outputs, with separate baselines for optimized serving and stock FastVideo inference.

Every embedded clip is formal repeat 1, selected before inspecting its quality and published without re-encoding. Play clips individually or start a comparison pair together. Each pair enables only one audio track at a time.

01 / PR evaluation on two baselines

Each PR is tested on the stack that executes its relevant code. These are additive changes to the existing work, with the attention algorithm and precision held fixed within each pair.

Optimized native serving

Median change: -0.009 s (-0.19%). A negative value is faster. Five samples describe this run; they do not establish a production tail-latency distribution.

Original FastVideo inference

Median change: -0.255 s (-1.11%). A negative value is faster. Five samples describe this run; they do not establish a production tail-latency distribution.

Original-stack latency breakdown

Mean wall-clock stage times across the five formal requests. These are pipeline timers, not an Nsight trace. Remaining time includes dispatch, inter-stage work, output transfer, and HTTP delivery.

Execution and output checks

Every arm passed the eight-rank execution audit and media decoding checks. Output equality and kernel repeatability are evaluated separately below.

Equality uses SHA-256 over decoded video or PCM audio for the fixed first formal repeat, not the MP4 container bytes.

PR #5027 workspace is not accepted as end-to-end lossless. All 32 transport byte checks passed and decoded audio matches, but decoded video differs: SSIM 0.969241, PSNR 40.884 dB. Repeats 1 and 2 are identical within each arm. The source of the cross-arm video difference remains unlocalized; these metrics describe the difference, not a subjective quality ranking.

The upstream regression compares each kernel with itself under concurrent copy traffic: 37,807 tokens, one head, dimension 128, 20 rounds, 100 independent 64 MiB copies per round, zero absolute and relative tolerance. The local API already selects CuTe DSL, so the newer optional backend keyword is omitted. This checks the stage-reuse race; it does not compare Sage numerics with BF16.

Download execution and correctness evidence

02 / Precision schedules on the optimized stack

Earlier same-day measurements are retained below. They are not substituted for the fresh PR baseline. Both variants use the same NVFP4 video decoder and BF16 RDMA transport. These earlier clips predate the PR #5317 synchronization fix; their timing and appearance do not establish robustness under concurrent memory traffic.

03 / Measurement protocol

Each arm has two excluded warmup requests followed by five formal requests. Timing starts immediately before a local HTTP request and ends after all MP4 response bytes arrive. Model loading, warmup compilation, and post-response media validation are excluded. Encoding and transfers performed by each pipeline are included. Chat interpretation, streaming queues, RTMP upload, and Twitch playback delay are outside this boundary.

RTF is request latency divided by actual video duration; a value below one is faster than playback. GPU clocks were not locked, and arms were measured sequentially. Small median differences do not establish a statistically significant speedup. Every dot is one formal request. The dashed line marks 124 / 24 = 5.167 seconds. Use the per-arm playback column when a pipeline emits a different frame count. Warmup and loading times are available in the results JSON.

The unchanged prompt schedules its third shot at 6.6 seconds, beyond this five-second request. The model can compress the shot timing. These outputs compare performance and generation under identical input; they do not validate strict execution of the full three-shot script.

04 / Configuration and interpretation

The optimized stack retains exact AdaLN caching, split QK/V overlap, MXFP8 SwiGLU fusion, tiled output projection, batched VAE decoding with gather overlap, fused output blending/RGB conversion, overlapped audio/video decoding, early AAC encoding, and chunked MP4 output. BF16 first-step weight prefetch is counted in request latency.

The first-step rollback retains pre-quantization student weights: 24 blocks stay on GPU and 26 use pinned CPU buffers with prefetch. Both optimized precision schedules still use an NVFP4 VAE. The audio decoder is unchanged, but DiT precision can affect jointly generated audio.

“Original” refers to the pinned FastVideo preview baseline used by the existing gallery (source 3196835; checkpoint revision b65818d), not every FastH3 release. Ulysses transports 38,168 rows in the original stack and 38,208 in native serving because their padding alignment differs. Original-source checks cover 1,284 files against the previously recorded hashes. The original full checkpoint and the optimized base-plus-adapter deployment are different artifacts; cross-stack differences also include sharding, residency, decoder precision, and output handling. Therefore a whole-stack timing difference is not an isolated quantization or PR speedup. Within each PR pair, those choices are fixed. The emitted H.264 bitstreams report CRF 18 for native serving and CRF 23 for stock FastVideo, so cross-stack visual comparisons also include different encoding settings; neither is lossless video encoding. One prompt and seed cannot establish a general quality ranking.

integrated_multimodal_description: [Shot 1] Cinematic live-action fantasy inside a small wooden teahouse suspended beneath a colossal moon, shown in a symmetrical wide shot through open paper doors. An elderly tea master in an indigo robe, with silver hair pinned high and a warm gravelly Mandarin voice (S1), pours glowing tea into two porcelain cups while a young female radio astronomer in a cream flight suit, with a clear fast Mandarin voice (S2), adjusts a brass receiver beside her. The camera pushes in with small amplitude at slow speed as lunar dust drifts upward instead of falling, and the tea master says: <d>[Chinese] The moon is not silent; it is only waiting for us to hear it.</d> [Shot 2] At 00:03.400, the camera cuts to a macro close-up of the tea surface. Concentric ripples become a precise star map, tiny points of light connect across the cup, and the receiver needle snaps toward a distant signal. The astronomer leans into the eyepiece, her face illuminated by moving constellations, and replies: <d>[Chinese] Signal locked.</d> [Shot 3] At 00:06.600, the shot cuts outside to a sweeping arc around the teahouse as the vast moon slowly opens into layered mechanical petals, revealing a luminous blue core. The two women step onto the balcony without changing clothes or positions relative to each other; the tea master raises her cup while the astronomer sends one narrow beam from the receiver into the core. The camera pulls out with large amplitude at slow speed until the teahouse becomes a warm point against the unfolding lunar structure.
overall_soundscape: Porcelain touches wood with delicate clicks above the soft hiss of boiling water and the papery flutter of the doors. A low receiver hum gains harmonics as the signal locks, followed by distant metal plates shifting across the moon and a soft pressure wave moving through airborne dust.
non_diegetic_music: Slow guzheng harmonics over sustained glassy synthesizer tones at 68 BPM. A restrained taiko pulse enters when the lunar petals open, then resolves into one long bowed-cymbal shimmer.