This is a map, not a verdict. Full-duplex interaction serving in vLLM-Omni is an active design — RFC #3745 is open, two implementations are landing from different directions, and several core questions are still being argued in the thread. The goal here is to lay out the shared ground everyone agrees on, then put the open decisions on the table — with their advocates — so the room has a common picture to argue from.

Short on time? Jump to Part 3. The decision matrix there is the agenda — nine questions in three themes. Parts 1–2 are the shared context you need to argue them.

The shape of the whole post, before the details:

flowchart TB
    sub["vLLM-Omni stage runtime<br/>request-oriented today"] --> prim["the session primitive<br/>RFC #3745 (proposed)"]
    prim --> A["Track A — native in-engine<br/>MiniCPM-o 4.5 · #3907"]
    prim --> B["Track B — orchestration<br/>JoyAI-VL · #4575"]
    A --> dec["nine open decisions<br/>Part 3 · the agenda"]
    B --> dec

What we mean by full-duplex

Full-duplex voice is the “Doubao / Gemini Live / GPT-4o voice” experience: no push-to-talk, no turn button. You can interrupt the model mid-sentence, it listens and speaks at the same time, a single conversation context stays alive for minutes, and barge-in lands under 300–400 ms. A new class of models — MiniCPM-o 4.5 (Omni-Flow — perception and speech on one time-sliced clock), Nemotron VoiceChat, SoulX-Duplug, Moshi, PersonaPlex on the audio side; JoyAI-VL on the vision side — are full-duplex by design: they perceive and respond concurrently.

They are the edge of a broader shift Thinking Machines Lab calls interaction models: a model in “constant two-way exchange with the user — perceiving and responding at the same time,” where “for interactivity to scale with intelligence, it must be part of the model itself.” When interactivity becomes native to the model, the serving stack has to match it.

Whatever the modality, an interaction model runs one loop: perceive continuously, and every tick decide whether to stay silent, respond now, or hand the hard part to a slower background model.

flowchart LR
    p["PERCEIVE<br/>audio / video / text<br/>streaming in"] --> d{"every tick:<br/>what now?"}
    d -- "nothing to add" --> sil["STAY SILENT<br/>keep watching"]
    d -- "time to reply" --> resp["RESPOND<br/>speak / act now"]
    d -- "too hard right now" --> del["DELEGATE<br/>hand to a background model"]
    sil --> p
    resp --> p
    del --> p

MiniCPM-o 4.5 calls the first two ⟨listen⟩ / ⟨speak⟩; JoyAI-VL adds the third, delegate. The rest of this post is about what it takes to serve that loop.

Part 1 — The common ground

This part is the picture nobody really disputes: why today’s engine can’t do it, and the shape of the primitive that the RFC proposes.

How vLLM-Omni serves a request today

vLLM-Omni decomposes an any-to-any model into a directed graph of stages — each stage an independently served engine with its own scheduler and batching, wired to the next through a unified connector.

Two substrate capabilities matter later: streaming stage output (async_chunk) — partial output streams to the next stage as it’s produced, so the moment the Talker emits a token, Code2Wav turns it into a waveform — and a control/data-plane-decoupled connector (SHM or Mooncake).

Qwen3-Omni is the canonical example: a three-stage Thinker(LLM) → Talker(LLM) → Code2Wav pipeline that already streams beautifully (Figure 3) — the Talker emits a token, Code2Wav speaks it, overlapped. But it’s still turn-based: you finish, then it thinks and replies. It’s the model the counter serves perfectly — one request in, one streamed answer out, KV freed at the end. Full-duplex is the same stage graph asked to listen while it speaks — and that’s where the counter breaks.

This substrate is strong — up to 91.4% lower job-completion time versus baseline. But it is request-oriented. A request goes prefill → decode → finish, and at finish its KV blocks return to the manager.

The impedance mismatch

Today’s engine treats a conversation like ordering at a counter; full-duplex needs a phone call.

flowchart LR
    subgraph R["A REQUEST — like ordering at a counter"]
        direction TB
        r1["you give your<br/>whole order"] --> r2["they make it"] --> r3["they hand<br/>it over"] --> r4["done — and<br/>they forget you"]
    end
    subgraph S["A SESSION — like a phone call"]
        direction TB
        s1["the call opens"] --> s2["both talk at once,<br/>for minutes"]
        s2 --> s3["you can cut in<br/>any time"] --> s2
        s2 --> s4["the call hangs up"]
    end
    R ~~~ S

Run a phone call through the counter and three things break:

The one-line framing everyone shares: the unit of work should be the session, not the request.

The proposed session primitive (RFC #3745)

The RFC proposes one primitive — DuplexSession — and a loop that never says “done”:

flowchart TB
    open(["call opens — the conversation gets its own memory (a KV lease)"]) --> listen
    listen["LISTEN — audio streams in; the model watches for your turn to end"] --> turn{"whose turn?"}
    turn -- "you're still talking, or you cut in" --> barge["BARGE-IN — stop speaking, drop the half-said reply, but KEEP the memory"]
    barge --> listen
    turn -- "you finished" --> speak["SPEAK — think, then stream audio back; memory carried across the turn"]
    speak --> listen
    listen -. "you hang up" .-> close(["call ends — memory released"])

The idea that makes it work: the KV cache is leased to the call, not to a single reply. A barge-in throws away the half-spoken answer but never the memory. The RFC sketches it as four thin layers — read the diagram top to bottom; it’s the path one audio chunk takes:

flowchart TB
    subgraph API["Protocol layer — OpenAI-Realtime-aligned"]
        WS["WS /v1/audio/conversation"]
    end
    subgraph ENG["Engine layer"]
        DS["DuplexSession registry<br/>TTL-GC · epoch · ring buffer"]
        SDC["StageDuplexClient<br/>open · push_chunk · signal_turn<br/>barge_in · close"]
    end
    subgraph SCHED["Scheduler layer"]
        DSCH["OmniDuplexScheduler<br/>session KV lease<br/>+ coalescing window"]
    end
    subgraph STAGE["Stage layer"]
        DV["DUPLEX_VAD — turn-taking"]
        AR["LLM_AR — duplex-append mode"]
    end
    WS --> DS --> SDC --> DSCH --> DV
    DSCH --> AR
    SDC -. "direct SHM chunk ring (bypass ZMQ)" .-> STAGE

Each layer fixes one of the three problems from earlier:

• Protocol layer — WS /v1/audio/conversation, an OpenAI-Realtime-aligned WebSocket carrying audio in and out on one long-lived connection. Deliberately thin: it just translates wire events into session calls and holds none of the hard logic.
• Engine layer — where the call lives. The DuplexSession registry is the new home for “a conversation”: it keys sessions by id, holds the barge-in epoch and the input ring buffer, and reclaims idle sessions by TTL (problem ③ — the engine finally has a notion of a conversation). The StageDuplexClient is the thin handle a model drives it through — open / push_chunk / signal_turn / barge_in / close.
• Scheduler layer — OmniDuplexScheduler does the two things the stock scheduler can’t: it holds the KV lease so a session’s blocks aren’t freed between turns (problem ① — no re-prefill), and runs the coalescing window, waiting a few ms so a momentarily-quiet stream still batches with its peers (problem ② — no batch drop-out).
• Stage layer — the model itself. DUPLEX_VAD owns turn-taking — is the user’s turn over? is this a barge-in? — and gates the LLM_AR stage, which runs in duplex-append mode: it adds the new chunk’s tokens to the kept KV instead of re-prefilling.

The dotted line is the latency shortcut: audio chunks don’t crawl down the layers over ZMQ — the StageDuplexClient writes them straight into the stage-0 process through a shared-memory ring, so only small control events (turn signals, barge-in) take the slow path. And every output chunk is stamped (session, turn, epoch); a barge-in just bumps the epoch and stale chunks get dropped — but the KV lease is never touched.

A model plugs in at exactly one seam — the StageDuplexClient:

class StageDuplexClient(StagePoolClient, Protocol):
    def open_session(self, sid: str, params: DuplexSessionParams) -> None: ...
    def push_chunk(self, sid: str, chunk: DuplexChunk) -> None: ...        # SHM ring, not ZMQ
    def signal_turn(self, sid: str, event: TurnEvent) -> None: ...
    def barge_in(self, sid: str, epoch: int, scope: Literal["current", "all"]) -> None: ...
    def close_session(self, sid: str) -> None: ...

One primitive, many model shapes

The reason a shared primitive is worth the trouble: full-duplex models share almost nothing structurally except needing persistent KV. Two audio models from the thread look nothing alike.

MiniCPM-o 4.5 works on a 1-second clock: a structured token group per chunk, learned ⟨listen⟩/⟨speak⟩, barge-in only at chunk edges. Nemotron VoiceChat works on a ~30 ms clock: one acoustic embedding per decode tick, one word + one “what I heard” token out, no boundary tokens, barge-in enforced engine-side.

flowchart LR
    subgraph M["MiniCPM-o 4.5 — a 1-second clock"]
        direction TB
        m1["every 1 s:<br/>append an audio chunk"] --> m2{"listen<br/>or speak?"}
        m2 -- speak --> m3["stream ~1 s<br/>of audio back"]
        m2 -- listen --> m1
        m3 -. "cut in only at the<br/>next chunk edge (≈1 s)" .-> m1
    end
    subgraph N["Nemotron VoiceChat — a 30 ms clock"]
        direction TB
        n1["every ~30 ms:<br/>one audio frame in"] --> n2["one word out +<br/>one 'what I heard' out"] --> n3["TTS speaks,<br/>one step behind"]
        n3 -. "cut in any tick —<br/>engine must enforce it" .-> n1
    end
    M ~~~ N

These different clocks set hard barge-in latency floors and force a duplex adapter to support three injection patterns, not one:

Part 2 — Two implementations, two directions

Two PRs are landing full-duplex from opposite ends of the spectrum. This part describes both, neutrally; the question of how they relate is in Part 3.

Track A — audio, native in the engine (MiniCPM-o 4.5, #3907)

PR #3907 extends MiniCPM-o 4.5 into a session-oriented audio stream over /v1/duplex and /v1/realtime?duplex=1, with a real audio-in → audio-out data plane inside the engine:

flowchart LR
    mic["your voice<br/>(PCM frames in)"] --> ep{"/v1/duplex or<br/>/v1/realtime?duplex=1"}
    ep --> sess["session actor<br/>(epoch + playback cursor)"]
    sess --> s0["Stage 0<br/>listen / speak decode"]
    s0 -- "speak: hand off hidden states" --> s1["Stage 1<br/>TTS / token2wav"]
    s1 --> out["audio streamed back"]
    s0 -. "listen: stay silent,<br/>keep the memory" .-> sess

The whole session is three monotonic quantities. Everything Track A does — listen, speak, remember, get interrupted — is governed by three counters that only ever move up. Read one full session as their joint timeline (the user speaks, the model replies, the user barges in, the model replies again):

The table is the design. Every column is boring except the barge-in column — and even there, watch what does not fall: the KV (③) and the committed memory (②) keep climbing. Barge-in ticks the epoch and tears down the downstream stages, but stage 0’s KV and the conversation memory survive untouched. “Drop the reply, keep the memory,” literally as a graph. Now the three, one at a time:

① epoch — the generation counter. begin_response stamps a reply with the current epoch; every emitted chunk checks is_stale(epoch); barge_in() does epoch++. That one increment invalidates every in-flight piece of the old reply at once — nobody chases scattered chunks, they self-drop the next time they look. It is an integer, not a bool: after two barge-ins you must distinguish live-generation 2 from dead-1 from dead-0, which a flag could never encode. And it tears down only stage_id > 0 — stage 0, which owns the conversation KV, is deliberately exempt:

flowchart LR
    bi["barge-in"] --> ep["epoch++<br/>0 → 1"]
    ep --> stale["stage_id > 0<br/>torn down"]
    ep --> keep["stage 0 KV<br/>kept resident"]
    stale --> drop(["in-flight reply<br/>self-drops (stale)"])
    keep --> mem[("conversation<br/>memory survives")]

② playback cursor — commit only what was heard. Four watermarks, each max-monotone: generated → sent → played → committed. The model races ahead, the speaker lags, and sent ≠ heard — so on barge-in only played_ms enters memory; the streamed-but-unplayed tail is discarded. That is why the model never believes it said something you didn’t actually hear:

flowchart LR
    g["GENERATED<br/>model produced it"] --> s["SENT<br/>streamed to client"] --> a["PLAYED<br/>you actually heard it"]
    a == "only this much (played_ms) enters memory" ==> mem[("conversation<br/>memory")]
    s -. "barge-in: drop the rest —<br/>sent but never heard" .-> x(["discarded"])

③ stage-0 KV — the memory itself, and the one that never comes back down. A single long-lived resumable request owns the conversation KV. It grows at ~12 tokens/sec of audio (1 s = 16000 samples @ 16 kHz → 10 pooled embeddings + <unit>/</unit>) and nothing ever frees it until the call closes — no compaction, no eviction, no sliding window. Barge-in keeps it; that is the entire point of preserving stage 0. The cost is that growth is linear and uncompacted, so the session’s horizon is bounded: context window ÷ 12 seconds of conversation.

All three ride one small state machine — the session moves between listening and responding, and barge-in is the edge that returns it without dropping the memory:

stateDiagram-v2
    [*] --> IDLE
    IDLE --> LISTENING: input.append
    LISTENING --> LISTENING: input.append · keep listening
    LISTENING --> RESPONDING: commit / response.create · begin_response (capture epoch)
    RESPONDING --> RESPONDING: input.append · listen WHILE speaking
    RESPONDING --> LISTENING: response.done (not stale)
    RESPONDING --> LISTENING: barge-in · epoch++ → old reply stale, stage 0 KV kept
    LISTENING --> CLOSED: close
    RESPONDING --> CLOSED: close

Two edges make it full-duplex rather than turn-based. The self-loop on RESPONDING — input.append keeps feeding the model while it speaks — is the duplex property; a half-duplex machine has no such edge. And barge-in is an epoch bump, not a state: “interrupt me” isn’t a transition you draw, it’s the generation counter (①) ageing out the old activity while stage 0’s KV (③) stays resident.

Net: #3907 declares the full capability surface (patterns, input modes, signal sources), but the scheduler-owned KV lease is a slot, not a mechanism — supports_core_kv_lease defaults to False and lease_active is bookkeeping that nothing acts on. Stage 0’s context is model-internal state (supports_model_internal_state), not a leased, migratable core KV. So the memory is resident but bounded: it can’t migrate across replicas, and because it is never compacted it runs until max_model_len is exhausted (the model’s max_position_embeddings × any rope_scaling, clamped by --max-model-len) — on the order of context-window ÷ 12 seconds of talk. Verified on H20: stale_audio_delta_count=0 (barge-in really drops the stale stream). That bound is exactly the seam to Track B, which throws the resident KV away and rebuilds context from compact text every tick — trading MiniCPM’s minutes-long, low-latency horizon for an unbounded one.

def barge_in(self) -> tuple[int, list[str]]:
    # Stage0 is the long-lived resumable request that owns the conversation
    # KV/context. A barge-in stops downstream output but PRESERVES stage0 so
    # conversation memory survives. Only stage_id > 0 are torn down.
    stale = [b.request_id for sid, b in self.stage_bindings.items() if sid != 0]
    self.epoch += 1
    self.pending_inputs.clear()
    self.stage_bindings = {sid: b for sid, b in self.stage_bindings.items() if sid == 0}
    return self.epoch, stale

class DuplexAdapterPattern(str, Enum):
    CHUNK_GROUP_APPEND = "chunk_group_append"          # MiniCPM-o 4.5
    PER_STEP_TENSOR_INJECT = "per_step_tensor_inject"  # Nemotron VoiceChat
    PARALLEL_FRAME_JOINT = "parallel_frame_joint"      # Moshi-class

class DuplexInputMode(str, Enum):                      # append is one mode among several (decision ⑤)
    APPEND_AUDIO_CHUNK = "append_audio_chunk"
    REPLACE_LATEST_CHUNK = "replace_latest_chunk"
    REENCODE_CONTEXT = "reencode_context"
    ROLLBACK_TO_CHECKPOINT = "rollback_to_checkpoint"
    TURN_COMMIT_ONLY = "turn_commit_only"

class DuplexSignalSource(str, Enum):                   # turn-taking from many sources (decision ③)
    MODEL_NATIVE = "model_native"
    EXTERNAL_VAD = "external_vad"
    CLIENT_EVENT = "client_event"
    SERVER_POLICY = "server_policy"
    DIALOGUE_STATE_MODEL = "dialogue_state_model"

def mark_generated(self, generated_ms): self.generated_ms = max(self.generated_ms, generated_ms)
def mark_sent(self, sent_ms):           self.sent_ms      = max(self.sent_ms, sent_ms)
def acknowledge(self, played_ms, committed_ms=None):
    self.played_ms    = max(self.played_ms, played_ms)
    self.committed_ms = max(self.committed_ms, committed_ms if committed_ms is not None else played_ms)

Track B — vision, orchestration over a plain server (JoyAI-VL, #4575)

The audio track isn’t the only one. JoyAI-VL-Interaction is a vision-first interaction model: an 8B Qwen3-VL-shaped model retrained so that deciding when to speak is a learned capability. Watching video at 1 Hz, every second it emits </silence> (keep watching), </response> (speak now), or delegates to a background model — the loop from the intro, triggered by what it sees.

PR #4575 lands its serving layer — and takes the opposite road from RFC #3745 and #3907. The RFC and Track A make the engine stateful: a DuplexSession owns the conversation and the scheduler leases its KV across turns, so context lives as resident KV inside the engine. #4575 inverts that. The engine stays a stateless, vanilla vllm serve; every bit of conversation state lives in a separate orchestrator process; and context is never held as resident KV — it is rebuilt from a compact memory every tick and replayed through vLLM’s automatic prefix cache. Three off-the-shelf parts, no engine surgery: a stateless model server, a Python session object, and the prefix cache.

So Track B’s whole game is prompt engineering for the cache: hold the conversation in a tiered text memory, lay it out so the head never moves, and let the stock engine do KV reuse for free. The rest of this section is how those pieces fit.

It lives under vllm_omni/experimental/fullduplex/, and the split is worth getting right (the README is explicit about it):

• joyvl/serving/ is the live path — an InteractionSession driving decision/ (the silence / respond / delegate policy) and a 3-tier memory/ (raw frames → text summaries → compressed long-term, shaped for prefix-cache reuse) directly, with bridges/ for pluggable ASR / TTS + background delegation.
• core/ (DuplexRuntime / DuplexSession / DuplexAdapter) is a model-agnostic scaffold — a demonstration of how a model would plug into a generic full-duplex framework, exercised only by tests today. Per its own README it’s “built for” fused-audio models like MiniCPM-o — which is exactly the irony in decision ① below: #3907 built its own instead of using it.

class DuplexAdapter(ABC):
    @abstractmethod
    def capabilities(self) -> DuplexCapability: ...
    @abstractmethod
    async def on_input(self, session, modality: str, data) -> None: ...
    @abstractmethod
    def respond(self, session) -> AsyncIterator[OutputChunk]: ...

    def should_respond(self, session) -> bool:        # default: always
        return True
    async def on_barge_in(self, session) -> None: ...
    async def on_playback_ack(self, session, cursor: int) -> None: ...

The per-second loop — the model decides

There are three independent clocks here, and conflating them is the easy mistake:

• tick = one frame (~1 s) — one model decision;
• chunk = 100 frames (~100 s) — the memory buffer (when summarization fires);
• query = a human-driven span that rides across many chunks.

Each sampled frame is one stateless POST. Turn-taking isn’t a separate VAD module — it’s the model choosing, every tick, between </silence> / </response> / </delegation>. The orchestrator wraps that choice with exactly two rules:

• Before the model: a force-silence gate — until the first query of the session, idle frames short-circuit to </silence> with no model call at all. Once a query is armed, the gate is dead and every tick runs a forward pass (even to decide “stay quiet”).
• After the model: dedup — a </response> whose text repeats the last spoken line collapses back to silence; an intervening genuine silence resets the comparison, so the same line can fire again once the scene has moved on.

The subtle, important part: a query arms continuous evaluation — it does not force a reply. “Tell me when the water boils” keeps emitting </silence> tick after tick while the pot isn’t boiling, then speaks the instant a frame shows it. The model re-judges every second whether the current visual evidence now answers the standing query.

Keeping the conversation contextualized

A two-hour stream never fits in the context window, so the orchestrator doesn’t grow the input — every tick it rebuilds a compact context prefix from memory and prepends it to the live frame. The 3-tier brain feeds it (with frame_seconds = 1.0, the clock below is in real seconds):

• short-term — the live chunk, raw frames (what’s happening now), up to 100 frames ≈ 100 s;
• mid-term — one text summary per closed chunk (100 frames → 1 summary), each tagged with its frame range;
• long-term — every 5 mid-terms roll up into one compressed block; a sliding window keeps the last 15 blocks ≈ 2 h.

A chunk actually carries two stores — working_frames (video only) and chunk.messages (the live transcript) — and at the 100-frame boundary they drain into different tiers: the video becomes a mid-term summary, while the query + its replies are archived into a separate Q&A history. The raw transcript itself is dropped. So a spoken reply lives in Q&A history, never in the visual summaries — the two halves of the conversation are remembered through two different channels and only recombine in the prompt prefix.

flowchart LR
    subgraph mem["3-tier memory"]
        direction TB
        st["short-term<br/>live chunk · ≤100 frames"]
        mt["mid-term<br/>1 summary / 100 frames"]
        lt["long-term<br/>roll-up every 5 · window 15 (~2h)"]
        st -. "100 frames<br/>summarize" .-> mt
        mt -. "every 5<br/>roll-up" .-> lt
    end
    mem --> pre["context prefix<br/>(byte-stable head)"]
    pre --> inp["model input, this tick"]
    frame["live frame + new query"] -. "appended, NOT spliced into the head" .-> inp
    inp --> model["JoyVL model"]
    model -. "head unchanged → prefix-cache hits" .-> pre

build_memory_prefix stitches these into one text block — video history (long-term + mid-term) + Q&A history + the standing query — that rides in front of the frame. Two properties make it cache-friendly:

• Its inputs are frozen for the whole chunk. Mid-term, long-term, and the archived Q&A only change at a chunk boundary (the summarizer runs in the background and appends), and the boundary resets the chunk anyway. So across the ~100 ticks within a chunk, build_memory_prefix returns the same bytes every time. The memory layout is also append-only at the durable end, so even across boundaries the leading lines stay identical and keep hitting the cache.
• It withholds the volatile query. A newly issued query is not spliced into the head; it rides in the current tick’s appended message instead, and only migrates into the head (as history) once its chunk closes. The query is the thing that changes most often, so keeping it out of the head is what prevents the most frequent cache-busting event.

Net: the head is byte-identical across ticks, vLLM’s radix prefix cache hits, and the hours-long history is never re-prefilled — only the one new frame at the tail. That’s Track B’s entire KV-reuse strategy, with no engine session and no KV lease.

def build_memory_prefix(memory, *, current_query, query_in_current_chunk,
                        keep_qa_history, current_chunk_index) -> str:
    sections = []
    # 1) video history = long-term digest + mid-term chunk summaries
    history = []
    if current_query and memory.long_term_memory:
        history.append(memory.long_term_memory)
    for e in memory.mid_term_summaries:
        history.append(f"<{e.frame_range}>\n{e.summary_text}")
    if history:
        sections.append(VIDEO_HISTORY_HEADER + "\n\n".join(history))
    # 2) past Q&A, archived before this chunk
    if keep_qa_history and current_query:
        sections.append(QA_HISTORY_HEADER + qa_lines)   # "[Q@t] … [A@t] …"
    # 3) the standing query — ONLY if it is not already in the current chunk
    if current_query and not query_in_current_chunk:
        sections.append(USER_QUERY_HEADER + "\n" + current_query.strip())
    return "\n\n".join(sections)

The two tracks differ on where the session lives and how KV is reused:

flowchart LR
    subgraph A["Track A — native in-engine (#3907)"]
        direction TB
        a1["session lives INSIDE<br/>the engine"] --> a2["KV leased by<br/>the scheduler"] --> a3["lowest latency,<br/>deepest change"]
    end
    subgraph B["Track B — out-of-process (#4575)"]
        direction TB
        b1["session in a thin server<br/>before a plain vllm serve"] --> b2["KV reuse via<br/>radix prefix cache"] --> b3["ships day-0,<br/>no engine surgery"]
    end
    A ~~~ B

Part 3 — What we’re deciding (the agenda)

This is the open part. Nine questions are live in the RFC #3745 thread; none is settled. Here they are in one view, then grouped into three themes — Structure, Semantics, Targets — each with the concrete code that makes the fork real.

Theme 1 — Structure: how many abstractions, and where? (①②⑥⑧)

The headline fact: the repo already holds session abstractions that don’t share code — and the one meant to be reusable (core/) isn’t even on JoyVL’s live path:

# #3907 (MiniCPM, native) — the live session:
from vllm_omni.engine.duplex import DuplexSessionRuntimeState
# #4575 (JoyVL, orchestration) — the live session:
from vllm_omni.experimental.fullduplex.joyvl.serving.session import InteractionSession
# #4575 also ships a generic scaffold, built "for fused-audio models like MiniCPM-o" —
# yet #3907 built its own and never used it:
from vllm_omni.experimental.fullduplex.core import DuplexSession   # demonstration / tests only

• ① converge into one model-agnostic core · keep two (native-audio vs orchestration-vision) · one shared API with two backends.
• ② treat native (#3907) and orchestration (#4575) as two backends of one core, chosen by latency tier · native as the destination, orchestration the day-0 stopgap · two independent tracks.
• ⑥ where the session object lives — orchestrator main path · the coordinator (yinpeiqi) · a separate layer · a new session-based orchestrator type (Gaohan) · a wrapper between engine and executor (Nightwing-77); boundaries flagged unclear (TKONIY).
• ⑧ whether every model moves under experimental/fullduplex/ as a capability-declaring adapter, or stays put (today #3907 is in model_executor + entrypoints; #4575/#4771 are under experimental/fullduplex).

Theme 2 — Semantics: turn-taking, KV lease, append (③④⑤)

Two of these are already enums in #3907 — so the fork isn’t “invent something,” it’s “which members become canonical and required.”

class DuplexSignalSource(str, Enum):   # ③ — who may declare a "turn"?
    MODEL_NATIVE = "model_native"
    EXTERNAL_VAD = "external_vad"
    CLIENT_EVENT = "client_event"
    SERVER_POLICY = "server_policy"
    DIALOGUE_STATE_MODEL = "dialogue_state_model"

class DuplexInputMode(str, Enum):      # ⑤ — is "append" the default, or one of these?
    APPEND_AUDIO_CHUNK = "append_audio_chunk"
    REPLACE_LATEST_CHUNK = "replace_latest_chunk"
    REENCODE_CONTEXT = "reencode_context"
    ROLLBACK_TO_CHECKPOINT = "rollback_to_checkpoint"
    TURN_COMMIT_ONLY = "turn_commit_only"

• ③ is DUPLEX_VAD the single owner · optional, since an end-to-end model self-VADs (tc-mb) · or one of many sources behind a TurnController (Sy0307)?
• ④ does the KV lease cover stage-0/thinker only — downstream stages are epoch-flushable (yinpeiqi) — or all stages? Under memory pressure: reject + TTL-GC · evict oldest · compress (sink+window)?
• ⑤ is append the default, or one declared capability among several, gated by session_mode: turn | duplex so the six existing TTS pipelines never regress (Sy0307, linyueqian)?

Theme 3 — Targets: scope and latency (⑦⑨)

• ⑦ single-session first — land the primitive shape before admission control / fairness (tc-mb, agreed by several).
• ⑨ pin each phase to a barge-in floor it commits to (≈1 s / 300 ms / 80 ms). Sub-300 ms needs sub-chunk cancel + audio_end_ms truncate + mandatory VAD (linyueqian). Keep core ↔ client transports (ZMQ + SHM) swappable for closed-loop function calling — run the thinker with no user input to produce a call, then “prefill in the middle” (vklimkov-nvidia). Lockstep models need a non-token side-channel payload + a frame-budgeted stop + a deploy-time prewarm pass (Liangtaiwan).

Phases, as proposed

The RFC sketches a phased rollout; how the phases map to the open decisions above is part of what’s being discussed.

gantt
    title Full-Duplex Session Architecture (proposed phasing)
    dateFormat YYYY-MM-DD
    axisFormat %b
    section Phase 0
    Req/resp bring-up (MiniCPM-o 4.5, #3907) :done, p0, 2026-05-12, 20d
    section Phase 1
    DuplexSession + StageDuplexClient + KV lease :p1, after p0, 14d
    section Phase 2
    DUPLEX_VAD + OmniDuplexScheduler + cadence :p2, after p1, 14d
    section Phase 3
    Barge-in epoch + WS protocol :p3, after p2, 10d
    section Phase 4
    SHM direct-ingress ring + HC batching :p4, after p3, 10d
    section Phase 5
    Generalize to World Models (#1987) :p5, after p4, 14d

Phase 5 makes DuplexChunk an opaque typed payload so a World-Model (#1987) observe→predict loop could reuse the same DuplexSession — one session system, not two. Full-duplex sits at P1 on the public roadmap (issue #2136).

Sources: RFC #3745 and its discussion thread; PRs #3907 (MiniCPM-o 4.5) and #4575 (JoyAI-VL); the MiniCPM-o 4.5 paper (arXiv:2604.27393, Figs. 1 & 4); the JoyAI-VL-Interaction paper (arXiv:2606.14777, Fig. 5); the vLLM-Omni systems paper (arXiv:2602.02204, Fig. 2) and the Apr-2026 meetup deck (Fig. 3). All views attributed to named contributors are theirs, from the public thread. Project: github.com/vllm-project/vllm-omni.