Skip to content

vllm_omni.engine.duplex.intermediate

DuplexIntermediateBuffer

Bases: TypedDict

Structured keys carried through model_intermediate_buffer.

The buffer remains a dict for scheduler and msgspec compatibility, but duplex-specific producers and consumers should use the helpers in this module instead of scattering nested string keys across serving, runner, and model code.

codes instance-attribute 

codes: dict[str, object]

duplex instance-attribute 

duplex: dict[str, object]

global_request_id instance-attribute 

global_request_id: list[str]

hidden_states instance-attribute 

hidden_states: dict[str, object]

ids instance-attribute 

ids: dict[str, object]

llm_output_text instance-attribute 

llm_output_text: list[str]

llm_output_token_ids instance-attribute 

llm_output_token_ids: list[int]

mel_spec instance-attribute 

mel_spec: object

meta instance-attribute 

meta: dict[str, object]

native_duplex instance-attribute 

native_duplex: bool

omni_payload instance-attribute 

omni_payload: object

prompt_token_ids instance-attribute 

prompt_token_ids: list[int]

request_id instance-attribute 

request_id: str

stream_output instance-attribute 

stream_output: bool

waveform instance-attribute 

waveform: object

build_duplex_append_prompt 

build_duplex_append_prompt(
    *,
    request_id: str,
    fence: DuplexFence,
    session_config: Mapping[str, object],
    runtime_config: Mapping[str, object],
    seq: int,
    turn_seq: int,
    payload: object,
    final: bool,
    prompt_token_ids: list[int],
    model_fields: Mapping[str, object],
) -> dict[str, object]

Wrap model-selected tokens and payload in the shared append envelope.

Models own token selection and extra worker fields; the framework owns request identity, sequencing and shallow config snapshots.

build_duplex_intermediate_buffer 

build_duplex_intermediate_buffer(
    *,
    request_id: str,
    prompt_token_ids: list[int] | None = None,
    output_token_ids: list[int] | None = None,
    output_text: str | None = None,
    stream_output: bool = False,
    native_duplex: bool = False,
) -> DuplexIntermediateBuffer

get_stream_request_key 

get_stream_request_key(info: dict[str, object]) -> str

get_tts_handoff 

get_tts_handoff(
    info: dict[str, object],
) -> tuple[object | None, object | None]

Read the canonical handoff, including the legacy flat aliases.

set_ref_audio 

set_ref_audio(
    buffer: DuplexIntermediateBuffer,
    waveform: object,
    sample_rate_hz: int,
) -> None

set_tts_handoff 

set_tts_handoff(
    buffer: DuplexIntermediateBuffer,
    token_ids: object | None,
    hidden_states: object | None,
) -> None

Store the AR-to-TTS handoff used by the full-duplex stage bridge.