Model plugin contract for full-duplex models.
One DuplexModelPlugin subclass per model binds what used to be two dotted paths (the engine DuplexRuntimeExtension and the serving ServingRuntimeAdapter). Everything runs engine-side now, so the plugin is loaded once by DuplexOmniEngine and handed to DuplexOrchestrator / DuplexSessionManager.
module-attribute ¶
dataclass ¶
Bases: DuplexModelSessionState
Framework-owned flag anatomy, implemented once.
The flag set above is the session runner's contract with the model (commit retention, deferred response/creates, silence-continuation bookkeeping); it is identical for every lockstep or frame-locked model, so a plugin only supplies its input packetizer as audio_buffer.
class-attribute instance-attribute ¶
committed_audio_operation_id: str | None = None
class-attribute instance-attribute ¶
class-attribute instance-attribute ¶
committed_audio_reserved_bytes: int = 0
class-attribute instance-attribute ¶
deferred_precreate_response: bool = False
class-attribute instance-attribute ¶
deferred_response_create: bool = False
class-attribute instance-attribute ¶
last_native_submit_monotonic: float | None = None
class-attribute instance-attribute ¶
pending_silence_owner_id: str | None = None
class-attribute instance-attribute ¶
dataclass ¶
Bases: ABC
Everything vLLM-Omni needs to know about one full-duplex model.
Engine policy (sampling params, append planning, output decisions) and session policy (capabilities, runtime configuration, per-session state, data-plane projection) live on the same object so a mismatch between the two halves is impossible by construction.
class-attribute instance-attribute ¶
class-attribute instance-attribute ¶
projects_intermediate_outputs: bool = False
class-attribute instance-attribute ¶
silence_continuation_sample_rate_hz: int = 16000
class-attribute instance-attribute ¶
silence_continuation_samples: int = 16000
Persist model-context history at a turn boundary. Default is a no-op.
This is not playback-ACK history. A model that keeps its own prompt transcript implements this; the session runner only decides when.
abstractmethod ¶
configure_sampling_params(
*,
runtime_config: dict[str, object],
defaults: tuple[object, ...],
) -> tuple[object, ...]
data_plane_context(
*,
epoch: int,
turn_id: int,
active_response_turn_id: int | None,
active_response_id: str | None,
auto_responds: bool,
response_format: str,
speed: float | None,
modalities: tuple[str, ...],
) -> object
The context handed to data_plane.project; the default is the generic dataclass.
abstractmethod ¶
decide_output(
*,
stage_id: int,
final_stage_id: int,
segment_finished: bool,
segment_token_ids: tuple[int, ...],
segment_output_metadata: dict[str, object],
output: object,
) -> DuplexOutputDecision | None
Output stages that may keep running after the next user turn starts.
Empty means a concurrent turn does not overlap a previous output stage. Shared lifecycle reads this instead of assuming a stage layout.
partial_stage_followup(
plan: PartialStageForward, req_state: object
) -> PartialStageForward | None
Optional second submit after plan has already been forwarded.
Default models have nothing to add. AURA uses this to queue the end sentinel only after the last sentence text is already resumable.
abstractmethod ¶
plan_append(
*,
request_id: str,
fence: DuplexFence,
session_config: dict[str, object],
runtime_config: dict[str, object],
seq: int,
turn_seq: int,
payload: object,
final: bool,
sampling_params: object,
) -> DuplexAppendPlan
plan_partial_stage_output(
orchestrator: object,
stage_id: int,
replica_id: int,
output: object,
req_state: object,
) -> PartialStageForward | None
Return a Talker update the orchestrator should submit, or None.
Default models do not split Stage1 text. The orchestrator owns the actual _forward_to_next_stage call.
async ¶
prepare_append_plan(**kwargs) -> DuplexAppendPlan
Prepare a plan; plugins may offload expensive work on owned snapshots.
prepare_prompt_config(
config: dict[str, object],
*,
state: DuplexModelSessionState,
payload: dict[str, object],
) -> dict[str, object]
Add model-owned context before planning an append on the session loop.
abstractmethod async ¶
prepare_runtime_config(
config: DuplexSessionConfig,
*,
model_config: ModelConfig | None,
) -> dict[str, object]
Return True to project this intermediate stage to the client.
Unlike decide_output, projecting does not short-circuit the pipeline: the stage output is still forwarded to the next stage. Default is off. Orthogonal to projects_intermediate_outputs (Qwen3 Stage0); this hook is per-stage.
release_concurrent_turn_requests(
*,
stage_id: int,
segment_finished: bool,
output: object,
context: object,
) -> bool
Return True when the next user commit may start while prior TTS drains.
The plugin chooses when that is safe. The runner must not hard-code a stage id. Default off. Unlike barge-in, this path must not cancel the old TTS.
runtime_config_after_model_output(
current: Mapping[str, object],
output_metadata: Mapping[str, object],
) -> dict[str, object] | None
Return a runtime-config patch after a model output is observed.
Plugins may use model-owned output metadata to retire server-side runtime state that has been consumed by the worker. The default keeps the framework unaware of model-specific metadata.
runtime_config_for_function_output(
config: DuplexSessionConfig,
current: Mapping[str, object],
item: Mapping[str, object],
) -> dict[str, object] | None
abstractmethod ¶
runtime_config_for_update(
config: DuplexSessionConfig,
current: Mapping[str, object],
) -> dict[str, object]
One silence unit as an append payload (pcm_f32le zeros).
Used by the runner's turn continuation and by the startup warmup; a model whose unit is not plain zero PCM overrides it.
ASR text to show as the user's words, or None.
Default models do not surface Stage0. AURA uses this for a spoken turn only; vision-follow commits stay off the transcript.
dataclass ¶
One downstream update the orchestrator should submit.
close_only is a final update with no new sentence. output is the model-built payload; the orchestrator does not interpret its text.
queue_close_after means this chunk still has text, but Stage1 has finished and an earlier sentence is already in flight. The text must be submitted resumable. A non-resumable submit is an end sentinel (StreamingUpdate.from_request returns None) and aborts that sentence.
load_duplex_plugin(
path: str, encode_audio: EncodeAudio
) -> DuplexModelPlugin
reject_changed_runtime_value(
new_value: object,
current_value: object,
*,
message: str,
code: str,
error_cls: type[
DuplexRuntimeConfigError
] = DuplexRuntimeConfigError,
) -> None
reject_private_runtime_keys(
extra_body: object,
private_runtime_config_keys: frozenset[str],
*,
message: str,
error_cls: type[
DuplexRuntimeConfigError
] = DuplexRuntimeConfigError,
) -> None
Reject client overrides while preserving each plugin's error contract.
validate_duplex_plugin_sampling(
plugin: DuplexModelPlugin,
*,
sampling_defaults: tuple[object, ...],
) -> None
Fail fast when the plugin cannot produce one sampling parameter per stage.