Skip to content

vllm_omni.diffusion.utils.video_decode

Decode video file paths to RGB frames.

Path suffix checks and bounded first/last-N imageio decoding. Model-specific conditioning and decode-budget stay in each pipeline's preprocess layer.

Currently used by Cosmos3 V2V preprocess and transfer control_path. Unifying with OmniVideoBackend / _decode_video_bytes is a follow-up.

VIDEO_EXTENSIONS module-attribute 

VIDEO_EXTENSIONS = {
    ".avi",
    ".m4v",
    ".mkv",
    ".mov",
    ".mp4",
    ".webm",
}

decode_path_video_frames 

decode_path_video_frames(
    path: str | Path,
    *,
    max_frames: int | None = None,
    keep: Literal["first", "last"] = "first",
) -> list[ndarray]

Decode a video file to RGB uint8 frames (H, W, C).

keep='first' stops after max_frames. keep='last' still scans the file and retains only the tail window.

is_video_file_path 

is_video_file_path(value: Any) -> bool

True when value is a filesystem path with a known video suffix.

Serving may leave uploaded references as temp paths such as /tmp/vllm_omni_video_reference_*.mp4 instead of decoded frames.