Skip to content

vllm_omni.config.speech_cache

API-process speech cache budgets, independent of stage engine caches.

SpeechCacheConfig

LRU limits, not preallocated memory. Zero disables the respective cache.

resolve_max_bytes class-attribute instance-attribute 

resolve_max_bytes: int = Field(default=4 * 1024**3, ge=0)

resolve_max_entries class-attribute instance-attribute 

resolve_max_entries: int = Field(default=2048, ge=0)

speaker_max_bytes class-attribute instance-attribute 

speaker_max_bytes: int = Field(default=512 * 1024**2, ge=0)