vllm_omni.diffusion.models.helios.cross_attn_cache ¶
Weakref-guarded LRU cache for cross-attention key/value reuse.
HeliosTransformer3DModel projects the text-encoder output into cross-attention K/V once per request and reuses the projection across the ~50 denoise steps of that request. The cache is keyed by a storage fingerprint of the source tensor (data_ptr / shape / stride / dtype / device / _version). That fingerprint is not unique across requests: once the source tensor is freed, PyTorch's caching allocator may hand its address to an unrelated tensor of the same shape, producing a false cache hit and cross-prompt contamination (request B silently reuses request A's projected K/V).
SourceTensorLRUCache stores a :mod:weakref to the source tensor next to the cached value and verifies the source is still alive on every lookup. An address reused by a different tensor still matches the fingerprint, but the original source's weakref is dead by then, so the lookup becomes a miss instead of a silent wrong hit. Within a single request the source tensor is held alive across all denoise steps, so legitimate intra-request reuse still hits.
SourceTensorLRUCache ¶
Bounded LRU cache keyed by a source-tensor fingerprint + liveness.
The key is derived from the source tensor's storage fingerprint (data_ptr / shape / stride / dtype / device / _version); a weakref to the source is stored alongside the value and checked on lookup so that an allocator address reused after the source is freed never yields a false hit. Stale entries (dead weakref) are evicted lazily on lookup and proactively on insert.
get ¶
Return the cached value for source or None on miss.
A miss is also returned when the source tensor that originally populated a matching entry has been freed (its weakref is dead), even if the fingerprint now matches a new tensor reusing the address; the stale entry is evicted in that case.