Preserve the present
Append visual embeddings to a fixed-capacity recent cache, retaining detailed evidence from the latest observations.
KeyRec: Bounded Visual Memory for
Streaming and Long-Video Understanding
KeyRec preserves recent details and historical events in bounded visual memory, with query-adaptive readout using approximately 10% of the dense decoder-facing visual tokens.
THE IDEA
Long videos need better memory.
Dense visual tokens accumulate as a video grows. Simply keeping fewer tokens can discard coherent event evidence or the fine details of what is happening right now.
KeyRec separates these two needs. A recent visual cache preserves detailed observations, while a bounded event bank organizes historical evidence. Incoming frames update both memories without knowing the question. When a question arrives, a text-only router allocates a fixed readout budget between them—without reprocessing historical frames.
The framework is training-free and operates on model-facing visual embeddings, supporting both modular Gemma VLMs and the encoder- and projector-free NEO-ov architecture.
01 / METHOD
Write once. Read for the question.
A recent cache and an event bank preserve complementary evidence under a fixed readout budget.
Organize the past
Propose novel events and use semantic similarity and temporal proximity to add, merge, or evict them within a bounded bank.
Read what matters
A text-only router chooses recent and event allocations, then retrieves by relevance or temporal coverage.
Same budget.
Different evidence.
Explore the five router states in the Gemma streaming profile. This illustration uses the paper’s allocation values; it does not run a model.
READOUT BUDGET224 tokens
Recent 160 Events 64
14 blocks × 16 tokens · Total budget stays fixed
For Gemma, KeyRec reduces retained visual context and language-model cost after visual encoding; it does not skip the vision encoder.
02 / RESULTS
Less context. Stronger streaming results.
Across three VLM backbones and four benchmarks, KeyRec achieves the best compressed result in 13 of 15 settings, including ties.
Accuracy (%)Higher is better ↑
All values are from the paper, with frozen backbones and matched frame sampling. Compressed methods use ≈10% of the dense decoder-facing token budget; KeyRec’s persistent candidate memory is ≈20%. Vanilla uses the full visual input.
View all 15 reported settings +
| Benchmark / subset | Backbone | Vanilla | STC-Pruner | STOM-CTR | CausalMem | KeyRec |
|---|
Bold indicates the best compressed result, including ties. “—” denotes an unsupported architecture.
SCALING TO LONGER INPUTS
Bounded context,
more room for video.
At 512 sampled frames, KeyRec remains feasible while dense Vanilla runs out of memory in the paper’s LongVideoBench evaluation.
Gemma 4 E4B · Paper Table 2. Peak memory includes weights, ingestion, and query processing.
| Frames | Vanilla | STOM-CTR | KeyRec |
|---|---|---|---|
| 64 | 17.09 | 15.05 | 15.06 |
| 256 | 39.23 | 15.47 | 16.17 |
| 512 | OOM | 16.57 | 18.93 |
The trade-off: KeyRec adds routing overhead and uses more candidate memory than STOM-CTR. Long-video compression is not lossless; see the full paper for latency and accuracy comparisons.
03 / A CLOSER LOOK
The answer is in the history.
A StreamingBench example with NEO-ov 2B shows how historical evidence supports an answer.
“Why can thieves enter the room without being detected?”
Router decision Heavy event
The question needs evidence from preceding events. KeyRec selects historical events for temporal coverage and combines them with recent evidence.
MODEL ANSWER
“Because the masters have all fallen asleep.”
The orange patches are retained visual evidence. Red borders in the paper figure identify selected events.
04 / EXPLORE THE WORK
Paper & resources.
Read the method, experimental protocol, ablations, and qualitative analysis.
05 / CITATION
BibTeX
@article{chen2026keyrec,
title={KeyRec: Bounded Visual Memory for Streaming and Long-Video Understanding},
author={Chen, Zihan and Rong, Xuejian and Wang, Xiaojuan and Gong, Boqing and Zicher, Adi and Pritch, Yael and Karnad, Nikhil},
journal={arXiv preprint arXiv:2609.32182},
year={2026}
}