KV Cache

Key-Value Cache: a memory structure in transformer AI models storing intermediate computations to avoid redundant processing during inference