KV-cache compression

A new paper on arXiv describes a self-pruning transformer architecture that combines extreme KV-cache compression with universal attention, promising major memory savings for large language models during inference. The KV-cache is one of the biggest memory bottlenecks in running long-context AI models, so techniques that shrink it sharply could make serving models cheaper and allow longer contexts on modest hardware. Early reactions are focused on whether the compression holds up against established methods in real workloads.