Articles by tag: long-context

TurboQuant: 3-bit KV-cache for LLM without losses

Learn how Google TurboQuant compresses key-value cache of LLM to 3 bits, speeding up inference 8x. Benchmarks on Gemma/Mistral, application in RAG. For developers.