Articles by tag: attention

Prompt Caching LLM: KV cache 10 times cheaper

Prompt Caching Breakdown: how OpenAI and Anthropic cache KV attention to reduce costs and delays. Technical details for developers, inference examples. Speed up your LLM queries.

Induction Heads on TinyStories: weak formation

Experiment shows why TinyStories suppresses Induction Heads in GPT-2. Analysis of SAE, metrics and comparison with Previous Token Heads. Study the impact of the dataset on ICL for your models.