Articles by tag: transformers

Large Language Models: Basics and Practice

Get to know language models, LLM, tokens, instruct versions, and multimodality. Practical launch of Qwen in Colab for developers. Start experimenting with open models.

Prompt Caching LLM: KV cache 10 times cheaper

Prompt Caching Breakdown: how OpenAI and Anthropic cache KV attention to reduce costs and delays. Technical details for developers, inference examples. Speed up your LLM queries.

Self-attention and multi-head in transformers

Breakdown of attention mechanisms: self-attention, cross-attention, multi-head with examples and PyTorch code. Theory, mathematics, practice for AI developers. Study scaled dot-product attention.

Neural Networks and Multiplication: SwiGLU in Transformers

Breaking down why perceptrons don't multiply and how SwiGLU solves the problem. Formulas, benchmarks, application in LLM. For middle/senior dev.