Search Articles — Sudonull

Search Results

In this project

LLM Quantization: 160GB Model on a Laptop

https://sudonull.com/llm-quantization-160gb-model-on-a-laptop

Learn how to quantize large LLMs to 4-bit without quality loss. Symmetric/asymmetric quantization, code, benchmarks. Run 80B models locally — developer guide.

From other projects

Trending Now