Main Tech Discovery Antistress
🇬🇧 English 🇪🇸 Español 🇩🇪 Deutsch 🇫🇷 Français 🇵🇱 Polski 🇨🇿 Česky 🇨🇳 中文 🇰🇷 한국어

Articles by tag: cpu-offload

Running MoE GPT-OSS-120B on 6 GB GPU up to 30 t/s

Running MoE GPT-OSS-120B on 6 GB GPU up to 30 t/s

Accelerating GPT-OSS-120B on a weak GPU with -cmoe and --fit. Efficient RAM usage for MoE models. Instructions for developers.