We use cookies to measure traffic and show relevant ads. By clicking Accept you agree to our use of analytics and advertising cookies.
Expansion of ruGPT3XL to 8k tokens via sparse attention and fine-tuning. PPL 13.00 on gazeta, Triton/SDPA optimization ×1.85. Instructions for developers.
