https://sudonull.com/rugpt3xl-with-8k-context-sparse-attention
Expansion of ruGPT3XL to 8k tokens via sparse attention and fine-tuning. PPL 13.00 on gazeta, Triton/SDPA optimization ×1.85. Instructions for developers.
Search Articles — Sudonull
Search Results
In this project
Trending Now
Docker Container Networking Explained: A Practical Guide
Docker container networking explained in depth: learn bridge vs host networks, DNS, port publishing, and how to build secure, scalable multi-container apps.
What Is a Vector Database? Beginner's Guide to AI Search
What is a vector database? Learn how vector databases power AI search, RAG, and recommendations. Discover key concepts and when to use them. Start here.
How Does Retrieval Augmented Generation Work? RAG Guide
Learn how does retrieval augmented generation work: RAG reduces AI hallucinations, grounds responses in real data, and transforms LLMs into trustworthy systems. Start now.
MongoDB vs PostgreSQL When to Use Each: Key Differences
MongoDB vs PostgreSQL when to use each: compare ACID compliance, scalability, schema flexibility, and query power. Choose the right database for your project.
Google Gemini Ultra 2.0: record performance and price
Google introduced Gemini Ultra 2.0 with a context of up to 10 million tokens, surpassing GPT-5. Learn about the breakthrough architecture, price of $0.0005, and impact on the AI market. Read the full analysis.
REST API vs GraphQL: Key Differences & How to Choose
Compare REST API vs GraphQL side-by-side. Learn which architecture fits your project with data on performance, caching, cost, and scalability. Make the right choice.
How to Learn Python for Beginners: Step-by-Step Roadmap
Discover how to learn Python for beginners with this step-by-step roadmap. Master fundamentals, data structures, and build real projects. Start your coding journey today.
PostgreSQL vs MySQL: Which One to Choose?
PostgreSQL vs MySQL which one to choose? Compare features, performance, scalability, and use cases. Make an informed decision for your project with this data-driven guide.
JavaScript vs Python Which Is Better: Full 2026 Guide
JavaScript vs Python which is better for you? Compare ease of learning, salaries, use cases, and career paths to choose the right first language in 2026. Start now.
Starlink Launch in Japan: Direct Smartphone Communication via Satellite
SpaceX and KDDI launched satellite communication for smartphones in Japan. Learn how Starlink Direct-to-Cell works, who can access it, and what hidden geopolitical goals this project pursues.