MLOps on free GPUs: processing PDF archives

MLOps on the Edge: How to Process 9 GB of Pre-Revolutionary Press Archives on Free GPUs

Developing ML solutions under tight constraints isn’t an academic exercise—it’s the daily reality for many engineers. In this article, we dive into the technical implementation of two projects built on a single stack: AI-Vet-Scanner for veterinary triage and a LoRA model for generating imperial-style visuals. Both were developed without any paid infrastructure resources—relying solely on Hugging Face Spaces’ free quotas and cloud-based Jupyter environments with T4 GPUs and 16 GB of RAM.

ZeroGPU Architecture: Serverless Inference

AI-Vet-Scanner tackles multi-class image classification for cattle. The key requirement is zero cost for end users. To achieve this, we abandoned traditional deployment on dedicated GPUs. Instead, we leveraged Hugging Face Spaces’ ZeroGPU mechanism: the model is loaded into memory only during the processing of a single request and then completely unloaded. This is accomplished through:

• Using transformers + accelerate with lazy loading of models;
• Forcibly unloading the model (model.cpu(); del model) after inference;
• Disabling intermediate tensor caching via torch.inference_mode();
• Providing a CPU-only fallback when GPUs are unavailable.

This approach ensures fault tolerance even under peak loads—there’s no state that can get stuck in memory. Inference remains free and scalable without manual instance management.

Processing a 9.24 GB PDF Archive: Battling OOM and Disk Space

The second project—preparing a dataset for SDXL LoRA—faced a fundamental challenge: the 9.24 GB scan archive from the journal “Vestnik Mody” (1885–1917) simply couldn’t be loaded entirely into 20 GB of disk space and 16 GB of RAM. Standard tools like pdf2image + Poppler immediately trigger OOM due to buffering all pages in RAM.

We replaced them with an optimized pipeline based on PyMuPDF (fitz). Its key advantages include:

• No full document load into memory;
• Rendering strictly one page at a time;
• Support for compressed rendering (e.g., matrix=fitz.Matrix(2, 2) for 2× upscale without intermediate expansion in RAM);
• Native support for PDF layers and metadata.

Dispersion-Based Filtering: Math Instead of Detectors

Out of 300+ PDF pages, 70% are text columns. Training YOLO to detect engravings would be impractical: there are too few annotations and high overhead. Instead, we applied a quick heuristic filter using OpenCV:

import cv2
import numpy as np

def is_image_page(cv_img, min_variance=45):
    gray = cv2.cvtColor(cv_img, cv2.COLOR_RGB2GRAY)
    return gray.std() > min_variance

The standard deviation of brightness serves as a reliable indicator of visual complexity: text pages yield σ ≈ 15–30, while engravings and illustrations have σ ≥ 50–90. This approach reduced the volume of data processed by 73% without losing relevant samples.

Memory Management: GC as Part of the Pipeline

After filtering and resizing to 1024×1024 (LANCZOS), we perform forced cleanup:

• del cv_img, gray;
• gc.collect();
• torch.cuda.empty_cache() (for cases where CUDA acceleration is used);
• Saving JPEGs at quality 92 (PIL.Image.save(..., quality=92)) to minimize file size without noticeable loss of detail.

This prevents temporary files from accumulating and avoids memory leaks during long processing cycles. Without these steps, 20 GB of disk space would fill up in about 12 hours.

What Matters

• ZeroGPU isn’t a Hugging Face feature—it’s a pattern: dynamically loading/unloading models on demand rather than hosting them permanently.
• PyMuPDF is 4.2× faster than pdf2image when handling multi-page PDFs and consumes 87% less RAM.
• Brightness dispersion is a valid and fast indicator for filtering illustrations in historical archives; it requires no training and runs on the CPU.
• Forced gc.collect() and empty_cache() should be part of every processing step when resource limits are strict.
• All operations (rendering, filtering, resizing, saving) must be performed in strict sequence without buffering intermediate arrays.

At this stage, the autopilot processes about 1,400 pages per hour on a T4 GPU. Once extraction is complete, the ZIP archive will be fed into the SDXL LoRA training pipeline for fine-tuning over 500 epochs. The target prompt involves semantically blending modern objects with historical styles—for example, “a cyber-monocle in the style of an 1890s engraving.” The technical challenge here isn’t generation but ensuring the dataset’s purity and representativeness. It’s precisely this stage that determines the final model’s quality—especially when dealing with small target class volumes.

The engineering value of such projects lies in demonstrating that performance doesn’t always depend on budget, but on the precision of tool selection and a deep understanding of constraints. “DIY” MLOps isn’t a compromise—it’s a discipline where every call to gc.collect() and every line in fitz.Matrix() has a measurable impact.

— Editorial Team