Wayland screenshot search without AI: scripts

Local Screen Shot Search in Wayland Without AI

Bash scripts enable screenshot capture with automatic OCR text extraction and content-based search—no neural networks or cloud services required. Designed for Wayland environments using grim, slurp, tesseract, and tofi. Each screenshot is saved alongside a companion .txt file containing extracted text for indexing and fast retrieval.

Required tools:

• grim: screen capture
• slurp: region selection
• tesseract: OCR engine
• tofi: search interface
• wl-clipboard and cliphist: clipboard management
• notify-send: desktop notifications

screen.sh: Capture & OCR Script

This script captures a selected screen region, saves it as PNG, and asynchronously runs tesseract to generate a text sidecar file. Supports both Russian and English language recognition.

#!/bin/bash
set -euo pipefail

SCREENSHOT_DIR="${SCREENSHOT_DIR:-$HOME/Pictures/screens}"
OCR_LANG="${OCR_LANG:-rus+eng}"
SLURP_ARGS=(-d -b 1B1F2866 -c 89b4faff -w 1)
GRIM_ARGS=(-t png -l 3)

die()  { notify-send "screen.sh" "$1" -i dialog-error -t 4000; exit 1; }
need() { for cmd in " $@"; do command -v "$cmd" &>/dev/null || die "missing: $cmd"; done; }

OCR function with result cleanup:

run_ocr() {
    tesseract "$1" - -l "$OCR_LANG" --psm 3 2>/dev/null \
        | grep -v '^[[:space:]]*$' \
        | sed 's/[[:space:]]\+/ /g; s/^ //; s/ $//'
}

save_sidecar() {
    run_ocr "$1" > "${1%.png}.txt" 2>/dev/null || true
}

OCR runs in the background: command -v tesseract &>/dev/null && { save_sidecar "$FILEPATH" & disown; }. Notifications are interactive—offer options to open the folder, edit in swappy, or copy text to clipboard.

The --ocr mode copies extracted text directly without saving a file.

ssearch.sh: Indexing & Search Script

Supports three modes:

• --index: process existing PNG files lacking .txt sidecars
• --list: output TSV with path, timestamp, and text snippet
• Main mode: fuzzy-search via tofi

Indexing logic:

if [[ "${1:-}" == "--index" ]]; then
    while IFS= read -r png; do
        txt="${png%.png}.txt"
        [[ -f "$txt" ]] && continue
        run_ocr "$png" > "$txt" 2>/dev/null || true
    done < <(find "$SCREENSHOT_DIR" -name "screenshot-*.png" -type f | sort)
fi

List generation:

if [[ "${1:-}" == "--list" ]]; then
    while IFS= read -r txt; do
        png="${txt%.txt}.png"
        [[ -f "$png" ]] || continue
        base=$(basename "$txt" .txt)
        dt="${base#screenshot-}"
        stamp="${dt:0:10} ${dt:11:2}:${dt:13:2}:${dt:15:2}"
        snippet=$(head -3 "$txt" | tr '\n' ' ' | cut -c1-120)
        printf '%s\t%s\t%s\n' "$png" "$stamp" "$snippet"
    done < <(find "$SCREENSHOT_DIR" -name "screenshot-*.txt" -type f | sort -r)
fi

Search pipes into tofi with fuzzy matching and 1100px width. Selected results copy text to wl-clipboard and open the image via xdg-open.

Setup & Integration

Install on Arch:

sudo pacman -S grim slurp tesseract tesseract-data-rus wl-clipboard libnotify swappy

Make scripts executable and bind shortcuts in Hyprland:

bind = , Print, exec, ~/wayland/scripts/screen.sh
bind = SHIFT, Print, exec, ~/wayland/scripts/screen.sh --ocr
bind = SUPER, F, exec, ~/wayland/scripts/ssearch.sh

For older screenshots: ./ssearch.sh --index.

Key Points

• OCR runs asynchronously—never blocks the UI; PSM 3 is ideal for mixed-content screens.
• tofi’s fuzzy-match handles OCR inaccuracies gracefully.
• Sidecar files are always created—even empty ones—for reliable indexing.
• Zero AI or cloud dependencies; pure Unix tooling only.
• Tailored for Wayland + tofi; adapting to GNOME/KDE requires replacing the launcher.

Limitations & Improvements

Tesseract struggles with small fonts in dark themes—test PSM/OEM settings. Search is purely string-based, no semantic understanding. For rotated screenshots, set up cron or systemd-timer to clean up files older than N days.

— Editorial Team