ipset and CrowdSec Server Protection from Scanners

Multi-Layer Traffic Filtering: ipset, Auto-Blocking, and CrowdSec for Server Security

An ipset-based filtering system with auto-blocking of suspicious IPs and CrowdSec slashes junk traffic from 12% to under 0.3%. Blocking happens at the Linux kernel level (L3/L4), before nginx and PHP-FPM even process requests. This cuts server load on production Bitrix sites, where scans for .env, wp-login.php, and config.php used to spike failures up to 99%.

Key benefits: full visibility into blocks, atomic IP list updates, and modular design for quick project-specific tweaks.

Advantages Over External WAFs and CDNs

External WAFs and CDNs shine for massive global apps, but for low-traffic local setups, self-hosted filtering is the way to go.

Top reasons to build your own:

• Total control: See exactly which IPs are blocked, by what rule, and for how long.
• Block before the app: No wasted nginx/PHP resources on trash traffic.
• Flexibility: Easily add rules for Bitrix-specific patterns like .env or .git.
• Simplicity: No extra layers bloating your stack.

Analysis runs on HTTP logs (L7), but packet drops happen in iptables with ipset at the kernel level.

Three-Layer Defense Structure

The system is tiered for step-by-step filtering.

Layer 1: Geo-blocking. Packets from irrelevant countries get dropped immediately—no HTTP parsing needed.

Layer 2: Auto-blocking suspicious IPs. Post-geo, it scans for patterns like wp-login.php, shell.php, .env. Culprits get banned for 2 hours.

Layer 3: CrowdSec. The bouncer feeds a global blacklist into ipset with TTL.

Modularity lets you toggle layers independently, minimizing false positives.

ipset: The Scalability Foundation

Plain iptables with thousands of IP rules means linear O(n) scans, tanking performance under load.

ipset uses hash:ip or hash:net for O(1) lookups:

-m set --match-set blocked_ips src

List updates are atomic via temp sets and swaps, with zero downtime:

ipset create "${country}_temp" hash:net 2>/dev/null || ipset flush "${country}_temp"
if [[ -s "$temp_file" ]]; then
    while read network; do
        ipset add "${country}_temp" "$network"
    done < "$temp_file"
    ipset swap "${country}_temp" "$country"
fi
ipset destroy "${country}_temp"

This keeps protection rock-solid even during bulk list refreshes.

Scanner Detection by Patterns

The auto-block-suspicious.sh script tails logs, zeroing in on shady requests:

• wp-login.php (even on Bitrix).
• shell.php, shoha.php.
• .env, .git, .aws/credentials.

IPs get auto-banned for 2 hours. Example: A barrage of 200+ /wp-login.php hits neutralized in 20–30 seconds, load normalized instantly.

Blocking stats:

• China: 8,802 IPs.
• Suspicious: 10 IPs (e.g., 4.193.97.168, 195.178.110.109).

CrowdSec Integration

The CrowdSec bouncer populates the crowdsec_blacklist ipset with TTL:

• Auto-syncs global signatures.
• Auto-expires on timeout.

CrowdSec bolsters custom modules, handling known threats without overhauling your logic.

Implementation Results

Metrics comparison (2025 no WAF vs. 2026 with WAF):

| Metric | No Protection | With WAF |

|------------------|---------------|-------------|

| Junk Traffic | ~12% | <0.3% |

| Load Average | 1.8–2.5 | 0.9–1.2 |

| Failure Rate | Up to 40% | 12–15% |

Scanning RPS spikes die at the kernel; analytics now bot-free.

Limitations:

• No defense against high-volume L4 attacks.
• Needs CMS updates (Bitrix).
• Weak against residential IPs.
• Not for global CDN-scale projects.

Key Takeaways

• Scalability: ipset delivers O(1) checks for millions of IPs.
• Atomicity: Swaps prevent filtering gaps.
• Modularity: Independent layers (geo, suspicious, CrowdSec) for precise tuning.
• Efficiency: Junk down to 0.3%, load average halved.
• Transparency: Full visibility, no black boxes.

— Editorial Team