Elasticsearch search errors: anti-patterns

Critical Elasticsearch Search Design Mistakes (and How to Avoid Them)

In Elasticsearch search projects, engineering teams repeatedly make the same foundational errors. These issues migrate across systems—regardless of team experience. This article examines them as anti-patterns: proven approaches that guarantee degraded performance, relevance, and operational cost.

Recognizing these patterns early is essential for mid- and senior-level engineers. While examples focus on Elasticsearch, the principles apply universally—to Solr, OpenSearch, or custom search stacks.

Ignoring Data Quality

Search starts with content—not algorithms. Dirty data is the #1 reason search systems fail. Duplicates, unstructured text, and inconsistent formats directly undermine relevance.

Common issues:

• No normalization of product names (e.g., "iPhone 14" vs. "Apple iPhone14").
• Uncleaned descriptions riddled with typos and extraneous characters.
• Duplicate records without deduplication.

Improving data quality delivers more impact than ranking tuning. This work falls to junior engineers—but demands a systemic approach: robust ETL pipelines, ingest-time validation, and continuous index quality monitoring.

Without clean data, even BM25 or neural rerankers won’t save you. In real-world projects, 70–80% of effort goes into data quality—not query logic.

Missing Requirements and Metrics

"Make it work like Yandex" is a classic vague brief. Without documented, measurable requirements, search becomes subjective—and teams skip defining business goals: top-K relevance, zero-hit rate, or click-through rate.

Key evaluation metrics:

• Precision@K — % of relevant documents in the top K results.
• Recall@K — % of all relevant documents retrieved in top K.
• NDCG — position-aware relevance scoring with graded relevance.
• CTR and conversion rate, measured per search session.
• Latency and QPS, under realistic load.

Manual evaluation of frequent queries doesn’t replace A/B testing or offline metrics. Without them, regressions go unnoticed: rankings degrade gradually, users silently abandon search.

The "King Query": Monolith vs. Scalability

One monolithic query handling filtering, aggregations, ranking, and pagination is a widespread anti-pattern. Its appeal seems obvious: consistency, fewer network calls. But the cost is steep.

Monolith drawbacks:

• Execution time >1s on complex queries.
• Maintenance hell: adding one feature forces refactoring the entire query.
• Sharding and replication become inefficient due to query size and resource pressure.

| Approach | Latency | Maintainability | Consistency |

|----------|---------|-----------------|-------------|

| King Query | 1–3s | Low | High |

| Micro-queries + caching | 50–200ms | High | Medium (with cache) |

Break search into stages: match + filter → enrich → rerank. Use ESQL or ingest pipelines for optimization.

Fuzzy Search as a Universal Fix

Relying on fuzziness for automatic typo correction is an Elasticsearch myth. For a query like "apliphone", Elasticsearch generates hundreds of variants—"applesiphone", "aplfone", etc.

Consequences:

• Recall increases, but precision plummets.
• Relevance degrades: noise floods the top results.
• Query cost multiplies 10–50×.

Tuning parameters won’t fix the root cause:

{
  "query": {
    "multi_match": {
      "query": "apliphone",
      "fuzziness": "AUTO",
      "prefix_length": 2,
      "max_expansions": 50
    }
  }
}

prefix_length=2 reduces noise—but doesn’t solve the underlying issue. Better solutions: frontend spellcheck (Hunspell/Yandex.Speller) and synonym graphs applied at ingest.

Nested Objects and Joins in the Index

E-commerce needs relationships: product → price → stock → reviews. Normalization via parent-child joins feels logical—but kills performance.

Problems with has_child/parent:

• Deep pagination breaks entirely.
• Scoring fails to aggregate correctly across relationships.
• Schema changes trigger full reindexing.

Recommendations:

• Denormalize: embed price and stock directly into the product document.
• Use separate indices (e.g., goods, prices) and join client-side.
• Reserve nested fields only for tightly coupled, low-cardinality data.

Key Takeaways

• Data > Algorithms: 80% of search quality comes from data quality.
• Metrics are non-negotiable: Precision, Recall, and NDCG must be tracked for every use case.
• Decompose queries: The King Query guarantees >2s latency.
• Use fuzzy sparingly: Only for rare terms—and always pair with spellcheck.
• Denormalize aggressively: Joins in Elasticsearch are a production anti-pattern.

— Editorial Team