DAPE.work
Login Register

Rejected Execution

Severity: High
Elasticsearch Version: 8.5.0

Problem

Search or indexing requests are rejected due to thread pool saturation

Root Cause

Excessive load causing thread pools to reach maximum capacity, leading to request rejections

How to Detect

Symptoms

  • Rejection errors in logs indicating thread pool saturation
  • High thread pool queue sizes
  • Increased response times and timeouts

Commands

GET /_cat/thread_pool?v
GET /_cluster/health
GET /_nodes/stats/thread_pool

Remediation Steps

  1. Identify overloaded thread pools and nodes
  2. Increase thread pool sizes via cluster settings if hardware permits
  3. Implement rate limiting or request throttling at application level
  4. Optimize queries and indexing operations to reduce load
  5. Scale the cluster horizontally by adding nodes

Prevention

  • Monitor thread pool metrics regularly
  • Set appropriate thread pool size limits based on hardware capacity
  • Implement proactive load balancing and request rate limiting
  • Optimize query and indexing performance
  • Scale cluster resources in anticipation of increased load

Production Example

curl -XPUT 'localhost:9200/_cluster/settings' -H 'Content-Type: application/json' -d '{"persistent": {"thread_pool.search.size": 50, "thread_pool.index.size": 30}}'
View all Elasticsearch errors