Your Data Layer Dashboard
The cat indices check is your first look at the data layer health. It provides a quick overview of all indices, their health status, document counts, and storage usage - essential for understanding your cluster's data landscape.
Among all Elasticsearch APIs, cat indices is probably the most frequently used by administrators. It gives you an instant snapshot of your cluster's data: which indices exist, how healthy they are, how much space they use, and how many documents they contain. This check forms the foundation of data layer diagnostics.
What You'll Learn
Index Assessment
- • Reading index health indicators
- • Understanding size and document metrics
- • Identifying index naming patterns
- • Spotting storage inefficiencies
Problem Detection
- • Finding oversized or undersized indices
- • Detecting index proliferation issues
- • Identifying stale or unused indices
- • Monitoring storage growth patterns
Cat Indices API Deep Dive
GET /_cat/indices?v&s=store.size:descSimple English Explanation
Think of this API as getting a "table of contents" for your data. It's like asking: "What books (indices) do I have on my shelf, how thick are they (size), how many pages (documents), and are they in good condition (health)?"
It's the first thing most administrators check when investigating data-related issues.
Sample Output
health status index pri rep docs.count docs.deleted store.size pri.store.size green open logs-2024.12.04 1 1 1234567 123 2.1gb 1.0gb yellow open metrics-2024.12.04 1 1 987654 45 1.8gb 900mb green open users 1 1 5432 0 12.4mb 6.2mb red open corrupted-index 1 1 0 0 0b 0b green open .kibana_7.15.0_001 1 1 156 3 1.2mb 600kb
📊 Key Columns
- • health: green/yellow/red status
- • status: open/close state
- • index: Index name
- • pri/rep: Primary/replica shard counts
- • docs.count: Total documents
- • store.size: Total storage used
🔧 Useful Parameters
- • ?v: Show column headers
- • ?s=store.size:desc: Sort by size
- • ?h=health,index,docs.count,store.size: Custom columns
- • ?bytes=b: Show exact byte values
- • logs-*: Filter by pattern
Common Index Issues and Solutions
🚨 Critical: Red Index Status
Immediate Impact
- • Data unavailable or partially accessible
- • Search results incomplete
- • Indexing may fail for affected shards
- • Potential data loss risk
Emergency Actions
- 1. Check allocation explain for specific shard
- 2. Verify node availability and disk space
- 3. Review cluster logs for errors
- 4. Consider restoring from backup
- 5. Use allocation reroute if safe
⚠️ Warning: Index Proliferation
Symptoms
- • Hundreds or thousands of indices
- • Many small indices (<1GB each)
- • Poor search performance
- • High cluster state overhead
Solutions
- • Implement data streams for time-series data
- • Use rollover policies to control index size
- • Delete old indices with ILM
- • Consider index templates for consistency
ℹ️ Info: Storage Optimization Opportunities
Optimization Targets
- • Large indices that could be split
- • Old indices suitable for cold storage
- • Indices with high deleted document ratio
- • Unused or forgotten indices
Optimization Actions
- • Force merge indices with high deletions
- • Move old data to warm/cold tiers
- • Delete unused indices after verification
- • Implement automated lifecycle policies
ElasticDoctor Analysis Implementation
🔍 How ElasticDoctor Analyzes Index Health
Health Status Assessment
ElasticDoctor automatically categorizes indices by health status (red/yellow/green) and provides immediate alerts for any indices with availability issues or reduced redundancy.
Size and Storage Analysis
Identifies oversized indices that may impact performance, calculates storage efficiency metrics, and detects indices with high document deletion ratios that waste disk space.
Index Proliferation Detection
Monitors index creation patterns to identify proliferation issues, recommends consolidation strategies, and suggests lifecycle management improvements.
Optimization Recommendations
Provides actionable recommendations for force merging, data tier migration, index lifecycle policies, and capacity planning based on usage patterns.
Index Management Best Practices
✅ Healthy Index Patterns
- • Use consistent naming conventions
- • Implement time-based indices for logs
- • Keep shard sizes between 10-50GB
- • Monitor and maintain green health status
- • Use index templates for consistency
💡 Monitoring Tips
- • Check indices health daily
- • Monitor storage growth trends
- • Set up alerts for red/yellow status
- • Track index creation patterns
- • Review document deletion ratios
❌ Index Anti-Patterns
- • Ignoring red index status
- • Creating too many small indices
- • Not implementing lifecycle management
- • Keeping unused indices indefinitely
- • Allowing unlimited index growth
⚠️ Performance Impact
- • Red indices affect search completeness
- • Too many indices slow cluster operations
- • Large indices can impact query performance
- • High deletion ratios waste storage
Index Health Mastery
Essential Insights
- • Health First: Green status should be the norm
- • Size Matters: Monitor and control index growth
- • Pattern Recognition: Identify optimization opportunities
- • Proactive Management: Prevent issues before they impact users
Action Plan
- • Implement daily index health monitoring
- • Set up automated lifecycle management
- • Create index optimization procedures
- • Establish capacity planning processes