[MB-58147] Slow down eviction when resident ratio reaches 0

Product: Couchbase Server
Component: index-service
Issue Link: MB-58147
Affects Version(s): 7.2.6
Fix Version(s): 7.6.0

Summary

• When the index resident ratio drops to 0% (no items remain in memory), the eviction process may continue running even though there are no items left to evict.
• In this state, eviction can enter a busy loop, causing excessive garbage generation and triggering frequent garbage collection. This leads to unnecessary CPU consumption.
• As noted in MB-58147, the eviction process should slow down when it reaches this condition to avoid high CPU usage and excessive garbage production.

Symptoms

• Constant eviction despite a resident ratio of 0%.
• Constant Garbage Collection despite no drop in memory.
• High CPU for the Indexer process.

Triggers

• Resident ratio reaches 0%

Verification

• Confirm that the resident ratio of indexes is at 0%("resident_ratio":0.00000) as observed in indexer.log files:

2024-06-21T12:33:38.276+00:00 [Info] prdcem_small/cem_small_generic_links_all_email_idx/Backstore#16999956971241817872:0 Plasma: Warning: not enough memory to hold records in memory. MemStats: {"memory_size":3286009,"memory_size_index":2761585,"buf_memused":2819626,"mvcc_purge_ratio":1.00307,"resident_ratio":0.00000,"alloc_size":13800490,"free_size":7752896,"items_count":5334673,"recs_in_mem":0,"reclaimed":7725094,"reclaim_pending":27802}

• Verify whether indexer memory usage remains stuck at a consistent level.

  

• Check if Garbage Collection (GC) is running continuously, even though memory usage is not decreasing.

  

• Determine whether high CPU usage consistently coincides with periods of frequent Garbage Collection.

  

Workarounds

• Increase the Index Service quota (which may require adding memory to Index Service nodes) to improve the resident ratio and address sizing requirements.