[MB-64968] Analytics service restarting on certificate refresh

Product: Couchbase Server
Components: Analytics-Service, Columnar
Issue Link: MB-64968
Affects Versions: 7.2.x, 7.6.x
Fix Versions: 7.2.8, 7.6.6

Summary

A certificate refresh can cause the Analytics driver process to restart, causing alerts, failing queries, and interrupting ingestion.

Symptoms 

• Analytics Service restarts when certificates are refreshed.
• Analytics queries will fail due to the restart.
• Ingestion from KV will be interrupted due to the restart. 

Triggers

• Triggered by any refresh of the certificate/TLS config for the cluster in question.

Verification

Look for the following log messages in the Analytics Service logging containing analytics driver has exited w/ exit status 33, indicating that the Analytics driver restarted:

2025-01-17T17:33:05.720+00:00 WARN CBAS.cbas tls config has been refreshed by ns server
2025-01-17T17:33:05.720+00:00 WARN CBAS.cbas restarting driver due to encryption setting change
2025-01-17T17:33:05.720+00:00 INFO CBAS.cbas immediate restart driver request rec'd; killing & restarting analytics driver
2025-01-17T17:33:06.130+00:00 WARN CBAS.cbas analytics driver has exited w/ exit status 33

After a restart, the following log messages will be present when CBAS starts up:

2025-01-17T17:33:06.142+00:00 INFO CBAS.cbas analytics driver is running; pid: 20783
2025-01-17T17:33:13.253+00:00 INFO CBAS.utils.ClusterStateManager [Executor-12:ClusterController] Cluster State is now ACTIVE

Workarounds

No workaround available.

This behavior is addressed by code changes in the fix versions.