[CXXCBC-699] SDKs built on C++ SDK versions below 1.1.1 can result in uneven number of connections across data nodes.

Product: C++ SDK
Component: SDK
Issue Link: CXXCBC-699
Affects Versions: 1.0.x, 1.1.0
Fix Versions: 1.1.1, 1.2.0

Summary

• When using SDKs built on top of the Couchbase C++ SDK (such as the Python, Ruby, or PHP SDKs) prior to version 1.1.1 (for example, 1.1.0 or any 1.0.x release), an uneven distribution of connections from the application to Data nodes may occur. This happens when the connection string contains a list of IP addresses instead of a Fully Qualified Domain Name (FQDN).

Symptoms 

• You may observe a significant imbalance in the number of connections from the application to different Data Service nodes. In one reported case, Couchbase identified twice as many connections to one node compared with the other two.

Triggers

• The issue can occur when the connection string contains a list of Data node IP addresses.
• It can also occur when an FQDN is used, but the DNS server does not properly randomize the IP addresses it returns for that FQDN.

Verification

• Check the metric kv_curr_connections (see Data Service metrics reference). If there is a noticeable difference in this metric across Data Service nodes, your application is likely affected.
• This issue impacts SDKs built on the Couchbase C++ SDK only. The following SDKs use the C++ SDK core rather than the older C SDK (libcouchbase): Python SDK 4.x, Ruby SDK 3.4.x, and PHP SDK 4.x. Previous versions of these SDKs should not be affected.

Workarounds

• The recommended solution is to upgrade your SDK to a version that uses Couchbase C++ SDK 1.1.1, 1.2.0, or higher.
• If upgrading is not possible, randomize the list of Data Service node IPs in your connection string before each connection attempt to balance connections more evenly.