[MB-66876] In Couchbase Server 7.6.x Magma storage stopped compressing values greater than 64 KB.

Product: Couchbase Server
Component: Data-Service
Issue Link: MB-66876
Affects Version: 7.6.x
Fix Versions: 7.6.8, 8.0.0

Summary

• A regression in Couchbase Server 7.6.x causes the Magma storage engine to stop compressing values greater than 64 KB. This occurs if SDK compression is disabled or when using Sub-Document Operations. As a result, you may experience increased space usage and slower data processing in any bucket that uses the Magma storage engine. For example, backup operations may take longer to complete.

Symptoms 

• Buckets using the Magma storage engine consume more disk space after you upgrade to Couchbase Server 7.6.x.
• Backups of these buckets take significantly longer following the upgrade.

Triggers

• This issue affects any bucket using Magma with documents larger than 64 KB after upgrading to Couchbase Server 7.6.x.

Verification

• Disk usage increases significantly after an upgrade to Couchbase Server 7.6.x.
• Backup speed decreases significantly after an upgrade to Couchbase Server 7.6.x.

Workarounds

The workaround for this issue is as follows:

• Increase the Magma configuration parameter to a value higher than the maximum supported document size (20 MB). In this example, set it to 30 MB. Use cbepctl utility to make the change against one node; beware that it would be lost after a node restart:

cbepctl <host>:11210 -b <bucket> -u <user> -p <password> set flush_param magma_min_value_block_size_threshold 30000000

To make the change persistent, set the parameter in the bucket’s extra configuration string:

curl -v -u <user>:<password> -X POST http://address:8091/diag/eval -d 'ns_bucket:update_bucket_props(<bucket>, [{extra_config_string, "magma_min_value_block_size_threshold=30000000}]).'

• Only new data written after this change benefits from compression. To apply compression to existing data, run compaction from the Web UI. Run compaction during periods of low activity, as it is resource-intensive.