Skip to main content
Docs Developer Videos Webinars Academy Blog Forum Contact

[CBL-7006] Updated Blob Replication Failures with Delta Sync in Couchbase Lite P2P

Yuvraj
Updated 
Product: Couchbase Lite
Component: couchbase-lite
Issue Link: CBL-7006
Affects Version: 3.2.x
Fix Version: 3.2.4

Summary

  • The main issue observed was that the Blobs (binary attachments) were not reliably getting replicated when Couchbase Lite Peer-to-Peer (P2P) replication (with Delta Sync) was done, especially after updating blob data was done on one of the peers.
  • Specifically, when a document containing a blob is updated on one peer and then replicated using delta sync, the receiving peer may not recognize the newly-referenced blob and does not fetch the updated blob content.
  • This causes the blob pointer in the replicated document to reference a blob digest that the device does not have, resulting in broken or missing blob access.
  • This issue does not occur if delta sync is disabled.

Symptoms 

  • Accessing updated blob data from a document replicated via P2P returns null, incorrect, or empty binary data when the application tries to read it.
  • Subsequent document updates that reference new blobs may correctly forward document metadata (like digest and length), but the actual blob content isn’t included in replication. This can be confirmed in logs - A log value of _mayContainBlobChanges=1 indicates that the update includes changes to a blob and triggers a fetch, while _mayContainBlobChanges=0 means no blob change was detected, causing the blob not to be synced.

Triggers

  • Delta Sync enabled for P2P replication between Couchbase Lite devices
  • A document with a blob is updated on one device using a newly-created blob (the pointer is updated to a new digest/reference).
  • The update is propagated as a document delta, not a full revision.
  • The new blob is not detected on the receiving device.

Verification

  1. Confirm that Delta Sync is enabled in P2P replication.
  2. Do a preliminary check to see if the documents match on both sides of the replication. Check and match document contents (For example, using the cblite tool on both sides' databases):
{
  "_id": "test",
  "blob_/audio": {
    "@type": "blob",
    "content_type": "audio/mp3",
    "digest": "sha1-iN9s20fWFBKYL4brbc6mB71Zkg=",
    "length": 16788
  }
}
  1. Check for the following log patterns during/after replication:

On initial (full) sync, blob is detected and transferred:

  • One of the peers (let’s say Device A) sends revision 1.
2025-05-12T18:08:50.959738Z Sync Verbose Obj=/DNRepl@236D5896628/C4RemoteRepl#159/Repl#160/Pusher#162/ Coll=0 Transmitting 'rev' message with 'test' #1-9f1fa405854ab28a92d9e6e3f1cbb0e50719697d
2025-05-12T18:08:50.959897Z BLIP Verbose Obj=/DNRepl@236D5896628/C4RemoteRepl#159/Repl#160/Connection#156/BLIPIO#157/ Sending 'rev' REQ #0 Z
  • The other peer (Device B) detects the blob (_mayContainBlobChanges=1 is seen) and transfers it.
2025-05-12T18:08:50.955141Z Sync Verbose Obj=/C4IncomingRepl#124/PsvRepl#130/Puller#144/IncomingRev#618/ Coll=0 _mayContainBlobChanges=1
2025-05-12T18:08:50.955153Z Sync Verbose Obj=/C4IncomingRepl#124/PsvRepl#130/Puller#144/IncomingRev#618/ Coll=0 _mayContainEncryptedProperties=0
2025-05-12T18:08:50.955317Z Sync Verbose Obj=/C4IncomingRepl#124/PsvRepl#130/Puller#144/IncomingRev#618/ Coll=0 Requesting blob (61359 bytes, compress=0)
  • Logs show the digest, and blobs getting received:
2025-05-12T18:08:50.960648Z Sync Verbose Obj=/C4IncomingRepl#124/PsvRepl#130/Puller#144/IncomingRev#618/ Coll=0 Finished receiving blob sha1-C0vIqN+E/ASYntH5PVnUQv1vXs= (61359 bytes)
2025-05-12T18:08:50.961285Z Sync Verbose Obj=/C4IncomingRepl#124/PsvRepl#130/Puller#144/IncomingRev#618/ Coll=0 All blobs received, now inserting revision
  • Logs also show the content:
2025-05-12T18:08:50.991038Z Query Verbose Obj=/QueryEnum#620/ --> [{"ParentId":"155","ProjectId":"51","Id":"test","DateUpdated":"2025-05-12T18:08:50.7737913+00:00","Type":"Audio","Environment":"Dev","blob_/audio":{"digest":"sha1-C0vIqN+E/ASYntH5PVnUQv1vXs=","length":61359,"content_type":"audio/mp3","@type":"blob"}}]

On subsequent (delta) sync, blob change is not detected:

  • Device B updates the document to revision 2:
2025-05-12T18:09:30.177866Z DB Verbose Saved doc 'test' #2-3e7709308f9987d7d9facd22a24101255d7b128d; body=294, extra=378
2025-05-12T18:09:30.177883Z DB Verbose Obj=/DB#427/ Saved 'test' rev #2-3e7709308f9987d7d9facd22a24101255d7b128d as seq 5
  • Device B encodes the revision as a delta (having a new digest and length) and sends it
2025-05-12T18:09:30.181071Z Sync Verbose Obj=/C4IncomingRepl#124/PsvRepl#130/Pusher#139/ Coll=0 Queueing rev 'test' #2-3e7709308f9987d7d9facd22a24101255d7b128d (seq #5) [1 queued]
2025-05-12T18:09:30.181157Z Sync Verbose Obj=/C4IncomingRepl#124/PsvRepl#130/Pusher#139/ Coll=0 Sending rev 'test' #2-3e7709308f9987d7d9facd22a24101255d7b128d (seq #5) [0/10]
2025-05-12T18:09:30.181581Z Sync Verbose Obj=/C4IncomingRepl#124/PsvRepl#130/Pusher#139/ Coll=0 Encoded revision as delta, saving 247 bytes:
    old = {"ParentId":"155","ProjectId":"51","Id":"test","DateUpdated":"2025-05-12T18:08:50.7737913+00:00","Type":"Audio","Environment":"Dev","blob_/audio":{"digest":"sha1-C0vIqN+E/ASYntH5PVnUQv1vXs=","length":61359,"content_type":"audio/mp3","@type":"blob"}}
    new = {"ParentId":"155","ProjectId":"51","Id":"test","DateUpdated":"2025-05-12T18:09:30.1739925+00:00","Type":"Audio","Environment":"Dev","blob_/audio":{"digest":"sha1-iN9s20fWFBKYL4brbc6mB71Zkg=","length":16788,"content_type":"audio/mp3","@type":"blob"}}
    Delta = {"DateUpdated":"2025-05-12T18:09:30.1739925+00:00","blob_/audio":{"digest":"sha1-iN9s20fWFBKYL4brbc6mB71Zkg=","length":16788}}
2025-05-12T18:09:30.181640Z Sync Verbose Obj=/C4IncomingRepl#124/PsvRepl#130/Pusher#139/ Coll=0 Transmitting 'rev' message with 'test' #2-3e7709308f9987d7d9facd22a24101255d7b128d
2025-05-12T18:09:30.181746Z BLIP Verbose Obj=/C4IncomingRepl#124/PsvRepl#130/Connection#117/BLIPIO#120/ Sending 'rev' REQ #0 Z
  • Device A receives the change; however, it does not detect any changes to the Blob ((_mayContainBlobChanges=0 is seen)) as seen below:
2025-05-12T18:09:30.189007Z Sync Verbose Obj=/DNRepl@236D5896628/C4RemoteRepl#159/Repl#160/Puller#165/IncomingRev#27770/ Coll=0 Received revision 'test' #2-3e7709308f9987d7d9facd22a24101255d7b128d (seq '5')
...
2025-05-12T18:09:30.189092Z Sync Verbose Obj=/DNRepl@236D5896628/C4RemoteRepl#159/Repl#160/Puller#165/IncomingRev#27770/ Coll=0 _mayContainBlobChanges=0
  • On reviewing the contents of the document, we see that the digest and the length are updated:
2025-05-12T18:09:30.221261Z Query Verbose Obj=/QueryEnum#27772/ --> [{"ParentId":"155","ProjectId":"51","Id":"test","DateUpdated":"2025-05-12T18:09:30.1739925+00:00","Type":"Audio","Environment":"Dev","blob_/audio":{"digest":"sha1-iN9s20fWFBKYL4brbc6mB71Zkg=","length":16788,"content_type":"audio/mp3","@type":"blob"}}]

Workarounds

  • Disable delta sync for peer-to-peer replications involving blobs. This will force full revision transfer, so all referenced blobs are evaluated and fetched as needed.
  • Upgrade to Couchbase Lite 3.2.4 or later, which addresses this issue.

Related to

Related articles

Comments

0 comments

Article is closed for comments.