Product: Couchbase Server Component: Eventing-Service Issue Link: MB-69282 Affects Version: 8.0.0 Fix Versions: 8.0.1, 8.1.0
Summary
Upon upgrading from 7.x to 8.0.0 with pre-existing Eventing Functions created, any Eventing function with a num_timer_partition JSON value set to a non 2^x^ value will not deploy and may cause an Eventing Service crash loop. Previous versions rounded the num_timer_partition value to the nearest power of 2, but in 8.x this now is strictly enforced. The conversion that happens during a 7.x to 8.0.0 upgrade may cause the Eventing Service to incorrectly reject the deployment.
Symptoms
- Rebalances may fail on the Eventing Service with a message like the following:
2025-11-04T11:53:16.463+08:00, ns_orchestrator:0:critical:message(ns_1@<IP>) - Rebalance exited with reason {service_rebalance_failed,eventing,
{agent_died,<0.106986.0>,
{lost_connection,
{'ns_1@<IP>',shutdown}}}}.
Rebalance Operation Id = b1b570d8eea710eb9e6468b3aa5ac0dd
- Eventing Service repeatedly crashes and restarts (crash loop).
- Within the UI, Eventing functions may no longer be visible.
- Within the UI, newly created Eventing functions post upgrade may appear to be in an undeploying state.
Triggers
- Upgrading from 7.x to 8.0.0 with existing Eventing functions where any of the Eventing functions have a “num_timer_partitions" value set to a non 2^x^ value or greater than 1024 (viewable if you export the Eventing function and view the JSON).
Verification
To verify that a function has an invalid num_timer_partitions value you should see the following log line within the service level eventing.log from a Couchbase Server Log Collection.
2025-11-25T14:28:08.055+00:00 [Error] eventing::main Error starting supervisor: unable to recover previous states: num_timer_partitions value should be map[1:{} 2:{} 4:{} 8:{} 16:{} 32:{} 64:{} 128:{} 256:{} 512:{} 1024:{}]. Exiting...
The Eventing crash loop behaviour aspect of the behaviour can be identified in the service level eventing.log from a Couchbase Server Log Collection. You should see the following log line shortly after the num_timer_partitions error.
2025-11-25T14:28:15.315+00:00 [Info] eventing::main started with config: NodeVersion: 8.0.0-true NodeUUID: <NodeID> HttpPort: 8096 HttpSSLPort: 18096 EventingDir: /opt/couchbase/var/lib/couchbase/data/@eventing, NodeUUID: <Node ID>, <Certificate & Key files paths>
You can use the following grep to identify the number of instances of this issue within Couchbase Server logs for all nodes. If this returns nothing or 0, then we would expect a different cause.
grep -i "num_timer_partitions value should be map" */ns_server.eventing.log -c
Similar to this you can identify the Eventing restarts, however this could return a non-zero number due to a number of other reasons. This is to determine if any restarts have occurred, possibly as a result of an invalid
grep -i "eventing::main started with config" */ns_server.eventing.log -c
Workarounds
There is an effective workaround available.
We would recommend you first copy/export your JavaScript code from your Eventing Functions before performing any mitigation steps so that you can refer to this later if needed.
In order to end the crash loop, all Eventing functions with an invalid num_timer_partition value will need to be removed. To do so all Eventing functions can be cleared using the following command which uses the cbevent tool located at /opt/couchbase/bin (within a Couchbase Server installation):
/opt/couchbase/bin/cbevent -flush -user <username> -password <password> -host <hostname>
You can now add back your Eventing functions using one of the following two methods.
Re-Create Eventing Functions:
Newly created Eventing functions on version 8.0.0 will by default have the num_timer_partitions set to 1024. Which satisfies the hard requirement. Since they are newly created, you will need to restore your eventing function settings and code.
Import Old Eventing Functions:
- For each Eventing function, open the exported JSON and change/add “num_timer_partitions":1024 (or any 2^x^ up to 1024)
- Then import the Eventing function using the Couchbase UI, this should. preserve the previous settings that it was exported with, alongside the additional change we made to the JSON.
Comments
0 comments
Article is closed for comments.