Root cause has been identified. If you are a node operator with `db_backend = "goleveldb"` and `app-db-backend = ""` set in app.toml, please update your config, specifying `app-db-backend = "pebbledb"` instead.
This issue stems from our use of the `types.DBbackend` compile-time variable, which was removed in Cosmos SDK v0.47. The value derived for `app-db-backend` on nodes that do not specify a value effectively changed between ZetaChain v18 and ZetaChain v19. Explicitly setting `app-db-backened = "pebbledb"` should enable goleveldb based nodes to return to service.
Posted Aug 15, 2024 - 18:58 UTC
Identified
Block production has resumed for the Athens Testnet.
Root cause is not known, but appears to be isolated to nodes with "db_backend = goleveldb" set in config.toml.
CrossChain transactions are currently blocked, but will resume within the next few hours after testing is completed.
Posted Aug 14, 2024 - 20:25 UTC
Update
The network has just under the 2/3s majority needed to restart. No root cause for failed upgrades on some validators has been identified. We have made contact with several validator operators and are actively working to them with them to restore their nodes.
We are continuing to investigate the root cause of the failed upgrades.
Posted Aug 14, 2024 - 18:49 UTC
Update
The network has just under the 2/3s majority needed to restart. No root cause for failed upgrades on some validators has been identified. We are continuing to investigate.
Posted Aug 14, 2024 - 17:18 UTC
Investigating
We've spotted that something has gone wrong. We're currently investigating the issue, and will provide an update soon.
Posted Aug 14, 2024 - 15:43 UTC
This incident affected: Testnet (ZetaChain Testnet).