Cosmos SDK v0.50.13 Security Upgrade
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Mar 12, 2025 - 23:00 UTC
Scheduled - To protect against a high criticality security update, ZetaChain has released v28.1.4 (https://github.com/zeta-chain/node/releases/tag/v28.1.4) for Mainnet and v29.0.3 (https://github.com/zeta-chain/node/releases/tag/v29.0.3) for Testnet to upgrade the networks to Cosmos SDK 0.50.13. We request that all node operators upgrade as soon as they can safely do so.

For Mainnet's v28 update, the only change being made is an update from Cosmos SDK v0.50.12 -> v0.50.13. A small TON related change is also included in the v29 update for our Testnet. For more information on the Cosmos SDK vulnerability, please see their security advisory here: [ISA-2025-002: x/group can halt when erroring in EndBlocker](https://github.com/cosmos/cosmos-sdk/security/advisories/GHSA-47ww-ff84-4jrg)

Additional Critical Information:
- Due to the low difficulty required to exploit this vulnerability, please apply the updates as soon as you can safely do so.
- Validator operators, if the chain does halt before we reach 2/3 voting power, please install the updated binary swiftly to restore the network.
- Guidance from the cosmos/cosmos-sdk release page states:
- This patch is not state-breaking, so chains can upgrade in a rolling manner. This does not have to be a coordinated upgrade.
- However, validators should upgrade as soon as possible when the release is made available.
- If the vulnerability is exploited before 2/3 is patched, the chain will halt.
Mar 12, 2025 23:00 - Mar 14, 2025 23:00 UTC
Uptime over the past 90 days. View historical uptime.
Mainnet Under Maintenance
90 days ago
99.93 % uptime
Today
ZetaChain Mainnet ? Under Maintenance
90 days ago
99.93 % uptime
Today
CCTX - Base ? Operational
CCTX - Bitcoin ? Operational
CCTX - BSC ? Operational
CCTX - Ethereum ? Operational
CCTX - Polygon ? Operational
Testnet Under Maintenance
90 days ago
100.0 % uptime
Today
ZetaChain Testnet ? Under Maintenance
90 days ago
100.0 % uptime
Today
CCTX - Base Sepolia ? Operational
CCTX - Bitcoin Testnet3 ? Operational
CCTX - BSC Testnet ? Operational
CCTX - Ethereum Sepolia ? Operational
CCTX - Polygon Mumbai ? Operational
CCTX - Solana Devnet ? Operational
Blockchain Explorer ? Operational
Frontend - Mainnet ? Operational
Indexer - Mainnet Operational
Frontend - Testnet ? Operational
Indexer - Testnet Operational
Testnet Faucet Operational
ZetaChain.com/docs/ Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Major outage
Partial outage
No downtime recorded on this day.
No data exists for this day.
had a major outage.
had a partial outage.

No incidents or maintenance related to this downtime.

Scheduled Maintenance
ZetaChain v29 Mainnet Upgrade Mar 24, 2025 15:15-17:15 UTC
v29 MAINNET Upgrade - ACTION REQUIRED for node operators:

We are upgrading to v29 on the ZetaChain Mainnet. Please make plans to upgrade your binaries at the estimated upgrade time. The governance proposal to accomplish the upgrade will be created on March 18th and the exact block height will be released at that time.

Please note that the upgrade is named v29 and Cosmovisor will expect the binaries under the cosmovisor/upgrades/v29/ folder.

Mainnet Upgrade Time: March 24, 2025, 8:30 PST / 15:30 UTC

Please return to this status page on March 18th for updated information included the upgrade height and which version of the binary to use.
Posted on Mar 11, 2025 - 19:39 UTC
Past Incidents
Mar 15, 2025

No incidents reported today.

Mar 14, 2025
Blocked Outbound Cross-chain Transactions on Zetachain Testnet to Solana Devnet
Resolved - This incident has been resolved.
Mar 14, 21:46 UTC
Identified - Cross-chain transactions are currently not being broadtcast from Zetachain Testnet to Solana Devnet. The team is investigating and will post updates here.
Mar 14, 18:36 UTC
Mar 13, 2025

No incidents reported.

Mar 12, 2025

Unresolved incident: Cosmos SDK v0.50.13 Security Upgrade.

Mar 11, 2025
EVM Chain Outbounds are currently delayed
Resolved - This incident has been resolved.
Mar 11, 22:39 UTC
Identified - The issue has been identified and we are working on getting the fix deployed on enough of our nodes.
Mar 11, 22:08 UTC
Mar 10, 2025
Zetachain v29 Testnet Upgrade
Completed - The Solana Devnet program upgrade has been completed, and the network is functioning as expected.
Mar 10, 21:03 UTC
Update - A CCTX observation delay in Arbitrum Sepolia was detected, and has been resolved. All pending Arbitrum Sepolia transactions will be processed on the network shortly.

Solana Devnet remains paused until a contract update can be completed.
Mar 10, 16:24 UTC
Update - The maintenance for Solana Devnet will be extended.
Mar 10, 15:13 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Mar 10, 14:30 UTC
Scheduled - v29 TESTNET Upgrade - ACTION REQUIRED for node operators:

Make sure to upgrade your binaries at the estimated upgrade time.

Please note that the upgrade is named v29 and Cosmovisor will expect the binaries under cosmovisor/upgrades/v29/ folder.

- Proposal ID: 92
- Upgrade Height: 9362500
- Release: https://github.com/zeta-chain/node/releases/tag/v29.0.1
Mar 10, 13:47 UTC
Zetachain v29 Testnet Upgrade
Completed - The scheduled maintenance has been superceded by the new maintenance here: https://status.zetachain.com/incidents/pk2wtqy5j806
Mar 10, 15:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Mar 10, 12:00 UTC
Scheduled - v29 TESTNET Upgrade - ACTION REQUIRED for node operators:

Make sure to upgrade your binaries at the estimated upgrade time.

Please note that the upgrade is named v29 and Cosmovisor will expect the binaries under cosmovisor/upgrades/v29/ folder.

- Proposal ID: 92
- Upgrade Height: 9362500
- Release: https://github.com/zeta-chain/node/releases/tag/v29.0.1
Mar 8, 17:53 UTC
Mar 9, 2025
Solana outbounds latency
Resolved - This incident has been resolved.
Mar 9, 14:59 UTC
Update - We are continuing to work on a fix for this issue.
Mar 9, 14:59 UTC
Identified - We've identified the cause, and are preparing a fix.
Mar 9, 14:28 UTC
Investigating - Outbound transactions to Solana are currently experiencing delays. We are actively investigating the cause of the issue.
No funds are at risk - all transferred tokens will be received once the issue is resolved.
Mar 9, 11:04 UTC
Mar 8, 2025
Zetachain V27 Mainnet Upgrade
Completed - The scheduled maintenance has been completed.
Mar 8, 17:42 UTC
Update - We will be undergoing scheduled maintenance during this time.
Feb 7, 05:18 UTC
Scheduled - v27 Upgrade: Important Upgrade Information

The v27 upgrade includes a performance enhancing but heavy IAVL store migration which requires special attention from validator operators.

Key Points:
• Validators using cosmovisor must pre-download the correct binary to their upgrades/v27/bin directory
• Validators are recommended to ensure "iavl-disable-fastnode = true" to be set in app.toml to avoid downtime.
• Failure to do this may cause your node to go offline for several hours while it processes the migration

Additional Technical Notes:
• Cosmovisor auto-download will not work for this upgrade due to the hash of a different binary being used for zetaclientd-linux/arm64 in the proposal
• Cosmovisor users should manually place correct binaries in cosmovisor/upgrades/v27/bin directory before the upgrade is reached
• Monitor the upgrade status in your validator logs
• Join the our nodes-general discord channel for real-time support during the upgrade

For additional support or questions, please contact the technical team through our official channels.
Feb 7, 15:00 UTC
Mar 7, 2025

No incidents reported.

Mar 6, 2025

No incidents reported.

Mar 5, 2025

No incidents reported.

Mar 4, 2025
Mainnet Degradation
Resolved - This incident has been resolved.
Mar 4, 23:01 UTC
Monitoring - Blocks are being produced reliably currently, though some validators have not yet resumed signing blocks. Restoring a node data snapshot is the most reliable fix presently if your node is affected.

We now consider ZetaChain Mainnet to have fully resumed service, though we are are continuing to monitor.
Mar 4, 20:37 UTC
Update - All delayed CCTXs have now been processed.
Mar 4, 20:34 UTC
Identified - CrossChain Transaction (CCTX) service has been restored.
Mar 4, 19:59 UTC
Update - The ZetaChain network is producing blocks, though 20-30 validators experienced a crash. If your node has halted, there is initial anecdotal evidence that restoring from a known good blockchain snapshot can restore node functionality.
Mar 4, 19:42 UTC
Investigating - ZetainChain Mainnet is currently not producing blocks and we are actively investigating the issue to restore services as quickly as possible. We understand the inconvenience and appreciate your patience as we work to resolve this.

For all status updates, please subscribe to our status at https://status.zetachain.com/
Mar 4, 19:11 UTC
Mar 3, 2025

No incidents reported.

Mar 2, 2025

No incidents reported.

Mar 1, 2025

No incidents reported.

← Incident History Powered by Atlassian Statuspage