Customer sites (CitexNode Deploy) unreachable
(29/10/2026, 10:16) · Resolved after 4 h 3 min
- Kundenseiten
- Builds
- Datenbanken (MariaDB)
- Dauerhafter Speicher
- Web-Knoten 1
- Web-Knoten 2
- Web-Knoten 3
Resolved
All CitexNode Deploy services have been stable since the restore. The incident is resolved.
Monitoring
The database cluster has been restored (all three nodes in sync, no data lost). Websites, databases, storage and builds of CitexNode Deploy are running again. We keep monitoring for a while.
Cause identified
The host is back online, the web nodes are running and websites without a database are reachable again. The MariaDB databases of Deploy projects do not come back on their own after the hard restart; we are restoring the database cluster now. Sites that use a database may show errors until then.
Cause identified
Cause identified: the data centre operator disconnected the web cluster's host, most likely triggered by the traffic of our planned move. The nodes themselves keep running and no data is lost. We have requested the host to be unblocked; the sites will come back on their own afterwards.
Investigating
The host running the web cluster has been unreachable since shortly after 10:15 CET. Websites and apps on CitexNode Deploy and builds are down as a result. All other services (dashboard, API, game servers, bots, mail) are not affected. We are in contact with the data centre.