We're satisfied with platform recovery post-upgrade.
Posted Feb 03, 2025 - 14:33 AEST
Update
We are continuing to see superior performance across the board. We will continue to closely monitor the database upgrade. We can confirm total downtime for the database upgrade was about 2 minutes.
Given we brought the database upgrade forward we will be cancelling the planned maintenance tonight. We are sorry for the disruption to your day and will share a post mortem at a later time.
Posted Feb 03, 2025 - 13:46 AEST
Update
We are continuing to see application performance return to normal levels across the board. We will continue to closely monitor the database upgrade.
Posted Feb 03, 2025 - 13:09 AEST
Monitoring
The database upgrade has been successfully performed and we are closely monitoring the application.
Posted Feb 03, 2025 - 12:53 AEST
Update
We have commenced the database upgrade.
Posted Feb 03, 2025 - 12:45 AEST
Update
The incident team has decided to bring forward the planned database upgrade that was scheduled for later this evening, to sustain the increased traffic. We anticipate there will be disruption to the platform for approximately ten minutes during the upgrade. We will commence the upgrade at 12:45PM AEST. We expect platform functionality to return to normal within minutes following the upgrade.
Posted Feb 03, 2025 - 12:38 AEST
Update
We've provisioned additional resources to process the reporting backlog. Reporting delays are being managed between 5-10 minutes.
Posted Feb 03, 2025 - 11:55 AEST
Update
Reports, exports, and various finance processes will continue to experience considerable delays as we monitor platform load.
Posted Feb 03, 2025 - 10:52 AEST
Identified
We identified a database query causing undue stress on the primary database. We're actively managing it's impact. Reports, exports, and notifications have been delayed, but are starting to be processed again.
Posted Feb 03, 2025 - 10:21 AEST
Investigating
We're aware of a high volume of timeouts causing disruption to a variety of platform behaviour. Active investigation is underway.