Affected service teams have reported full recovery, impact is mitigated and work continues internally to ensure database resources remain in a healthy state and the incident does not recur.
Nov 27, 05:29 UTC
Database recovery appears to be a success and we believe we are seeing recovery on all services that were affected by the original outage starting ~4:10 UTC. We are monitoring recovery and working on restoring other related systems (that are not currently affecting production traffic) to a similarly healthy state to prevent a recurrence.
Nov 27, 05:21 UTC
We're expecting to potentially start seeing recovery in about 10 minutes based on database metrics. We will continue monitoring.
Nov 27, 05:16 UTC
Impact began at ~4:10 UTC and team executed remediations promptly.
We have identified the cause of the issue and a database recovery process has been in progress for ~45 minutes. We are working on identifying options for expediting the speed of this process, and also for obtaining an estimate for when impact will be resolved.
Will update with specific assessment of impacted API routes shortly.
Nov 27, 05:06 UTC
We are currently experiencing a backend database issue that is causing downtime and increased error rates for some API endpoints, item adds, and Link flows. We will update momentarily with more details about specific impact and expected recovery time.
Nov 27, 05:01 UTC