Summary

On April 23, 2024, causes.benevity.org experienced an outage from 14:27 MDT to 16:45 MDT. The outage was caused by an incomplete database operation that slowed site performance, effectively preventing users from accessing the site. The outage was resolved by repairing the database operation.

Impact

Users were unable to access the Benevity Causes Portal (causes.benevity.org) during the period of the outage.

Root Cause

Prior to the outage, an update to the Causes Portal was released. The update included a database operation that did not complete as expected. Pre-release testing of the update did not discover the issue. The incomplete database operation caused a significant decline in the site performance of Causes Portal, at which point users began to experience difficulty using the site.

Future Mitigation

An additional step to the Causes Portal release process has been added to prescribe additional review or testing when a release includes changes affecting high-traffic database operations.

Timeline of Events

• 14:27 MDT - Site performance declines after Causes Portal release
• 15:24 MDT - Database performance decline identified
• 16:01 MDT - Root cause identified (incomplete database operation)
• 16:31 MDT - Issue remediated by repairing the incomplete database operation
• 16:43 MDT - Incident resolved; systems fully operational