On December 1 and 2, 2022 Benevity Grants experienced intermittent periods where some users would have presented with an error page when accessing resources within the site. The logging service that is used by Benevity Grants was experiencing higher than expected volumes of traffic, which resulted in downstream impact to the Benevity Grants application. Resources have been increased on the logging service and the service has been fully isolated from Benevity Grants so that any impacts to the logging service do not cause service interruptions for users on Benevity Grants.
Benevity Grants users were able to login to the site, but may have experienced intermittent error pages being displayed when accessing resources within the site. During this time, Grants applicants also intermittently received error messages that their application forms failed to submit.
During a period of higher than expected traffic, some instances of an internal logging service ran out of resources, and were unable to handle incoming requests. The Grants application detected the issues with the logging service and attempted to automatically restore the service. During the recovery, some instances of the Grants application displayed error messages to users as they were navigating the site.
Resources were increased for the logging service to ensure it could keep up with heavy demand during high traffic periods for the Grants application. Additionally, the logging service was reconfigured and isolated from the Grants application so that errors in the logging service will not impact or interrupt user traffic to the Grants application.