Versaic: Login failure - unexpected server error

Summary

On Monday, June 23, 2025, at approximately 08:03 Mountain Time (MT), a disruption occurred during a routine service update to the Versaic application platform. The issue affected user access to both the Client and Admin portals, resulting in login failures and an error message display. The incident was resolved within approximately one hour, with all services fully restored by 09:05 MT.

Impact

From 08:03 MT to 09:05 MT, users attempting to log in to the Versaic Client or Admin portals were unable to access the system. Instead of reaching the application’s dashboard, users encountered a message indicating: “An unexpected server error occurred.”

During this period:

• No user data was lost, corrupted, or compromised.
• The issue was isolated to the login functionality; the underlying data and platform services remained secure and unaffected.

Root Cause

The interruption was caused by an unexpected failure during a database update as part of a scheduled deployment. Although automated system checks are designed to detect such issues before any changes are released to the live environment, this particular failure mode was not identified in time. As a result, the new application version was released, causing the login issues experienced by users.

Our technical teams were immediately engaged upon alert and began diagnosing the issue. Within approximately 20 minutes, the primary cause was identified, and remediation actions were promptly executed. Full functionality was restored by 09:05 MT.

Future Mitigation

To help prevent similar incidents in the future, we are taking the following steps:

• Enhancing failure detection mechanisms within our deployment processes.
• Introducing additional validation checkpoints before changes are made live.

Timeline of Events

• 08:03 MT - Initial alert received
• 08:20 MT - Major contributing cause identified
• 08:25 MT - Notable action taken
• 09:25 MT - Incident resolved; systems fully operational