DonorPerfect Slowness and Errors Reported
Incident Report for DonorPerfect
Postmortem

Yesterday at approximately 8:44 am EDT, we were alerted by our monitoring and support teams that some clients were experiencing slowness and/or errors in DonorPerfect. We quickly identified that the issue was due to an issue with our Constant Contact integration, and affecting a subset of clients only on certain servers. We began addressing and disabling that integration pending further investigation by 9:00 am, and restored service to the vast majority of clients by 9:21 am. As we continued monitoring the issue, we also saw some issues related to slow gift listings for some clients, which we believed were residual to the initial issue, and which we investigated and addressed by approximately 10:30 am. We continued to monitor throughout the remainder of the morning to ensure no other residual issues remained and declared the issue resolved at approximately 11:32 am.

The root cause of the slowness was ultimately traced to an exception condition within our Constant Contact integration that we had not experienced before, and had not occurred in our testing. We wrote, tested, and applied a patch yesterday to prevent it from re-occurring, and are working to continue to improve the overall performance of the constant contact integration.

We know you count on reliable service from DonorPerfect, and we apologize for any inconvenience this issue might have caused you. Thank you again for being a DonorPerfect customer.

Posted May 27, 2020 - 09:01 EDT

Resolved
We believe we have addressed all performance issues and are marking it resolved for now. We have some additional investigation to do regarding the root cause and what plans we're going to make to ensure it doesn't happen again. We apologize for this inconvenience. Thank you.
Posted May 26, 2020 - 11:35 EDT
Update
We have resolved the reports of the errors we saw earlier this morning, but still seeing undesirable load times on pages like the gift screens. We apologize for this inconvenience and we're working hard to make it right. Thank you.
Posted May 26, 2020 - 10:17 EDT
Investigating
We are currently investigating the cause of the slow response times and errors clients are reporting. It's our top priority to resolve and we'll keep you updated on our progress here. Thank you!
Posted May 26, 2020 - 09:11 EDT
This incident affected: DonorPerfect.