All the background jobs are up and running and the system is back to full processing.
During this outage, which occurred for a period of time between 1:00 PM EST and 1:40 PM EST, subscription processing for sites was temporarily paused, resulting in delays in subscription renewals around that time frame for sites.
Any API calls, and signups/subscription creation attempts for sites during that time frame would have failed, and will need to be retried. As background jobs were paused during resolution of this issue, delays in export jobs and other background jobs are expected.
This issue is now fully resolved.
Posted Nov 21, 2023 - 16:47 EST
Monitoring
A fix has been implemented and background jobs are being ramped back up to full capacity. We are currently monitoring to ensure full resolution implementation is successful.
Posted Nov 21, 2023 - 15:34 EST
Update
Subscription processing, which was temporarily paused while identifying the issue, has resumed. Performance is still degraded and most background jobs system-wide have been temporarily paused. Our developers are still working on fully resolving this matter.
Posted Nov 21, 2023 - 15:06 EST
Identified
We have identified the underlying cause of this issue. The system is back up and operational, but performance is still degraded. Our developers are working to fully resolve this matter.
Posted Nov 21, 2023 - 13:56 EST
Investigating
We are urgently investigating an outage with the Advanced Billing system. We will provide more information as soon as possible.
Posted Nov 21, 2023 - 13:34 EST
This incident affected: Maxio Advanced Billing (Main Application, Billing Portal, API, Chargify.js (Maxio.js), Scheduled Payment Processing, Email Sending, Documentation Site, Integrations to 3rd Party Services, Webhooks, Maxio Hosted Pages, Data Exports).