Quickreply Page - Delay in automated drip campaign messages – Incident details

All systems operational

Delay in automated drip campaign messages

Resolved
Partial outage
Started almost 3 years agoLasted about 23 hours

Affected

QuickReply.ai

Partial outage from 7:35 AM to 6:30 AM

Drip Campaigns

Partial outage from 7:35 AM to 6:30 AM

Updates
  • Resolved
    Resolved

    We have fixed it for the current events. The past events will slowly be processed.

  • Investigating
    Investigating

    Our systems delayed processing of events as we were experiencing slower responses from upstream services. This lead to accumulation of events which further slowed down the event processing as the criteria was based on events in to events out ratio. This was built to avoid messaging failures at BSP and Meta's end. But lead to a vicious cycle of slowness. We have taken note of this and now resuming the events which are stuck for long. The events which has a sequence of messages, might see only the last intended message and the previous ones will be discarded. The campaigns which had an immediate message to be sent were not impacted. Only the messages with delays were impacted.