[Resolved] Link between data centres down - Reduced redundancy

Started on 15 March 2021 at 2:13 PM. Resolved after about 2 hours

Affected

Inbound Calls
Outbound Calls
  • Investigating
    15 March 2021 at 2:13 PM

    We are investigating this incident. One link between our data centres has gone down and is flapping. We are still fully operational and running over secondary links. Currently not service affecting.

  • Identified
    15 March 2021 at 2:21 PM

    We are continuing to work on a fix for this incident.

  • Identified
    15 March 2021 at 2:23 PM

    We are continuing to work on a fix for this incident. Now service affecting and calls being diverted to DR.

  • Resolved
    15 March 2021 at 2:33 PM

    We just resolved the issue! We've resumed service. Due to the dropped link between data centres, traffic had to be rerouted, which caused a short service disruption.

  • Identified
    15 March 2021 at 2:46 PM

    We are continuing to work on a fix for this incident.

  • Monitoring
    15 March 2021 at 2:59 PM

    We are currently monitoring our network after we experienced a link issue between data centres at around 2 pm this afternoon.

  • Monitoring
    15 March 2021 at 3:17 PM

    We are monitoring the network. Currently, everything looks stable and calls are connecting. If you see any calls stuck, please report these to support@nta.co.uk

  • Resolved
    15 March 2021 at 3:55 PM

    The platform and network infrastructure are now stable; we are marking todays incident as resolved. The issue was caused as originally reported by a flapping network link between the data centres. We disabled and rerouted the traffic but by the time this was completed, too many calls were affected which caused the service to be disrupted. The moment we detected this issue, our upstream carrier for this link was contacted and we are now waiting for their findings. Our API may be delayed for several hours due to this. We apologise for the disruption and the inconvenience this may have caused you and your customers.