RFO 16/06/22

We became aware of a number of issues affecting us and many other providers too. This had an impact on our service and therefore we took emergency action to mitigate the ongoing issues. We engaged our Disaster Recovery system and rerouted in excess of 8500 inbound calls each with a duration greater than 10 seconds. To alleviate the issues we disabled our hosted platform and BLF notifications temporarily, which enabled outbound and internal calls to be made. Once we were confident the issues were resolved we fully restored our services.

An example of a couple of issues from suppliers:- DWS:- Dear partner, Hi To advise many services were affected today by an issue with a peering company lumin/lvl3. This is a peering partner who deals with hand-off traffic to the wider net for many suppliers. Between 13:15 and 13:45 there was a period of no traffic passing over their network. This was resolved by passing peering traffic to a different peer partner. This is not a DWS specific issue and likely affected multiple suppliers within the UK.

June 16: Problems at BT (British Telecom) BT (British Telecom) is having issues since 09:00 AM GMT. Are you also affected? Leave a message in the comments section! We apologise for any inconvenience caused.

All services have now returned to normal and an RFO will be issued tomorrow after further in-depth investigation.

We have now restored service to the portal and BLFs. We are continuing to monitor and investigate the issues that led to today's outage.

We have re-established inbound calls, some services such as the portal and BLF's are still disabled as we continue to monitor the issue. We will continue to provide updates as these services are re-established.

To advise many services were affected today by an issue with a peering company lumin/lvl3.

This is a peering partner who deals with hand-off traffic to the wider net for many suppliers. We cannot confirm this is the reason for our outage, but we are led to believe it has had an effect.

We are continuing to monitor this issue, outbound and internal calls should be fine. Inbound calls are still being routed via disaster recovery to help reduce load. We have also temporarily disabled BLFs.

We are continuing to slowly restore services and keeping the load on our servers low in order to ensure a stable recovery. You may notice some service restoring slowly, with the exception of the portal.

We are currently investigating this incident. You may experience calls intermittently working as we try to bring our service back.

The Web interface/portal has been temporarily disabled to reduce the load on our servers to enable calls to be re-established. Please note we went into full disaster recovery mode 25 minutes ago. We hope to have this issue resolved shortly.