Service Outage: 19:10 - 21:30 BST

A power failure at our data centre resulted in a complete loss of service at 19.10. This caused all of our servers to shut down unexpectedly and then restart. We then had to run full diagnostic checks to make sure they were running correctly before bringing the service back online at 21.30 BST.

The service has now been fully restored, with no loss of data or other negative side effects. Latency may be a little bit higher than normal over the next few hours while caching is restored on the live system.

We apologies for this unexpected outage, and will be following up with our network provider to discover the cause, and why secondly power supplies were not called into use.

Thank you for your patience during this outage.

==============

UDPATE: 10:12 BST (GMT +1)

Because the power failure caused our machines to shut down unexpectedly, our disk array which contains uploaded files restarted in 'read only' mode as a precaution. This means we have to run full disk integrity checks before we can go back to 'write' mode. Only 50% of sites have been affected by this, and we will repair them 10% at a time.

In any event, all content on these disks is backed up both on and off site, so in event of any errors in the disk, we can fully restore the files.

We will let you know when this is complete.

==============

UDPATE: 14:00 BST (GMT +1)

The service is now fully restored.

This means that all websites should now:

  • Be fully accessible
  • No longer display a "500 Internal Server Error" message
  • Permit files to be uploaded normally
  • Enable forms to be submitted without error
  • Allow pages to be saved correctly

Now that the service has been fully restored we will begin our investigation into the causes of the outage and how we can ensure that this is not repeated. We will ensure that you are kept informed of our findings as well as the solutions.

If you are still encountering problems related to this outage (see above) you may wish to:

  1. Flush or clear your cache/temporary Internet files within your web browser. This will ensure any error pages saved in your browser will be cleared and your website reloaded from fresh.
  2. Contact support using the query form and give us as much detail as possible about the error e.g. any error messages, when it occurs, what pages it occurs on etc.

Thanks again for your patience, understanding and support with this issue. It is very much appreciated.

==============