98 Server Issue – Status Updates

Status Update as of 6pm GMT (11.30pm IST / 1pm EST)

More than 90% of the sites have been restored now. We are now working on the final list of eight domains or so, which had corrupted backups and we could not recover. We are now working with the data center to connect the old secondary hard disk as the secondary hard disk now. (When we got a new primary hard disk, we moved the old primary hard disk as the secondary hard disk to start the restores). We hope to get the last good backup from these and restore these sites.

In any case, we expect to complete the remaining handful of sites within the next hour or so.

We will provide the next update in two hours from now and hopefully this nightmare will be behind us.

————————————————

Status Update as of 3pm GMT (8.30pm IST / 10am EST)

More than 50% of the sites have been restored.

Some of the site backups have been corrupted and hence we are manually copying from the original location (and not the backups). In these cases, we might have to manually recreate the email ids and database user ids, though the good thing is that we are making sure the emails and the data are not lost.

Due to the above issue our restoration is taking longer than planned.

In any case, we expect to complete the remaining 50% of the restorations within the next two hours or so.

We will provide the next update in two hours from now.

————————————————

Status Update as of 1pm GMT (6.30pm IST / 8am EST)

We have received control of the server and now we are restoring the accounts from the backups.

As mentioned earlier, we will notify each client by email as soon as their site is restored.

We expect this process to be completed in the next two to four hours.

Will update within the next two hours on this.

—————————————-

Status Update as of 12 noon GMT (5.30pm IST / 7am EST)

The data center has confirmed that the server will be online any minute now. As soon as that happens, you should start seeing your sites back online.

If you have an emergency and your site is not up yet, please contact us at support@auroinfotech.com and we will make sure your site is back up immediately.

Even otherwise, we expect to get all the site back up within the next few hours. Each client will receive an email as soon as their website is up.

We will update you again within the next hour, as soon as we take possession of the server.

———————————————

Update as of  10am GMT (3.30pm IST / 5am EST)

Please note that the server is offline now. The data center is replacing the primary hard disk right now. We will update as soon as the server is back online. Once it is online, We will starting our restore process and you should see your sites come back online.

We will provide the next update in two hours from now.

————————————————-

Status Update at 9am GMT (2.30pm IST / 4am EST)

Please note that the server will go offline at 9.30am GMT (3pm IST / 4.30am EST).

We expect to restore the sites on the new hard drive within four hours from then.

Please standby for our next update.

————————–

Original Post:

Over the past three days, one of our servers (.98 server) has been having continuous hardware issues. In order to resolve this, the data center team has tried various solutions such as replacing the RAM, doing FCSK on the drives, replacing the chasis, etc.

At this point, they are recommending a complete formatting of the hard disks, which will cause the entire set of files to be lost.

In order to avoid losing all the content, we have identified an alternative solution.

We managed to get the server back up temporarily and have managed to get a backup of all the sites on the home directory of the respective sites.

The data center has arranged for a brand new hard disk ready for us to use now as the primary drive. They will load this disk and then load the current primary disk as the secondary so that we can restore all the sites back.

We expect this process to have a downtime of about twelve hours and have come up with a plan to minimize the same.

Please standby for an update on this.

Regards,
Server Support Team