We have some issues with our web5 server that we're working through at the moment. Things should be cleared up shortly.
-Tony
UPDATE (06/19) 08:15 am: Websites back up.
UPDATE (06/19) 10:10 am: Some problems with email still - working on it.
UPDATE (06/19) 11:50 am: We encountered an issue with the web5 server earlier today. The issue is being dealt with and your website should be up without any issues. Although the server issue has absolutely nothing to do with your website in particular, we might have to restore a copy of your website once the web server issue has been addressed.
For this reason we would recommend that you hold off making any changes to your website until further notice and that IF you have to make a change to your website that you keep track of those changes so that you can re-do them once the back-up have been restored.
This is NOT a security issue, compromise or hack. It has to do with the file system.
We expect the restore from backup to happen tonight and although there might be some downtime it will only affect your website during our night time hours.
Our plan is to do a database backup right before the work is done tonight and then when the work is complete, we will reload the database for your website so that any information is not lost but, like we said, if you can hold off on changes for a day or so, that might be best.
We will keep you in the loop and get back to you shortly with an update and when you can start working on your website again. We're going to try to get this done as quickly as possible and we're bringing our staff in to work extra hours and we apologize for the inconvenience.
UPDATE (06/19) 12:50 pm: We may have to go to a backup file from a day or two ago when we do the work tonight. We're going to try to do a database backup before that restore and then restore the database after that, so the text changes might stay in place but any files uploaded (like images) may have to be uploaded again or something. We'll see how this all shakes out but we want to prepare people just in case.
UPDATE (06/19) 4:30 pm: Bad news... the server work cannot be scheduled for tonight. Stayed tuned as we get this scheduled. We'll update this thread again soon.
UPDATE (06/19) 7:12 pm: We're going to have the server work done on Saturday, 06/28 starting a 8 pm CDT. We need to use the backup that was taken Wednesday night, which is why we were asking everyone to hold off on updating their websites. We realize that our clients cannot wait over a week to do this, so we're going to handle your website maintenance requests for free until then. This applies to all clients that have hosting on web5. What we'll do is accept your requests when you send them in to tickets@webstix.com, do the work (no charge) and then keep track of the changes so that we can re-do those changes after the server work is complete.
UPDATE (06/20) 1:59 pm: Any changes to email (creating accounts, changing passwords) will also need to be re-done after the work is done on the 28th.
UPDATE (06/23) 12:35 pm: Regarding the server work, it's going to happen this coming Saturday the 28th. It was not last Saturday. I guess that wasn't clear because I have a number of our clients asking about that today. We're sorry if there was any confusion
We know this is a way off. The options we had were to either have this work done this coming Saturday or else have the entire server offline last Friday from 4 am to 1 pm. We felt that second topion was not really the best option.
Go ahead and make changes if you want. We don't want to stop you from doing that. We ARE going to take a database backup before the work begins on Saturday. This means most changes you do will still be there once the server comes back online and once we restore the database. The changes that might get lost is if files are uploaded to the server - like images or PDFs.
If you can stick to content updates and keep track of the files being uploaded to the server, then that should work fine. If you want to send us a list of pages you've changed and the files uploaded, we can then help make sure everything's fine on the other side of this work. We're also offering to do these changes for you to help make sure changes are being kept track of.
Our main website is also on the same server, so we're dealing with the same thing as everyone else. 🙂 We're trying to make this as simple and painless as possible. If there's anything we can do, please let us know.
UPDATE (06/25) 10:04 am: On Saturday, the server will be down starting at 8 pm. It'll be down throughout the night time hours and will hopefully be back up around 6 am on Sunday as the drives are changed out and the backup restored. We'll take a database backup at 5 pm on Saturday, so please do not make any updates past that time. When the server comes back up, please give us some time to restore the databases. This may take another hour or more on Sunday morning. Thank you for your help during this work and we're sorry for any inconvenience.
UPDATE (06/26) 11:22 am: GOOD NEWS! We've found another way to do this work where it looks like we won't have to go back to 6/18 for a backup. We should be able to use the backup that completes on Saturday morning instead.
Even so, we'd still like to know what's being updated so that we can confirm everything transfers fine to the new drives.
Here's the plan:
The point at which the server comes back online Sunday may vary as it depends on the speed of the restore.
This is probably one of the best times of the week to do this kind of work since load on the server and website traffic is about the lowest of the week.
During this time, we'll have our domain name (webstix.com) temporarily redirected to another server with this message showing on a single page. Any updates will be posted there.
UPDATE (06/28) 7:08 pm: The server work has begun - web5 is down tonight and it'll be back up in the morning.
UPDATE (06/29) 7:01 am: There was a problem with one of the backups, so it's going to take longer than we expected. When we get an ETA, we'll let you know.
UPDATE (06/29) 7:47 am: It's looking like 3-4 hours yet.
UPDATE (06/29) 10:35 am: The backup is stil being restored onto the new drives. There's probably about an hour left and then once that's done, we'll work on restoring databases. We'll then bring up websites. Email will be available once the restore is complete. Please hold tight.
UPDATE (06/29) 11:24 am: For some reason, the restore has slowed down a little. Please keep watching for updates.
UPDATE (06/29) 11:50 am: It has really slowed down and now it looks like it might be another 4 hours... sorry. This job has top priority with our host. Go out and enjoy this nice day.
UPDATE (06/29) 2:38 pm: We're getting there. There has been progress. Hopefully not long now.
UPDATE (06/29) 3:05 pm: Files have been restored back but configuration needs to be checked and rebuilt first.
UPDATE (06/29) 4:25 pm: Almost there - the server's back. Some services are up. They have to finish configuring the web server and restore databases.
UPDATE (06/29) 6:25 pm: We appreciate your patience. Websites are up now. We're checking everything else over (including email). There were some unexpected issues - for example, the database had to be reinstalled and some of the backup restore processes failed. We have our team looking over websites. If you notice anything, please let us know.
UPDATE (06/29) 7:05 pm: Email should be working properly now.
