The Apology From FastMarket Regarding Last Week's Downtime

First of all, I would like to say "Happy Thanksgiving" everybody. Here is the quote from its admin regarding last week's down time.

I want to post an explanation of last week Tue-Fri most stupid downtime of FAST Market sites. The stupid it was because it should not ever be. We planned the upgrade of our database server, so that new software features could be deployed.

Everything should have run smooth and unnoticed by members. But then, in the midst of the process, several software incompatibilities were found, shame on the team, shame on me for not detecting the possible problems before the installation begin. To push the blood pressure further up, when we tried to rollback changes and use previous version until all problems fixed, we only found that this was already impossible. The attempt to rollback had screwed because of the temporal interdependencies existed between two databases: forum and members.

Otherwise speaking, first we planned to upgrade both databases one by one, but because of the situation with screwed rollback, the upgrade of the forum should be done at the same time as the upgrade of members area. Ohsh.

So, we had to put both servers down totally (because logged-in members could interfere and corrupt forum data and because members area data could have been corrupted already because of the incompatibilies) and continue fixing compatibilities problems under the extreme pressure. Apparently this was not so fast.

After upgrade, we manually rechecked all the transactions occured in the Friday morning, and after commiting everything, had started the sites.

We are deeply distressed by the situation had happened and I will strive for such issues never arised again.

Now the good news.
First, no started deposit/withdrawal transaction nor interest counting was harmed anyhow, all the interests earned during downtime were successfully transferred to member accounts, well with the slight schedule delay, of course.

Second, as we have upgraded both servers, now we can quickly deploy all the new features that are been completed in our softsmiths.

Stand by!

Best regards
Jason Moon,
IT Dept Cheif