Arghhh....
Weird things do happen at the most inappropriate times... this is what we called Murphy's Law -__-
Ok, its the 7th month of the lunar calendar, a lot of people said that those cannot be explained must be due to "that" reason -.- And today really witnessed (or almost witnessed) something like that >_<
Friday night we stayed till quite late to perform some testing of our system (it can be roughly divided into Application Server, Web Portal, Client Software and Database).
After troubleshooting a few problems we were able to come out with a stabilized version that we agreed will undergo further testings the next day (Saturday)... and so we packed our things and returned home...
The next day when I reached office, we began migrating the whole system to our QA server (to make sure everything was fine), weird things begin to happen...
First, the Web Portal failed to function (we only migrated the affected modules, the portion where the problem occurred has not been changed by anybody)..
then not long after, the database stopped functioning...
to add to the final spice, the Client Software failed to function too...
*PANIC PANIC PANIC*
After one hour of troubleshooting and backtracking the versions (by rolling back to the previous day's working copy) still didn't solve any problem.. we began to suspect something is not right T__T
Just then, it was 5.00 pm and we've been spending 2 hours on these problems without any progress...
ZJ (my team lead) decided enough is enough, we shall switch to debug mode and troubleshoot one by one...
(another 15 minutes later)
debugging begins...
we found out, for the Web Portal part, somebody changed the code without commenting (with 3000 lines of code in a single class... if you don't comment, we cannot find it :D), and that portion was changed after we migrated to test the stabilized version the previous day...
(*fixed one problem*)
(*then ZJ found more problem on the client software, so he decided to do programming himself, while I troubleshoot the mysterious Client Software problem*)
(another 15 minutes...)
"Argh!!!!!!!!!!!!!!! Stupid error!"
I found out why the Client Software failed, its not a bug in the software application, but instead a database parameter was set wrongly, due to the script generated with error... so devious, and once again it was proven that the mechanism i wrote has no problem, contaminated data is the problem (well, i'm proud of myself :D)
Almost at the same time, Zj found the problem and fixed a serious bug before user finds out...
As for the database error? Well, our QA server's C drive was only left with 2 mb space -__- sql server log cannot tahan anymore, all the transaction log and data log failed to function... zj reinstalled the sql server on another drive to solve this problem (pathetic eh....)
in the end, because of all these stupid mistakes... it became so eerie that how come all the wrong things happened at the same time T__T
damn scary... hahahahahahaha
hope monday's migration will be a success, my heart can't take such news anymore, hahahaha :D
没有评论:
发表评论