The Woes of Testing Uncovered....greenspun.com : LUSENET : TimeBomb 2000 (Y2000) : One Thread
Hello Forum, I am writing today to tell about something interesting that I noticed... I heard, and subsequently, read about the two airport failures; one in New York and one in Chicago. As I read the New York Times' article on it, I came across a scary little factoid. (I'm going from memory here, because I can't access the Time's archive files) -- One of the airport's spokespeople said the the system that failed was installed back in late February (I think they said the 28th) Well, the system didn't malfunction until a couple of days ago! That means that over two months have passed between the time of connection and the actual failure. What does this say about the importance of testing? What does this say about companies and governments that keep pushing back deadlines? I think that it means unexpected things are going to happen in these programs, and its possible that years of testing may be necessary to bring them up to par. In the New York airports that had the failures, the new program switched back to the older (non y2k-compliant) system after an error occured. The whole thing crashed during the attempt at an interface between the old and new. Yet, after the turn of the millenium, there is no switching back to the old system. What if they put the "newest, and final" system online in July and get it going for a while and then have it crash in December? Whoops! Only a couple days to go, we can do it, we can do it! Maybe... I'm holding my breath. Jake Ryder.
-- Jake Ryder (NightRyder@NanoNet.net), May 09, 1999
"What" it says is that no matter how much you test, there is always the potential of missing unknown problems. Face it, you cannot guarantee code reliability regardless of the amount of time you have to "test". As i posted earlier, testing isn't "turning a knob". It can take weeks or even months to set up some tests because of the complex relationship betwen data. Speaking from a pure big iron perspective "test what you can imagine, then let 'er rip in production"! The magic show begins around 4 am...
-- branchregister14 (firstname.lastname@example.org), May 09, 1999.
Right on! In non-trivial cases, testing does not find all of the errors. The rest emerge slowly over the next few years or even decades. That's one key reason why we have legacy systems...they are mostly debugged! We can never assume that there will be a completely debugged system if only testing is used.
-- Mad Monk (email@example.com), May 10, 1999.