Got this y2k update in my email.

( snip )

Remember last newsletter when I wrote about the four million gallon sewage spill in California? We listed articles about the spill in our Bug Bytes section of our site. One of the people I met at the conference was Larry Schellhase who is on the Y2K team for the Department of Public Works for Playa Del Rey, California. He knew all about the spill and took me to task for calling it a year 2000 related problem. Even though it happened during a Y2K test, it actually had nothing to do with Y2K, he explained to me. Then he gave me the complete report on the spill. Technically he is correct in that there was no Y2K issue that caused the problem. This is what happened when the bypass gate closed and dumped the sewage.

For the test, they opened the bypass gate and closed the gate to the sewage plant, then powered everything down and went on generator power. When this happened, the bypass gate closed, even though the command had been sent to keep it open. The reason it closed was that the UPS (Uninteruptible Power Supply) for that computer was down for maintenance. This was not considered to be a problem because if the computer goes off, it will simply reboot when the power comes back on. The reason the gate closed was that the computer had been programmed to close the gate as the first step of the reboot. This was a serious mistake in the program. With sewage, you want the gates to fail open. Programming them to fail shut is like programming prison cell doors to fail open. This command was written by some forgotten programmer following some sort of reasoning process in the early 1980s.

Since this incident happened during a Y2K test and part of the problem was in the software, I am still calling it a Bug Byte and I'm glad it was well publicized because it shows just how important it is to test these systems. I expect companies all over the world will find similar problems that aren't really Y2K related, but are still an accident waiting to happen.

-- kevin (innxxs@yahoo.com), July 10, 1999

Answers

Thanks for the post, even though I'm not a 'doomer' information like this is one of the reasons I hang around. Nice to see some substance.

Bryce

-- Bryce (bryce@seanet.com), July 10, 1999.

And the same kind of weaseling was done with the Peach Bottom nuclear plant Y2K incident a few months ago. "Human error" during Y2K testing, not Y2K itself.

Look, folks, if it walks like a duck, quack likes a duck, looks like a duck ... then it IS a duck!! Pure and simple.

Not every Y2K problem is going to literally be due to some hardware or software component failing to correctly handle the year 2000 rollover. In many instances it will be some "non-Y2K function" that has perhaps never been done before that will suddenly rear its ugly head (as in the 2 instances named) due to Y2K. (Easy example: a failure occurs in a computer program due to an unforseen side effect of Y2K remediation. The handling of dates was flawless, but there was a new -- non-Y2K -- bug introduced during the remediaton process.)

Splitting hairs for the benefit of PR may have a calming effect now. But when oil pipelines explode, nuclear plants shutdown, and sewage bubbles up everywhere, I doubt if too many people are going to care whether it was due to a true blue certified Y2K problem or not.

-- King of Spain (madrid@aol.com), July 10, 1999.

Hey kevin--

--*This command was written by some forgotten programmer following some sort of reasoning process in the early 1980s.*--

This one sentence speaks volumes to the Y2K problem. When I was a programmer in the sixties, there were very few languages i.e.; SPS, Fortran, Cobol, RPG, BAL, PL/1, and probably 1 or so I've forgotten in 30+ years. Heck we even wired boards to perform sorts and accounting functions. Sometimes we wrote routines in Hex just to prove we could. So this was pretty much at the beginning I'd say. There was no great push for documentation and little in the way of notes within the program to explain logic reasoning.

This program you write about was compiled in the 80's, and their was still the lack of documentation, and debugging tools. Today, there are an estimated 600 languages used worldwide, and variations of many of those, and guess what? There is still a huge void in documentation and debugging capacity in the common rule set for flowcharting and management of systems. True there are vastly better tools today than in the beginning, but how many programs contain "zero" bugs.

Part of the problem exists in the end user attempting to "force" the program to perform tasks not originally designed into the package. I know of many a business that accomplished the user specific *wish list* tasks by manipulating the package in strange ways. It worked sometimes and other times the bugs would not appear immediately, say until the end of year calcs. or calling a subr that hadn't been used before. The point is, that many hidden flaws existed that could not have been culled out until failure actually occurred.

When I worked for Tandy in the 80's, the patch books became binders that became libraries of fixes. The scary thing is, while attempting to discern the fix through visual inspection of the code, many of the old conventions and tools from the 60's were still being used! But, by attempting to use a vanilla package or routine in the updated version of an accounting program, it would fail. Many rewrites later, the whole thing would be scrapped.

If I extrapolate these early failures, with the vast networks of embedded chips and processes, it is little wonder that even the most talented genius types can not agree on the potential of Y2K.

-- Michael (mikeymac@uswest.net), July 10, 1999.

King of Spain:

I find your viewpoint a bit problematic.

On the one hand, neither of these incidents (Peach Bottom or Van Nuys) was a y2k problem directly. In other words, y2k could have come and long gone and neither problem would have arisen. On the other hand, both of these problems were time bombs -- incorrect programming that would have caused the cited problems under the correct circumstances ever since they were written. In both cases, the problems were uncovered because y2k testing (rather than y2k bugs per se) introduced conditions unusual enough for the bugs to strike. Actual rollover would *not* have instigated either problem.

I think it's true that y2k will introduce a wide variety of unusual conditions, giving long-hidden bugs an opportunity to strut their stuff. But it really doesn't appear that all that many such bugs are lurking out there. Quite a bit of testing has occurred by now, and relatively few newsworthy screwups have accompanied them. So it's a real concern, but probably not a bit one (unless it's *your* water supply or nuke, of course).

-- Flint (flintc@mindspring.com), July 10, 1999.

Flint, wellcome back guy! Nice to have you around. I mean it.

You know why? You are the ACID test Flint. No one can afford to slice it as thin as you do. Not even yourself I think.

(1) 'Indirect Y2K' consequences ARE y2k for Crisssssake!!! What you call 'indirect consequences' Flint is the life and soul of the Y2K monster!! The very NATURE of Y2K includes spillover effects, domino effects, cascade effects, what-have-you effects. Flint, come on guy anyone can tell that you know this !! Just what are you trying to prove? Where is good old plain common sense Flint? It's like saying "Floods aren't that bad. If kids don't go to school and trade is suspended indefinetly it's not a Hydrogen and Oxygen problem"

(2) Y2K will make your "correct circumstances" pop up everywhere! Can't you see how pervasive Y2K is? We are not talking about computers Flint, we are talking about LIFE itself (Nicholas Negroponte).

(3) "Extensive testing" you say??????????????????????????????????? Bwwaaaaaaaaahhhhhaaaaahhhhhhaaaaahhhhaaaa!!!!!! WHERE???????????

-- George (jvilches@sminter.com.ar), July 10, 1999.

George:

Yes, the testing is taking place. Most are ho-hum. Every once in a great while, something goes very wrong. Then we read about it. Then the weak of mind (sorry, but it's true) hold up the ONE thing that failed, remain blissfully unaware of the zillion tests that went smoothly (and received no publicity), and extrapolate disaster from that SINGLE bad example.

At some point, you need to grasp the sheer scope of what's being done (to the tune of hundreds of billions of dollars), and the small number of bad results (so small we are able to worry *every one* of them to death here on this forum), and get a good picture of the *real* pervasiveness of these bugs.

And you can't do this at all, so long as you ignore every truckload of apples that has no bad ones, and call the entire truckload bad if you find a single bad apple in it. You can read about every fatal car accident (in your area) in the paper. You can't find a single word about all the millions of trips made without accident. If you didn't know from personal experience how rare these were and only read the paper, you'd conclude that driving a car was certain death. And this is exactly what you're doing with y2k, because you lack perspective.

-- Flint (flintc@mindspring.com), July 10, 1999.

Sun-Tzu. "one or two raindrops are a distraction but have little power. A million raindrops are a FLOOD and have indomitable power"

And in the words of the famous philosopher MomCat "If 4 millions gallons of sh*t are headed your way, it matters little if the mistaken programming had to do with ought-ought, or fail-closed vs. fail-open... get thee to higher ground."

-- Linda (lwmb@psln.com), July 10, 1999.

I have perspective allright Flint. I can see the sh*t coming our way. The problem is that you either lack information or common sense (or both). There are 50 thousand mainframes loose out there, 100 million computer systems, and 50 b-b-billion embedded chips.

Jurisdiction: The whole wide world. Impact: YOU (it's globalized Flint, sorry) Bottom line: Prepare and help others prepare as much as YOU. Otherwise it's useless. You are NOT helping any Flint.

-- George (jvilches@sminter.com.ar), July 10, 1999.

George:

I'll keep monitoring what's really going on, and you keep chanting your catechisms, and we'll both be happy. How's that?

-- Flint (flintc@mindspring.com), July 10, 1999.

George,

Playing that broken record again.

You've gone from acusing Flint of not having any common sense to him being responsible for people not preparing for Y2k.

Jeez, let's stick to the topic at hand.

I don't think anyone needs to question if Flint has a handle on the consequences of Y2k.

And, Flint is right. That really wasn't a Y2k error. It could have happened anytime the power went off.

And I do think companies are testing, because if they don't there isn't much chance they will be around after 2000. And Flint's analagy about the car wrecks is appropriate, I think.

Besides, companies are hush about anything with Y2k because they don't want to be sued. What they say can be used against them.

-- JAW (clueless@pollyanna.com), July 10, 1999.

OT

Linda

Another Sun-Tzu student eh? Greatest book writen IMHO. Email me if you wish to chat tao.

-- Brian (imager@home.com), July 10, 1999.

JAW (clueless-pollyanna)

Flint's analogy about the car wrecks is very POOR. Actually, invalid.

It would be valid if applied to computers before Y2K. As if I doubted in IT's efficacy or convenience simply because (before Y2K) they crash every so often. That is the situation JAW, not what Flint is talking about. If Y2K did not exist, I would not doubt for a second how useful computers are, despite the fact that they may crash every now and then. By the same token, I do not doubt that cars are useful and convenient despite the fact that there is a car wreck every now and then. In order to make an analogy with car wrecks we would all need a Y2K equivalent phenomenon applied to cars, which we have not.

If not fixed and adequately tested, with supply chain wide import and export of data, computers and embedded systems will fail because Y2K is pervasive. Cars today are not subject to any equivalent phenomenon and consequently are an invalid analogy JAW.

Still, I like having Flint around. He is the pollyanna acid test. And if this is as good as pollyanna philosophy can do to justify the state of Y2K things to come, it is obvious that Y2K is alive, kicking and growing.

Furthermore JAW, the rest of the world does exist you know and it's just as exposed to Y2K as the US. Actually today's globalized economy means that THEIR problems are OUR problem. Testing you say? Ha,ha. Many are not even aware, including all of Italy, lots of Germany, most of Russia, Brazil, China, etc.

JAW, being a clueless pollyanna is dangerous you know, not only to yourself but also to everyone else. Talking about car analogies, it's like wreckless driving.

Take care and be aware

-- George (jvilches@sminter.com.ar), July 10, 1999.

I saw a news article that also stated that a y2k bug did not cause this spill, that it was an "old" programming error. Also that the spill was not 4 or 5 million gals, but 1.5 million or somthing like that. Still a lot of sewage, but nice to see a few facts...good work kevin. Regards,

-- FactFinder (FactFinder@bzn.com), July 10, 1999.

Flint correctly points out that human errors of judgment, as well as mechanical errors, have occurred in the past, and probably will continue to occur into the foreseeable future. Some of these events have caused considerable damage (TMI, Chelyabinsk); some have resulted in many deaths (Bhopal, Chernobyl).

Y2K, insofar as I understand what I know of it, increases the likelihood of such events, to some as yet unknown degree.

I find no comfort in this.

-- Tom Carey (tomcarey@mindspring.com), July 11, 1999.

Tom, particularly taking into account that there are 50,000 mainframes loose out there, 100 million computer systems, and 50 b-b-b-billion embedded microchips, with remediation time expired, incomplete worldwide testing if any, and with hardly any time left to do anything other than contingency planning and convincing your neighbors to prepare.

Is it that hard to figure out?

-- George (jvilches@sminter.com.ar), July 11, 1999.