Last week, Cory H asked for an explanation, from an "optimist", on why they are optimistic about Y2k. He included a list of qualifications.

Without going in depth, my experience meets his qualifications. If someone wants a detailed explanation, fine, but I won't include it here.

Also, for reasons stated on that thread, I won't be sending this post off to Cory for inclusion in his WRP. However, I did think it was a worthwhile effort, so I am posting it here, in a forum that allows an actual "dialogue", as opposed to a "monologue" response.

Based on his requested qualifications, this post addresses large, enterprise systems, of the Fortune 500 - 1000 variety. It does not address a myriad of other Y2k issues. Again, only large enterprise systems will be discussed.

Issue - Addressing Y2k

I do not believe you will find an organization in this area that has not addressed Y2k. Be it through replacing systems or remediation, I have yet to see any evidence that organizations of this size are completely ignoring Y2k.

This point is important, and I believe is the essence of de Jager's turn towards more optimism. Y2k, as a technical IT issue, is not difficult to fix. But it has to be addressed. Most of the earlier Y2k work revolved around "awareness" of the issue. I believe it is fairly safe to say that organizations of this size are aware, and have been addressing Y2k.

Given they are addressing the issue, the argument then falls to two basic areas:

1) Will "enough" get done (fixed and tested).

2) Will failures overwhelm the organization.

Will "enough" get done

There seems to be an underlying assumption that organizations did not begin Y2k efforts until recently (past year). While they may have not begun remediation until recently, it has been my experience that many organizations began addressing Y2k beginning in the 1994 timeframe. However, they addressed it by replacing systems, not remediating them. Since 1994, for example, SAP has installed over 20,000 instances. SAP replaces exactly the type of large, enterprise wide systems Cory is talking about. The boom in SAP has been directly related to Y2k.

And SAP only has about 50% of the market share in client-server enterprise systems. BAAN, Oracle Financials, and others have been installed as well.

These facts tend to be ignored when looking at the scale of the Y2k effort, based on things like lines of COBOL code, etc. Not all systems have to be "fixed", and a large number of systems that would have been fixed were replaced.

That, however, did still leave a large Y2k remediation effort to be performed. And organizations faced a dilemna in scheduling these repair efforts. Once a system is "fixed", an effort still must be undertaken to keep the system "fixed". That is, it would be a large duplication of effort to repair all systems in say 1996, and test for Y2k compliance. 3 years of modifications would have to be continually tested for Y2k, along with normal testing. And in many cases, Y2k tests involve maintaining a separate "time-machine" instance, if you will. In essence, most organizations were not in a position to maintain a separate Y2k testing environment for 3+ years. And would not got to the effort of performing full Y2k testing, then have 3 years of modifications, only to basically undergo the same process again.

The strategy mostly used is to time the completion of repairs, such that Y2k testing can be performed, which can then be followed by some form of "software maintenance" freeze. To be honest, the target dates floating around 6 months ago, re a 12/31/98 date for completion, were always unrealistic in my eyes. I had a hard time believing an organization of any size would be able to "freeze" maintenance for a year. My best guess was 6 months tops. Which seems to be where we're headed.

I believe the 12/31/98 targets did, however, serve a vital purpose. Much of the pessimistic view stemmed from past "software metrics", indicating development projects tend to come in late. Now, I have many reservations about the applicability of these metrics, since they are based on development projects, not maintenance projects. But that aside, taking them at face value, I believe the quotes are to the effect that 15% of projects are late. OK, but setting an early deadline of 12/31/98 allowed projects to be late, and still be done in time for Y2k. Assuming they were staffed and managed towards this deadline, whether they made it or not, a large percentage of the work would still have been done. A large portion of the postulated 15% late will, in fact, complete work prior to Y2k. And even those that don't completely finish, will have the vast majority of work done.

Will things be missed? Probably, but not at the level of applications in total. Sometimes, Cory gives the impression that there are mystery jobs running, that no one knows exists. Hardly. Production schedules define precisely what jobs are run, when they run, etc. The IT people know what runs. Sure, there may be some programs that are run by request, that may be missed. But my guess is any that are missed are run so infrequently, as to not jeopardize the business is any way, if they are basically Fixed on Failure.

What about interfaces? I classify interfaces at two levels; explicit and implicit. Explicit interfaces physically produce a file, to send data from one system to another, either internally or externally. Again, these are very well defined, and would be extremely hard to miss. One misconception I keep seeing is regarding potential problems when "one" system expands dates, and another uses "windowing". But folks, it doesn't work that way. You don't just change an interface file definition, without agreement on the other end. It just isn't done. And nothing prevents a system that expanded dates from accepting an interface file with two-digit years, and "windowing" the dates in.

The other misconception centers around incompatible "windows". While this may be a problem in some rare instances, I cannot think of an example, other than birth dates. For the vast majority of dates passed, things like posting dates, invoice dates, delivery dates, billing dates, etc., it just doesn't matter if one system uses a pivot year of 20 and another uses a pivot year of 30. And won't matter, for another 20 years. I know, this isn't the "correct" way of thinking, but if we're truly facing a crisis of such huge dimensions, then the priority is on immediate problems. Period.

Implicit interfaces are not nearly as well defined, and as such are my chief area of concern. These involve applications that access other applications files, or in some other way directly connect to another application. Job scans can provide most of this information, but I have done projects where these had been missed, mostly when an application reads a backup copy of another's files, as opposed to the actual files. In my opinion, this is the major area of concern.

So, in essence, yes, I believe that the vast majority of IT systems in these enterprises will be addressed. Admittedly, in situations where organizations are focussing only on "mission-critical" systems, you must rely on their call on what truly is critical. But then, they are in the best situation to make that call. I expect all major applications to be addressed, and a vast majority of remediation to be completed.

Which leads to the next question:

Will failures overwhelm the organization

The real issue.

First, what I'll call residual errors, or secondary errors that are introduced through modifications. Definitely a problem. As virtually anyone experienced can tell you, the absolute first place you look when a production error occurs is at what has recently changed. The salvation here is that, these errors are happening now, and have been happening, in a spread out fashion, as systems are reimplemented. These account for the majority of the few errors reported publicly, and the probably thousands of errors that never make it out of the IT department. Somehow, the fact that these errors are not publicly evident is construed into some sinister "cover-up". I mean, that is the whole point of production support and maintenance, to catch and fix errors before they become evident outside the organization. That these are happening now is a "good" thing; they are spread out, and addressed prior to Y2k.

No one denies there will be a large spike of errors on rollover and for the next few days. Potential failures exist throughout the year. But again, these will be spread out, and addressed as normal production failures. The real concern is the spike of errors on rollover. My confidence in organizations ability to handle the spike stems from the following:

1) I've seen many projects go live, with huge spikes in errors on implementation. In some cases, error rates are 40-50% higher than under normal conditions. And, in the case of many SAP implementations, these do indeed cross most if not all functional areas of a business. They are handled because support knows it will happen, and is staffed to handle them. Again, the same can be said for the rollover. Everyone knows when it will happen, and I expect most IT people to be spending many sleepless nights over the rollover. Part of the job. But we deal with it, and we keep the business running.

2) Analogous to a normal failure, the absolute first place to look for errors on rollover will be with date logic. Despite some statements, IT support is very good at triage, and determining the order in which errors must be addressed. The type of error that may cause a long-term down would be in an application/system that had not been addressed. If a situation were to occur, where for example application A reads files from another application that had expanded dates, application A could very well be hosed for an extended period, if it had not been at least initially addressed. This would roughly correlate to a "design" failure, which is why I stressed at the top the importance of "addressing" the Y2k effort across applications. Assuming the application has undergone remediation, fixing individual failures in areas of code that were missed or incorrectly changed can be accomplished fairly quickly.

The spike in IT errors on rollover will not be as high as some think. Gartner Group estimated that 8% of Y2k related errors will occur on rollover. Many errors have occurred already; more throughout this year, and still more throughout next year. I hesitate to play with statistics, but I believe Capers Jones estimates that at top, 11-12% of the code in the most vulnerable systems, financial, contains date-related logic. On rollover, using Gartner Group, only 8% of that is susceptible to failure. Subtracting out code that is actually fixed, or required no fix at all, you maybe can start to see why I don't expect a major collapse.

No doubt, some loss of efficiency will occur. This won't be a walk in the park, especially for those in IT who have to deal with it. But not a collapse. Will it cause a recession? Who knows; certainly not me. But I've apparently lived through at least 3-4 recessions already, with no real scars to show.

Anyway, if you made it through this, take your shots. It was at least an opportunity to put together many thoughts I've had in one spot.

-- Hoffmeister (hoff_meister@my-dejanews.com), April 13, 1999

Answers

Good job Hoff, I'll leave the critique to the mainframers (although I noticed it didn't take long for you to launch into the SAP dialog).

Also, could you post a summary of your work experience as it relates to Cory's specs.

-- a (a@a.a), April 13, 1999.

I'll probably have some pretty intense questions and comments but think this will be a deservedly long-lived thread and want to join a first in thanking you for taking so much time to set out your thoughts in a connected fashion.

-- BigDog (BigDog@duffer.com), April 13, 1999.

'a':

Sorry, can't get away from SAP.

As to experience:

16+ years of IT. It ranges from Series/1 to AS/400 to Mainframes to Unix. The Mainframe is pretty old, 8+ years ago, not counting my first two SAP implementations on R/2.

The SAP experience pretty much handles his large database experience. COBOL used to be my blood; in a past life, worked with Pansophic Systems on a code-generator called TELON, which used to be pretty popular. The guts was a set of Assembler Macros that actually generated the COBOL code. It also generated PL/1, if I remember right, but I never touched that.

I even did a stint in EDP Auditing, which covers his disaster-recovery qualifications.

Anything else?

-- Hoffmeister (hoff_meister@my-dejanews.com), April 13, 1999.

Hoff is in a good organizations. There ARE good organizations out there. I've been in some dealing with this problem over the last six years. There are also medicore and bad organizations out there.

-- noel goyette (ngoyette@csc.com), April 13, 1999.

Hoff,

Granting everything you say, you still overlook at least some key Y2K issues.

One, you are looking at fairly up-to-speed Y2K US businesses. Here, the problem is likely to be what you mentioned in passing, ie, loss of efficiency. As I have said in various presentations, I expect businesses to face one of three negative alternatives (if they aren't ready, that is):

A. Meltdown: out of biz, babe. See ya, wouldn't wanna be ya. B. Shutdown: we're takin' a two-week holiday; out to lunch til Jan 15. C. Slowdown: probably the most common. You'll get your product or service, but it will take an extra two weeks (incidentally, multiply this throughout the entire global economy- get the idea?)

Now, permit me to point you to a thread posted earlier today: banks, at least some banks, are still running behind schedule. Okay? Now, these are *US* banks, first to the plate. No Euro distractions, no distractions from a regional financial crisis (translation: Asia). Recently, the British FSA warned of the possibility of having to shut down 12 "household names" among Brit financial firms. Government regulators don't come out & make such claims without some reason. Further, there's skepticism in the London financial district about the true state of readiness of the Continental banks & financials. And as far as Japanese banks go- well, the Japanese gov't has not reported promising results, to put it mildly (although at the same time, the Japanese do have the advantage of a not beginning fiscal 2000 until 4/1/00).

Further, as the Senate Y2K report pointed out, the real problem in the US is with *medium*sized bidnesses. This has been my (limited, anecdotal) observation as well. Small biz can turn it around fairly quickly; but medium-sized biz's can have a much tougher roe to hoe. This apparently is true worldwide, ie, medium-sized biz's are behind the curve. And any reading of case studies will tell you what will happen to these businesses (including small ones) if they don't get sufficient remediation & testing done in time (ie, bankruptcy is not out of the picture at all).

And I'm not even touching on questions of shipping, ports, airport readiness (yes, in the US, as well as globally), certain possible communications problems, the potentially pervasive impact of the rotten Y2K situation of the health care sector (one-seventh of the US economy), the water situation, etc.

I realize that this goes well beyond the scope of what you wrote. However, my point is that Y2K, as a situation/crisis/whatever, goes far beyond the question of US big business mainframe readiness. And as a general rule (though not completely), most surveys have relatively considered US big biz ahead of the curve (although Forrester, last I checked, thought 15% of the Fortune 500 were going to have a rocky transition).

Just my thoughts. Are we having fun yet?

-- Drew Parkhill/CBN News (y2k@cbn.org), April 13, 1999.

A very well written view and I would encourage you to pass it on to Cory. However, I will take issue with some of your underlying assumptions. First, you postulate that the large corps intentionally timed their repairs so they wouldn't incur ongoing Y2K/maintenance expense. I have detailed knowledge about four of the top Fortune 100 (and I do wish I could post the names but my hands are legally tied). Of these four, the late start was due to combinations of unawareness and political infighting. None of them got the assessment phase finished before July of last year. When the scale of the problem became evident, various managers got fired/downsized/exiled and outside groups were brought in. No "timing" element with these guys. I have anecdotal insight to maybe another six companies in the Fortune 500. Except for one corp that got seriously started in late 96/early 97, basically the same scenario. Further, if the right standards were put in place, ongoing maintenance changes aren't that much of an issue in my opinion. You could always schedule a Timemachine shakedown for late 1998/early 1999 figuring the number of new problems (and there would be some) would be minor. As an aside, I don't personally view Y2K code change issues as either purely maintenance or purely developement. It is somewhere in between. As you have pointed out, some "fixes" were new installations of varying code types. Thats developement in my book. However, aspects of windowing could be considered maintenance of a pre-existing code structure. Most large corps will have both things going on. You are right (and many PC people can't follow this) but date windowing and date expansion are not exclusionary. If you are installing new code, you will probably opt for four digit date handling (you should!). However, interfaces may require some serious negotiation. Anyway, maybe the other Fortune 490 are different, but the ones I actually know about did not time anything. I'll give you credit for a really novel Polly theory. Companies are late in Y2K compliance because they planned it that way! Hmm, doesn't that make their initial Y2K budgets and statements fraudulent?

"No one denies there will be a large spike of errors on rollover and for the next few days."

Exactly!!! A large spike in probably every major software application within the company and its subsidiaries! Thats the key point. The virtually simultaneous failure of multiple subsystems. Finding a given problem is such an environment is very hard (which interface file, what was the job stream exactly, what production files were changed prior to the error, whats the backup point etc). I will quibble about a "few" days as well. Some of the cumulative, but not instant abend failures, may well not show for weeks or longer. You agree as to the spike but not its consequences. I understand that. From what I can see, there just won't be enough mechanics to fix all the wrecks quickly.

This one post is getting too long and supper is waiting so until later..

-- RD. ->H (drherr@erols.com), April 13, 1999.

---I honestly don't think that you "get it". In all the technical fixes and workarounds that you mentioned, the one glaring obvious fact is that the "fixers" had FULL USE of EVERYTHING ELSE in the whole wide world working while they were doing their repairs. None of them(or you) was dealing with massive traffic jams-deadlock-loss of power, no water to the buildings, family at home freaking out because they were home alone unprotected while the power was out and the fires were starting to spread, the toilets backing up all over the house, etc. It is indeed possible for any, even very technically involved, "problem" to be fixed--that's why you guys make the big bux. Because you can do this- sorta. I give you windows 98 for an example of that-something even the most ignorant of puter folks are aware of. The difference is, fifty zillion of these unexpected problems are going to be happening ALL AT ONCE, so that the "normality" of even a really hairy glitch will be compounded by 1000's of % of other things you haven't had to deal with ever before. And no, NO ONE, REPEAT ZERO, not ANYONE HERE, no matter what their IT credentials has ever experienced this. It's the "all at the same time" death of a thousand cuts. You can bandaid all you want, until the capacity for new cuts appearing swamps your ability to keep tearing open packets of bandaids. Specializiation continues to restrict viewing of the big picture, and it's been my experience that the more specialized and credentialed someone is, the more they lose sight of the over-all problems. For every one computer glitch that needs to be fixed in emergency mode, there most likely will be hundreds more. Who's going to be working in a building with zip juice, no water for a sprinkler system, and no central air exchange? How many new big office buildings can you even open the windows on? How many big buildings have the ability to shut off their grey and brown water drain systems if the main recipient of the sewage-the municipal plant-can't handle it? How many guys will actually try to even go to work if there is a systemic failure, leaving their well-unprepared-families-cuz-daddy-said-no-big- problems-will-happen totally at the mercy of the mobs? All those answers fall into the category of no, none, ain't possible, who you kidding, etc. In fact, this won't be the death of a thousand cuts, this will be the death of 50 zillion mosquito bites, ALL HAPPENING AT THE SAME TIME. I'm sure you can swat some mosquites, but I see little evidence that you can swat a zillion simultaneously. If you can, why do we have continuuing software glitches in even the most modern of puters and programs? It's all connected, stand-alone-only exists in the narrow world of specialists--brilliant people, highly specialized-and they have jobs that require that sort of mindset--they couldn't do their job otherwise. You need to be, my humble opinion, a committed generalist by mindset in order to "see" the big enchilada. I'm sure very dedicated, high IQ, experienced and skilled hardware and software and managerial types will be working as hard and dedicatedly as they can, right up to and through the rollover--and than that will be it. They'll be big crashes, mediums, and zillions of smalls--then the full nature of the interconnectedness will sink in. and one by one, then in twos and thress, then whole departments and buildings full will try and make their way home-the golden age of Hitech as God over. Good luck and better skill to all who trust in the system, and fail to prepare. Government is a joke, and short term quarterly profits have doomed everyone to a retreat back to barbarism. I hope all the mercedes drivers and long exotic vacation takers live long enough to look back, see where they messed up, and come to the awareness that some things were more important than ego, the corporate ladder, and having a house with more bathrooms than you really need. We have squandered our future as a society for short term power, ego and material possessions.

-- zog (zog@avana.net), April 13, 1999.

Zog,

Yup.

I can't say that I dissagree with you.

I do, however, have to question your sense of totality. It is the very thing I struggle with in making up my own mind how bad it will be. A lot of the failure scenario makes sense to me. Though I find it difficult to quantify the reaction of society to this failure.

I honestly don't think I want to go there.

Father

-- Thomas G. Hale (hale.t@att.net), April 13, 1999.

Back again. Supper was surprisingly good for beans and rice! Now where were we? First, Zog - Hoff simply doesn't give creedence to a widespread utlity failure. Its just not in his equation at all. For the moment, lets just say the lights and water stay on. Lets concentrate on large enterprise systems and how they react to change.

"First, what I'll call residual errors, or secondary errors that are introduced through modifications. Definitely a problem. As virtually anyone experienced can tell you, the absolute first place you look when a production error occurs is at what has recently changed. The salvation here is that, these errors are happening now, and have been happening, in a spread out fashion, as systems are implemented."

You are correct. The "residual" errors are being caught - some of them. You ignore the errors that lie waiting in the code that is only executed post-2000. You know, the other virgin half of all those IF statements! Those corps which skimp on testing believe that, by putting code back into production now, they in effect use the production environment as a test environment and fix on failure (FOF). And they further believe, like you, that they can catch and fix all problems one at a time. I believe it is a foolish choice. Until you run code through at least a month's production cycle( daily updates and all), you really can't say much about how solid the code is performing. The corps I mentioned earlier all internally say they will be done 4th qtr 99. Very little solid test time - just a few "key" dates and not against the full size databases. Very little intentional error laden data - all sanitized and ready to run clean. Lot of middle managers already pointing the finger at "other" peoples data as a "problem". Maybe I'm paranoid, but it sure sounds like a prescription for failure.

Analogous to a normal failure, the absolute first place to look for errors on rollover will be with date logic. Despite some statements, IT support is very good at triage, and determining the order in which errors must be addressed."

Hoff, there is date logic in virtually every program in many job streams. In fact, there is date logic in the JCL (of whatever system). There is also behind-the-scenes date logic at the systems level.( Cory became famous for fixing the X'000197AF systems date logic problem. There are real problems at that level which, in some companies, won't get addressed properly. The result will be a systems crash and a lot of dead in the water computers.) Anyway, in a usual large scale app, there might be a couple dozen programs run sequentially. Not unusual for program # 3 to produce a file used by program # 18. Might not be obvious where the problem is right away. Further, it depends exactly what screwup occured. The fix could involve multiple programs, changing file definitions/sizes/packing. Granted, a good geek can fix these problems - but only one at a time. There is also the mix of real time vs batch applications. The real time stuff will demand priority, but the batch systems are needed to update various background data on an overnight basis. I would suggest that in some corps, prioritization is going to be difficult. I hope this suggests a vary unique environment because thats the point Cory makes over and over. These corps have NEVER had to deal with this kind of failure environment. It is not analgous to the common random failures seen in every corp.

"The spike in IT errors on rollover will not be as high as some think. Gartner Group estimated that 8% of Y2k related errors will occur on rollover. Many errors have occurred already; more throughout this year, and still more throughout next year. I hesitate to play with statistics, but I believe Capers Jones estimates that at top, 11-12% of the code in the most vulnerable systems, financial, contains date-related logic."

Now you are contradicting your earlier position that there "there will be a large spike of errors on rollover". Or at least thats the inference. I don't know where Caper Jones gets there numbers. My own experience is that a key element of most financial systems is date handling. Its in almost every program. In that sense, I would say its in about 90% of the code. Notice I didn't say that date handling itself is 90%! I suppose if you counted lines of code you might conclude that date handling is 7 - 10% of the total code, but that is very misleading. The function of 90% of the code is controlled or influenced by accurate date handling. As for the 8% error rate on actual rollover, I think that is fantasy. Most IT experts will grant the large "spike" in rollover errors (as you did). It would seem to follow that this is more than 8% of the total.

Lastly, let me reiterate what has been said before by Yourdon. We expect 80% of the corps to do okay. Its the other 20% thats the big worry. Particularly the mid-size company that supplies vital parts to meag corps X, Y and Z. We also have very grim views of the companies in the 2nd/3rd world that supply the US with cheap (fill in the blank). In many cases, these are companies that had previously manufactured in the US, but now manufacture only in Mexico, Brazil, Malaysia etc. When you add the total picture together, it spells huge economic problems for us.

-- RD. ->H (drherr@erols.com), April 13, 1999.

Sorry, guys, limited time tonight. Some responses, more detail tomorrow:

Drew:

Cory asked for some fairly specific criteria, implying to me the area he wanted addressed. A full-fledged Y2k essay would have to approach the size of the Senate Report, and be written by a host of people. That wasn't the intent here.

Zog:

See above. Also, as RD says, in my opinion, widespread power outages of any length are becoming extremely unlikely. These types of Mad-Max scenarios may make interesting speculation, but just aren't likely enough to cause me concern. Anyway, again, that wasn't the issue of the post.

RD:

Companies are not late in Y2k compliance until the rollover occurs, and they're not ready then. Maybe I have been fortunate in the corps I've worked with. My current client is a sub-Fortune 250 corp. They started Y2k efforts in early 1997, and will complete in May. My roots, if you will, are with a sub-Fortune 100 corp. They started their Y2k effort in 1992-1993, with an SAP R/2 installation to replace their Wholesale Billing system. These early efforts grew out of an earlier COBOL conversion, if I remember correctly from OS COBOL to COBOL II. (If I remember, it had something to do with a CICS upgrade. Again, my mainframe is old and hazy). In any case, the inventory of COBOL source was incomplete, and pointed out which systems should be replaced, as opposed to modified. (Besides the fact the apps were 20+ years old). In any case, I've worked directly with 4 Fortune 500 companies, and have friends working on other SAP projects at 3 others currently, including our local power company. From every indication, all of these companies have Y2k well in hand.

I disagree that it as simple as merely "scheduling" a Time Machine shakedown for a 1998/1999 timeframe. This type of setup is time-consuming, and expensive. I doubt many corps would go thru this effort twice.

You're right, the 10% referred to actual code, not counts of programs. But it is still valid, I believe. Date handling errors on rollover are in a subset of 10% of the code. Other areas are month-end closings, etc., that won't be immediately affected, and make up the rest of the 10%. "Large" is a relative term; yes, there will be an increased number, a number that I will probably consider too large, when faced with them. But I have no doubt they will be fixed.

In essence, my main point is my optimism does not stem from an unrealistic view of just how well systems run. Precisely the opposite. It is the experience I've had keeping systems seemingly falling apart up and running, that leads me to believe the same for Y2k.

-- Hoffmeister (hoff_meister@my-dejanews.com), April 13, 1999.

Thanks to Mr. Hoffmeister for starting a very interesting thread, contributing a thoughtful DGI viewpoint, and challenging the GI programmers here to debate the possibilities of Y2K. As a confessed non-programmer, reading these kinds of thoughtful debates seems to add to my understanding of the problem and my preparation requirements. I also appreciate hearing how things are going in the Fortune 500 (or Fortune whatever). I don't need company names and I understand your confidentiality issues. This is just a matter of doing good business.

Thomas Hale, in his response, brought up an interesting question about how people are going to react to Y2K problems. This is the kind of sociological question that doesn't seem to get much practical discussion. My own opinion about how the South Central Riots happened has been that the intense media coverage of a couple of isolated and small events (seemingly involving less than a twenty people in total) made the riots happen. Specifically: the intense (almost hysterical) media coverage of the non-response of the police and of crimes being committed on camera and the hysterical speculations of journalists made the riots happen. As if the journalists had a wish for something big to happen, their dark imaginations and their sympathetic "insight" into racial problems seemed to give life to the fires and looting.

There are other factors too. But this struck me as I watched tv. I kept thinking that the hysterical journalists should just shut up and go back to regular programming. It didn't happen. Things got worse. Y2K riots would be different than the South Central Riots. Y2K riots wouldn't be about getting some things you had on your Christmas list (car radio and speakers, booze and snacks, electronics, video tapes, and groceries), Y2K riots would be about getting things you think you need to keep alive. The South Central Riots was not a black thing, others were just as involved-- if not more so. But Y2K riots would be an everybody thing. And I don't think everyone would be polite either.

Last week, my business partner was returning from Washington, D.C. by metro train when she experienced train delays for the third day in a row. The Farragut West station was crowded with people waiting for the trains. It was so crowded that people on the edge of the platform (next to the tracks) were almost falling into the track. People were feeling "claustrophobic," angry, and frustrated. They pushed each other around and crowded tightly without any regard for the safety of others or themselves. The escalators were stopped and crowded, thus making it impossible to leave the platform. When the trains started coming through (an hour later), people on the platform edge were screaming for those behind them to give them space to step back from the oncoming trains. The Metro officials and police were unable to respond to the situation. This is how educated and professional people react to a minor inconvenience (one hour delay) in Washington, D.C.

There are also other things to consider in trying to predict people's reactions. Hopefully, some of you will find this an interesting topic to pursue further. Perhaps, another thread would be more appropriate.

Sincerely, Stan Faryna

-- Stan Faryna (info@giglobal.com), April 13, 1999.

Dear "Hoff,"

I found your post quite well-written. In fact, in an earlier post I raised a few of the same issues, however, without your expertise or experience. Perhaps Mr. Yourdon will weigh on the oft-debated issue of applying metrics to a remediation project.

Regards,

-- Mr. Decker (kcdecker@worldnet.att.net), April 13, 1999.

Hoff,

Agreed- you weren't doing an overview. My basic point was simply that even if all Big US Co's were really in swell shape, we'd hardly be out of the woods.

And, just for the record, I know of one Big US Co which I have been repeatedly told by sources independent of each other which has some pretty serious stuff on its hands- mainframe stuff, would be my guess.

Note also, again, the banking story today. The banks have repeatedly found that Y2K has been more expensive, more time-consuming, and more difficult to test than they expect.

And, let's not forget: according to one Y2K project manager from UK/NZ, big international banks are putting untested code back into use (because testing was too much of a hassle). Oh, joy.

-- Drew Parkhill/CBN News (y2k@cbn.org), April 13, 1999.

Don't worry, Drew, the code will all be tested. Thoroughly.

-- Flint (flintc@mindspring.com), April 13, 1999.

Flint,

Oh, you're right about that! With the world financial system as a focus group :)

-- Drew Parkhill/CBN News (y2k@cbn.org), April 13, 1999.

Hey Flint! Have you hugged your meme today???

-- Robert Sturgeon (rsturgeon@calwest.net), April 14, 1999.

I hate to sully this thread as a non-mainframer, but just to thank Hoffmeister for raising the level of thinking and analysis for us all.

And welcome, Zog, if Zog ye be, over from other boards. You'll find this an excellent bunch of folks to hang out with.

Zog's post questions the point of doing big corp remediation of business systems if infrastructure goes down or stays third-world spotty. But this is to divert us into arguments between first-order and second-order breakdowns. As long as there is a reasonable (or unreasonable) faith in infrastructure, corps must remediate or be eaten in Y2000 "Engulf and Devour" merger mania.

And utilities must remediate their production and delivery systems or be blamed as the "guys who lost Western Civilization". I would be surprised if major industry groups are not probing government now for emergency nationalization plans in case of utility blackouts, brownouts, parts shortages, or billing and insolvency problems.

Aside from infrastructure-caused shutdowns, most of the y2k effects we hear that remain seem to be economic side-effects of shortages, shutdowns, layoffs, etc. The question becomes "How bad a recession/Depression", but this doesn't seem to be teotwawki as in Mad Max, just as in Herbert Hoover. There is probably a recovery path with more bounce than happened in the 30's, but that would be hard to describe here in brief.

The argument between Hoff and Zog is one of different views of the recovery or breakdown spirals, and they both present us with spiralling viewpoints. For Hoff, the problems are out in the open quickly and in small numbers, the low point is reached -- and known -- soon, public confidence returns, and so the recovery spiral prevails almost from the first. For Zog, the problems feed upon one another, and one industry's first-order breakdowns become another (possibly-compliant) industry's second-order breakdown. The spiral is thus downward, and people pull their efforts back, waiting for the low point to be reached, lest their early efforts be swept away. This is the basic Infomagic feedback loop downward, with many systems more added to the fall.

An interesting side (?) scenario, of course, is the breakdown of SOCIAL systems because of, or in anticipation of, y2k. And with anticipation comes the possibility of breakdowns pre-December 31, and of breakdowns without much computer failure at all.

Banking is the prime canary in the coal mine for this category, and it has been hard for me to think of other examples since this one carries so much of the societal load in finance, supply, and subsistence. But just the fact of cash and FDIC covering only about 4% of bank deposits soon appears to many observers to be an upside- down pyramid waiting to fall one way or another. y2k becomes a side issue in the perpetual vulnerability of fractional reserve banking to loss of public confidence.

Another example might be based upon current public fear of the IRS which props up tax revenues, propping up T-bond prices, which help prop up the stock market. The reversal of all this might be imagined, but it appears the IRS will not be seriously computer- challenged until next year, so they might not contribute to the downward spiral until well after rollover, BUT, they also might be seen as giving a SEPARATE blow to recovery (much as we hesitate to associate the IRS with the recovery of anything) from a "bump in the road" outcome in early 2000.

y2k, rather than a "U"-shaped upward recovery, or a "waterfall" arc downward, might be a sine wave oscillation, with various industries/big corps/agencies each contributing their ups and downs to public welfare/confidence at different points during the year. A general factor of public weariness going into Winter 2001 might be imagined, too, especially if background unemployment creeps upward through the year.

-- jor-el (jor-el@krypton.com), April 14, 1999.

Darn! Sorry about getting long-winded, but you get me thinking about Systems, and... Never having been mainframer, or more than 5 years inside large corps (15 years ago), I find myself trying to peek inside the black box of large systems through the eyes and experience of posters like several in this thread.

So I go on about the Systems I wonder about, provoked by your clear presentation and questioning about the Systems you know and wonder about. Thanks for the clarity and provocation.

-- jor-el (jor-el@krypton.com), April 14, 1999.

Although I'm way out of my league here, what Jor-el said about unemployment and recession brought to mind something Pastor Chris wrote on his BB in response to a question on 'mission critical' :

"Ponder the fact that millions of people are employed to operate non-mission critical systems. For the most part, these systems have not had any attention. Millions of non-mission critical systems will fail. You can figure out the rest; massive unemployment.

If you work in a small to medium to large company and nobody is looking at your computer system or worksite for Y2K related problems, either the company is stupid, or they have determined your job not to be mission critical."

I'm pretty sure that gov't is the largest employer here Canada, and that they are the ones most likely to fix only 'mission critical'. It doesn't make for happy thoughts...

-- Tricia the Canuck (jayles@telusplanet.net), April 14, 1999.

jor-el--yes, sort of as you described, but I think it's worse than that, because I think, based on what I've seen and heard about-official and anecdotal-that there won't be any of the iron triangle up and running much at all. I hear and read official spin--then I talk off the record and in person with programmers and techs who are all preparing big time with the large checks they are getting now. I talk to suppliers, some of their main customers are upper management types in utilities, etc. gov.org keeps re-classifying what's mission critical and what isn't-with an aside there of why the hell do we even need non mission critical systems. I hear from folks anecdotally about themselves or relatives who report serious plans and training going on for draconian martial law measures, including confiscations of various good and means of production. I talk to genny suppliers, who have tales of gov.org buying up a lot of "things"-things needed for a big crash. I've heard from folks directly in various orgs and corps who relay the info that their office y2k assessment consisted of a boss coming into the room and asking if their peecee's are y2k compliant-while they slowly nod their heads and say "uh-I guess so"-whereupon aforementioned boss walks out with another puter checked off for the yes-man syndrome in ladder office politics to be continued.

This whole deal has folks so scared that it's like kids in a graveyard after watching a scary movie--they keep telling themselves back and forth that there are no ghosts--when the "ghost" of y2k is quite real, menacing and looming near. And no one wants to be "wrong"-not in their job, or in their dealings with family, or in their outlook of "what might be"-so they will continue to reassure themselves that no--it just CANNOT be, because I SAY SO. To follow on an earlier thread that has passed rapidly around the net, -I have looked out the window, and yes those are elephant tracks in the yard, those are piles of elephant dung, and by telling myself that there's probably an elephant, I can afford to be wrong--the evidence is such for me. Folks looking at the elephant signs, and convincing themselves because there has never been an elephant here before, that elephants cannot exist, that there just can't be an elephant--well, I guess they'll find out-the "signs" certainly look like an elephant to me. My preps and views will insure my safety--not doing them because of vague assurances by thoroughly untrustworthy large corporations and governments is just foolish. They have everything to lose by telling the truth, and nothing to gain. And individuals who base their reputations on being the end-all of "fixing" have a lot to lose as well, so, they just have to be right-they have no choice. I really don't see where any top level puter pro working and making large wads of dineros in "fixing" would ever just throw up his or her hands and go to the boss and say it's just hopeless--too little time, too interconnected, too many unknowns. They'd be fired on the spot, most likely, and shunned by their peers, and ridiculed and worse. They would have to admit that perhaps, just perhaps, their lifes work might be in vain, and no one can do that, it's just too hard. i can stand outside and look at it though, and see the elephant and it's not as threatening to me, it's just there, and I can deal with it.

I've been put in the position before, various jobs, where I was just flat out over my head. Sure I tried to muddle through, but eventually it just got to be too much. The y2k problem is, my opinion, of such an overwhelmingly large scale, that anyone who puts their trust in the "fixers", and ignores the very real dangers of the "fixers" being wrong-is just playing with fire and their families safety. Being a pro dgi just won't cut the mustard. I can afford to be wrong--absolutely no biggee for me--but dgi's, if wrong, will be putting everything they hold near and dear to the complete mercy of uncontrolled chaos. There is no managed chaos. The "system" works, or it doesn't. You car's engine works, or it doesn't. 99% of a car can be just dandy, and 1%, in any of a number of critical areas, will leave you a pedestrian. Our electric grid is already stretched to the max. Where I live we get advisories and dirty power on hot days. I already live in the land of the blinking 12's from power outtages. My phone line is so rotten, even with the repair guys out several times, that all I can squeek by with is a 19.2 connection. We get the hint of a snowflake and shelves in the stores are stripped bare that evening, of many critical items. A jury verdict 3,000 miles away turned a functioning city into a seething animal mass. No, I am NOT optimistic, not at all. I have never trusted "they", given "they's" past record. "They" is about to fall flat on their collective butts, and be run over by the vehicle of reality. But "they" cannot ever admit this, even unto themselves, because they would then have to admit that "they" are, after all, fallible. No corporation admits that they are wrong--they "settle out of court for an undisclosed sum without admitting guilt"--no government admits it's ever wrong--it just passes legislation absolving itself of blame for events that haven't even come to pass. Or it uses it's power to attack the accusers--with whatever weapons it might have. So far, from the gov and corporate "they's"--it's MY fault-because I'm a "food hoarder" or "I'm gonna cause a run on the banks" or I'm a "gun totin survivalist whacko"--why don't "they" just call me nigger or jew and be done with it? The mass media demonization is daily, and it's from the same "they's" who created, promoted, and lacked the ability to either fix or report truthfully on the problem that's now my fault. Well-"they" and their apologists and appeasers can byte me. It's an elephant, where elephants have never lived before. Ha!

-- zog (zog@avana.net), April 14, 1999.

that was a good post zog.

-- a (a@a.a), April 14, 1999.

I keep following the money, the god of the corporations. Companies hate allocating funds for zero gain projects. According to budgets, many are far behind. Many have underestimated costs. They need more money. They were unaware of how much work was needed. The money says that they will not be ready. At this point, their god cannot save them. They have allowed time to become their enemy.

-- Mike Lang (webflier@erols.com), April 15, 1999.