Excerpt from "An Application Of The NERC Report" by Paul Milne:

"So where are they in reality? In reality they have twice as much work to be done as they have done and they have zero time at all for testing, a full half the job."

Posted on Ed Yourdon's TimeBomb2000 at: http://www.greenspun.com/bboard/q-and-a-fetch-msg.tcl?msg_id=000OHU.

-- Anonymous, January 15, 1999

Answers

Perspectives are based on an individual's trust factor, as well as other things. If Mr. Milne's basic premise is that everything self-reported in utility 10Qs (concerning where they are in fixing critical systems) is untrue, then there is no proper argument to be made. In that case, we can't assume to know anything about what progress there is - or is not. Personally, I have never observed that any group of people perennially lies their butts off all the time. Some, yes, but this doesn't incline me to tar everyone with the same brush.

If the premise is that the reports are not lies, but that utilities' are defining project compliancy estimates in a misleading way, then there is a counter argument to be made. If I understand Mr. Milne's assessment in the post, what he is saying is this: If a utility has 200 mission critical systems, only a portion of those will actually need to be fixed and the rest will require "little or no remediation". Yet those already compliant systems will be included in the utilities percentage of completion - which in effect hides the actual work they have left to do. As in if 150 of those critical systems needed to be fixed, and 50 didn't, then the utility would report they were 50% finished if they had fixed just 50 out of the 150 which really needed fixing. (50 not needing remediation added to the 50 which were fixed = 100, or half of the total systems.)

This argument presumes that utilities are looking at all their systems as a whole and are issuing percentage reports *including* any systems that have been assessed as not having Y2K problems. I have found utility 10Qs which did note the rate of percentage of systems needing fixing versus those that don't - in their Assessment Phase. In the Remediation Phase of utility projects, however, the percentages given are usually worded such as "conversions" done and "conversions" remaining, or "implementations" completed versus "implementations" done, or "modifications made" and "modifications still to be done". These words do not apply to systems which did not need fixing.

Some utilities estimated the work done versus work remaining in man-hours, and TECO Energy wrote: "Sixty-five percent of the renovations to the transmission and and distribution system have been made, which represents 55 percent of the work required to achieve Year 2000 readiness for this part of the project."

You do not "renovate" that which doesn't need to be, and if you're trying to falsify completion percentages then you don't specify them using words which all indicate "fixes". You also don't tell people that 65% of the fixes equalled only 45% of the work. While I also have a great deal of skepticism concerning corporate reports, I do not believe that utilities included systems in their completion estimates that did not have any Y2K problems to begin with.

I do agree, however, with Mr. Milne's statement, "Usually though, the last part of the job is hardest. Assurances from vendors fall flat, customers cross you off their list, programmers burn out, unanticipated problems crop up, etc etc."

-- Anonymous, January 15, 1999

Bonnie,

If the post is "over there", where might you expect to find the response? ~ Very Big Grin ~

Luv ya big!

-- Anonymous, January 15, 1999

Critt, this was not one of my "Duh!" moments. *smile* I answered Mr. Milne's post here because I thought perhaps some of this forum's readers might like some different thoughts after reading the link. I simply didn't think it was necessary to go head to head with Mr. Milne on the Yourdon forum. Bet you had a good chuckle, though!

-- Anonymous, January 16, 1999

Bonnie,

Clearly, not a "duh" moment.

I understood your intent - the two forums satisfy different needs.

The problem for me is that I hold both forums within one conceptual model. In my head, they are effectively seamless. A little cumbersome, but still seamless.

I'm having the dickens of a time remembering thread content over on TimeBomb. I spend too much time jumping around locating references and responses. I figured it might be safer for my personal well-being to poke at you rather than issue a "Policy Statement" (yeah, that'd make me real popular)

I have to give this some more thought...

~C~

-- Anonymous, January 16, 1999

Thank you Bonnie for your reasoned observations.

Mr. Milne's logic may be appropriate in the conventional sense, but Y2k is not a conventional problem, as he well knows. Had there been the belief from the beginning that all systems could be renovated in the alloted timeframe, then his conjecture on methods of reporting progress, at least to the internal command chain, would be reasonable.

By defining "mission critical" systems, utilities and their IT and non-IT personnel recognized that not all systems could be renovated prior to 1/1/2000. Consequently, they are now in a triage mode, concentrating on systems vital to their continued functionality and existence.

In a triage mode, the criticality of a system is not weighed against the complexity of its renovation. Essential elements are addressed according to their functional importance. In actuality, there may be an inverted curve with relation to Mr. Milne's rationale. We understand that triage is an extension of assessment. Proper assessment ranks systems according to their importance to business continuation. Exceptions to the rule exist, as in waiting for parts for complex systems may delay their renovation, but the tiering of systems stands. Systems will be fixed according to their ability to give the business the best chance of continuing through the date change.

We do not know, but estimates of renovation and testing may be influenced by the emphasis on remediating the most complex and critical systems first. Again, there is no information in the NERC report to support or reject this hypothesis.

The main failing of Mr. Milne's observations is that the parameters he uses to define his argument do not exist with respect to Y2k. When in the history of Information Technology has a project meant the survival of a business beyond a specific date? Certainly there have been instances that without updated projects, companies could lose market share, but nothing that equals the magnitude of Y2k.

It the mandate of the IT and non-IT specialists to do all they can to prepare their utilities for the date change, and not to renovate systems in such a way to promote due diligence through the accounting practices of NERC. Frankly, it is my belief that they don't care what the NERC numbers say, since they are constantly up for interpretation.

Additionally, Mr. Milne's arguments in this presentation speak nothing of embedded systems. Timeframes for dealing with embedded systems projects may not conincide with IT or mainframe correction timeframes. According to an ITAA report released 12/31/1998,

(Embedded systems consultant) "Dave Hall says companies still in the process of fixing their embedded software projects are not necessarily behind the Y2k curve. According to Hall, bug fixing within embedded systems does not require the "massive amount of testing" required with large scale legacy systems."

Again, whether this applies to utility systems and how it influences their timeframes, we do not know. It is clear that the study of Y2k is still an emerging science, and one that will be studied for years, albeit in retrospect. My point is that since the NERC numbers can be interpreted many ways, their usefulness in determing progress of the industry is in question. It is fair and proper for Ms. Camp to use their stastistics to point out a variance to the conclusions presented by NERC. For Mr. Milne to draw further conclusions is just personal speculation, which is certainly welcomed and thought provoking. However, I believe his conclusions lack contextual foundation.

-- Anonymous, January 16, 1999

Take a closer look at the embedded systems problem you just described with respect the comment that companies may not be behind/late on embedded systems. Maybe each system does not require much testing, but what are your options if the widget fails.

1) Get a new widget. Ostensibly this has to be from the same source as the first. Most embedded systems are proprietary and do not share function or interfaces with like systems. If the the company that originally produced the widget has a Y2K compliant one and it is in stock, then the salesman would already be beating on the door trying to sell a new one. If the salesman hasn't been around, chances are that there is no replacement for the widget or the widget is out of stock. However, if you can get a Y2K version of the widget, this is a quick and no-brainer update.

2) Go with a different source for the same widget. In rare cases, the widget may be second-sourced (meaning same function and interface), Y2K compliant, and in-stock.

3) Get a different widget that performs the same function but does not share the same interface as the original widget. In this case, drop-in replacement is not possible. Either one must create a translation layer to translate the new widget interface to the old controlling system or replace the controlling system. The difficultly and expense of these options will vary on a case by case basis. One thing that is known for sure: new systems are a pain to install and configure. They always take more time, dollars, and manpower that initially anticipated.

4) Do without the system. In this case, you will find out if it is really mission critical.

All in all I'd say the embedded system problem has a fair potential for complex and difficult solutions.

-- Anonymous, January 16, 1999

In response and clarification to D. Smith:

Thank you for your comments. I generally concur with your observations. However, I believe that David Hall's statement regarding testing was from the post-remediation perspective. I could be wrong, but that's how I take it at this point. If someone can demonstrate that his comments were directed at pre-renovation testing, then I will glady stand corrected.

The report I received from ITAA was an email, therefore I cannot establish a direct link. You can contact their site and the following link and request it. It's "It's ITAA's Year 2000 Outlook", Vol. 3., Mo. 47.

The link is:

http://www.itaa.org/transact/2koutlokksub.htm.

Good luck.

-- Anonymous, January 16, 1999

Charlie Register writes:

"The main failing of Mr. Milne's observations is that the parameters he uses to define his argument do not exist with respect to Y2K. When in the history of Information Technology has a project meant the survival of a business beyond a specific date? Certainly there have been instances that without updated projects, companies could lose market share, but nothing that equals the magnitude of Y2k."

If this is your primary critique, I disagree profoundly. I'll cite three of many possible reasons:

1. Recent surveys have shown that utility companies do not place Y2K on their list as a critical problem.

2. Cf Yourdon's recent article, "Deja Vu". There is no evidence, sadly, that Y2K is following anything other than a classic IT trajectory.

3. Software Magazine's November issue describes an audit of a remediated system, following its certification internally. The audit concluded that, had the certified system executed in 2000, it would fail. Obviously, we can't extrapolate wildly, but this is a factual situation and the author rightly makes some sound judgments related to other Y2K work.

Evidence could be multipled a thousand-fold (companies and localities that have decided to fix-on-failure, etc).

Can you cite ANY factual evidence to support your claim? Otherwise, your critique of Milne is without substance.

-- Anonymous, January 16, 1999

BigDog:

Your disagreement is welcomed. I will not offer evidence to the contrary because I am not qualified to do so. Computer systems is not my area of expertise.

Mr. Milne's assertions impressed me as being an examination of human nature more than an exercise in Information Technology timeline analysis. That is the element of the "trust factor" that Ms. Camp eluded to in her previous post.

I still contend that the parameter's of the past don't have to apply apply to an event the magnitude of Y2k. If companies are serious about remediation and triage is employed correctly, then we cannot conclude that a company stating 50% readiness of systems still has 75% of the job to complete.

If companies are not taking Y2k seriously, as you contend, and are not addressing critical systems by ranked importance, then the issue is academic.

We could debate the degree of emphasis Y2k is receiving by industry and government across the nation, but you I would find me an unfitting opponent and to be honest, it would be quite fruitless. Ultimately, you will have to take your own personal perceptions and experiences and plan for Y2k-related disruptions in relation to the degree that you trust the information you have garnered. To that I wish you grace and good luck.

As stated before, it was not and is not my intent to question Mr. Milne's observation concerning the conventional approaches to software projects. And I am aware there is a degree of concern that the Y2k issue is not being addressed in the manner that would comfort us all.

Are some, maybe many, companies treating Y2k renovation simularly to previous IT projects? Probably. I have no particular expertise in this area and therefore am not qualified to speculate, or offer evidence to the contrary.

I was hesitent in offering my initial observations, not so much because of the uncertainity of my thoughts, but because my understanding of computer systems is still quite limited.

-- Anonymous, January 16, 1999

Regarding the LAST POST. Please accept my apologies. That was a tremendous example of poor editing. Suffice to say I did not delete the passages beyond "I wish you grace and good luck." Good thing I'm not in code remediation, isn't it?

This is simply redundant and poorly phrased and should have been deleted prior to posting. If you ever needed evidence of the lack of my computer expertise, and lack of proper sleep, then look no further than this.

Again, please accept my apologies.

-- Anonymous, January 16, 1999

Milne certainly has, uh, some feelings from time to time about people. So do I. In fact, on Yourdon's forum with Milne's thread, I'm the guy who raised issues of "lying". Milne didn't

Right or wrong, Milne's effort was to take the data AS IT WAS REPORTED (just as Bonnie did) and to draw certain inferences from data. Granted, he made an assumption about how companies "look at" their total set of systems and report them, but he took the reported data as given.

Both Milne's point and Bonnie's rejoinder and Milne's rejoinder-to-Bonnie's-rejoinder deserve some more thought. I'm still not entirely sure who I agree with. When I do, I'll add my meagre coupla cents.

To your other points, briefly:

"If companies are not taking Y2K seriously, as you contend." This is not what I said, although the survey of utility companies is certaimly shocking. My point was and is simply that there is no evidence for believing that Y2K is going to call forth a new kind of effort than we have seen historically. And I have seen nothing in Rick's analyses to suggest that utility companies are making such an unusual effort.

"Are some, maybe many, companies treating Y2k renovation simularly to previous IT projects? Probably. I have no particular expertise in this area and therefore am not qualified to speculate, or offer evidence to the contrary."

With all due respect, Charlie, you are now writing columns for Westergaard. It is hardly a secret to the columnists of that web site that the answer is not, "probably", but "yes". In other words, neither you nor I need to speculate, assuming you take Lord, Porlier, Eddy and others of your new colleagues work seriously.

You are correct that the preparations one makes as a result of this are a personal judgment.

Please forgive me if you think I am trying to treat you as an "opponent". I'm really not. What we say matters and what you say matters very much since you now have a public platform. Please be careful to speak with the greatest accuracy. And correct me if I have misinterpreted your remarks. I view Y2K as a very serious and dangerous matter and believe that we are doing a great service on these newsgroups when we act and speak as though truth and accuracy matter. They do. Don't reduce what I say to "your opinion, my opinion." BTW, I'm not looking for another apology :-)

-- Anonymous, January 16, 1999

[My comments in brackets. Also posted to Yourdon]

FIRST MILNE POST

Company A is one of those companies that is doing 'better'. They have 1000 systems and 200 are Mission Critical.(I won't even try to skew this by using a bad one) It has fifty percent of its Mission Critical systems on the remediated side of the list and the other half to be addressed. Let's say that they began coding Jan 1 1998. One year out of the remaining two has elapsed.

[Ill take your assumption here but, arguably, most began coding June 1998 or later. Lets ignore how stupid that is, even though it might appear that the coding is going more rapidly than you imply. Coding is the least interesting part of this anyway]

To begin with, there already appears to be a problem. They have half the code finished and half to go. that would leave no time at all for full integrated testing. But, that is just another issue. Let's believe, for the moment that we are in a fantasy world and they don't need to test making it even better for them.

[Lots of unit testing and much integrated subsystem testing can take place in parallel though you are correct about fully integrated system test, which most utility industry dudes say cant be done for utilities anyway without shutting down, no? But I understand you arent dealing with testing here, which will indeed be a laugh riot later this year]

. There is NO company, having arbitrarily drawn a line down the middle, having said it had fifty percent complete, that would not have also included the other 25% in the report of remediated systems. No. They would, at that point, actually be 75% complete. We all know, for a fact, that any remediated or substantially remediated system would AUTOMATICALLY be in that first half and dispensed with immediately. This is unquestionable. If you want to debate that point , then read no further.

The situation is that they do, in fact, have all the substantially remediated or remediated systems in the first half. The reality is that they DO in fact have twice as much work to do.

Lets try to work this thing backwards to find out what their REAL schedule and rate of progress should have reflected. We know that the second half of the project will actually take TWICE the time as the first half because the first half actually included half of 'those' systems already done.

[Your analysis doesnt account for the fact that, even with coding apart from testing, honest coding work ALWAYS slows down as the completion percentage rises. I know from other posts that you are aware of this and realize youve oversimplified. I say this here, though, because even if one quarrels with your core assumption that only 25% of mission-critical systems were done in 98 (hey, make it 35% 45% or the original 50% if you want), they still wont get there in 99 even if we focus on code alone. Code completion is itself non-linear.].

So where are they in reality? In reality they have twice as much work to be done as they have done and they have zero time at all for testing, a full half the job.

[No, lets now assume that they are doing lots of unit and subsystem testing in parallel but that they need *only* 20% of overall project calendar time for system testing. Were still into the end of 1Q 2000 at the end of a very cold winter]

No matter how you slice it, these companies are in big big big trouble.

[WE are in big trouble]

FIRST CAMP REPLY TO MILNE:

[My comments in brackets]

Perspectives are based on an individual's trust factor, as well as other things. If Mr. Milne's basic premise is that everything self-reported in utility 10Qs (concerning where they are in fixing critical systems) is untrue, then there is no proper argument to be made.

[From what I could see, Milne took the self-reported data as accurate. His point is that there is no way to get to compliance in 99 based on the self-reported data]

If I understand Mr. Milne's assessment in the post, what he is saying is this: If a utility has 200 mission critical systems, only a portion of those will actually need to be fixed and the rest will require "little or no remediation". Yet those already compliant systems will be included in the utilities percentage of completion - which in effect hides the actual work they have left to do. As in if 150 of those critical systems needed to be fixed, and 50 didn't, then the utility would report they were 50% finished if they had fixed just 50 out of the 150 which really needed fixing. (50 not needing remediation added to the 50 which were fixed = 100, or half of the total systems.)

[Yes, this is my understanding of his argument as well, but they would report 75% complete: 50 didnt need fixing, 50 were fixed; 50 still to go]

This argument presumes that utilities are looking at all their systems as a whole and are issuing percentage reports *including* any systems that have been assessed as not having Y2K problems. I have found utility 10Qs which did note the rate of percentage of systems needing fixing versus those that don't - in their Assessment Phase. In the Remediation Phase of utility projects, however, the percentages given are usually worded such as "conversions" done and "conversions" remaining, or "implementations" completed versus "implementations" done, or "modifications made" and "modifications still to be done". These words do not apply to systems which did not need fixing.

 You do not "renovate" that which doesn't need to be, and if you're trying to falsify completion percentages then you don't specify them using words which all indicate "fixes". You also don't tell people that 65% of the fixes equalled only 45% of the work. While I also have a great deal of skepticism concerning corporate reports, I do not believe that utilities included systems in their completion estimates that did not have any Y2K problems to begin with.

[Bonnie, is your statement above your "belief"? I know how diligent you are with the numbers. What percentage of reporting companies explicitly distinguished mission-critical systems that needed renovation from those mission-critical systems that were determined to have no such need?]

MILNES REPLY POST TO CAMP: [My comments in brackets]

(Milne)  I made no mention whatsoever about the utilities lying in my post as the basis for my points. Reading comprehension is required. My entire argument was based on the fact that they have reported 'x' amount done and have left out a vital piece of information. And that is, out of how much has been done, how much of that is attributable to actual remediation? This is AMPLY explained in my post and totally ignored by you.

[Milne is correct here. That was his question.] (Camp from earlier post) .. This argument presumes that utilities are looking at all their systems as a whole and are issuing percentage reports *including* any systems that have been assessed as not having Y2K problems.

(Milne) They are, Their reports on their remediation are of their OVERALL work. They are NOT reports based upon ONLY what they are remediating as if to say thay began with 10% compliant systems and then report that they are 40% done instead of 50% done.

[I agree with Milne here, barring Camp reporting statistics from report as per my query to her above. In similar projects in past, we would logically do the following: assess systems; determine which are mission-critical; determine which dont need any work (Im simplifying) and immediately put them in the remediated bucket; begin work on the ones that need fixing and report them as remediated as we completed them, etc. We werent purposely "front-loading" things (the good boy and girl geekettes dont lie by and large; that comes further up the chain), just delighted to get some bang from our assessment as soon as possible. But it sure did have the subsequent effect on schedules that Milne described in his original post.]

(Camp from earlier post) I have found utility 10Qs which did note the rate of percentage of systems needing fixing versus those that don't - in their Assessment Phase.

(Milne) They are in the wild minority.

[Again, Bonnie, can you quantify this for us?]

BIG DOG GOES BOW-WOW IN BRACKETS

[My take on why this entire thread matters: compliancy percentages are the whole ball-game in the media charade about Y2K. "Bozo Inc is 80% compliant and says not to worry, theyre confident." Yawn. "Taking all agencies of the Federal government as a whole, they are 73.454% compliant. No need to prepare. Go to sleep." Yawn. Yawn. Im beginning to get drowsy .

Bonnies work has been awesome. Cowles too. Im relying on it, for goodness sake: they better be right! Milnes post is an extension to that work, not a challenge, rebuttal or flame of it. Or am I out to lunch? It helps us understand even more clearly why the NERC report doesnt hold water. And, more generally, it helps us understand that compliancy percentages are largely a shell game. Thats why Milne dropped testing out of the picture, though testing is where the compliancy story will be for the post Y2K historians.

Some companies (oh, how few, how few) use serious metrics because they understand that accurate numbers are an extraordinary competitive advantage. Capers Jones has built a career on trying to get executive management (not IT mgmt, execs) to understand this. They understand it conceptually but refuse to require it of IT because there is a major front-loaded cost in collecting metrics before the huge payoff down the road shows up. And, yes, lots of IT geeks resist it (lets be honest) because its a heck of a lot easier NOT to be measured ("quality code coming right up, boss  yeah, sure"). Try getting code inspections accepted by an IT organization!

Bonnie must use the numbers she has. So must Milne. So must we all. If I can do it without throwing up, Im going to try to write a short, serious paper on the whole compliancy game. Tens of millions of fellow citizens are failing to prepare because of the compliancy charade.

Help me with those number, Bonnie.]

-- Anonymous, January 16, 1999

Big Dog, I'd say we're all in the Y2K awareness-promoting effort together or we wouldn't be posting on these sites at all. Most of those who access this forum, or any other, do so because they have concerns about the potential problems the Year 2000 may bring. Many are already preparing to the extent that is within their ability. All would probably like to have a crystal ball to see into those IT offices and corporate headquarters. (Especially if we could hook that crystal ball up to a television camera and broadcast the news. *smile*)

We can pry and poke at details, but the fact is none of us is going to *know* exactly how the Year 2000 plays out until it arrives. We can't even *know* if we'll still be here and alive on Jan. 1, 2000. You mentioned that you were "relying" on Rick's and my work. I don't think you should rely on anything but your own judgement. There is no infallible answer or argument about Y2K details; there is no "THIS is the answer which will quantify everything I want to know or tell me exactly how much I should prepare." Are those kinds of answers available for anything in the future?

I can claim no omniscience in knowing the intent, competence, honesty, diligence, defining standards, or lack of them, of the hundreds of people involved in remediation projects in utilities, or any business for that matter. My *guess* is that there are as many differences in handling Y2K projects and reporting as there are people involved.

What I do know is that I have had communication from people who are concerned the direction of this thread is unproductive, as well as communications which indicate it may be regarded as some sort of gleeful duke-it-out entertainment. For my part, I conclude with this:

Whether a person thinks the utility estimations are fairly accurate or whether they believe those estimates are inherently false, I believe the end of the reasoning over how compliancy statistics are reported boils down to "we will have problems" and "we will have problems" for each view.

Isn't that what we're all trying to say?

-- Anonymous, January 17, 1999

I believed the thread was very productive and I *never* have a shred of interest or desire to "duke it out" over something as important as this subject. I was completely sincere and still am in trying to understand Milne's and your post on this thread and in trying to discover from you the requested numbers. But I understand that you want to bring the discussion to a close as per this last post.

Thanks again for your diligent efforts.

-- Anonymous, January 17, 1999

To BigDog:

I appreciate your thoughts. I, too, am not interested in taking on a adversarial role. Misinterpretations? There are a few, but I will set them aside. The continuation of string beyond a certain point strikes me as counterproductive as it will do neither of our viewpoints justice.

Let me add this as a final clarification. Please understand that I, too, view Y2k as an extremely serious issue. It was my concern over how the traditional media, an area that I have expertise in, was mishandling the issue that prompted me to offer 'letters' to Westergaard. They were kind enough to consider my comments worthy of posting, to wit I am honored. I will continue to concentrate my efforts there and hopefully my analysis will be constructive.

I very much appreciate your time and dedication to this issue. Again I wish you grace and good luck.

-- Anonymous, January 17, 1999