Paula, care to comment on these comments by Senator Bennett?

Are you aware of all the work that was done to make him believe this to be true? Or is he just trying to calm us down?

NOVAK: Senator Bennett, there are an estimated 50 billion embedded chips in the world, at -- I've read that six percent of them may be defective, but because of the Y2K, but if only one percent are bolluxed up, how much of a problem would that be for a variety of things not working?

BENNETT: Yes, one percent would be an enormous disaster. And when I first got into this the most responsible estimates I had were two to three percent failure. There are some folks that have said as six percent. But I don't really think it was ever that high.

We are now being told by the chip makers, and I've spent a lot of time on this, that -- two things:

Number one the figure is too high. The real figure for failure is going to be something like two-tenths of one percent, instead of one percent or two percent.

And secondly, that there's a very real difference between failure and failure. [Depends on what the definition of "is" is!] In other words the chip can fail, but the device that it's connected to can continue to operate because many times the date function is non- essential. So, we've spent a lot of time worrying about that, checking on it.

I can give you some horror stories because we know the horror stories. But it now appears that this is not going to be as bad a problem as we first thought . . . .

Here's the site with the full text:

http://www.cn n.com/TRANSCRIPTS/9912/11/en.00.html

-- Duke 1983 (Duke1983@AOL.com), December 13, 1999

Answers

Conflicting info to confuse everybody ...

-- do they (know what@they're.doing?), December 13, 1999.

After we worked out how much it would cost to fix the problem, we decided that it wasn't so very important after all.

19 days until we find out. Until then, all is spin.

-- Servant (public_service@yahoo.com), December 13, 1999.

Lets see, I've got ten fingers. One percent would be one finger. In the event I lose that one percent, I won't operate as I did, right? Not to mention the fact that during the time I am adjusting to the loss, I won't be functioning well. In fact for a time I might not be functioning at all!

I seriously wonder if some of these people know the difference between a computer chip and a "chip" in Las Vegas.

-- Richard (Astral-Acres@webtv.net), December 13, 1999.

I suspect it may be a serious error to assume that anyone in government or out of it knows what's going to happen on and after the rollover. Or even the degree of disruption to plan for. Like most of us, most of "them" are human, are not clairvoyant, are not noted for extraordinary wisdom, and are unable to foretell future events.

Anyone can see, and say, what might happen. We've been doing that here for months now.

-- Tom Carey (tomcarey@mindspring.com), December 13, 1999.

Duke,

I noticed that Senator Bennett had changed his perspective on embedded when he gave a talk at an IQPC Workshop in Arlington, Virginia on March 17 of this year. I could not believe what I heard him say and indicated that in the Q & A, challenging him to explain the grounds for his statements. He indicated that he was basing what he was saying on what he was being told by corporations. I spoke with him after the talk and urged him to take another look at the sources of his information as well as his conclusions. I urged him to focus on the highest hazards sectors where the stakes are far higher and the percentage of potential failures far higher as well. He said that he was taking a close look at the chemical sector (he had indicated so such concern in the presentation he had just made).

The following are various thoughts regarding Senator Bennett's apparent perspectives on embedded systems.

Senator Bennett does not appear to have kept up with the potential sea change in understanding that has taken place since November 9, 1999 at the President's Council on embedded issues. But, then even the President's Council does not seem to fully comprehend the implications of the November 9 meeting with embedded systems experts and the subsequent press release by the Secretary of Commerce, the posting of an article on the subject at the NIST website by Gary Fisher of NIST and Michael Cherry of Century Corporation, and the statement issued by John Koskinen that can be found on other threads on this forum. Perhaps, Senator Bennett has not studied the results of the November 9 meeting. Perhaps, no one has brought them to his attention. It may also be that there are opposing views on the subject among his senior Committee staff. It may also be that the Senator has not been fully briefed concerning the opposing points of view either within the Committee staff or outside.

Aside from the possibility of being ill-informed or uninformed, another way of explaining Senator Bennett's change in perspective is simply that he has succumbed to wishful thinking.

Another way of explaining the Senator's change in perspective is that he has never fully comprehended the nature, scope, and seriousness of the embedded problem.

Another way of explaining the change is that he has adopted the mainstream inclination to focus on probabilities rather than stakes. He apparently is minimizing the seriousness of millions of embedded systems malfunctioning, some of which will have immediate or near term consequences and others of which will have with longer term consequences. Those who take the "probabilities" side of the "stakes" vs "probabilities" argument typically minimize the seriousness of malfunctioning systems. They typically minimize the implications of such malfunctioning and failures for public health and safety, the sustainability of the environment, and the integrity of the social fabric. Values come into play here, as well as understanding, knowledge, experience, and common sense.

Denial can be playing a role in his change of perspective. One Chernobyl in the U.S. would have horrific consequences. Indeed, Three Mile Island, had it proved catastrophic in 1979, could have had devastating consequences affecting an area that could well have included the nation's capitol. Several Bhopals or even mini-Bhopals scattered here and there would also be unthinkable. The mere recognition that such catastrophes are possible here as well as in other parts of the world, can have psychologically paralyzing effects on those who recognize the possibilities. This can include persons in roles of public responsibility. It can be easier for a public official to take the path of least resistance and simply try to convince oneself that it cannot happen here. In a word, denial can be the easiest way out.

In my conversation with Senator Bennett in March, when he indicated that he had changed his perspective based on the information that he had been getting from corporate contacts, I told him that the percentages of failure were sector specific. He seemed not to be aware of that. I told him that I would be happy to meet with him and his staff and share information with him. I also said I would send him another copy of my White Paper since he seemed not to have seen the copy I had provided his office in February. I had met before and after with members of his staff and I continue to raise questions of him in public meetings.

Another reason for his change of perspective could well be a small "p" "political" one: those in political roles who feel that they may have gotten too far out on a limb and wish to be closer to the mainstream can rectify that situation by backtracking and distancing themselves from previous statements and perspectives. This certainly seems to be the case with Senator Bennett. In his most recent appearance on Y2K in September at the National Press Club, Senator Bennett even renounced and belittled his own formerly held perspectives. I have reason to believe that his wish to align himself more with the mainstream has played a major role in his about face. Standing up for what one believes to be true seems to be an increasingly rare attribute among those in public life. Standing up for what one believes in is an attribute of leaders who exhibit statesmanship. Senator Bennett himself had exhibited such statesmanship in his landmark speech on Y2K on July 15, 1998 at the National Press Club. Senator Bennett, of course, is not the first to have backtracked in this way. Social, psychological, and political pressures can cause many to change their convictions. When a political figure is also uninformed or ill-informed about a complicated issue, it far more likely that that individual will succumb to such pressures.

It should also be noted that the most recent reports and efforts of the Senate Committee have continued to include a range of damning findings involving embedded problems. This can been seen in the final report of the Committee and in the intermittent concerns of the Committee for the chemical sector (dating particularly from around February of 1999) and since early October 1999 for nuclear power plants. Senator Bennett has alluded publicly to such concerns, even held hearings. He has spoken of instances of failures and expected failures. When speaking in sound bites, he appears to either be forgetting what he knows, ignoring its significance, or otherwise minimizing its importance.

Perhaps, his statements simply reflect the ruminations of an individual who has tired of the subject and is worn down by the daunting nature of all the problems that he and his Committee have uncovered.

It seems very difficult for the Senator to connect the heartfelt concerns that were especially apparent last year with his subsequent intermittent concerns and more recently awakened interest in chemical sector problems and possible nuclear sector problems. He appears not to be able to come up with a coherent perspective on these matters. It is as if he were several different people making statements which were in no way consistent with each other now or over time. It is not always possible when one speaks in soundbites during media appearance to reflect the breadth of concerns that one has. It would be helpful if he were interviewed by someone who was aware of his changing perspectives and the inconsistencies in his statements and in the findings of the Committee. It would be helpful if he were interviewed by someone who could call upon him to reconcile the contradictory nature of what he is on the record for doing and saying.

Perhaps he is not aware of the inconsistencies. Or perhaps he is aware of the inconsistencies and he does not think it important that he try to clarify his thinking or his pronouncements. Perhaps, he has tried and not been able to reconcile the inconsistencies and he is simply putting the best face on the problem, so as not to panic the public.

I hope these thoughts are helpful.

-- Paula Gordon (pgordon@erols.com), December 13, 1999.

Thanks, Paula, for your quick and thorough response, as usual.

I take it from the direction you came at the question, that you don't personally believe there is any basis at all for Bennett's more optimistic veiwpoint?

Also, I have seen alot mentioined about this "sea change" in the white house thinking on embeddeds because of a November meeting. Anywhere we can get more information or discussion what that meeting was all about and some of the results?

Or is the best we have suppostion because of some of the change in public pronouncements?

Thanks again, Paula.

-- Duke 1983 (Duke1983@AOL.com), December 13, 1999.

Duke,

Thanks for your kind words.

You wrote:

"I take it from the direction you came at the question, that you don't personally believe there is any basis at all for Bennett's more optimistic viewpoint?"

Not an iota. In fact, the statement by the Secretary of Commerce, the NIST material, and John Koskinen's subsequent statement in late November would likely raise anyone's concerns who was rightward of a 7 on the impact scale.

You also wrote:

"Also, I have seen alot mentioned about this "sea change" in the white house thinking on embeddeds because of a November meeting. Anywhere we can get more information or discussion what that meeting was all about and some of the results?"

I will attach an excerpt from other writing I have done on that subject at the end of this message, plus a copy of the statement that John Koskinen offered in late November. I would not say that these development have influenced White House thinking. Mr. Koskinen's views have been changed and some members of the Council. I don't see any evidence that the President knows about the statement or, if he does know about it, that he understands its implications.

Attachment 1:

Excerpt from "December Comments and Impact Rating" (See the "Comments, Essays, & Op-Ed Pieces" page at http://www.gwu.edu/~y2k/keypeople/gordon)

..The Presidents Councils Perspective Changes on Embedded Systems:

...On November 9, 1999, the Presidents Council and the Office of Management and Budget convened a meeting involving a small group of embedded systems experts. The result of that meeting was reflected in part in a press release that was issued by the Secretary of the Department of Commerce. On the same date, the National Institute of Standards and Technology issued an article that focused on embedded systems issues. The Secretary of Commerce urged that efforts need to be redoubled to test for year 2000 computer problems that are hidden away in a variety of machines other than computers. See http://www.nist.gov/y2k/embeddedarticle.htm and http://www.nist.gov/public_affairs/releases/g99-204.htm The Chairman of the Presidents Council was questioned about the November 9 meeting at the Press Briefing held on the occasion of the release of the Councils Final Assessment Report at the National Press Club on November 10. A New York Times reporter wrote the following of the exchange that he had with Mr. Koskinen after the formal Press Briefing had concluded.

'Another concern, which Koskinen said he was briefed about on Tuesday at an Office of Management and Budget meeting with computer specialists, is that some computer systems that do not appear to track the date may nonetheless have date-sensitive microchips in them. Those systems also have to be tested and plans must be made to handle breakdowns, Koskinen said.' From: http://www.nytimes.com/library/tech/99/11/biztech/articles/11year.html

According to an embedded systems expert who is acquainted with Mr. Koskinens change in perspective on this issue combined with my own knowledge of what was determined at the November 9 meeting, the quote should more correctly have read (needed changes indicated in all caps):

'Another concern, which Koskinen said he was briefed about on Tuesday at an Office of Management and Budget meeting with EMBEDDED SYSTEMS specialists, is that some EMBEDDED systems that do not appear to track the date may nonetheless have date-sensitive microchips in them. Those systems also have to be tested and plans must be made to handle breakdowns, Koskinen said.'

I would add these major and continuing concerns regarding embedded systems failures. The first is from my Part 2 of my White Paper:

When embedded systems fail, they can fail in a variety of unpredictable ways. Small, seemingly insignificant failures can trigger other system failures." [From Page 40 of Part 2 of my White Paper: "A Call to Action: National and Global Implications of the Year 2000 and Embedded Systems Crisis". See http://www.gwu.edu/~y2k/keypeople/gordon.]

I would also add that the timing of the triggering of other system failures cannot be readily predicted since the environment in which the failures are taking place is dynamically changing. Once the failures have occurred and have triggered other failures, the root causes of the initial failure can be hard if not impossible to determine.

Understanding embedded systems is crucial to understanding the crisis nature of the situation that we are in. The absence of understanding of embedded systems has played a major role in the governments approach to addressing Y2K. In my view, the failure of the Administration to recognize from the outset the importance of consequences of the malfunctioning of embedded systems has resulted in an extremely flawed approach to addressing the problem and a failure understand its complexities, along with a failure to recognize the crisis nature of the problem.

The Presidents Council has failed to give adequate attention to the highest risk, highest hazard systems, plants, sites, pipelines, facilities, etc. The Presidents Council has failed to take the action that it should have taken to help ensure that impacts that can be expected as a result of malfunctioning embedded systems in highest hazard, highest risk sites, plants, facilities, systems, pipelines, refineries, etc., etc. would be minimized to the extent humanly possible.

Even with the late recognition concerning the seriousness of embedded systems problems as of the November 9 meeting, no major initiatives involving embedded systems have been apparent on the part of any agencies or departments of the Federal government apart from the statement of the Secretary of the Department of Commerce. The important implications of the November 9 meeting and the subsequent press release and article at the NIST website, seem not to have been recognized or shared with the President, the Secretary of Agriculture or the Secretary of Energy, based on remarks they have made since the November 9 meeting.

Attachment 2:

John Koskinen's Statement Regarding Embedded Systems and His Perspective Concerning the Results of the November 9th Meeting

[I have added my comments in all caps in the text of the list of "final statements". I have also numbered the "final statements". They are not numbered in the original text.]

PRESIDENT9S COUNCIL ON YEAR 2000 CONVERSIONS MEETING ON Y2K EMBEDDED SYSTEMS Tuesday, November 9, 1999 American Society of Association Executives Building 1575 I Street, Washington, DC

(Statement issued by John Koskinen circa 11/29/99)

Participants in the meeting included technicians that had done work in the bio-medical, defense, electric power, gas, manufacturing, oil, shipping, and telecommunications industries. To help with the discussion, an agenda was provided with discussion statements concerning the types of embedded systems potentially atY2K risk, difficulties in testing for such embedded systems and fixes for problems found. Those statements were revised during the meeting and the agreed upon final statements are presented below, along with a brief summary of the discussion that led to the final statement.

Types of embedded systems found to have a Y2K risk:

[1] Final Statement: Embedded systems are at risk of problems during Y2K rollover if they conduct a calculation that depends on a representation of the date. The date could be in "relative" or "absolute" form.

The participants presented a number of specific cases where they had found Y2K problems in embedded systems. Several of these involve calculations of time increments inside an embedded system without the date being displayed or apparently used. In these instances an embedded system calculates the time interval by subtracting seconds from seconds, minutes from minutes, hours from hours, and calendar dates from calendar dates.

All except one of the examples were large, complex processes where embedded systems inter-relate with each other and, in some cases, with external computer systems. The one example was of a stand-alone embedded system that was unconnected to others that did not apparently involve dates. That example lead to a discussion about the need for a continuous power source being available for any such devices to function, and it was pointed out that in some sectors there are many such devices, but that few problems had been found in them.

There was considerable discussion of potential failure rates of embedded systems. Estimates ranged from a 1 - 2% potential failure rate of processes containing embedded systems in some sectors to 4 - 6% in others, but no conclusion was reached.

COMMENT: IN SOME SECTORS, THE POTENTIAL FAILURE RATE IS FAR HIGHER. SOME OF THOSE SECTORS INCLUDE THE SECTORS THAT POSE THE GREATEST RISK TO LIFE, PUBLIC HEALTH AND SAFETY, AND ENVIRONMENTAL SUSTAINABILITY. PG

......An important distinction was made between failure of an embedded system, which may not cause a process or device to fail in operation, and failure of a process or device due to an embedded system. The former represents the estimates above, and the latter is much less prevalent.

The remainder of the discussion during the meeting focussed on large, complex processes that contain embedded systems. The question of having a real time clock or access to a clock was discussed and examples were presented where the time was set by a process controller and transmitted to other embedded processors involved in the process. Other examples of problems were discussed where time was used apparently to calculate relative increments (e.g. day of the week) as opposed to absolute dates.

When embedded systems will fail:

[2] Final Statement: Where possible, all mission critical systems should be tested end-to-end, whether or not the systems appear to have date sensitive functions. Failure to do so means a small level of risk has been assumed that, at minimum, should be addressed with a contingency plan.

COMMENT: THE USE OF "SMALL" TO CHARACTERIZE THE LEVEL OF RISK MINIMIZES THE SERIOUSNESS OF THE FACT THAT THERE IS RISK. THE RISK MAY INCLUDE SOME HIGHLY SENSITIVE "SAFETY CRITICAL" SYSTEMS IN NUCLEAR POWER PLANTS, CHEMICAL FACILITIES, REFINERIES, HAZARDOUS MATERIALS SITES OR FACILITIES, PIPELINES, WATER SUPPLY SYSTEMS, AND SEWERAGE DISPOSAL PLANTS. PG

The discussion that lead to this statement began with a presumption that embedded systems involved in calculating time increments, as well as those that apparently computed dates, are at Y2K risk. During the discussion the statement to "test mission critical systems whether they have a date function or not" was almost agreed to, until it was pointed out one can only test those types of devices with end-to-end testing.

This statement was focussed on mission critical systems because it is difficult and expensive to conduct such testing. The term mission critical systems was used to include safety critical systems as well as other systems where the cost of failure would be high. Therefore, while the statement says the risk of failure is low, the impact of any such failure would be high.

COMMENTS: THIS GETS US INTO THE STAKES VS PROBABILITIES ARGUMENT. THE STATEMENT REFLECTS A FOCUS ON THE PROBABILITIES SIDE OF THE ARGUMENT. PG

...The statement also recommends a contingency plan to help mitigate risk -- such a plan should not be viewed as an alternative to testing because detection of a failure may be difficult and a failure could cause substantial collateral damage before it is detected.

[3] Final Statement: The majority of failures of embedded systems are expected to occur on or about December 31st through January 1st. However, simply turning a system off during that time frame is generally not a solution.

COMMENT: THIS STATEMENT SEEMS TO ME TO BE HIGHLY QUESTIONABLE AND ALSO MISLEADING. THIS STATEMENT DOES NOT REFLECT AN UNDERSTANDING OF

~ THE FACT THAT THE TIMING OF FAILURES IS FAR MORE COMPLICATED THAN THAT (FOR INSTANCE, SEE MARK FRAUTSCHI'S PAPER ON EMBEDDED SYSTEMS. SOME ASSESSMENTS, INCLUDING GARTNER GROUP ASSESSMENTS, HAVE PROJECTED PROBLEMS OVER TIME, NOT ALL AT ONE TIME.)

~ THE RESULTS OF FAILURES, INCLUDING MULTIPLE CASCADING FAILURES, WILL NOT NECESSARILY BE IMMEDIATELY APPARENT.

~ THERE COULD INDEED BE ROLLING WAVES OF DISRUPTIONS AND DISASTERS THAT GO ON FOR MONTHS OR YEARS OWING TO EMBEDDED SYSTEMS FAILURES THAT OCCUR ON TOP OF IT-RELATED DISRUPTIONS. IT MAY BE IMPOSSIBLE TO TRACE THE ORIGIN OR THE SEQUENCE OF THE FAILURES AND DISRUPTIONS AND ALL THE MORE DIFFICULT TO UNDERSTAND HOW TO PROCEED WITH REPAIRS. PG

The discussion explored the question of whether the time of primary risk of failure was during the rollover time. It was generally agreed that the vast majority of failures in embedded systems are likely to occur over that period. On the specific question of whether Greenwich Mean Time would be a time of high failure, it was stated that most failures would likely occur at 12:00 local time, although some would also occur on Greenwich time.

During the discussion, there was a concern raised that the statement may lead to the ineffective solution of turning off systems during the rollover period. Therefore, the specific admonition not to rely on that work-around was included in the statement.

[4] Final Statement: One can have two apparently identical systems of which one will not have a Y2K problem but the other will have operating difficulties. However, the chances of this are small.

The likelihood of failure of one of two identical systems, as described in this statement was considered to be very small, but, again, it was agreed that all mission critical systems needed to be tested.

Difficulties in testing for embedded systems at risk:

[5] Final Statement: Organizations that have relied on a device manufacturer9s declaration of Y2K compliance are at risk if they do not keep up with the most recent manufacturers9 statements.

COMMENT: VENDOR CERTIFICATION IS OF QUESTIONABLE SIGNIFICANCE IF AN EMBEDDED SYSTEMS IS LINKED TO ANOTHER SYSTEM OR SYSTEMS THAT ARE NOT COMPLIANT. PG

The discussion concerned cases where testing had brought into question manufacturers9 statements of the readiness of their products. A number of instances were cited where problems had been found both externally by users that had tested and by manufacturers themselves. While the changes needed to remedy such problems have normally been made quickly available, the concern was expressed that many organizations were not aware of or taking advantage of those fixes.

[6] Final Statement: Some interconnection problems among embedded systems can only be revealed by end-to-end testing.

The discussion concerned how to test for problems in embedded systems. There was considerable discussion of difficulties of testing in operational environments and the risks and complexities of end-to-end testing. However, a number of examples were cited to show that one could not find all potential problems in complex, interconnected embedded processes without end-to-end testing.

Fixes:

[7] Final Statement: Anyone taking a fix-on-failure approach for Y2K, particularly with embedded systems, runs a significant risk of collateral damage and a difficult recovery.

There was little discussion leading to this statement. Remedying the kinds of Y2K problems participants had found in embedded systems was difficult and time-consuming.

[8] Final Statement: After a full and careful technical assessment, there may be administrative or operational workarounds to many Y2K problems involving embedded systems.

COMMENT: IT WOULD BE USEFUL FOR ANY FUTURE ITERATIONS OF THESE STATEMENTS TO INCLUDE AT LEAST SOME REFERENCES INVOLVING THE HIGH RISK EXAMPLES THAT WERE IDENTIFIED.

While simply turning a system off during the rollover is not normally an effective administrative work-around, in some instance it could be. Similarly, setting the year back so that Y2K does not occur may be a work-around in some instances. However, before using these or any other ways to work-around the Y2K problem, all agreed that a thorough assessment of the full implications of the work-around was necessary.

[9] Final Statement: Even those that have conducted thorough testing need to develop contingency plans for mission critical processes and exercise them.

There was little discussion of this statement, in light of the earlier statements that indicate the risk of Y2K problems.

COMMENT: THE IMPLICATIONS OF FAILURES AND THE NEED FOR NOT JUST DEVELOPING AND IMPLEMENTING CONTINGENCY PLANS, BUT DOING THE SAME FOR RESPONSE AND RECOVERY PLANS AND ACTIONS ARE NOT ADDRESSED AS THEY SHOULD BE IN THESE STATEMENTS. THESE NEED TO BE CARRIED OUT WHILE ALSO CONTINUING TO ADDRESS ASSESSMENT, REMEDIATION, AND TESTING CONCERNS. PG

[END OF John Koskinen's statement and my comments in CAPS regarding his statement]

***************************************************************

The evening panel program that will be held on December 16 at the Washington Post will be addressing many of the issues raised in this thread. The topic of the panel is as follows:

"It's Not Over 'Til It's Over and It Could Go On for Years: Determining Y2K and Embedded Systems Priorities ~ The Need to Continue to Prevent and Minimize Impacts Now and Into the Future."

I will submitting updated information on the program shortly. In the meantime you can find information at http://www.greenspun.com/bboard/q-and-a-fetch-msg.tcl?msg_id=001xgV

There is always a chance that C-SPAN will decide to broadcast the panel live. Anyone can call or send requests to C-SPAN's viewer services asking that they broadcast the program (7 PM to 9 PM EST). Information on how contact C-SPAN is included in the thread just noted.

-- Paula Gordon (pgordon@erols.com), December 13, 1999.

GW Y2K PANEL 12/16/99 "It's Not Over 'Til It's Over and It Could Go On for Years..."

-- Ashton & Leska in Cascadia (allaha@earthlink.net), December 13, 1999.