On 1/1/99 the Chicago Tribune ran an article entitled "Millenium Bug Lurks with Time Running Out". The article appears to be fairly right on, siting many known problems. They interviewed the Com-Ed people who had this to say about the Nuclear Plants:

>In one especially topsy-turvy twist from Y2K, nuclear power plants, >some of the most complex machines around, are turning out to be >relatively free of Y2K Bugs.

>Com-Ed had previously planned to fix y2K problems at eight of its >reactors by next summer and wait to update the remaining two. But >company officials now say Y2K problems are so minor that all will be >fixed by mid-year.

I would like to know what your take is on these statements. Would you consider them accurate? I

-- Anonymous, January 03, 1999

Answers

With over 13 years nuclear power plant operating experience, I consider these statements very accurate.

Only minor, secondary systems such as plant security and radiation monitoring would be effected by date sensitive computer chips. The majority of plant safety and control systems rely on analog and not digital technology.

Eric Holland

-- Anonymous, January 04, 1999

Eric,

I'm sorry, but I don't believe you...

The NRC did an audit of the Seabrook nuclear plant, and this is what they reported:

(see http://www.nrc.gov/NRC/Y2K/Audit/Y2K50443.html for the full report)

There were 12 "items" that contributed to plant safety, 7 software, 5 embedded. 7 of these items (no breakdown of HW/SW) were okay, 4 needed to be fixed, and one needed to be replaced.

There were 13 items that could cause the entire plant to trip, 3 SW, 10 HW. All 13 need to be fixed.

There were five items that could cause a reduction in plant effeciency, 1 SW, 4 HW. All five need to be fixed.

There were 159 items that were related to regulatory requirements, 101 SW, 58 HW. 72 are okay, 72 need to be fixed, 11 need to be replaced, and 4 will be eliminated.

There were 617 business critical items found, 319 SW, 298 HW. 322 are okay, 142 need to be fixed, 111 need to be replaced, and 42 are going to be eliminated.

That doesn't sound to me like "only minor, secondary systems"...

Am I missing something here?

Jon

-- Anonymous, January 04, 1999

Ok,

I see your statistics and I am personally preparing for the worse. The way I see it, it's better not to take chances. However, as a reality check, Missouri, Texas, Illinois, and Springfield (just from this site) all have made reports that they have been testing and they were all surpised to find less "bombs" than they anticipated. That leaves three options,

1.) they are right and they have a handle on it, 2.) they are right and they are completely messing up their testing or 3.) They are lying.

I am an optimist and see no reason for number 3 so we can conclude that either they are actually on top of it or they are idiots. Given that while I have been on a Y2k Project for over a year, I haven't the slightest idea how electric plants work or how they handle dates. Usually when I don't know something, I have to rely on others opinions and facts. They generally comes in an artical, a letter, a book, a webpage or a report. I see a major discrepency here. One side says we have no chance of having power, the other says we are half way through testing and have revealed relatively few errors. Why shouldn't I trust the electric companies ability to find and fix their problems? Why should I believe a bunch of lay people who most of whom probably have not spent one hour on a Y2k Project testing computer systems?

-- Anonymous, January 04, 1999

Folks, there is not as much disparity of information in the above as it might seem, and it is not necessarily a case of certain data being wrong and others right. When we narrow our focus to specific statements, we often miss the "whole". Is it possible that statements from some nuclear plants that they have found less problems than expected can be combined with other less promising info so that both aspects are true? Yes.

Consider this. According to the International Nuclear Safety Center, there are now 105 operable U.S. nuclear reactors (if I counted from the chart correctly). These reactors were built at different times, have had differing technological updates, and also all have individual aspects of their construction. Not all of them buy equipment from the same vendors, nor do each of them have the same engineers or computer professionals on staff. Nor did they all begin their Y2K projects at the same time, or proceed in the same ways.

We can, and should, expect there to be differing levels of complexity and time frames for each Y2K project. All nuclear facilities are not created identical; nor are the people managing the facilities. The same for non-nuclear generating and any other infrastructure entities. I have no problem accepting at face value that the fixes in some places are proceeding quicker than was expected. I also take note of other facts which indicate this in itself does not guarantee a no-problem scenario where the power stays on across all the grids.

If you read the comments submitted to NERC by electric industry personnel in answer to being asked what they consider the greatest obstacles they're facing, (report dated 12/9/98), you will find many areas which are stated as concerns by those actually working on the problem. A lot of these fall into the Human Resources category:

"availability of technical resources for the assessment and testing of embedded systems" "resources and time to perform testing, remediation, and contingency planning" "maintaining sufficient resources to complete Y2K" "our greatest obstacle is having the right resources available to perform Y2K work" "filling open engineering positions" and "internal competition for key personnel to support project activities" "difficulty scheduling outside vendors due to their enormous workload" and "availability of subject matter experts" and "staffing and volume of items to check"

I won't go into the many other comments about the difficulty of testing embedded systems, or the worries about vendors supplying critical patches in time, or all the concerns about the readiness of external partners and interconnection systems - or the serious worries expresed about telecommunication reliability, and being able to upgrade SCADA systems in time, etc. etc. The link to the NERC November Monthly Summary and Database was provided by Rick on the 12/28/1998 "Where's the December Nerc Report?" question, and you can read all these comments from industry insiders yourself.

The point is that it's completely logical for some projects to be ahead of the average curve, and for some to be behind. We can be happy for the good news about some nuclear facilities, and also be cautious knowing that "some" good news from part of any basic infrastructure set may not be enough to prevent severe overall consequences. Keep your eyes on the broad picture and the reports of progress, or lack of progress, will not be so confusing. I could say, "I have wood for my woodstove" and that sounds promising - unless I have 10 sticks of wood instead of the 10 face cords expected to be needed.

-- Anonymous, January 04, 1999

Thanks Bonnie. I had a momentary lack of reason. I work in the Y2k industry and for some strange reason am surround by the biggest group of sceptics. They have the "My PC is ok, so I am ok" view. The individual utility company pages I have been two list two major concerns, Embedded systems and Suppliers. The first is very complex and the second could yet cause the system to fail. Even if the whole plant is compliant, if it can't get it's fuel, forget it. Forgot.

On another Note Bonnie, The Abrams report states something to the effect that the US Government is not aware of any major labor shortage. "But in truth, we have not heard from all the hundreds of companies that we interact that there is a work force shortage." If the NERC report, which I will read next, states that they are having HR problems, wouldn't this be a significant contradictions?

MB

-- Anonymous, January 04, 1999

Bonnie,

I agree completely with your points. The point I was trying to make, and didn't make clear at all in my posting, is that the line "Nuclear plants are mostly analog, and aren't for the most part affected by Y2K" cannot be accepted at face value.

I've seen that quote repeated over and over, and, although it may be true in some cases, we have documented proof that in at least one case it is not.

Jon

-- Anonymous, January 04, 1999

Matthew, I have also noticed a seeming contradiction about the availability of skilled people for Y2K projects. There are articles stating some consulting firms which specialize in Year 2000 remediation are downsizing because they are not getting the contracts they expected. Yet all you have to do is check out any headhunters site, and the need for computer systems professionals is rampant, with many openings not being filled.

My personal explanation for this is based on my husband's and other "geek" friends experiences. The contradiction can be summed up by this sentence: "I can't hold my most skilled people, or find replacements for them AT THE SALARY I AM ABLE TO PAY."

Or you could say, "at the salary I want to pay" in many cases. One thing I have noticed going through the utility 10Qs is that the corporations which state they hired outside specialist firms to handle their remediation (versus using only in-house IT people) tend to be ahead in their fixes. In-house employees generally have Y2K work added onto their regular work load, and are not always experienced in the types of remediation work necessary. Bottom line is it's always a matter of getting the authorization to spend the bucks. Companies tend to wait until the point is reached where they see no option but to pay the extra money, and that point is usually when a project or systems are messed up bigtime and they're scared. Irregardless of Y2K, salaries for skilled computer people and engineers have been rising faster than the corporate bureacracy can process or keep up with.

An example: One subsidiary of a Fortune 500 company lost two very good systems people to a consulting firm who offered them almost twice what they were making. The subsidiary was prohibited from offering them the same because of the corporate pay structure plan in place at that time. As a consequence, one of the positions is still unfilled after a year and a half (with other workers trying to take up the slack) and the second position was filled by a less experienced person, unfamiliar with the business' systems or protocols. *****

Jon, I understand and it's a good point. We're all getting hit with the equivalent of television "sound bites" and have to be reminded that there are no cookie-cutter answers to the Y2K problem. To be fair, it also works both ways - the "planes will fall from the sky" bit, being one example. We're all going to have to use a lot of common sense when we encounter "Y2K bites"!

-- Anonymous, January 04, 1999