Latest NERC report online (24 hours early!)

greenspun.com : LUSENET : Electric Utilities and Y2K : One Thread

NERC has placed the latest report to DOE online (24 hours ahead of when it was scheduled to be there). You can get to it at:
ftp://ftp.nerc.com/pub/sys/all_updl/docs/y2k/4-30-y2k-report-to-doe.pdf
Of course, as usual, you'll need the Adobe Reader to view it.
And also, as usual, I'll provide some useful/useless (depending on your point of view) commentary after I've had the opportunity to review it.
(Note to "regular" - I deleted your dupe post of this link simply so we could have any comments consolidated in one thread - no offense intended.)

-- Anonymous, April 29, 1999

Answers

None taken. I have 2 questions/comments.
1. At the bottom of p. 32:
Of those APPA respondents who have tested critical components, over 60% indicate results that show Y2k would not cause interruption of electrical service.
Am I correct in interpreting this to mean that nearly 40% would have service interruptions?
2. In the Appendix, over 20 CEMS (Continuous Emissions Monitoring Systems) Components are listed as Exception Items. All justifications are for "Vendor availability". The expected date of deivery (?) ranges from 6/30 to 11/1, with most indicating September or October. Is this delivery schedule doable? What about installation and testing? Is enough time allotted? Are there multiple vendors for these systems?
Sorry, that's more than 2 questions. Anybody have any answers?

-- Anonymous, April 30, 1999

I am just starting to go through the report. As I go through it, I get the sense that they don't really know for themselves how things are. They seem to be relying on reports. I guess I was expecting them to do some investigation.
It is also frustrating to me because I would like to assess the risk, but this report is stressing the elements that seem to pose little to no risk. At certain points I find myself asking, "So why are they even looking for problems if none exist?"
I'll keep reading...

-- Anonymous, April 30, 1999

O.K. I feel better. They just came out and said it. This is from the "Customers should" section:
"Thanks to the efforts of electric utilities to prepare and the anticipated minimal impact on electric operations, the risk of electrical outages caused by Y2k appears for Y2k to be no higher than the risks we already experience."
...
"Power supply decisions should be based on the risk exposure of a customer on a year-round basis, rather than the anticipation of any single event, such as Y2K."
I take this to mean that there will be no Y2k problem worth preparing for. It apears that not only were the doomsayers were wrong, the alarmists and many of the Y2k investigators were as well.
It must not have been nearly as big a problem as it seemed at first.

-- Anonymous, April 30, 1999

It must not have been nearly as big a problem as it seemed at first.
-- Reporter (reporter_atlarge@hotmail.com), April 30, 1999.
you're kidding...right?

-- Anonymous, April 30, 1999

declan's take on the nerc report.
http://www.wired.com/news/news/politics/story/19419.html
2000 Looms for US Nuclear Plants by Declan McCullagh
3:00 a.m. 30.Apr.99.PDT One-third of US nuclear plants will not be finished with Y2K repairs in time to meet the industry's self-imposed summer deadline, says a report due for release on Friday.
Of the 66 operating nuclear facilities, 20 said that they will need extra time to rid their computer systems of Year 2000 glitches, according to the most recent quarterly report from the North American Electric Reliability Council (NERC).
[...]
But the group also says hospitals; telephone companies; and gas, water, and sewage facilities "should review their emergency power supply provisions and procedures." So should large corporations.
[...remainder snipped...]

-- Anonymous, April 30, 1999

We upgraded our CEMS System during our annual unit outage in January. This was primarily to upgrade to the compliant version of SCO Unix. We also took the opportunity to replace the 486 server with current Pentium technology and to add a warm standby server to the system at the same time.
We purchased the PCs and SCO software, set them up on-site for initial testing and then shipped them to our CEMS Vendor for installation of the CEMS package. The installation took about two weeks followed by return shipment to our site. On-site vendor installation took one day. EPA required database verification tests were run remotely by the vendor after our plant came back on-line and was providing realtime emissions data.
This was a fairly painless process. Credit goes to our excellent vendor and our internal PC/LAN Analysts.
Jim

-- Anonymous, April 30, 1999

marianne: I am not really kidding; I am kind of confused. I want to believe that NERC is telling us the truth and give them the benefit of the doubt. I realize that I might have been too alarmed by Y2k. I have learned a great deal in the process though. I have learned about how good it is to prepare. I plan to continue preparing for any and all emergencies in the future. (And encourage others to do so)
Although I would be surprised if NERC is lying, it is still possible that they may be wrong. So, I plan to prepare for any emergency including one caused by NERC being wrong. It's my level of alarm that is changing.
One thing seems certain to me. If they are wrong, many people will probably not be prepared as a result. I assume that they are fully aware of this. I think that they have the authority and the ability to make sure that what they are saying won't cause people to be unprepared. They have chosen to say that there won't be noticeable problems, so I feel obligated to extend them some trust unless it is verified that they are wrong. Maybe I'm missing something. How do you feel about this report?

-- Anonymous, April 30, 1999

Kindly Sir Reporter,
From your reply: "One thing seems certain to me. If they are wrong, many people will probably not be prepared as a result. I assume that they are fully aware of this. I think that they have the authority and the ability to make sure that what they are saying won't cause people to be unprepared. They have chosen to say that there won't be noticeable problems, so I feel obligated to extend them some trust unless it is verified that they are wrong. Maybe I'm missing something. ..."
I recommend you invert your concern: their specific intent is to create a sense of trust, calm, relaxing, do-nothing, feel-good atmosphere - despite the present mix of good news, okay news, bad news, and terrible news. They are very definitely "aware" of what they are doing, they are also presented with a situation (from the administration) that prevents them from
1) independently presenting information to the public without getting it politically cleared through Koskinian's office.
2) independently analyzing or criticizing the sources of the information, or results of the analysis from those sources - they must toe the party's line, which is to prepare the public and the media for a "bump in the road" - nothing else, nothing worse.
3) "they" - the people who write the actual reports, who do the presentations, who control the office - are scared stiff of a public panic. Most specifically, they will do anything, say anything, predict anything, and use any propaganda gambit available - with no moral restraint of any kind.
This opinion cannot be "verified" - since the troubles most ikely occurring from Y2K disruptions have not yet occurred. BUT their opinion(s) and "hopes" (that nothing will happen that would harm anyone) likewise cannot be verified.
Trust. Give me any reason to trust them. Give me any justification of their trust in the past. Give me any reason to believe that "they" have your best interests at heart, and not the political future of the administration. Give me 20 predictions that they have made in the past 6 years to indicate "they" can accurately predict this future.
Were they correct about the budget? Tax revenues? Deficits? The war(s) in Somalia, Bosnia, and 29 other spots around the globe? Were they correct in discussing welfare reform? Health care reform? Tort reform? The "Era of Big Government is over?"
It can only be "hoped" (prayed) that they are right. Now, what indicates that they are correct? Nothing. What indicates that this administration (who controls the committee's political marching orders) has never lied in the past about anything that portends bad news, news of criminal activities, news of political scandal, news of troop unreadiness, news of republican initiatives, news bout school lunch programs, budgets, taxes, or anything else?
Therefore, we can estabish that the administration has no regard for the public, for the good of the country, or anything past the next day's headlines, and tommorrow's poll results. So, who should you trust? The political beings who are telling you (despite their own words in their own reports) that nothing will happen? Or your own research - which indicates that this many utilities (and hundreds of thousands of other businesses) are scheduled to finish very, very late in the year?
If you "trust" these people, what will happen if there are significant interruptions - of anything? It doesn't only affect power - but food distribution, HVAC, food packaging, warehousing, water, oil & natural gas distribution, collection, metering, pumping, monitoring,
Say - five to six days, or a period of 24 days of irregular outages of various services affecting various areas for various periods of time in various degrees of severity? Your bet now: where are the disruptions going to be? What will be affected? For how long? Who will affected in what ways?
My theory is: we don't know, but if you (and I) will be mentally and physically ready for soem disruptions, we can manage through.
Koskinian's and Clinton's opinion is: we don't want a run on the bank before 2000, therefore we want the public not to do anything, or else they will withdrawal all their money from the banks. Therefore, we must ensure the public believes nothing will happen.
The ONLY consistent message ever put out by the administration about Y2K is: "We don't want mass panic, we don't want a run on the bank. This fervent wish/hope/fear is shortly thereafter followed by: "Y2K is hyped up by right-wing, gun-toting, money-grubbing, religious fundamentalists who are proclaiming the "End of the World"."
Nothing else matters. To them. Your family's health and security should matter. To you.

-- Anonymous, April 30, 1999

I have a question for Rick and Bonnie and any other game sleuths:
To jump from 44 percent to 71 percent complete for testing and remediation is more than substantial. Remember the discussion regarding the "exception reports"? As I recall, if a facility could demonstrate it was ready "except for" something to be completed during a maintenance outage, or due to a vendor problem (I believe?) then NERC would put them into the "ready" column, and monitor completion of the exception event. I wonder whether the 71 percent completion rate includes facilities that are yet incomplete due to "vendor availability"? That seem to be the implication of the exception reporting rule, and may have contributed to the jump in numbers.
This came to mind because on page 23 the report states (regarding nuclear facilities): "Remediation is on average 71 percent complete. Twenty facilities have identified specific components for which remediation is scheduled to be completed after June 30, 1999."
20 of 66 is over 30 percent. How can remediation be 71 percent complete while over 30 percent are scheduled to be completed after June 30, 1999?

-- Anonymous, April 30, 1999

Oops.... 44% to 75% overall. The 71% number was nuclear...

-- Anonymous, April 30, 1999

reporter, i believe that the industry 'insiders' that post to this forum are telling us what they believe to be true...but i do not necessasarily believe that they are representative of the industry as a whole.
as for outright lying, i believe that there are *some* running this industry that would 'massage' the truth a tad while hoping for the best.
having said that, let us assume, for the sake of this post, that the electrical industry is totally prepared for y2k. remember marcella's 'connect the dots' post?
we will only address one dot here...communications.
this was excerpted from the epri's yk imbedded sytems information report. it was delivered to the senate special committee on the year 2000 on july 12, 1998.
begin text: the tlecommunications industry will be a primary objective of our effort to achieve inter-industry cooperation. while many utilities have their own internal electronic communications systems, all utilities depend to some degree upon the telecommunications service providers for inter-utility communications within individual reliability regions. initial efforts at contingency planning for y2k induced events have made us very aware of the *critical* dependence of electric power industry response plans upon electronic communications. we are sure that reliable electric power is a critical feature of the contingency plans of the telecommunications industry. to begin the dialog, we are currently designing a pilot program for utilities and their telecom providers to sit down and discuss their mutual dependence and identify areas of common concern and possibly joint action. end of text.
the following was taken from the recent international administration report put out by the department of commerce.
begin text: communication channels the global communications infrastructure includes the services to transmit voice, data, and video information by wire, radio broadcasting[including cellular], and satellite; electronic mass media services[e.g., radio and television broadcasting and cable tv]; and online computer information services, including the internet. it also encompasses traditional mail and delivery services an the printing and publishing industries, which are less susceptible to critical y2k failures. the individual entities that make up this infrastucture vary greatly. they include a wide range of disparate and competing entities, including public and private companies, government regulatory bodies, international telecommunications alliances, and foreign and domestic carriers. reliable communication services are essential to the smooth operation of the global trading system. these services are heavily used by the companies that must maintain close contact with their overseas networks of subsidiaries, customers, and suppliers. this infrastructure's **VULNERABILITY** has been demonstrated in the past. in april 1998, *software* errors in a telecommunications switch caused temporary disruptions in the services of a major u.s. long-distance carrier; as a result, thousands of automated teller machines malfunctioned and some retail establishments lost their ability to authorize credit card transactions. in may 1998, a communications satellite control system failed, temporarily halting pager service for millions of people. as the scope and number of communication channels expand, dynamic and interdependent communications technologies become more complex. there are a myriad permutations and combinations of routing possibilities to transmit information. a phone call from washington to london may travel one path the first time and a different path one minute later. the industry has built technical redundancies, alternative network routing capabilities, and carrier back-up plans into the network to avert service disruptions. however, _the network's complexity and size make it *difficult to predict* if and where y2k-related communications disruptions could occur,_ and they increase the difficulty of testing the networks for y2k compliance. source: world information technology and services alliance countries with high teledensity, the number of main telephone lines per 100 inhabitants, have greater volumes of communications software, switches and other equipment that could contain y2k problems[see table 5.] there are developed countries where 85 to 95 percent of the population have ready access to a telephone, as opposed to less than 10% typically in many developing countries. end of text . reporter, i can synthesize this data for you if you like but i do not feel that it will be necessary. epri states the *critical* dependency of the electrical industry on the telecommunications industry.
the international trade report notes the *vulnerability* of the tlecommunications industry...in particular, those with high teledensity[they tend to get a bit more honest and careless when they when they are discussing problems of a global nature.]
this is our achilles heel...the u.s. has the largest, most sophisticated, most redundant telecommunication system in the world. do you still feel that the y2k problem is overstated?
and that is just *one* dot...m

-- Anonymous, April 30, 1999

I haven't read the report, but will borrow this quote from Reporter's report. "...the risk of electrical outages caused by Y2K appears to be no higher than the risks we already experience." Think about it. This means they assume every potential problem will be found and fixed. Dick Mills, an expert who I think is considerably more optimistic than Rick, would be the first to tell you that is impossible. I believe he uses 85% as a realistic figure for problems found and fixed, with the great majority of unfound/unfixed problems easy to compensate for, but not necessarily all. So what is going on in their heads when they make statements like that?

-- Anonymous, April 30, 1999

Two things bother me about this report right off the bat...first, the statement refered to by "Reporter" above. In the previous NERC report, dated 11 January 1999, we read the following on page A-8 under the heading "What is the Nature of the Y2K issue in Electrical Systems?", quote:
"In most respects, the electric industry faces the same Y2K challenges as every other industry, and even small businesses and individuals. Y2K anomalies could lead to the malfunction of software programs on mainframe computers, servers, PCs, and communications systems. Corrupted data could be passed from one application to another causing erroneous results. In the electric industry, this means computer programs used for accounting, administration, billing, and other important functions could experience problems."
"Of greater concern, both in the electric industry and elsewhere, is the pervasiveness of the Y2K bug in embedded chips."
"A single circuit board can have 20-50 of these chips from various manufacturers. Because of the diversity of chip suppliers, one vendor may use a different mix of chips even within devices labeled with the same name, model number, and year. Many of these chips have built in clocks that may experience date change anomalies associated with Y2K. The difficulty is in identifying all of these devices, determining if they have a Y2K problem, and repairing or replacing those that do."
Now then...how do we justify, in a matter of 3 months, going from that to "no higher than the risks we already experience...severe wind, ice, snow, floods...yadda yadda yadda.
Secondly, after that January report was released, someone found the actual spreadsheet and posted the addy. I downloaded it and had all the actual numbers, answers to questions, etc. Since the numbers didn't exactly jibe with the rosey summary report, the report was picked apart in short order by folks a lot smarter than me. This time around, I see they have provided "spreadsheets" but they are composite summaries rather than the actual numbers. Lesson learned I reckon.
Don

-- Anonymous, May 01, 1999

",,,,,, They are very definitely "aware" of what they are doing, they are also presented with a situation (from the administration) that prevents them from ,,,,,,,,,,, 1) independently presenting information to the public without getting it politically cleared through Koskinan's office."
Now I'm even more concerned if Koskinan has to censor everything........

-- Anonymous, May 01, 1999

Regular,
In answer to your questions.
1) Yes, I would interpret the 60% number to mean that 40% of all the distribution companies found at least one embedded device/system/software that would have caused a power interuption. Of course that also means that 60% did not find a single embedded device/system/software that would have caused a power interruption. Another data point that the Y2K problem in the electric industry is not as wide spread as it could have been.
2) Delivery schedule for CEMS. The delivery daes later in the year should not be a problem, but the point the report also made was that these systems won't shut down an operating power plant, nor cause emissions to increase. Therefore the impact of being late is relatively small.
bob

-- Anonymous, May 02, 1999

marianne:
I really don't know if their confidence is overstated. One thing I do know is that I don't have the ability to get the answers like NERC does. I don't think that anyone has the ability to get answers in the industry like they do.
Perhaps they have determined that the communications that they need to keep plants operating have been found to be ready after taking the time to do some observation. They are the responsible ones.
If they are wrong, they will live with the consequences, and I honestly believe that no mere litigation limitation will keep the problem at bay if death is involved. Judges have a way of taking a second look. I would speculate that there is a good chance that if they are wrong, they will pay one way or another.
My sense of alarm is still decreased by news that NERC has found little problems with the US power system and that they were willing to say so.
If it can be concisely proven that they are mistaken, my confidence in this organization will be shattered as it is for the FAA. We will find out in eight months.
Untill then, I plan to prepare, just in case they aren't quite right.

-- Anonymous, May 03, 1999

reporter
at this point in time i am suffering from a serious case of y2k malaise. whereas before, in the early stages, i read everything that i could get my hands on and researched and cross referenced with a fervor normally reserved for the religious zealot, i now half heartedly skim over the latest spin and resulting responses.
one can not maintain that level of intensity for a protracted period of time without suffering an advanced case of terminal burn out. at the same time i try to force myself to keep abreast albeit in a more cursory fashion. why do i bother? well...just in case someone like yourself starts having reservations or in the event the spin becomes so intense that the 'newly GI's might refrain from preparing and become lulled into a state of complacency.
remember...less than 7% of all government systems will be functioning, we are talking about 'mission critical' for everyone reporting, be it the electrical industry, telecommunications, shipping, the oil industry, banking, and all others.
koskinen has the cleanest mind in the country...he keeps changing it! the disparate and contradictory reports coming from the faa are just one example.
when someone like yourself, who has consistently questioned the information being put forth, suddenly capitulates and falls prey to the spin, it sends the wrong message and inadvertently bolsters the information being put forth by the 'spinsters.'
we all know that it is not going to be business as usual...but *NOONE* knows how bad it will be nor how long it will be.
i am totally prepared, have been since last year...but there are many out there who are either 'on the fence' or just coming to grips with the issue.
we cannot roll over and play dead, someone must keep asking the questions and attempt to verify or negate the veracity of the answers we receive.
if you can get just one family to prepare that normally would not have bothered it was well worth the effort you put forth. there are many lurkers out there, we are prepared...many are not.

-- Anonymous, May 03, 1999

I have several questions: At page ii, "Key Result 2: Minimal Operational Impact Discovered", it is stated that 'fewer than 3% of items tested before remediation have any difficulty at all with Y2k date manipulations. The types of devices that do experience trouble with Y2k date manipulations exhibit mostly nuisance errors... In most cases, Y2k does not affect primary device functions related to keeping generators and power delivery facilities in servcie and electricity supplied to customers.' It sounds that Y2k has no important impact on electric power. However, in the "Key Result 3", it is stated that "Y2k is real". It seems confusing. I would like to know: (1) Is the 3% figure trustworthy? (2) Do electricity generation facilities differ a great deal so that the Y2K impact on one plant could be significantly different from another? (3) Could such study results in US equally apply to electric utilities elsewhere, e.g., China, SE Asia, Russia? Thanks a lot for responding.

-- Anonymous, May 04, 1999

marianne:
Please don't 'cave in' because of me. I am just showing my willingness to trust those in charge. That doesn't mean that anyone should quit preparing or asking questions of doubt.
I still retain two very solid facts:

People Lie
People Make Mistakes
There is no way to get around these facts. This is why panic will most likely happen over Y2K. No one has been able to convince the public that bad stuff won't happen, like they usually are able to. This is because the experts are saying bad stuff is more likely to happen this time and until now, I haven't seen any body like NERC come out and say that the experts are wrong. That still doesn't change the expert opinion; however. Until experts see 'light', the only prudent thing is to prepare and doubt. I wan't to extend my trust to NERC as a sign of my willingness to be wrong and respect authority given to them by our government. But if they are wrong, I may never listen to them again.

-- Anonymous, May 04, 1999

I thought things had settled down with all the assurances about the elecric power situation. Strolling through your site after a big vacation, I am gripped with new found fear after reading the NERC report. I await your opinion on it, but Marianne and others caught what gave me the headache. The telecommunications situation.

-- Anonymous, May 07, 1999

Moderation questions? read the FAQ