Rick's analysis of GAO report

greenspun.com : LUSENET : Electric Utilities and Y2K : One Thread

I spent some time going through the GAO report mentioned in previous threads, and offer the following comments. There's much to like about the GAO report, and much to whine about, whether you're a lay person or industry insider. Again, if you haven't read the report itself and have opted for the CNN FN "Readers Digest Condensed Version", I urge you to download the entire report off of GAO's website. Links were provided in earlier threads.

First off, the report appendices are a great primer for those of you who are not that familiar with the hierarchy of the electric industry, acronyms such as SCADA, or what an embedded device microprocessor board looks like.

The report, as I noted in a previous thread, relied in some measure on the November 1998 NERC stats. Before anyone jumps on the age of the source material, it is important to understand that the GAO report was actually presented to Sen. Bennett's committee in February of this year, and is just now being published. At the time the report was finished, the data from November 1998 was the most recent available to GAO (no NERC stats were published in December, and January's were issued too late to be considered for this report). Also, GAO did not rely exclusively on the NERC statistics - GAO staff did some investigation and analysis on their own. With respect to NERC and DOE statistics, though, GAO "did not validate the accuracy of reported information". So, in essence, the industry self-reported data that was reviewed was taken at face value. I certainly would have hoped that GAO, of all government agencies, would have done a bit of data validation prior to relying on that data.

The scope of the report is somewhat limited. GAO states that the U.S. electric power industry is comprised of 3,200 electric utilities, with about 700 of the utilities operating power generation facilities. According to Edison Electric Institute, there are approximately 7300 companies that either make, transmit, distribute, or market power in the U.S. The GAO report did not consider the impact of Independent Power Producers (IPP's) on the overall health of the North American electric network. IPP's are non-regulated power generating companies, and are not subject to any regulatory oversight (ie. a state Public Utility Commission or federal regulatory agency). In calendar year 1998, IPP's supplied around 10% of the net electric power generation in the U.S. (source: DOE Energy Information Agency, http://www.eia.doe.gov/cneaf/electricity/epm/html/epmt3p1.html). NERC expressed concern in their January report, and "conference minutes" leading up to that report, that IPP's were generally not reporting readiness information to NERC. When you consider that a company such as U.S. Generating Co., which acquired many power plants in New England during the last year and is now the second largest power generator in New England, is an IPP, and there's no oversight or regulation of their Y2k effort, it's concerning that the GAO report did not cover IPP's. (Also, for those of you who care, U.S. Gen is a subsidiary of PG&E, and currently operates nearly 6000MWe of power nationwide.)

The basis for the GAO report is centered around "computer control systems and embedded systems that are susceptible to Year 2000 failures." The report completely ignores the business necessity of backoffice IT systems, and the role that these systems play in short term (but not immediate) corporate health. There are also backoffice systems that are mission critical from a standpoint of, as an example, customer service and dispatch.

In an apparent attempt at a risk assessment of embedded systems, GAO focused almost exclusively on Supervisory Control and Data Acquisition (SCADA) systems (page 3). SCADA is certainly important to daily operations, but not absolutely critical. And what they missed about SCADA and EMS is important:

  1. The site uniqueness of each installed system that makes a "cookie cutter" industry level analysis impossible
  2. The critical need for "clean management" of the platforms and software once thoroughly tested and certified

The GAO report also places a lot of weight on NERC oversight of the electric industry Y2k effort. Remember, NERC only has 30 permanent people on staff, including secretaries, receptionists and the like. Their Y2K program management is contracted out. In my opinion, NERC does not have the organizational depth to "closely monitor the status of those facilities that may be at risk of failure, and increase its supervisory activities". (Pg. 3) And, not to cover ground that's already well tread, but keep in mind that NERC has absolutely no regulatory or legal authority to force participation or compliance by any specific electric company. NERC even issued a press release today, lamenting the fact that "The existing voluntary system for setting and encouraging compliance with industry reliability standards for the transmission system is not sustainable in today's increasingly competitive electricity industry." Simply translated, this means that they have no legal hammer with which to compel any individual electric company to do anything.

The suggested actions listed on Page 4 are certainly important, but there is no direction or suggestion on how to implement the actions. The recommendations also ignore the harsh reality of advance planning and scheduling for plant outages within the electric industry. And it is interesting that, even though the report was prepared two months ago, as of yesterday, a public utility commission contact of mine was unaware of any initiative by DOE, GAO, or NARUC (National Association of Regulatory Utility Commissioners) to "encourage state regulatory utility commissions to require a full public disclosure of Year 2000 readiness status...".

In reviewing the appendices to the report, many of the errors in the body of the report were magnified, but there's some good stuff in the appendices. The "Ownership of Electric Utilities" graph on page 14 does not include IPP's or power marketers. The "Primary Energy Sources" graphic on page 15 is flat out wrong when considering the entire U.S. The discussion of embedded systems and vulnerabilities on pages 21 through 23 is good, though I'm sure there's some in this forum that would argue the validity of GAO's conclusions regarding embedded systems. Most concerning to me is the statement on page 30:

The readiness of a large percentage of the distributors [municipal power agencies and rural electric co-ops] is uncertain because about 73 percent of the municipal and about 23 percent of the cooperative distributors did not report their status.

Is it lost on anyone that almost 1500 muni's and 200 co-op's didn't report their status? Do you live in an area served by a municipal power agency? I think this stat should concern you... I've already covered some of the issues on page 24 (Planned Industry and Regulatory Actions). What struck me were the two statements regarding actions to be taken by industry associations against agencies / companies that did not meet target dates:

"NEI did not plan to take any specific action..."

"APPA and NRECA did not specify in their report any actions..."

No surprises here - these associations have no authority to take any actions. More importantly, the Y2k role that these associations have been thrust into is totally foreign territory. Simply put, these associations can't take any specific "actions". They don't know how, and don't have the teeth to do so if they did know how. This is not a criticism of NEI, APPA, NRECA, or even NERC - these agencies (with the exception of NERC) have never been placed in a situation where they were responsible for the performance of their respective memberships. It's as an uncomfortable position for them as for the industry segments that they represent.

Let's talk about government owned power companies. By the GAO numbers, government power agencies are a basket case. But interestingly, when prodded by GAO, all of a sudden these agencies have "popped to", and committed to bettering the schedules that they've previously published. Now think about this for a minute - GAO holds a great deal of power over these agencies - and suddenly, they're feeling the heat. So, of course TVA, BoR, and BPA are going to "commit" (as much as any government agency) that they can meet enhanced deadlines. Look at the figures on pages 32 through 35 - then remember that TVA (and to extent BPA) have been fairly crowing about their successes in the press. In particular, pay attention to page 35. The bottom line, going from a 4th quarter completion date to a 2nd quarter completion with the stroke of a pen is PFM (and I'm not going to explain that acronym) - GAO says it will be so. Thus, it is so.

Pay attention to page 38. This was certainly an eye opener for me:

TVA officials explained that five fossil plants will not meet the industry target date for readiness because they will not make the Year 2000 renovations at these plants until the next scheduled outages for maintenance and repair in November and December 1999. Because these fossil fuel facilities represent about 26 percent of TVA's electricity capacity, Year 2000 failures could have had a significant impact. In response to our concern with the risks associated with this schedule, TVA officials examined the incremental costs of rescheduling the outages for these plants and decided to move the scheduled outage to June 1999 for three plants, including two with the highest generation capacity.

It took GAO prodding of TVA for TVA to do this simple math. Prior to GAO intervention, 26% of TVA's capacity was not due to undergo Y2k remediation until NOVEMBER AND DECEMBER of this year. Remember, TVA has been all good news. There's more than a few threads in the euy2k.com forum regarding TVA trumpeting "advancing the clocks" on some of their plants. Go back and do a search of the forum on the keyword, "TVA", and see what plants they advanced.

As I finish this, I'm going to avoid discussion of nuclear plants and the NRC, for the moment. That will be the subject of a separate thread in the next few days. There's a lot of things about the GAO report and NRC information on nuclear power and Y2k that is simply not jibing, but I've got to work on that one a bit.

So, there it is. You asked for it; that's my assessment of the GAO report. Is there any reason to be encouraged? At the beginning of the report, GAO states:

While the electric power industry has reported that it has made substantial progress in making its equipment and systems ready to continue operations into the Year 2000, significant risks remain.

You can believe it or not. The choice is up to you. The need for you to do a critical analysis of your own local situation has never been clearer.



-- Anonymous, April 22, 1999

Answers

Oh, and one other thing that's important in assessing the validity of the GAO report. I picked this up in the TimeBomb 2000 forum tonight:

When asked, at the monthly WDCY2K meeting tonight why there's a difference between GAO reports and the President's Y2k Council reports, Senator Bennett replied (and I paraphrase):

"GAO answers to Congress. John Koskenin answers to the White House."

-- Anonymous, April 23, 1999


Rick,

Thank you kindly for your post. Question: I thought I read somewhere that you would be involved in a public forum soon. Someone wanted C-SPAN to cover it. What about that?

:)

-- Anonymous, April 23, 1999


The event you're referring to actually occured this week in Quincy, Mass. Media coverage was sparse, as most of the network attention and cameras were focused (as was certainly appropriate) on events in Colorado. A summary of Quincy meeting was posted elsewhere in the forum.

-- Anonymous, April 23, 1999

Excellent analysis. Thanks for your work. I agree the TVA numbers are more than a little suspect.

-- Anonymous, April 23, 1999

thank you and thank you again...

-- Anonymous, April 24, 1999


Rick, I had caught the reference to the high percentage of muni's and coop's whose status was unreported, and the primary energy sources graphic had also made me frown. But I would have missed many of the things you pointed out. Your experience and ability to look at so many separate, and seemingly disparate bits of information and to present the broad industry implications of all those different aspects is exceptionally helpful. My sincere thanks for a fine, fine, analysis!

-- Anonymous, April 24, 1999

Rick: At a recent Y2K forum in Houston, several VIPs from the utility companies and city hall spoke. Seems like Texas has no problems. The head of Texas Public Utilities was quoted as saying, "January 1st will be no different than a day like today." I asked the City Hall representative if she would leave her 80 year old parents who were dependent on medical devices [plugged in] in the city of Houston during holidays. "Absolutely," she replied, "I'll be here and I've been here for thirty years." No problemo for Tejanos.Si?

-- Anonymous, April 26, 1999

Thanks Rick for all your time. Graham

-- Anonymous, April 26, 1999

Moderation questions? read the FAQ