Scada Internal Database Error Burns Three Young Boys to Death

greenspun.com : LUSENET : Electric Utilities and Y2K : One Thread

I realize that this story is not electricity oriented, however I believe that the SCADA systems are probably similar. Can anyone tell me how similar a pipeline vs. an electrical scada system is? I would think they're probably similar in that they gather information on flow rates and stresses related to the distribution of the electricity. Can anyone comment on EXACTLY how Scada works in the grid?

And Remember, "Hubris kills".

National alert from pipeline accident Regulators urge review of computer systems

Friday, July 9, 1999

By SCOTT SUNDE SEATTLE POST-INTELLIGENCER REPORTER

Breakdowns in the Olympic Pipe Line Co. computer system just before and during last month's deadly accident in Bellingham have so alarmed federal regulators that they have issued a nationwide warning.

The federal Office of Pipeline Safety issued the warning this week to the 2,000 operators of liquid and natural-gas pipelines in the United States. It urged them to make sure that computer systems used to operate and monitor pipelines are working properly.

The advisory details a series of computer failures on June 10 around the time Olympic's 16-inch line leaked up to 277,000 gallons of gasoline into Bellingham creeks. Gasoline vapor later exploded in flames, and two 10-year-old boys and a teenager were killed.

After the accident, Olympic acknowledged that its computer system crashed on the afternoon of the accident. The computer problems may have kept Olympic personnel from reacting quickly to the leak, regulators said.

The computer system is known as SCADA -- supervisory control and data acquisition. Such systems are common in the industry, though they may have been built at different times by different manufacturers.

All such systems go under the generic name of SCADA.

Some companies, including Olympic, add to their computer systems leak-detection equipment. Olympic's uses such information as temperature and pressure to detect leaks.

But investigators with the Office of Pipeline Safety have determined that Olympic's computer system broke down on the day of the accident.

"Immediately prior to and during the incident, the SCADA system exhibited poor performance that inhibited the pipeline controllers from seeing and reacting to the development of an abnormal pipeline operation," regulators said in their advisory.

The Office of Pipeline Safety is part of the U.S. Transportation Department.

Regulators did not name Olympic in the advisory. But Patricia Klinger, a spokeswoman for the Office of Pipeline Safety, acknowledged that the incident mentioned in the advisory and prompting the warning was Olympic's Bellingham accident.

The message to other pipeline operators, she said, is to "take extreme caution."

"We don't want to see this repeated."

Gerald Baron, an Olympic spokesman, said the company believes federal regulators are being prudent in sending out the advisory to pipeline operators.

Baron could not discuss the details of the computer problems and cautioned against focusing on computer difficulties or any other single factor as a cause of the accident.

Regulators believe Olympic's computer system typically operated at 65 percent to 70 percent of capacity.

But on June 10, the system had an internal database error. That error, plus the demands put on the computer by the leak, "hampered controller operations," the advisory said.

"The combination of the database error, the inadequate reserve capacity of the SCADA processor and the unusually dynamic changes that occurred during the upset condition appear to have combined and temporarily overburdened the SCADA computer system," regulators said.

"This may have prevented the pipeline controllers from reacting and controlling the upset condition on their pipeline as promptly as would have been expected."

Regulators also said that modifications made to the computer system after it was installed may have caused it to malfunction.

The Office of Pipeline Safety ordered Olympic on June 18 to find out what went wrong with its computer system and correct it. It also ordered the company to make a comprehensive review of its SCADA system.

Those demands came as part of a corrective order that closed the upper 37 miles of the 400-mile pipeline. Regulators also ordered the company to undertake several safety modifications and reviews.

The Office of Pipeline Safety may soon issue additional orders regarding Olympic's pipeline , Klinger said.

--------------------------------------------------------------------------------

P-I reporter Scott Sunde can be reached at 206-448-8331 or scottsunde@seattle-pi.com



-- Anonymous, July 10, 1999

Answers

this is precisely what many of us on this forum have been saying... s#*% happens and with y2k we will have much more of the same s#*% happening.

from the article: The combination of the database error, the inadequate reserve capacity of the SCADA processor and the unusually dynamic changes that occurred during the upset condition appear to have combined and temporarily overburdened the SCADA computer system," regulators said.

m] it is the combination of errors and unforeseen situations that make it impossible for *ANYONE* to determine the way anything will function at the rollover. how many "unusual, dynamic changes" will take place during y2k... let your imagination do the work for you.

article] "This may have prevented the pipeline controllers from reacting and controlling the upset condition on their pipeline as promptly as would have been expected."

m] operator errors... something that has been beaten to death on this forum. we are only human inspite of the protestations of the engineers to the contrary. now imagine the same situation in a nuke... running manual because they lost the normal monitoring devices. and puhleeese, do not insult anyone's intelligence by saying there are no parallels.

the article] Regulators also said that modifications made to the computer system after it was installed may have caused it to malfunction.

m] and what are we doing as a result of the threat of y2k?... we are changing almost *everything* in every industry where systems are an issue.

article] Breakdowns in the Olympic Pipe Line Co. computer system just before and during last month's deadly accident in Bellingham have so alarmed federal regulators that they have issued a nationwide warning.

m] it's only just begun.

-- Anonymous, July 10, 1999


Is there a link for this story?

-- Anonymous, July 10, 1999

linda,

this from the seattle post-intelligencer:

link

-- Anonymous, July 10, 1999


linda,

you might want to check this out also. while i was roaming around the newspaper archives i found this... note how many other refineries and industries were impacted by the failure. will we be able to work around catastrophes in this fashion after the rollover?

from the article: The rupture and fire cut off two refineries from the pipeline, leaving it with only two other supplying refineries in Anacortes.

That has left the pipeline operating at 45 percent of its daily 14 million-gallon capacity. Already, refineries are using more barges to ship fuel to customers and may increase such shipments, depending on how much regulators order pressure in the pipeline to be reduced.

prior

-- Anonymous, July 10, 1999


It's one thing when computers aren't working at all: you know they're not working, and you can take appropriate action (if any).

As we see here, it's quite another thing when computers aren't working properly: you might not know they're not working properly, and you might very well take no action or inappropriate action.

Unfortunately, three innocent young people lost their lives in what I'm sure must have been a quite horrible fashion because appropriate action had not been taken.

Jim Smith keeps reminding us, "Hubris kills". I remember how hard that hit me the first time I read it. Sadly, I think we are going to find out just how true that is in the coming year. Not just the hubris of engineers and programmers and economists, but the hubris of our whole technological society that asserts, without a moment's reflection, that we can't be so stupid as to let two slipped digits mess up the good life.

-- Anonymous, July 10, 1999



Moderation questions? read the FAQ