1/30/2001 Summary of an Engineer's Observations Regarding the Status of Ongoing Y2K-Related Embedded Systems and Complex Integrated Systems Problems

Preface:

I am attaching a summary of observations that I have drafted. The summary is based on observations that an engineer has shared with me. I am sharing these summarized observations for several reasons:

1) because of the relative absence of first hand accounts concerning what is actually going on regarding Y2K-related embedded systems and complex integrated systems problems;

2) because I have heard some similar off the record accounts from other engineers; and

3) because I feel that right now such off the record observations provide the best information and roadmap to further inquiry that we have.

Perhaps, those who are in a position to do so, will come forward, at least off the record, and help enlighten the public and those in positions of public and private sector trust and responsibility concerning the significant role that Y2K-related embedded systems and complex integrated systems problems are having in a variety of sectors, including the energy sector.

*******************************************************************

During the last week of January 2001, I received some information from a seasoned engineer who has been working "on the frontlines". The identify of the engineer cannot be disclosed since the individual's job security could be jeopardized.

The individual shared information concerning the many Y2K-related problems that he is continuing to see. (I have not met the engineer in person and do not know his or her real name and will refer to "him" as "he" in this summary.)

Also, rather than quote the individual directly, I am summarizing most of the information that he shared with me.

~ Several of the companies that he has worked with have had extremely serious data corruption problems. After much effort and temporary successes in dealing with these problems, the data becomes corrupted again.

~ With respect to the grid, he feels certain that the energy crisis will become increasingly apparent this summer. In his view there have been large numbers of failures involving energy systems. In these instances, he says that workarounds are often not possible. He notes that turning clocks back and going to manual have resulted in some cascading failures and time delays.

~ He notes increasing reports of problems with dirty power and low power and instances of involving the total failure of electrical equipment.

~ He also talks about what he feels is a direct correlation between solar storms and hardware failures.

~ He says that those working "on the frontlines" are being threatened with the loss of their jobs if they speak up about what they know.

~ I had told him that it was my sense that people at the top of private sector organizations do not seem to comprehend the extent of their Y2K-related embedded systems and complex integrated system problems. He said that of the persons he comes across, less than 20% of those who work with complex systems understand the systems and keep up with changes and that only a small percent is able to address problems effectively. The others don't really understand what is going wrong with their systems.

~ I asked him how large a role he thought Y2K-related embedded systems and complex integrated systems problems were currently playing in the evolving energy crisis. He said that he estimated that 70% of the failures involving the energy sector, and communications (among others) are directly the result of Y2K. He estimated that 20% of the failures could be due to human error on the part of those trying to deal with the problems. He said that those individuals often only have enough ability to deal with normal activities and that they have insufficient understanding to deal with anything that departs from the norm. He estimates that the other 10% of the problems is owing to normal hardware failure, user problems, and environmental issues.

~ He said that manual override and date resetting have been used when automated production systems and SCADA systems have failed. He said that it is not uncommon when he is replacing a system to be told by the client that he has to put in an old date or the application will not run. He added that many of these applications are old and that large networks over the past decade can be composed of a mix of upgrades, networks, and applications that are out of sync. Owing to these problems, he estimates that the country is running at 65% to 70% of last year's production rates on the average.

~ I asked him about problems in all of the high hazard sectors: oil rigs, refineries, oil and gas pipe lines, nuclear power plants, nuclear reactors, chemical plants, hazardous material facilities and sites, electric power plants, water purification plants, waste treatment plants, trains, planes. He responded that most of these have fixed what they could; fixed the rest on failure when possible; or, if the expertise is missing, attempted to make the failing system work manually. In situations where a system is run 24 by 7 and where there is an apparent problem, he says that there is only a narrow window of time during which the system can be analyzed and repaired. Sometimes when there is an apparent problem, but where no hard errors have occurred, he has been asked to replace hardware. When new hardware does not fix the problem, going to partial manual override becomes the only remaining option. He also noted that in many networked environments, date/time is sent in packets and when there are systems broadcasting an old date along with current dates, the data can be corrupted or miscalculated.

~ He said that he has not found anyone who is willing to talk about what is happening, even off the record. He said that some of his more aware customers are asking him what he is seeing and asking questions about the power crisis. He thinks that they are beginning to catch on.

~ I asked him if he knew of any cases involving high hazard sectors where the problems are being publicly recognized AND linked to Y2K? He said that Y2K is never mentioned in explanations as a cause of problems. Instead "silly" explanations are offered and most people take these explanations as fact.

~ I asked him what his prognosis was for nuclear power plants. He said that he was told prior to the rollover by someone in a position to know that in instances that his information source knew about, clocks were turned back where there was a possibility of potential problems and failures. He said that this only works for a time as the interconnectness of these system runs too fast for individuals to keep them going. In his view, the production task has become very costly negating most, if not all profit. In addition mechanical/electronic failures are extremely costly. He said that he felt that many nuclear power plants were running well below capacity due to the failures and owing to manual operations. He feels that they do not seem to be making much progress getting back to normal and that in the end those plants will become too expensive to run.

~ I said that I have been hearing about shortages in the pharmaceutical industry and ask him if he thought this might be related to problems with manufacturing processes. He said that there are manufacturing problems and that too many bugs have slowed manufacturing processes. He added that there is a major shortage of computer components and that the parts that are available are often parts that have been put back in stock even though they do not work. He said he has found the same to be the case when it comes to other technology companies and parts vendors.

~ Regarding health care system problems, he said that they are having all kinds of issues, including claims that are getting rejected for no valid reason, accounts that are coming up blank, or billing where charges and services are being doubled.

~ Regarding air travel, he said that air travel is having its share of Y2K issues. He also feels that solar storms are having an impact on air travel and that Y2K coupled with solar storms have triggered many of the problems that have been occurring.

~ I asked him what he thought about the possibility that manhole cover explosions might be caused by irregularities in transmission. He said that the manhole issue is a very interesting one and that he feels that it is due to electrical power cables overheating and creating a gas that results in an explosion. He thinks that this is probably due to the use of manual power overrides.

~ He said that every time there are major solar flares, he notes an increase in CPU, memory and disk drive failures. He notes that the incidence of failing modules is very high owing to their density, a factor that makes them more sensitive to the effects of solar storms.

~ I asked him if he knew of any cases where problems involving data degradation were being publicly recognized AND linked to Y2K. He said that not one company is going public. The usual explanation is that the company is having "computer problems" and that "the system is new."

-- (die@hard.doomer), January 31, 2001

Answers

I am an engineer in the electricity industry. I will not reveal my name or the company I work for even though my job security may not be comprimised.

The company I work for has had no serious data corruption problems due to Y2k, although we did experience some minor issues early on.

In my view there have been few failures due to Y2K,and those that did occur did not affect generation or supply.

I note that there may have been reports of dirty power, or even system overload, but these were due to high demand, lack of storage in hydro lakes, and no new power plants being built. The grid in some areas is reaching its capacity, but that was apparent 10 years ago.

I have not been threatened with the loss of my job, and I have not threatened anyone in this manor either.

Y2K is not mentioned as the cause of any problems for one very simple reason. IT IS NOT THE CAUSE.

My name may be false, but my email address is real.

-- Rolyat Mloclam (taylorm@es.co.nz), January 31, 2001.

Interesting how this "engineer" knows about problems from several industries: power, communications, pharmaceutical manufacturing, health care systems, air travel, oil rigs, chemical plants. Oh, and don't forget, he is an expert on solar flares too. What type of "engineer" is he? I suggest that at most he is a "social engineer".

What a crock.

-- Buddy (buddydc@go.com), January 31, 2001.

I was going to ask if she knew what date it is, but that was the first thing she "summarized".

At what point will this woman realize she's making an ass out of herself? Does she actually think she's going to get a job in THIS administration?!?!

rotflmao.....

-- (PatriciaS@lasvegas.com), January 31, 2001.

He said, she said, or is it 'them'?

What a buncha hooey.

If we get nuked it wont be due to y2k.

-- sumer (shh@aol.con), January 31, 2001.

I think Ms. Gordon has something there.

Fortunately it isn't communicable.

-- Fool Me Once... (howe9@shentel.net), January 31, 2001.

I have to second (third, fourth, fifth, etc) what all of the above wrote, except to disagree with Buddy. More like a Sanitation Engineer. After all one of the first things you learn when dealing with computers is: Garbage in, Garbage out.

Maybe we should all vote for the stupidest or funniest comment in the polemic. Mine is for one about the manhole explosions:

“~ I asked him what he thought about the possibility that manhole cover explosions might be caused by irregularities in transmission. He said that the manhole issue is a very interesting one and that he feels that it is due to electrical power cables overheating and creating a gas that results in an explosion. He thinks that this is probably due to the use of manual power overrides. “

Ah yes, those ole irregularities in transmission. They’ll get you each and every time.

-- The Engineer (spcengineer@yahoo.com), January 31, 2001.

Oh great!

And this is a new pair of pants too.......

Now we have to remember that THEY did say we would have imbedded problems all the way into or around 2027.......remember????? :-)

Deano

-- Deano (deano@luvthebeach.com), January 31, 2001.

Patricia, I must disagree with you. Gordon has gone WAY past making an ass of herself. I don't know what her level of insanity is called.

-- Uncle Deedah (unkeed@yahoo.com), January 31, 2001.

afterthought

All kidding aside this is about the saddest thing I have ever seen. I don't know what her professional reputation was before Y2K, but it must be about dirt by now. And still she spirals ever down....down......down.......

-- Uncle Deedah (unkeed@yahoo.com), January 31, 2001.