Embedded microprocessor report

greenspun.com : LUSENET : TimeBomb 2000 (Y2000) : One Thread

Goodness, I'm on a roll today.

I just got off the phone to a candidate of mine who is the engineering manager for a Japanese owned Tier One Automotive supplier in Southern Michigan.

It turns out that he is currently in charge of the engineering/shop floor Y2K remediation. He said that it was very fortunate that they do not have a full blown SCADA system or a fully integrated system, or they'd be sunk.

Now, the news I thought was important. I asked him about the "hidden clocks" in embedded controls systems, especially those that are part of "black boxes".

He said that absolutely he has found hidden dates/clocks that would have shut down production. He has had to reverse engineer systems down to the individual microchip level to find these, however. By the way, they use mostly Allen Bradley PLC's with custom software.

So, good news and bad news:

The good news is that some of these problems are being found.

The very bad news is that they exist and many organizations/companies either are not aware of them, or do not have the time, knowledge, and/or the expertise to identify and correct them.

Sorry, this time I'm not giving out the name of the company.

-- Jon Williamson (jwilliamson003@sprintmail.com), July 01, 1999



Thanks for taking the time to update us on this. We need all the information we can get...

-- Nabi Davidson (nabi7@yahoo.com), July 01, 1999.

Do us a favor and please see that you can get some product names for us. So we can verify this.

Nothing personal... just tired of trolls...

-- STFrancis (STFrancis@heaven.com), July 01, 1999.

Ah, STFrancis, I am not sure how long you have been around here but Jon is not a troll.

-- Mike Lang (webflier@erols.com), July 01, 1999.


Mike Lang is right. Jon provides quality information. In the past he has backed it up with names and supporting evidence when he was not bound by confidentiality agreements. He cannot always be as free with the info as he was in an earlier post today. Rest assured, he is not a troll.

-- Prometheus (fire@for.man), July 01, 1999.

I don't know how familiar you are with custom manufacturing, stamping, and assembly machines. Here is a brief summary of how they are designed and built:

These machines are ordered from machine tool companies which design and build them from scratch. Usually from a "blank piece of paper".

The machine tool companies often subcontract out parts of the design, especially the physical controls (industrial computers, PLCs, hydraulics, pneumatics, etc.) or the programming.

The controls themselves are often built of purchased subcomponents. There may be hydraulic controls from company A, pneumatic controls from company B, a computer vision system from company C, custom code from company D (which may contain prepackaged "objects" or chunks of code), and PLC's or Programmable Logic Controllers.

It will normally, once the machine is installed on the shop floor, take from 6 weeks to forever to get everything "debugged" and working together. I was a machinist on one of these new machines in Texas about 19 years ago. Over a million dollars of capital investment and it never did reach full production.

In this case, the PLC's are made by Allen Bradley, which has been very up front in admitting they have some date problems.

The bottom line is that I don't have model numbers of component chips. No model numbers of machines, either. Most of them are custom designed.

I did not ask my candidate for permission to quote him. To make it stickier for me, I am currently looking for some candidates for this company.

-- Jon Williamson (pssomerville@sprintmail.com), July 01, 1999.

Thanks for the support, folks. Stfrancis, I understand exactly how you feel. I posted earlier today about two southwest michigan companies in bad shape, and offered to provide the names privately. In fact, I sent them out to about 30 people.

-- Jon Williamson (pssomerville@sprintmail.com), July 01, 1999.

A key to discovering weather an embedded system is running hidden code is the amount of clock freq inputs going into a processor controller. Denoted on schematics as xtal 1, xtal 2 xtal 3 etc. If more than one clock signal is being inputed to run a chips timing string there is a good chance its running a secondary propietary internal operational firmware that is burned in by the vendor and is not acessable by the end user. Chip vendors do this on their high end embedded systems to keep competitors from easily copying them.

The microprocessor industry is fiercely protective of their designs this is why they are not willing to share the info about the hidden firmware code they encript into their products. Competitive advantage is like war to these companies and they dont care who gets injured so long as they dont lose the competive edge. The y2k bill before congress will let the big guys get off easy for years of looking the other way when they knew y2k would cause problems. Too costly to re-tool,It's going to break anyway so lets make money now and will deal with it later, was the attitude that prevailed among the captains of industry.

Since they had some of the politicans in their pocket they didnt need to worry about legal retribution there will be none. It's like war they think they can manage it until someone gets their hands on a nuke and then the game is over.

To our congressmen reading this post and voting on the y2k bill. If we dont hold these corporations publicly and legally accountable to fix their y2k problems in a set time frame our nation will be lost for good. The stakes are enormous. Life and death of planet earth and no laughing matter. If they cant fix there systems, which it appears that many will not be able too, they need to shut it down early. To prevent catistropic loss of life in systems that pose a great danger to the public's safety. Examples of these would be plants that manufacture toxic chemicals, Bio weapons and medicines,military munitions,tritium and other highly enriched radioactive materials,nuclear power plants,and any other processesm that involve immediate public life safety concerns if the plant malfunctions. If they are not ready as of today they should be on contingency operations until the peak of the y2k crisis is resolved. This is the only proper resolution to the y2k challenge.

Failure to look out for the best interest of the health and safety of our citizens is an act of iresponsibilty and treason against the office that you were elected to. Please vote right in the eyes of God and our county. Do the right thing no matter how much money is layed at your feet.

-- y2k aware mike (y2k aware mike @ conservation . com), July 01, 1999.

Yummy.... crow...

Sorry Jon.. but have grown a little weary with all these trolls around. I guess my B.S. Radar is on ultra sensitive right now. Did not mean to accuse you of being a troll....

And also not open my big mouth about some of the things I don't really know a "lot" about...

Now.. do I want my crow barbecued or sauteed...?

-- STFrancis (STFrancis@heaven.com), July 01, 1999.


Have you had any experience in refining environments that you can share here? I'm looking for any info along those lines. Or can you put me in touch with a source who has.

-- B. Lew Horshoe (g_gecko_69@hotmail.com), July 01, 1999.


Truly, not a problem. I understand where you're coming from.

As for the crow, no need for the "figurative". And the literal bird doesn't taste bad at all. Of course, neither do pigeons, coyote, or turtles, either.

B. Lew Horshoe

Not at this time. I've been thinking of doing some research in that area. Just haven't taken the time. I don't normally work with Petrochem. The only hint I've had was that when an Electrical Engineer I talked to had looked at Dow Chemical's embedded systems in Midland, Michigan (on contract), he sold his house (40 miles down wind) and bought one 120 miles upwind. He clammed up when I asked if I could quote him or if he would talk to anyone else.

-- Jon Williamson (jwilliamson003@sprintmail.com), July 01, 1999.

Allen Bradleys have no y2k bugs that stop them from operating, but some models that have real time clocks have minor date problems (leap year problem during cold - shutdown- rollover for example). From my experience it is rare to use a date in the application program for a control function, usually its date stamping/logging. Could your contact be talking about some other device? I too would like the manufacturer/model number of the device. True failures have been found, but they are very rare, and many people (even those involved in y2k) exagerate the severity of the problem.


-- FactFinder (FactFinder@bzn.com), July 01, 1999.

When the 80286 Intel chip was introed in the early 1980's, Process control types went crazy! Just 15 years earlier, we were using $100K XDS 16 bit "midi" computers for Direct Digital Control in refineries and Power Plants.

The more conservative approach was Digitally Directed Analog control. Now, we had a 286 chip with ultra fast memory (16megs directly addressable), scads of I/O horsepower and other goodies unavailable earlier. And those Chemical Engineers put them EVERYWHERE!

Comes January, the lawsuits begin no matter what the congress does. Hell, it already has! Today GTE filed suit against FIVE insurance companies to recover the costs of $400 MILLION Y2K remediation!!

-- K. Stevens (kstevens@It's ALL going away in January.com), July 01, 1999.

Moderation questions? read the FAQ