Disturbing news: Embedded chips (controllers), using date or not--a problem

greenspun.com : LUSENET : TimeBomb 2000 (Y2000) : One Thread

From J. D. Solomon, "Debugging the year 2000," Year/2000 Journal (y2kJournal.com)January/February 1999:

"The Year/2000 problem will impact both Information Technology (I/T) and nonI/T systems, which include building (security, heating, ventilation and air-condition, elevators, etc.), transportation, power production and water/sewer systems. In general, if non-I/T systems use a microprocessor (and most do), there'll be potential Y2K problems. . . .

"'My infrastructure system doesn't use a date function, so I don't have a problem.' False. Most microprocessors, or chips, have some type of date function, whether it's used or not. It's just as cost-effective to make a chip with a date function than not to. So most chips include both time and date. Unless the date function is "disabled" before installation, the chip is using the date function whether the equipment specifically uses it or not."

The author is J. D. Solomon, P.E., "Year/2000 program manager for Trigon Engineering Consultants in Raleigh, NC. . . ."

-- Old Git (anon@spamproblems.com), January 26, 1999

Answers

Please give some examples how an UNUSED date function can cause a controller or similar device to fail.

-- Dr. Roger Altman (rogaltman@aol.com), January 26, 1999.

Now that I have learned Pollyannaspeak (thanks Paul Davis, Troll Maria, and all), let me try to see what we can do here.

Just because there is a date, that does not mean that there will be a problem in 2000. Anyway, it would be in BINARY rather than DECIMAL, so there would be no problems. If anyone can PROVE that there is such a chip with a date that has this problem, then need to post the chip's serial number, date of manufacturer, where it is used, and how. If they cannot do this, then this PROVES, ABSOLUTELY PROVES, that they do not exist and that there is no problem.

Whew... I can see where pollyannaspeak can really take its toll on the brain.

-- King of Spain (madrid@aol.com), January 26, 1999.

Dr. Altman, my post above is a direct quote from the journal cited. The way I read it is if, say, an elevator is controlled by an embedded chip, it may still fail because the controlling chip ITSELF is date-dependent, even though the piece of equipment shows no date on its electronic display. The chip will fail, hence the equipment will fail. I have e-mailed the article's author with your question.

-- Old Git (anon@spamproblems.com), January 26, 1999.

Your Highness...I'm afeerd there's a flaw in yer logic there.

-- SeesBeyond HerNose (OneHandClappin@year00.net), January 26, 1999.

But fail WHEN? I'm assuming when it reaches 2000 or so, but wouldn't the date need to be set, and by whom and when? Does the manufacturer set the date (and if so, does the system need its own battery backup to maintain the date)? If randomly set because the date "didn't matter", does that mean they are time bombs set to go off at some completely unforeseeable time (and if so, has that been happening)?

-- Brooks (brooksbie@hotmail.com), January 26, 1999.


Let's wait on this until the source responds to the e-mailed questions! (It's lunch time in Raleigh, NC.) I'm not a techie so I can't give you proper answers. One reason we have this forum is so we can either confirm or debunk theories and opinions. It would make me very happy to know that my battery-operated glucometer will still work, even though the controlling chip uses only the day and month for the date display.

-- Old Git (anon@spamproblems.com), January 26, 1999.

arrrghhh...this has been rehashed so many times before. We need a search engine dog garn it!

- True that the great majority are stamped with date/time functions for cost effectiveness.

- Debated furiously on this forum whether they will fail if the system doesn't need the date funtion, and/or doesn't display it.

- Was generaly agreed and understood that the actual time/date of failure would depend on when the chip was made/put on the shelf.

- Every experts debate this issue ad nauseum, bottom line is still No One Knows Any Thing For Sure. No physical demonstrations/testings has ever been done to demonstrate irrefutably that systems said to be non-date dependent will trip on roll-over (as far as I know.)

I'm just not motivated right now to dig out the old threads again, becomes old and a bore after a while. Perhaps someone else could do it.

-- Chris (catsy@pond.com), January 26, 1999.


More info on embedded chip/system topic at

Mark Frautschi

and also see

Critique of Mark Frautschi's Report



-- Anonymous (anonymouse@anonymous.com), January 26, 1999.


"But fail WHEN? I'm assuming when it reaches 2000 or so, but wouldn't the date need to be set, and by whom and when?"

Yes! This is key. Chips which have "hidden" internal date counters might be set to any date - by no means the "correct" date. So the chip will fail, but nobody knows when.

Personnaly, I feel that it is GOOD that the chips are often set with the "wrong" date. At least things won't all fail on the same day. Now, about those chips that DO use date...

-- Anonymous99 (Anonymous99@anonymous.com), January 26, 1999.


For Brooks and everyone else:

Frautschi Paper

-- Jack (jsprat@eld.net), January 26, 1999.


From: "Solomon, JD" To: [Old Git's real name and address] Subject: RE: Year/2000 article Date: Tue, 26 Jan 1999 13:00:42 -0500 X-Mailer: Internet Mail Service (5.5.1960.3)

Old Git had it right. The intent of the statement was to make the average layperson aware that even though there may not be a computer, software, or even a method to manually enter a date, the piece of equipment/system may be using microprocessors that are storing date specific data or using date specific data in some internal calculations. One of the primary problems that we see when facilities perform their own in-house inventories is that their field people ignore the equipment which doesn't have a means for a person to enter a date. The intent was to "debug" that perception.

-- Old Git (anon@spamproblems.com), January 26, 1999.


As an exercise, I've tried to design a device that will fail solely because it contains date functionality, even though it makes no use of this functionality. So far, I can't imagine how to do this. It might be possible, but it's not easy.

I have been involved in the design of several devices that do have and use date functionality, even though the user cannot tell this by examining the device, nor would the user have any reason to suspect that dates (or even time) were used at all.

Potentially, these devices can fail, though not as used in any of their implementations I'm familiar with. The time/date of failure is undefined, and could be anywhen.

-- Flint (flintc@mindspring.com), January 26, 1999.


bottom line is...these things are gonna fail in wierd and woderful ways...and people will die as a result. I'll bet the farm on it.

-- a (a@a.a), January 26, 1999.

Flint, one of the problems with the arguments that you (and others) seem to always present is that you are always generalizing from the specific to the general, rather than the correct way: general to the specific. I don't doubt that your own experience with embedded systems leads you to believe that such Y2K failures would be highly improbable -- in fact, if you think about it, that is exactly why, percentage wise, they expected number of Y2K failures is in fact quite small.

But as I am sure that you are aware, the history of the source code for embedded systems/chips/devices/whatever is comparable to that of COBOL and other languages -- it was up to whomever was writing it, and whatever mood they happened to be in at the time, as to how it got written. There was (and is) no standard that is adhered to. Without such a standard specification on how to handle a particular condition (e.g., the inability to handle a date that does not "fit"), it is anyone's guess what will happen, unless you find it and look.

So, we just don't know what will happen, and the chance that a small chip could cause big problems is a genuine concern.

-- Jack (jsprat@eld.net), January 26, 1999.

The story I've heard about unused date functions goes like this: If the date is stored in two digits, then when the chip adds 1 to 99 to get 100, you get an overflow condition--that extra digit goes to an adjacent byte in memory that was being used for something else, with unpredictable results. I'm not an engineer and don't have a verifiable source for this, but in software this sort of thing certainly can happen.

-- Shimrod (shimrod@lycosmail.com), January 26, 1999.


A chip may have functions A&B&C&D another chip may have functions C&D&E&F. The company may only wish to use functions C&D. So it has a choice to buy either chip 1 or chip 2. it buys chip 2 becuase it is cheaper. Now, they do not bother to test the chip because it 'does not need' to know the date. However functions E&F *DO* know the date and can malfunction even though there was no NECESSITY to use a date function at all.

Many systems are not being tested on this basis and it is a huge mistake.

-- Paul Milne (fedinfo@halifax.com), January 26, 1999.


Paul Milne's statement is true, although his specific fears are unfounded.

Almost every IC used in a computer has anywhere from a few to many alternate or additional functions not used in the specific implementation. A PC probably uses less than 20% of the functions the chips actually provide. The RTC itself contains multiple unused functions (binary mode, daylight savings, multiple clock rate selections, several others). If it were even remotely true that an unused function represented any hidden danger, no computer would ever work at all!

The danger of a truly unused function suddenly jumping up and pissing in the soup is nonexistent. If the date really isn't used, it really isn't any problem.

BUT...

The lack of any external indication that the date is used, and our inability to dream up any earthly reason why the date *might* be used, doesn't mean the date is not used. Sad to say.

If the date is misused, it indeed represents a danger. If the misused date must synchronize with calendar time for any reason, the malfunction will (if not found and fixed) occur at or shortly after rollover, depending on the accuracy of the time being kept. If the device with the misused calendar-synchronized date is used in a critical implementation, the fallout can be catastrophic (but usually local, exceptions being power distribution and communications).

I wouldn't be surprised by a thousand serious incidents up to Bhopal level, worldwide. You don't need to be paranoid of everything containing silicon to recognize the clear and present dangers we face, and the enormity of the task of finding these few needles in a huge haystack.

The sheer perversity of embedded y2k failures will tend toward a maximum.

-- Flint (flintc@mindspring.com), January 26, 1999.


I believe Flint has neatly hit upon the potential problem. If a given embedded "device" (may consist of several "chips" on a board) has a timing function and is either slaved to an RTC or has an internal clocking functionality, it may fail. The device may well not externally represent a date or time as such. If I were searching for problems, I would look for processes that require timing functions as part of the design.

-- RD. ->H (drherr@erols.com), January 26, 1999.

I can't figure out what will happen to us, because I still have some fundamental unclarities, namely: * Where do the chips get their electricity from? * If a chip has a clock, how does it stay accurate over it's life of a hundred years, especially, with all those occasional power failures happening?

-- Obin (Obin@fla.net), January 26, 1999.

To RD and anyone interested in this stuff:

I am intimately familiar with one particular device. There are tens of thousands of these in the field. The device contains no clock/calendar chip, keeping track of the time via a periodic interrupt. It will fail at rollover.

This device keeps its time in sync with the real world by means of periodic radio messages. These messages tell the device what to do, and each message contains a time component. The *only* purpose of this timekeeping is as part of an encryption scheme to prevent the device from being fooled by bogus messages. It's not obvious that the time is embedded in the message, and the message is ignored if the embedded time component is too far off. Otherwise, the device has no need of any timing whatever.

Yes, I have a compliant version (now in production), but the cost of upgrading each device exceeds the cost of the device itself! Fortunately, the failure of these devices will cause very limited functional distress (but of course you need to buy a new one).

I wrote all the firmware, yet I myself would not be able to come up with a test that would demonstrate that this failure is imminent or even possible.

Well, live and learn.

-- Flint (flintc@mindspring.com), January 26, 1999.


Several comments in this string have questioned how a date can be programmed into a controller without any sort of user interface or control present. One project I worked on in the last couple of years was a chip burn-in and test system for THE major chip maker in the world (starts with "I"). After the silicon dies had been placed into the chip carriers, our machine raised a set of leads to the connections and began testing.

One of the functions tested was **the clocks**. Load with the current date and time and as the tests ran, check for time function errors. And once the clocks are set for the first time, the information for that production batch is the "creation date" for any chip in that batch. The date and time became part of the batch and serial number information for the chips.

If the chip is installed into any piece of equipment and there is no new date loaded, it defaults to the cration date for starting information. Depending on the model of the chip, it may even have an internal power source (usually internal capacitor) inside the plastic chip body that will allow time functions to run, even when there is no power applied to the chip or any circuit it's installed in.

More than just the chips themselves, some complete pieces of equipment have no user accessable controls for setting time or any other function. But during assembly there are ways that programs and time data are loaded. Often the connectors are removed during assembly and there are no signs of how anyone could ever program such a device.

These days, I'm finding it more common in military and industrial systems that the connectors are so miniaturized, that to look at them you can't recognize them as being programming ports. But if the product manufacturer plugs the board into their test and programming fixture, the connections are made and the dates, times and all programming features can be controlled by an external system.

WW

-- Wildweasel (vtmldm@epix.net), January 26, 1999.


With advancing age it becomes more and more difficult to assimilate technical information, especially if your brain has never had to deal with it. Through this thread, though, I've got a glimmer of understanding of the scope and complexity of the embedded chips question and I'm grateful to you all for helping me towards the light. Thank you.

-- Old Git (anon@spamproblems.com), January 27, 1999.

Git, this might be an example of the type of chip this author is talking about.

It comes from an email from from Roleigh Martin to C.K. Houston, which she posted 1/25/98 in a thread on Silicon Investor.

Processor doesn't use the date, but the date is still there

>>Would you post an example of the type of Embedded failure you are looking at please.<<

This email I received from Roleigh Martin, an expert in this area, might answer your question.

"I was a little disappointed in that one spokesman who said they had nothing to worry in their hardware, because their devices do not work at the date level. That showed ignorance of how these devices sometimes work.

There are plenty of electronic devices who only CARE about millisecond intervals, but whose electronics for calculating the millisecond difference pulls a time value from a computer clock that might look like this to the software:

YYMMDDHHMMSSTHK where YY=2 digit year MM=2 digit month DD=2 digit day HH=2 digit hour (military time) SS=2 digit seconds T= 1 digit tenths of a second H= 1 digit hundredths of a second K= 1 digit thousandths of a second

and the device figures out the gap in time (in milliseconds from) invoking a built-in function that sends two values of YYMMDDHHMMSSTHK, called YYMMDDHHMMSSTHK[present], YYMMDDHHMMSSTHK[past]

and the device always assumes that YYMMDDHHMMSSTHK[present] minus YYMMDDHHMMSSTHK[past] = positive number difference

However the above assumption will fail in 1/1/2000 if the past value is before 1/1/2000.

So anytime I hear a spokesman say, there is no worry because we only deal in milliseconds, etc., I consider that spokesman ignorant of what is the nature of the beast.

NOW, NOT ALL DEVICES WORK THIS WAY -- some only deal in ticks, and represent a past tick with a positive integer, and a future tick with a larger integer, and true "time" is never dealt with -- those devices will have no problems.

Also devices such as the above that accomodate the century rollover will work. Those devices that represent the year as 4 digits will work. And in some cases, some devices will not encounter such subtractions that lead to negative numbers, because they might not be "doing anything" on the rollover and their history span might be only "seconds or minutes ago".

But some devices will fail and have been proven to fail."

Roleigh Martin http://ourworld.compuserve.com/homepages/roleigh_martin

How likely it is that such a chip actually works with a clock that's set to the correct time, I have no idea. Seems like it would "just depend", not making anyone's life easier.

-- Debbie Spence (dbspence@usa.net), January 27, 1999.

ARGH. Yes, my brain is fried anyway from all this embedded stuff, but no reason to fry everyone else's too. :-) Trying again, "There are plenty of electronic devices who only CARE about millisecond intervals, but whose electronics for calculating the millisecond difference pulls a time value from a computer clock that might look like this to the software:

YYMMDDHHMMSSTHK where

YY=2 digit year

MM=2 digit month

DD=2 digit day

HH=2 digit hour (military time)

SS=2 digit seconds

T= 1 digit tenths of a second

H= 1 digit hundredths of a second

K= 1 digit thousandths of a second

and the device figures out the gap in time (in milliseconds from) invoking a built-in function that sends two values of

YYMMDDHHMMSSTHK, called

YYMMDDHHMMSSTHK[present],

YYMMDDHHMMSSTHK[past]

and the device always assumes that

YYMMDDHHMMSSTHK[present] minus

YYMMDDHHMMSSTHK[past] = positive number difference

However, the above assumption will fail in 1/1/2000 if the past value is before 1/1/2000." [If my HTML doesn't work this time, oh well, Tomorrow is another day!]

-- Debbie Spence (dbspence@usa.net), January 27, 1999.


Old Git and WW and others on this thread - Thanks!

I had no reason to doubt my colleague at our electric company when he told me that they had Y2K issues with the oil pressure units on front end loaders but it bothered me that they couldn't just disconnect and reconnect the battery causing the date to default to the "epoch" or start date. (Just like all those stupid little clocks at home - Microwave, VCR, etc. reverting to 12:00 when the power goes out).

Internal capacitor - interesting. Potentially disturbing, but interesting. Thanks again.

Hey wisdom seekers, you should be checking out this thread.

-- Another Lurker (112277.2114@compuserve.com), January 28, 1999.


For another lurker,

The problem you raised about the disconnect and reconnect battery cables trick does have a possible solution. (* = "Insert standard legal mumbo-jumbo here".) When working with a device such as a Programmable Logic Controller (PLC) on the workbench, there are certain leads on the potential problems chips that can be momentarily jumpered together to discharge any internal capacitor (*).

That said, the principle does apply to larger systems. I have seen procedures in some auto repair manuals that instruct how to do a "forced reset" of automotive computers. After both battery cables are disconnected from the battery, they are clipped together with a low value, high wattage (say one ohm, 10 watt) resistor and allowed to drain out any internal chip voltage storage systems (*). Twenty-four hours time is the figure that comes to mind.

For an automotive application, this would erase all volatile memory (requires power to be retained) functions, such as time elapsed since chip creation date and any "self-tuning" data that the engine management systems store from the last time the vehicle ran. Not to mention things like maintenance logs, mileage and radio station presets. Non-volatile memory, such as PROMs containing the engine computer's operating program would be unaffected by such a procedure.

Now far be it from me to make a recommendation that your friend look into something like this (*), but it's worth keeping in mind as an action to take before 01/01/2000. But wouldn't he like to be his employer's "Y2K hero"?

As far as whether this type of procedure can be used in an industrial environment, it probably depends on how critcal inter-connected devices are over sharing exact time and dates. If a plant has nothing but stand-alone systems, then this type of procedure might be a lifesaver. If the plant has lots of connected systems, then resetting all those systems to their various creation dates could almost be as disasterous as Y2K. I say almost because at least with pre-Y2K dates, the systems won't have locked-up devices that may not be able to be reset and made functional again.

Now what I've discussed are items that have been mulled over around our lab at work for possible ways out of the Y2K mess. My employer is working pretty hard to get all of our problem-prone systems replaced. We've got some pretty old tools and test systems in our plant and I honestly don't think we can get replacements for all those that need replacing. But we do have a fighting chance to keep going, if we can at least get the systems to keep running. Let the auditors argue over handwritten dates and times on test print-outs instead of the computer generated ones.

And look for a lot of industries to come to the realization that these types of procedures might be the last, best hope for avoiding total failure of their systems.

And if your friend does suggest trying these at work, tell him good luck (*).

WW

-- Wildweasel (vtmldm@epix.net), January 28, 1999.


Moderation questions? read the FAQ