ATM Problems due to Y2K upgrade

greenspun.com : LUSENET : TimeBomb 2000 (Y2000) : One Thread

I know people have posted about every little ATM problem or screwed up phone bill or whatever. These folks usually take a "Hmmm, I wonder?" kind of attitude, and then the Pollies say "Not every screw-up is a Y2K screw-up, you screw-ball..." or words to that effect.

This kind of story reminds us that some of these screw-ups may indeed be, at least in a secondary sense, due to Y2K in some circumstances. Unfortunately, we may never know the full extent of this kind of thing. This example is, afterall, something that most people have contact with, a very "public" kind of failure. How many of these kinds of stories go unreported?

Y2K Repair Leaves Cash Machines on the Blink (Steve Woodward, The Oregonian) "A software glitch rattled up to 5 percent of the nation's automated teller machines for more than two weeks earlier this month. The glitch resulted in failed machines, slow transactions and incorrect bank balances for tens of thousands of ATM transactions.

The mistakes occurred after a little-known Texas company in late June finished installing a new computer system designed to process ATM transactions properly in the year 2000..."

The glitch affected as many as half of the 17,000 ATM terminals for which Affiliated Computer provides transaction processing. Those terminals are located largely in merchant locations, such as convenience stores, bowling alleys and fast-food restaurants...

All ATM processors were required by federal regulators to be Y2K-compliant by June 30, meaning that their computer systems had to be able to process dates in the year 2000 and beyond. ACS began installing a new operating system in January, upgrading an old system that was not able to properly process dates beyond 1999. The company had been moving terminals systematically onto the new system without incident, until it moved the last of the terminals on the last weekend in June.

At that point, some machines broke down, stalled or rejected cardholders' requests. Other cardholders received the proper amount of cash from the machine, but the bank accounts were wrong.

The problems were isolated to terminals that used a certain kind of device "driver" and high-speed, dial-up connections with ACS, said Joni Floyd, the executive vice president in charge of the project. The sudden influx of 17,000 ATM terminals, 3.5 million cardholders and a transaction rate of 20 million per month exposed weak points in the new software, Floyd said.

Connor said ACS had set up a testing laboratory with more than 15 types of ATM terminals but acknowledged that lab testing can't find every potential problem.

"Until you can touch every single terminal out there," he said, "you can't test every conceivable transaction out there..."

-- pshannon (pshannon@inch.com), July 28, 1999

Answers

Not same company, but same topic. Only connection that I know of so far is that ... and EDS are both based in Dallas.

WASHINGTON - May 26, 1999 (Reuters) - U.S. banking regulators Friday announced an agreement with a major regional electronic funds transfer network, TransAlliance L.P., to speed up its lagging Year 2000 preparations.

TransAlliance, which processes ATM and debit card transactions for nearly 800 banks, thrifts and credit unions in the western United States and Canada, committed to complete Y2K testing of the computer systems serving all of its customers by JUNE 30, the deadline set by Federal banking agencies for all financial institutions. The company processes around 42 million electronic transactions every month ...

The company agreed to provide U.S. banking agencies with reports on its strategy for meeting the deadline, its progress in doing so and its contingency plans for dealing with any problems that might occur after the millennium date change.

The agencies involved are the Federal Reserve, the Federal Deposit Insurance Corp., the National Credit Union Administration, the Office of the Comptroller of the Currency and the Office of Thrift Supervision ...

TransAlliance, a partnership between a group of banks located mainly in the U.S. Northwest and computer services giant Electronic Data Systems Corp. (NYSE:EDS - news), said it was committed to getting its Y2K preparations on track ...

''We are pleased that we could agree with the regulators on the steps we are taking to ensure Y2K readiness by June 30, 1999. We are confident we will meet this date,'' its president and chief executive officer, James Benson, said in a statement.

In terms of the agreement, if TransAlliance fails to meet the deadline, its customers will be free to break their contracts with it and switch to another service provider ...

The company had undergone a management shake-up and was fully focused on meeting the federal deadline, Merrick said.

''There's a new management team that's been put in place within the last three weeks,'' he said. ''They're taking the full responsibility for getting it fixed, and it's their number one priority.'' ... http://dailynews.yahoo.com/headlines/tc/story.html?s=v/nm/19... Was on Yahoo - but in archives now

-- Cheryl (Transplant@Oregon.com), July 28, 1999.


Thanks pshannon!

BTW, there is now a new TBY2K category for:

Computer/IT Y2K-Related Glitches (New)

As requested by several posters.

Diane

-- Diane J. Squire (sacredspaces@yahoo.com), July 28, 1999.


pshannon,

>then the Pollies say "Not every screw-up is a Y2K screw-up, you screw-ball..." or words to that effect.

Not *only* pollies say the first seven words ("Not every screw-up is a Y2K screw-up") of that response.

GIs who've done programming and analysis for ATMs and the networks to which they're connected say that part too.

After having investigated and solved hundreds of problems in ATMs, their programming, and the programming of networks which run ATMs for eleven years, I can authoritatively testify that not every ATM-related screw-up is a Y2k screw-up.

I think it's a disservice to genuinely-Y2k-related problem investigation to encourage the idea that a substantial proportion of ATM problems are causally related to Y2k. Only a small proportion of ATM problems are Y2k-caused.

>This kind of story reminds us that some of these screw-ups may indeed be, at least in a secondary sense, due to Y2K in some circumstances.

The screw-up described in the story above is basically a replacing-an-old-system-with-a-new-system type of screw-up. That the replacement was necessitated by the need for Y2k compliance is, judging by the details presented in the story, coincidental, not causal. There is nothing in the story to support the idea that any of the problems described are Y2k-related. I have witnessed each of the types of ATM problems mentioned above, and there are plenty of possible non-Y2k causes for each.

I would agree that insofar as Y2k considerations lead to a higher than usual rate of software changes or to a lower than usual thoroughness of software testing, then the overall rate of software problems could be higher now than it otherwise would have been. But in order to responsibly use this argument, one must also acknowledge that many companies have been spurred by Y2k considerations to make improvements that they otherwise would not have, to our long-run benefit. E.g., many companies have done a complete software inventory for the first time ever. Many have upgraded sooner than they otherwise would. Many organizations have prepared or updated emergency contingency plans to a level never before achieved. It hasn't been all bad.

My main message is: Let's not jump to categorize problems as Y2k-related. View them with the consciousness that Y2k is a potential cause, but not necessarily a probable one. Too much readiness to blame Y2k leads to discounting of genuine Y2k-caused problems.

There's already far too much Y2k-denial out there. Please don't feed that with overreadiness to attribute glitches to Y2k.

Some of you may see this message as indicating that I've defected to the Pollies, but to my mind what I'm doing lately is switching my emphasis to countering unwarranted overreaction, whereas earlier I put more emphasis on pointing out possibilities for potential Y2k problems.

>The glitch resulted in failed machines, slow transactions and incorrect bank balances for tens of thousands of ATM transactions.

I saw each of these types of problems long before there were any Y2k connections.

>The glitch affected as many as half of the 17,000 ATM terminals for which Affiliated Computer provides transaction processing.

This tells me that the glitch probably did not occur in a date processing module used in common by all types of terminals.

A competently-designed transaction processing system would separate terminal-specific modules from modules which could be used by all types of terminals. I would expect only a bare minimum of date processing to be in the terminal-specific modules.

>The company had been moving terminals systematically onto the new system without incident, until it moved the last of the terminals on the last weekend in June.

This is a _STRONG_ indicator that the problems were not Y2k-related. (Not an absolute guarantee, but a _STRONG_ indicator.)

>At that point, some machines broke down, stalled or rejected cardholders' requests. Other cardholders received the proper amount of cash from the machine, but the bank accounts were wrong.

Each of these types of problems could have many, many possible causes that had nothing to do with Y2k.

It's possible that a Y2k bug could cause each of these symptoms, but to put Y2k at the head of the suspect list is a big mistake.

>The problems were isolated to terminals that used a certain kind of device "driver" and high-speed, dial-up connections with ACS

This makes it virtually certain that the faults lay in terminal-specific program modules.

-- No Spam Please (nos_pam_please@hotmail.com), July 28, 1999.


No Spam, Thank you for bringing your technical expertise to this thread, I really appreciate it. When you say that "glitches" that don't occur in date processing modules are not Y2K "bugs", I think I do understand what you're saying. But, this gets into a whole gray area for me. It seems that many of the Y2K-related problems we hear about are not technically due to errors in date processing. Rather, they have been the result of pressing new systems/equipment into service, perhaps because older non-compliant equipment or software cannot be remediated or it is not cost effective to do so.

With large numbers of agencies and entities implementing new systems to meet compliance or readiness goals as the deadline draws near, it's logical we will hear more of these kinds of problems. A very small local example: A small answering service company merged with a larger one when it was realized that the expense of remediating both of their non-compliant systems was not feasible. Instead, they purchased new equipment. The new equipment has been plagued with NON-Y2K technical problems that have drastically affected their ability to provide adequate customer service. They have lost close to 25% of their customer base in less than 2 months, and I have serious doubts about their continued survival. No, not a true Y2k problem due to errors in date handling. But, if not for Y2K the two companies would not have merged, and they would not have purchased new systems with a need to beat a deadline. They may go bankrupt as a result. We're going to see (have seen)a lot of this. If we don't call it Y2K, what do we call it?

-- RUOK (RUOK@yesiam.com), July 28, 1999.


RUOK,

You're welcome. Thank you, in turn, for returning to a point which I didn't cover to my own satisfaction either:

>It seems that many of the Y2K-related problems we hear about are not technically due to errors in date processing. Rather, they have been the result of pressing new systems/equipment into service, perhaps because older non-compliant equipment or software cannot be remediated or it is not cost effective to do so.

Before Y2k was a common topic of conversation, putting new systems or equipment into service to replace older systems or equipment often (the more complicated the system, the more sure this was to happen) involved going through a period in which users had to endure various inconveniences or disruptions because the new system or equipment did not do what it was "supposed" to do or did not emulate functions of the old system or equipment in the way it was "supposed" to do.

My doctor's medical clinic recently changed its system for handling appointments and incoming patients. When I arrived there yesterday, signs saying things like "Please pardon the extra time needed for us to handle your paperwork while we get used to our new computer system" greeted me.

Now, suppose it were 1989 instead of 1999. (This is a reasonable supposition for purposes of example because medical clinics had computerized admissions procedures back then, too, and said similar things to their patients just after a new system was isstalled back then.) Then one would not expect (or even suspect!) that the change had been made for purposes of replacing a non-Y2k-compliant with a Y2k-compliant one. I think the average patient might have thought something along the lines of "Oh, another darn computer change fouling up things again. Take a deep breath and stay calm." (Others, raised in a different environment than I was, or less calm than I am as I write this, might have used more profane terminology.)

What is my point? That the basic nature of the interaction between people and computers remains the same today as it was ten years ago, and whenever a new computer system is unleashed upon the users, there will be various problems or at least delays while the operators get used to the new way of doing things. *This* is what I meant in my preceding posting when I wrote "The screw-up described in the story above is basically a replacing-an-old-system-with-a-new-system type of screw-up."

If one wants to indicate a connection to Y2k, I'd prefer something like, "This is basically a replacing-an-old-system-with-a-new-system type of screw-up that occurs now rather than next year because the replacement was necessitated by the Y2k deadline."

(Well, I wouldn't prefer that it come out that stilted or wordy.)

IOW, the problem is not Y2k-related but its *timing* is Y2k-related.

>With large numbers of agencies and entities implementing new systems to meet compliance or readiness goals as the deadline draws near, it's logical we will hear more of these kinds of problems.

... and sometimes it will be useful, for clarity of thought, to distinguish between (a) Y2k-related problems and (b) non-Y2k-related problems whose timing is Y2k-related.

>A very small local example: A small answering service company merged with a larger one < SNIP > The new equipment has been plagued with NON-Y2K technical problems < SNIP > No, not a true Y2k problem due to errors in date handling. But, if not for Y2K the two companies would not have merged, and they would not have purchased new systems with a need to beat a deadline.

So they have a non-Y2k problem whose timing is Y2k-related.

>They may go bankrupt as a result.

Yup. Y2k-related _timing_ of problems can be as deadly as the problems themselves independent of timing.

>We're going to see (have seen)a lot of this. If we don't call it Y2K, what do we call it?

Separate the problem from its timing. Attach Y2k to the appropriate part.

-- No Spam Please (nos_pam_please@hotmail.com), July 30, 1999.



Moderation questions? read the FAQ