Question: Has Anyone Here Actually Worked For A Competent Software House?

greenspun.com : LUSENET : TimeBomb 2000 (Y2000) : One Thread

My point is this:

Having worked as a programmer, analyst programmer and "consultant" during my 11-year illustrious career for, amongst others, "The World's Favourite Airline", I can honestly state the following:


So there we have it. An industry operated by computer "professionals", just like a government is run by political "professionals". Errm...

Is there anyone here who can point to a truly professional software house? One which does not take short-cuts at every opportunity simply to under-bid rivals or meet deadlines? One which by some miracle has managed to get a few dozen competent managers and programmers together under the same roof?

I believe I know what we're up against here and that's why I'm making preparations far away from it all. Over to you.

-- Y2KGardener (gardens@bigisland.net), September 02, 1999

Answers

This might surprise some on this board but I have worked for good orgs and (I'd like to believe) been an excellent programming manager as well. After all, computers .... do ... almost .... work! Which is rather amazing considering that much of what you post above is indeed true in many places.

BTW, we end users will be the Y2K impact testers too, eh?

-- BigDog (BigDog@duffer.com), September 02, 1999.


* * * 19990902 Thursday

Y2K Gardener:

Touche! My sentiments, exactly.

The only "structured"/"modular" application code I've _ever_ been successful at designing and/or coding, I've had to do on my OWN time, out of eye-shot of the "managers" and customers. That is, I'd do the rough design and pseudocode outside of the office or when no one else was around.

By developing an app incrementally in modules, I was able to find some eager ( usually young ) user willing to try to "break" {TEST} the app concurrently during the development process. This, I've found is an excellent strategy for delivering production applications that are virtually "bullet proof", "fool proof," and extremely robust. The payoff for the client is easy to maintain and enhance software applications. They love it!!

By utilizing module library management tools, re-using drop-dead proven code is a sure bet to quicker and robust development of enhanced or new applications.

Management is happy with the results, they simply do not want to acknowledge OR PAY FOR the integrity of these methodologies.

*sigh*

What to do?! Get out on your own and insist on implementing good, proven processes; then, stick to it!!

Thanks for illuminating the muddled reality of most software development environments.

Regards, Bob Mangus

* * *

-- Robert Mangus (rmangus1@yahoo.com), September 02, 1999.


I'm with you Gardener, I've been programming for 18 years, and here are *my* experiences:

The number of truly competent programmers I have worked with can be counted on the fingers of one hand.

The number of truly competent managers I have encountered is precisely one. And he got fired.

Most programmers should not be left unattended with a keyboard - systems would be better off if they just stayed in bed. On maintenance, this is true, but I worked on software development - we HAD to code.

"Structured program? What's that?" Most programmers couldn't write a well-structured program if their lives depended on it. Yep.

This is how it works in real life: (i) Key in a humungous lump of code and brag about how many lines per day you are capable of coding; (ii) run said piece of code; (iii) repeat step (ii) after adding extra flags, counters and temporary variables until it looks good on the screen. Also make sure you use obscure variable names, such as "p" for a rectangle, and "s" for a point.

"Testing? What's that?". I have never, ever encountered a good and sufficient test plan. Have you? Yes I have - but I had to write it. For a 2 million dollar project, I asked for 8 full time testers (on two platforms), and 30 "beta testers". I got 1.25 testers (2 people part time each), and 5 beta testers. I asked for 1200 man-hours of testing (not including beta testers), I got 185. I asked for 26 different machine variations, I got 6.

Most programmers regard a single run of a program as being a sufficient test. Some are satisfied upon encountering no compilation errors. True, but sometimes there are so few programmers on a project they do not have time to run the program more than once.

"Managers" are in many cases simply failed programmers who have got lucky. [or are politically more astute]. They have not got a clue about the systems they are supposed to be overseeing. "Specs" are (re) written upon trying out the software for the first time. They also do not know the difference between a "requirements definition", "requirements specification", "functional specificatioun", "testing specification", etc. etc.

Manager: "Hey, programmer, has this been tested?" Programmer: "Yup". Manager: Tick.

Manager: "Hmmmm, it's 7:15, who is still sitting in their cube? Gee, Jolly isn't in his cube, he must be a piker." Tick.

The fact is, most software is tested by the end users after it has gone live. Ever tried version 1 of an MS product? Ever been the first to use a new software package or upgrade?

One other thread: Object Oriented programming has made the mess WORSE not better! It takes much longer to get up to speed because of gratuitous complexity in most Object Oriented (OO) programs. On one project I worked on, there was a program written for Macintosh and for Windows 95. The Windows program had 12 engineers for all aspects of its programming, the Macintosh had 2. The Windows program weighed in at 1.25 Meg; the Macintosh program was 325K. The Macintosh program had several more features, and we had to remove features to make the Windows version not be perceived as too far behind.

The Macintosh used a small, tight OO framework. The Windows systems used MFC. This is not a slam against Windows, all things being equal, the difference was the absurd complexity of the MFC, and the extra overhead necessary to deal with it.

'nuff said.

Jolly

-- Jollyprez (jolly@prez.com), September 02, 1999.


Gardener - you've never worked with moi :)

-- Andy (2000EOD@prodigy.net), September 02, 1999.

In my 15+ years of programming I've come across maybe five really good programmers (two I met 'off the job' at user groups). I've had only three good managers, and believe it or not, one was a lawyer (head of the legal dept. at a big brokerage house). He stayed out of my way, gave me an unlimited budget (no joke!) and backed me up whenever users made unrealistic demands. Come to think of it, the other two 'good' managers did basically the same thing. The remaining managers I've dealt with (either working for directly or because they were part of a larger project) were complete screw-ups and made Dilberts boss look like a member of Mensa.

Overall, most programmers are just nine-to-fivers. _VERY FEW_ actually care about what they are doing. I can't tell you how many times I've looked at someone else's code and said to myself "This person actually got PAID to write this crap?!?" I can think of only one or two times when I took over someone else's system that I *didn't* have re-write the whole damn thing from scratch.

-TECH32-

-- TECH32 (TECH32@NOMAIL.COM), September 02, 1999.



LOL

Sounds like a bunch of frustrated programmers here. And Y2K Gardener sounds so much like Ed Yourdon its eerie.

There are plenty of competent programmers out there. Doesn't sound like any of them are here though.

Competence in managers is not necessarily based on technical expertise. It takes good people skills to be a competent manager. Most of the best technical people I've ever known would make lousy managers.

Structured programming has evolved. Sounds like you need to take some training.

Lines of code? I've never met a single programmer who brags about writing lots of lines of code. Most I know might brag about how few lines it took to solve a problem.

All programs must be tested. Just because there isn't an anal-retentive narrative testing plan doesn't mean the darn thing doesn't work.

Have you ever solved a problem with the resources given to you, or do you just bitch about lack of resources until they cancel the project? You can't make any money if you can't under-bid your rivals.

Nothing's perfect, and if we insist that it should be we'd never get anything done.

Before you light your flame thrower, ponder these two points:

1. You've just insulted an entire profession.

2. I don't give a flying frig about coming back to this thread to argue the point further.

-- Buddy (buddydc@go.com), September 02, 1999.


(1) Competent software house? I've never seen one that I would give that title to... I've seen some real disasters, though!

(2) Outstanding manager. Yes, I worked for one (but it was in the technical training department!). I've had a few good ones (very few).

(3) Outstanding programmers? I define them as people I would hire and trust if I were in a crunch situation, and include those I trained (several universities and industrial training). Maybe a dozen (but that could be charitably high) all told. That's over a third of a century...

-- Mad Monk (madmonk@hawaiian.net), September 02, 1999.


Ever want to find a really lousy programmer? All you have to do is dig up something you yourself wrote oh, 3 or 5 years back, and make a major modification to it.

And you find that your nice clear documentation might have meant sense to you the day you wrote it, but no longer. And your nice logical structure fights any enhancement tooth and nail. And when you start going through the code line by line, you wonder what the *hell* might have been going through your mind to write this stuff.

Yes, yes, I know this doesn't apply to anyone here. Unless you've had to do this, and you're honest, of course.

-- Flint (flintc@mindspring.com), September 02, 1999.


Gawd,

Its like finding out how sausages are made.

-- R (Riversoma@aol.com), September 02, 1999.


R,

Otto von Bismarck, the nineteenth century German chancellor, had a quote like, "The people rest more soundly at night not knowing how their sausages and laws are made." To that we now need to add software.

-- Mr. Adequate (mr@adequate.com), September 02, 1999.



Bob Mangus - I fully agree: the few projects which are well written are those which programmers do "off their own backs" after hours with no time constraints. In my case, many of the most useful and popular features of the systems I worked on were created by myself in my own time, only being introduced when complete. Even then I had to fight unimpressed managers in order to incorporate these features - it was the satisfied customers that I listened to.

Jolly - Agreed, there are a lot of noble specifications, test plans and so on out there... but just try to get the necessary time/manpower. As for the OO fad, I completely agree: just before I left my former employer they went heavily into it. Result: software output was nil for at least 6 months while programmers fiddled with their objects, properties and inheritances and wrote pages of code just to add A to B or read a record.

Andy - Well, we all know about your programming prowess: I should have mentioned that in the question!

TECH32 - That's the worst part of the job, taking over another programmer's "steaming pile of dung" and having to make sense of it. More often than not, the "steaming pile of dung" is the only (non- functioning) "spec".

Buddy - I'm truly flattered after being compared to Ed Yourdon. Thanks! As for insulting an entire profession, I'm just calling it as it is from my viewpoint. If you read the question, I did ask if anyone had had positive experiences - so far there's not been much. Instead of using the phrase "people skills" for managers, I would suggest the alternative of "bullshit*ing" skills". In my experience the most common services carried out by managers is explaining to clients (a) why they are being charged $X0,000 more than the quote; (b) how come the system is Y weeks late; and (c) explaining to programmers why design flaws are in fact programming errors. Regarding structured programming - by 'evolved' do you mean OO languages? If so, 'devolved' sounds nearer the mark. You can't make any money if you can't under-bid your rivals. That's precisely the point I'm making: the software industry is a profit-above-anything-else industry, just like most others. Quality comes in at a poor 2nd or 3rd. Nothing's perfect, and if we insist that it should be we'd never get anything done. Precisely my point also: like so many you don't seem to realize that computer systems *have* to be pretty near perfect to work. The computer ain't going to second-guess what you really wanted to say. Y2K remediators are probably "getting a lot done" according to your definition... but we will soon see what the quality of that work was. I'm not optimistic.

Flint - Sorry, I'd have to disagree. I don't want to cast aspersions about your own programming skills, whatever they may be, but I for one am more than happy to pick up programs I wrote 2, 5 or 10 years ago be they 100, 1000 or 10000 lines and make amendments. My former colleagues were equally happy to do so.

R & Mr. Adequate - Precisely. I haven't eaten a sausage in 18 years.

-- Y2KGardener (gardens@bigisland.net), September 02, 1999.

I couldn't believe it when I landed that first job as a second shift computer operator, while I was still in high school, 31 years ago. I was actually getting paid to play with this stuff! Sometimes I still feel the same way, I can't believe that I get paid for my hobby.

This business has been good to me. I have a fair amount "in the bank". I live in a pretty nice place. I have met people that have become life long friends.

There are those of us out here that do care. I spend extra time testing. I spend extra time commenting the code. Sure, my boss knows that I could get it done quicker, but he also knows that the extra time will result in a better product.

OK, enough tooting my own horn. Is the computer business any different than any other business? How many people don't care, do 9-5, and as long as the check gets cashed every Friday, all is well with the world?

Sadly, I agree with much of what you say.

Tick... Tock... <:00=

-- Sysman (y2kboard@yahoo.com), September 02, 1999.


And one other thing. 30 years ago, when computers were in glass rooms, before every Tom Disk and Harry started programming in the latest greatest language, I think the "quality" was much better than today. Just my $.02 <:)=

-- Sysman (y2kboard@yahoo.com), September 03, 1999.

To be fair, although I've also seen a decline in quality over my 25 years of programming, it's mostly due to market forces.

The fact that Netscape, for example, can throw together a buggy browser and give it away, and become a many-billion dollar company in the process proves that demand far outweighs supply. In a market like that, you would expect quality to go to hell. If crap sells, why spend the time on quality?

Many programmers are also essentially self-taught. With no standards or discipline from their education, and none enforced by their peers or their employers, where would it come from? Programmers are pretty idealistic as a group, and like to think of themselves as producing good products, but it's a rare person who will take the time to polish something when his customers, managers and friends just don't care.

But these problems are eventually self-correcting. Either large software projects get such a bad reputation that they stop being created, or else something like Y2K comes along to kill off a lot of demand. In the smaller software business we will have in a year or two, ability to deliver a quality product may finally become an required skill.

And as for personal experience, I have worked with lots of well-intentioned programmers, many of whom could at least get a program to run reliably. Style or structure is rarely a consideration though. And some organizations do test, just never enough (you can't really test out a lot of coding errors anyway.) And I've had a few good managers, although I continue to think that as a group they are just overpaid administrators and baby-sitters.

As for Microsoft V1.0 code, I have to agree with you there. They must test the stuff somewhat, or else it wouldn't work at all, but I'm appalled at what gets through! I don't know how many times I've been reduced to screaming "Microsoft, burn in Hell!" at my computer after trying to use some of their stuff....

-- You Know... (notme@nothere.com), September 03, 1999.


Flint: LOL! Yep, it's true. If major modifications are needed then every module seems to need a re-write. Why is that? (Murphy's law)

The reason 3 year old code looks so bad (hopefully) is because I'm that much better now versus then.

All forms of business work on a "cut every corner" basis where ever they can. That's what's going to make the next 12 months so interesting....

-- James D (hunt122@hotmail.com), September 03, 1999.



All the above is true for sure. Whats even worse is when all the "talented" programmers (some of whom have been at a company for 15+ yrs designing systems from scratch) are told that "they take too long" to produce a finished product to the customers, henceforth - all code will be COTS and YOU gotta customize/maintain it!!

Naturally, all the ones worth their salt show management their "significant digit" and book for greener pastures, leaving the duds and "soon-to-retires" to handle the load. So what happens then? Can you say "Consultants"? (at 3x recently departed programmer cost).

Of course, sitting in on the meetings where the programming section bosses outdo each other by promising the most absurd due dates for projects just to look good to CEO is memorable too.

-- JCL Jockey (WeThrive@OnStress.com), September 03, 1999.


Flint,

I hear ya, but agree with James, we get better with age. OTOH, I still use "code" that I wrote 25 years ago. My LOG macro for example.

A one-liner to send messages to, and get answers back from, the operator. This is much easier than the DTF for the console, OPEN, CLOSE, GET, PUT logic that VSE (DOS at the time) requires. Sure, I've "updated" it over the years, but haven't touched it in at least 10 years, and the guts are still "same as it ever was".

I've collected quite a "library" over the decades. Not quite OO, but you get the idea.

Tick... Tock... <:00=

-- Sysman (y2kboard@yahoo.com), September 03, 1999.


S/W dev can be reduced to 3 constraints: good (high quality), fast (short delivery time), cheap (low cost).

You can pick any 2.

Realistically, you can only optimize for 1 of those.

I'm not certain for which constraint most of the Y2k remediation effort has been optimized, but I'd venture a guess it's not for the "good".

-- Nathan (nospam@all.com), September 03, 1999.


I remember in one of my courses (data structures? been a while now) I made a radical suggestion, since I was older and had been around the block a couple times. Seems we wrote some program (in Pascal, C wasn't invented then) for the assignment one week, and for the next week we were instructed to add an enhancement. Fine. For the next week, we were asked to add another enhancement. Not trivial enhancements, either.

So I suggested that rather than enhance our own code, we all hand our code to the person next to us in a big circle, and do the enhancement to *their* code. That's more like the real world. And nobody could do it in a week!

But we learned a whole lot about documentation, and meaningful variable names, and structure. I think the exercise made us all better programmers, but I sure didn't make any friends that way.

Learning to program well is a slow process even if that's your goal. And in the real world, the goal is to meet deadlines and good programming happens by accident if at all. Anyone who claims that they've *always* been a great programmer is kidding someone, probably themselves. Yes, some people grasp it more quickly and easily, and some are just terrible, but everyone can improve. A lot.

-- Flint (flintc@mindspring.com), September 03, 1999.


Good point Flint.

So why does the human race have a gazillion languages?

Why does modern s/w have the same?

I am constantly asked by nieces and nephews which "languages" to study...

Read back to prehistory - develop a feel for what "those" that didn't want development to achieve, achieved...

I have high hopes for you Flint... one day...

-- Andy (2000EOD@prodigy.net), September 03, 1999.


IT is a young technology. Some mistakes have been recognised in the last decade or two. Many others (probably!) haven't been, yet. This explains a lot, including the way that if a programmer is honest with himself, anything he wrote a decade ago sucks. (And probably still is useful, and probably needs maintenance from time to time, and irritatingly it will almost always be more attractive to keep the old crock alive than start over!)

Think back as far as you can, to servicing of still older mechanical equipment. Did you not get very irritated, finding that two or three different sets of all-but indistinguishable screw threads and spanner sizes were in use, rather than today's almost-universal "M" series? Well, that is the end of a long journey. Decades earlier, there were probably a dozen screw thread familiies. At the dawn of the industrial revolution, every works made its own screws, all incompatible. It took well over a century for this to be recognised as a mistake and finally(?) rectified.

Or think about this history of bridge-building, which is littered with heroic and disastrous failures. Without willingness to push at the limits, we'd still be stuck with stone arches spanning maybe 150ft at best, and rope suspension bridges that you could cross only on foot.

As for managers, I can't agree more. They are the ones who choose cheap over well-engineered every time, who defer maintenance until too late, who "downsize" or "outsource" all their competent staff out of a job, who refuse to listen to warnings that "it won't fly" until megabucks are squandered and indeed it doesn't fly, whose only skill is finding someone else to blame... If Y2K is anyone's fault, it's the managers'.

I am of course generalizing. There are some good managers out there. Just not enough of them!

-- Nigel Arnot (nra@maxwell.ph.kcl.ac.uk), September 03, 1999.


There is a push-pull between programming as artistic endeavor and as engineering discipline that has never been synchronized, though the best programmers do it as a matter of course without even thinking about it.

"In the large" (huge organizations), the goal has to be engineering, because it offers some hope (however faint) of developing a repeatable process that can be estimated, tested and quality-checked. However, "quality" seems to be found "in the small" and that same engineering approach (think CASE -- computer-aided software engineering and, to a lesser degree, OO) proves too blunt an instrument.

One of the reasons that "application environments" like SAP (packaged software in general) prove attractive to large organizations is that they promise a certain degree of quality (modest, IMO) if you surrender entirely to their constraints-world view about design- implementation. They also promise mitigation against the semi- competence of managers and programmers. (Again, OO as well but more broadly than an application environment). But once you hit the wall because the real-world requirements break the model, you are in deep doo-doo.

Very few programmers and fewer managers can work efficiently outside the wall. This is one of many reasons why FOF for Y2K is alarming, especially for mainframes and especially for operating system breakdowns or breakdowns that are at the somtimes fuzzy barrier between OS and app. There just aren't enough competent guys around to man the barricades. And some of them are GIs heading for the hills ...

-- BigDog (BigDog@duffer.com), September 03, 1999.


Y2k Gardener:

The question you raised is interesting, yet the conclusions you draw tend to lump all programmers into one group. Some of the large consulting firms tend to hire new college grads and bill them out at exorbitant fees, counting on the prestigious name of the firm to "sell them." These folks see a VERY small portion of the pie offered to the large firms for their work. I really don't know much about their "screening" process, as I tend to stay away from them. They typically hire folks to be employees, but occasionally will hire independents and pass them off as employees to clients.

Other firms may/may not "screen" their folks well. I've sometimes had four different telephone technicals from a firm before they'd submit me to a client. I've also GIVEN telephone technicals for firms that are looking for others. From a programmer's standpoint, I MUCH prefer to go to a client through a firm that screened me well versus one that simply looked at my resume.

It seems to me that the same work ethics that apply to other professions are inherent in programmers. Some were raised to work an hour for an hour's work and some have quite another attitude entirely. I think it was on this forum that someone mentioned a supervisor comparing a programmer to their husband who was a truck mechanic. The poster seemed to feel this was an invalid comparison, but if you're a damn good truck mechanic with strong work ethics, I'd equate that to a damn good programmer with strong work ethics. I UNDERSTAND the complexity of programming. I've done it all my life. I have NO CLUE how much is involved with truck maintenance, but to the inexperienced, EVERYTHING seems complex.

Regarding the post wherein the code you're working on is shoddy, be very careful to whom you express this. I worked once at a small firm doing contract work for a grocery-supply company nearby. I was asked to make changes to a program and made the mistake of saying aloud, "Who wrote this piece of..."...just to find out that the programmer beside me, (named project manager before I came aboard) had written it. Actually he had NO PROBLEMS with my rewriting it once he realized that I had eons more experience than he.

Bigdog:

Do you actually KNOW any competent programmers "heading for the hills?" I don't. I've even seen Scott Olmstead backing off on his previous predictions.

-- Anita (spoonera@msn.com), September 03, 1999.


Y2K Gardener is not me, and I am not he ... but who knows? Maybe we're identical twins separarated at birth, just like that movie played by Danny DeVito and Arnold Schwarzenegger...

One point to keep in mind about this discussion: there's no reason why anyone should believe that all programmers are created equal, in terms of their abilities. If you rounded up a thousand random adults, how many of them could play baseball well enough to survive on a major- league team? One of our problems is that almost anyone can call himself/herself a "major league programmer", and then go to work in companies that are either unwilling or incapable of measuring them to see whether or not they can perform like Sammy Sosa.

A long time ago (March 1968), in a galaxy far, far away, the Communications of the ACM (the monthly journal of premiere professional computer society in the U.S., which few programmers belong to) published an article (the authors were Sackman, Erickson, and Grant, for those of you who want to track it down) in which they gave a bunch of experienced programmers the same task to design, code, and test. What they found was a 25:1 ratio between the best and worst in almost all relevant areas -- e.g., how long it took to finish the assignment, how efficient the resulting programs were, and how many bugs the programs contained. As I recall, they found that the best programmers were best in ALL categories -- but aside from that, the other significant finding was that there was no significant correlation between years of experince in the field, and actual ability to write a decent program.

Even if an organization has a reasonable "filtering" process in its recruiting and interviewing procedures, there is likely to be a 10:1 ratio between the best and worst, especially in a large organization where you're going to get some statistical variety. In a small organization (e.g., the Silicon Valley startup), the interviewing process tends to be much more haphazard, and less likely to distinguish between real technical talent, and the ability to "BS" about one's alleged talent.

Given this enormous variety of so-called talent, success or failure in the programming organization is going to depend a LOT on the managers, and the procedures/processes/methodologies they employ. By analogy: if you have a small team like the Chicago Bulls prior to Jordan's retirement, then it might be possible for them to win even if there is no coach to organize their efforts (but notice how important everyone felt Phil Jackson was, and how many of the Bulls' problems were blamed on top management). But if you have a thousand random individuals assigned to task of harvesting potatoes from an acre of land, they're not going to do a very good job is there is inadequate management to assign tasks, monitor performance, test for undiscovered potatoes, etc.

So that leads us to a different question: how competent, sophisticated, and "mature" are the software development organizations themselves -- by which we mean primarily the management, not the technical geeks? There are a lot of ways to measure this, and an enormous amount of controversy surrounding it -- but for purposes of this discussion, one credible way of looking at it is the Software Engineering Institute's (SEI) "Capability Maturity Model" (CMM). The SEI-CMM has five levels, ranging from 1 (in which anarchy reigns) to 5 (in which a formal software process has been institutionalized, the process is carefully measured on a project-by-project basis), and the measurements are used to continuously improve the process).

As of a couple years ago, approx 75% of US software organizations were at level 1 on this scale; about 10% are at level 2 (sometimes known as the "tribal folklore" level, in which a process for software development does exist, but has never been written down. In such organizations, new programmers have to learn the organization's process by osmosis -- i.e., sitting beside a senior programmer and watching what he/she does). About 8-9% were at level 3 (a defined, documented, institutionalized process exists), which leaves 1-2% for level 4 and 5.

Examples of organizations at the very top of the scale are Motorola's software development organization in Bangalore, India; most of the software groups within Hewlett-Packard; the software group at NASA's space shuttle operation; etc. I think there are now about five level-5 software shops in the U.S.

A high level on the SEI-CMM does not guarantee success, but certainly improves the odds; by analogy, a baseball player with a high batting average is not guaranteed to hit a home run next time he's up, but the odds are better. With most Y2K projects, a level-4 or level-5 team would be highly assured of success; and unless the size and complexity of the project was overwhelming, I would feel fairly confident of a level-3 team succeeding.

With a level-2 team, things get trickier. Generally speaking, a level- 2 team has a statistically predictable chance of success as long as it continues working on the same kind of project that it worked on before. But if it tries to jump from a batch-programming environment to a client-server environment, or from traditional application development to Internet/Web development -- well, then the existing tribal folklore probably doesn't work, and the team may fall flat on its face. To the extent that a Y2K project is substantially different than whatever the level-2 project team had been doing before, there may be difficulties - - by which I mean that the project team might miss its schedule, budget, and resource estimates by a wide margin.

Level-1 teams, by their nature, are "random" when it comes to predicting their outcome. Sometimes they succeed spectactualarly, sometimes they fail miserably; it's like Russian roulette. A significant point is that if the project is a small one, and IF the team recognizes that it's getting behind schedule, then it can often overcome its problems by brute force -- i.e., by going into a "death march" activity of heavy overtime, etc. But that doesn't usually work on large projects.

All of this explains why I've maintained my "deja vu" attitude about the success of Y2K projects. For 8-10% of the software organizations, it will be a piece of cake -- because they operate at a level 3, 4, or 5 on the SEI-CMM scale. For the 15% who are at level-2, it will be difficult and risky (because Y2K is probably different enough from their traditional projects to require some changes in process), but certainly not impossible. Meanwhile, though the bottom 75% who are floundering along at level 1 are going to have serious problems; they may well think that they're 99% done, right up until the deadline of Jan 1, 2000 ... but they really don't know.

Ed

P.S. re your comment about Microsoft: many outsiders strongly believe that MS is at level-1 on the SEI-CMM scale. But given the intensity of their culture, you could make a plausible argument that, at the very least, they are at level 2 -- i.e., there IS "the Microsoft way" of writing software. Indeed, there have been several books written about Microsoft's development process; but it's not entirely clear whether that process has been formalized and institutionalized within the organization, so it's hard to say whether it qualifies for level-3. A few years ago, I had a chance to ask several Microsofties in Redmond about all of this, and most of them had never heard of SEI-CMM. The managers, on the other hand, were generally aware of SEI-CMM, and many of them argued that Microsoft is at level-5. As far as I know, the company has not been formally assessed by the SEI.

-- Ed Yourdon (HumptyDumptyY2K@yourdon.com), September 03, 1999.


Ed:

All your statistics aside, proper screening COMBINED with work ethics are the traits that will make/break Y2k. Work ethics can be reviewed via references. I have no doubt that my references include things like "She's a "heads-down" programmer, who will give you eight hours work for eight hours' pay."

I ALSO remember coming in one weekend to finish up some programs of which I was responsible. I had the flu that weekend, and my project leader knew it. I finished 2 of the 3 programs and HE stepped up to the plate and finished the third so that I could go home and get some sleep. It's called TEAMWORK, Ed.

-- Anita (spoonera@msn.com), September 03, 1999.


As a programmer with nearly 20 years experience, I must unfortunately agree that the problems you point out are all too frequently true.

The reasons are many but perhaps the biggest source of the problems we see are caused by the fact that the information sciences are in their infancy. The architecting of complex digital information systems is less than a hundred years old and there is a tremendous amount we simply don't know. To a great extent we are true pioneers and colllectively, we are going places that our predecessors couldn't even have dreamed of.

But this science is new and, like any new science, there will be failures. Even 6,000 years of experience building bridges could not stop the Tacoma Narrows Bridge from collapsing due mostly to an incomplete understanding of the forces of nature.

Wait until we've been designing software for 10,000 years. Then we'll have something to brag about.

All new sciences have their failures. Many people died in the early days of aviation due to an incomplete understanding of aerodynamics. Even today, our understanding is not as complete as it should be. But does this imply that we shouldn't have ever attempted 'heavier-than- air' flight? I think not.

It is also true that many problems can be caused by an unwillingness to apply what little we do know. A great many air crashes can be attributed to pilot error, specifically poor judgement calls and an unwillingness to 'go by the book'. So it also is with the design and implementation of software systems.

Lest the non-technical participants here think that incompentence is universal in our trade, I would like to point out that some organization do a very fine job of applying what we do know about the design of information systems.

In my own personal work experience, General Electric Medical Systems Division stands out as an organization that treated the design and implementation of software systems with great care.

I haven't worked for them for several years and really can't speak to the situation there today but at the time I did work for them, they were extraordinary with their attention to detail and their application of software design methodology. I would not hestitate to undergo a medical diagnostic procedure with their equipment. I have a deep respect for many of the developers and analysts I met there.

Unfortunately, these types of organizations tend to be the exception. The primary reason is financial. Properly designing and implementing complex information systems is an extraordinary expensive and labor intensive task. So corners are often cut. Sometimes, cutting those corners makes sense. At other times those decisions can be disasterous.

The science is moving forward. People like Ed Yourdon, C.J. Date and many others have helped to develop methodologies and modeling approaches that, if followed, can improve our chances of successfully implementing complex information systems. In the future, many others will build upon the work that they have done. Still, the science is in its infancy and accidents will happen...

Arnie

-- Arnie Rimmer (Arnie_Rimmer@usa.net), September 03, 1999.


Y2K Gardener

I have worked for a couple of professional software houses (by your definition, which I agree with). Out of about 25 companies that I've worked for, that's not a very high percentage. Whether or not this is coincidence, both of the good ones were very small.

I agree with the rest of your comments. Anita

Do you actually KNOW any competent programmers "heading for the hills?" I don't. I've even seen Scott Olmstead backing off on his previous predictions.

I have moved from Dallas to a house in the country directly because of Y2K. As to whether I'm a competent programmer, my employer and a number of others seem to think I am; at least, they're willing to pay a lot of money to have me work for them.

-- Steve Heller (stheller@koyote.com), September 03, 1999.


Reference Buddy's comment about "insulting an entire profession", I guess he's REALLY annoyed at Gerald Weinberg:

"If builders built buildings the way programmers write programs, then the first woodpecker to come along would destroy civilization." -- Weinberg's Second Law

Programming is currently a craft and an art which may someday become an engineering discipline. One hopes that a large number of people don't die in the software equivalent of a "construction failure" in the meantime.

-- Mac (sneak@lurk.hid), September 03, 1999.


Competent? Yes. If what is meant by competent is that software is created and maintained that mostly does what is required to keep the business afloat.

The capabilities and motivations of programmers and the people who manage them probably vary about as much as any those any of other occupational group you'd care to consider.

Is there room for improvement? Obviously. But the same can probably be said of accountants, advertising copy-writers, doctors, auto mechanics, teachers, carpenters, police officers, Senators, and so on.

There are, no doubt, some truly disfunctional programming shops. But I've never had the misfortune to work in one. My experience covers over fifteen years during which I've worked for a Fortune 500 corporation, a city government, and a public educational institution. By and large, I've found that my co-workers do care about their work, are quite often frustrated by various management policies and decisions, have neither state-of-the-art nor stone age resources at their disposal, and generally succeed in getting the job done.

Let's not forget that computer programming is still a relatively youthful field. People have been building bridges a lot longer than they've been writing accounts receivable systems, and there was a time when a significant percentage of newly built bridges would collapse. Perhaps Y2K will be an important factor hastening our maturity.

As has been pointed out, Y2K is not a technical problem so much as a social one. Many programmers have been working diligently, and occasionally with considerable ingenuity, to solve Y2K technical problems. I expect that many software systems that have undergone remediation will fail in various ways, as is the case with most software projects; however, the worst residual bugs will eventually be rooted out and fixed.

The most serious effects of Y2K will, I believe, be felt in cases where there has been a lack of understanding on the part of top management as to (1) the extent of the effort required to fix these technical problems, and (2) the degree to which enterprises are interdependent.

Both kinds of failures can be attributed to human beings, but there should also be, I think, a recognition that people are increasingly having to deal with greater complexity in the world. More is happening today, and at a faster pace, than ever before. To cope, we tend to specialize, and in doing so we lose some of our ability to see the big picture.

To avoid future problems like Y2K, and some that will dwarf Y2K in terms of their seriousness and complexity (e.g., the eventual death of our biosphere), we need to be able to create better institutions on a macro scale, somehow incorporating wisdom that so far mainly seems to manifest on a micro scale.

I could go on and on with this, but I guess the point I'm trying to make is that we need to resist the impulse to assign blame to scapegoats. There's more than enough blame for everyone, and we will need to work together constructively to keep human civilization from being a brief, interesting, but ultimately unsuccessful experimental episode in the history of the universe.

-- Peter Harlan (pharlan@efn.org), September 03, 1999.


A wondeful clear and relevant discussion.Good to see professionals talking so. As an interested observer (physician) all this is not reassuring. Too bad this thread is buried in all that other less than useful information. It also explains why my office software which was promised for june 1998 is still in testing.I am now making appointments for the year 2000 in a book in handwriting.And I plan to keep doing so even when my software is fixed. If my little program is in trouble-how many more are also?

-- Jan Czarnecki (janczar@air.on.ca), September 03, 1999.

Moderation questions? read the FAQ