Why software projects are always late (part 1)

Software projects are always late because they are not easy, and because people who should know better do stupid things.

It’s a question I’ve been asked many times over the past two decades and, let me tell you, quite often the question skates uncomfortably close to an accusation.

Well folks, after one frowning face too many it’s time to take a good hard look at some of the many reasons why projects go down the drain.

Projects in trouble tend to stay in trouble

Let’s start with something the software industry has known for almost forty years. If a project is in trouble, throwing money and people at it will almost always make things worse.

“Adding manpower to a late software project makes it later”

— Fred Brooks

It doesn’t matter how many times a project team is exhorted to redouble their efforts and work smarter, it doesn’t matter how many carrots are dangled or sticks brandished, it doesn’t matter how many incentives are waved around or (more commonly) how many pizzas are ordered for working late into the night, the basic facts about project deadlines and project scope won’t be changed by adding more people.

The one thing that does change when new people arrive is project complexity – it goes up. Apart from anything else, the number of lines of communication is multiplied. It’s hard enough getting ideas clear with one person. Adding a horde of newbies to a stressed out team won’t make that any easier, not even a little bit.

Of course, sometimes replacing a burned-out team of developers after they’ve been worked into a coma will temporarily increase velocity, but good luck sustaining that approach.

Unhelpful Metaphors

Software projects suffer from poor metaphors. Actually let me rephrase that. The metaphors are just bloody awful. Developing software is not like building a brick wall, you can’t count the bricks and extrapolate a finish date. It isn’t a factory situation.

If you think it is a factory situation, it leads you eventually to wonder why all the programmers can’t be replaced with code-generating machines. Don’t laugh, this problem is still taken seriously.

Nor is it a house, a sudoku puzzle, or a chicken dinner. It’s software, it’s code, and there’s nothing quite like it.

Perhaps software development is more like craft, and programmers should be more like craftsmen (proud of their work, taking time to do it right) but while this metaphor is more appealing than a brick wall, it’s still imperfect. For instance, everyone agrees it’s a poor craftsman who blames his tools, yet no sane programmer will spend much energy defending the monstrosity that was Visual SourceSafe. Take away the Visual Studio IDE and the productivity of many good programmers would fall through the floor. Apart from a few notable exceptions, programmers need the best tools.

Gardening is probably my favourite metaphor, with programmers as gardeners who tend the code base, who pull weeds (refactor and debug), plant seeds (add features), and train the growing plants (refine features as they get feedback). This metaphor works because plants are unpredictable, just like software requirements

But gardeners are lucky – they have hundreds of years of research and experimentation to build on. Programmers have a few decades of hackers mucking about pretending to have everything under control – our state of the art is Agile, which let’s face it is a blatant acknowledgement that things always go wrong, that things take longer than expected, and that no one really knows what they want from software.

Perhaps Donald Knuth has the right metaphor, perhaps programming is indeed Art of some kind. True, Art is not the impression you get reading his four (five?) mighty volumes, but I’m too humble to argue with the professor.

Poor management

Yes, programmers, this was probably the first thing you thought of, right? Management doesn’t understand programming, management interferes, management makes all the worst decisions and we would all be better of if they just left us the hell alone. Yes?

The realists know that someone has to bridge the divide between development and sales, and unless you’re in a tiny development shop, that divide is going to be bridged by a manager.

It’s great if the manager is also technical, but it’s more likely that management rises from finance or marketing or the boss’s family.

Aside: It’s my opinion that the best and most useful thing an “enterprise” project manager can do is run interference. The worst thing a project manager can do is interfere. (Don’t get me wrong, I’m only talking about interfering in matters they know nothing about.)


Ignoring risks

Ignoring software project risks really belongs in the same category as poor management, but I’m calling it out because it isn’t just managers who ignore risks. Have you ever worked on one of those projects where a comprehensive list of risks is produced (at great effort) and then filed with all the other risk logs, never to be seen again?

Treating all projects the same, regardless of risk, is plain stupid. There must be some kind of cognitive dissonance thing going on here, because it happens all the time.

That’s enough purging for one post. Next I’m going to look at outsourcing, wishful thinking, excessive optimism, and promising an end date before setting the scope of the project.


Update: Part 2 is here.






Posted in Software Management

Ed Guiness

I am the author of Ace the Programming Interview, published 2013 by John Wiley and Sons. In 2012 I founded SocialCoder.org, a volunteering organisation for programmers. I have been a professional programmer for more than 20 years, and a hiring manager since 2004.

Ask me anything.