Programming expertise, it’s all about the turtles

When I’m asked at an interview to talk about my weaknesses, I usually say that apart from being a perfectionist (said with irony) I can sometimes be too analytical.

Being analytical as a programmer is a major advantage, but I’m told by those close to me that every now and then I can be a royal pain in the arse.

Take for example how I reacted to this sage advice from Scott Hanselman, that you should know what your library is hiding from you.

“…it’s useful to remember that you’re buying into an abstraction over something.”

Which is fine advice for the normal person (the person who might make a note to read up on multipart form data sometime, uh, maybe) but my  reaction led me to turtles.

I’ll come back to turtles later.

Just like kitchen gadgets and the postal system, software is sweet when it works and can really create disorder when it doesn’t.

Part of the wonderful thing about working computer systems is how they present us with a simplified view of the underlying system. In other words they present us with an abstraction.

The better the abstraction the more we come to depend on it, and the less able we are to cope without it.

Our ancestors used to make fire with sticks, but with few notable exceptions none of us can do that now.

In almost the same way, many programmers using the Microsoft stack of technologies cannot write code without Visual Studio, even though their predecessors made do with a text editor, a compiler, and make. Some have gone as far as asking if Visual Studio might rot the mind.

Of course, real programmers cut code with a magnetised needle and a steady hand while the truly hard core do it with butterflies.

This is not to criticise my fellow coders; we get leverage from development tools just as we get leverage from abstractions. We almost never need to revert.

Almost never.

Charles Petzold, renowned programmer and author, wrote a book called Code: The Hidden Language. What’s good about this book is that it goes much deeper than your average book for dummies. Petzold describes computer systems from the ground up – he starts with how communication can be encoded as ones and zeros, works his way through logic gates, to calculating machines, to low-level languages, and eventually arrives at modern computer systems and the internet.

It’s a real eye-opener if your introduction to computer systems was writing HTML for the web, and a book I highly recommend.

I mention Petzold’s book because it shows how many layers of abstraction we take for granted, even if we think ourselves “techie”.

Back to the turtles…

A well-known scientist (some say it was Bertrand Russell) once gave a public lecture on astronomy. He described how the earth orbits around the sun and how the sun, in turn, orbits around the center of a vast collection of stars called our galaxy. At the end of the lecture, a little old lady at the back of the room got up and said: “What you have told us is rubbish. The world is really a flat plate supported on the back of a giant tortoise.” The scientist gave a superior smile before replying, “What is the tortoise standing on?” “You’re very clever, young man, very clever,” said the old lady. “But it’s turtles all the way down!”

— Stephen Hawking in A Brief History Of Time

Whether it was Bertrand Russell or a Hindu that came up with the image, an infinite regression of turtles seems a good metaphor for the question of how deep a programmer should dive beneath the immediate abstraction.

At one extreme the programmer can ignore the turtles, and struggle like an inverted turtle when the underlying reality fails to cooperate.

At the other extreme the programmer can disappear down the layers of abstraction in an endless quest for understanding.

Somewhere between these two extremes there is an appropriate balance, and I hereby confess that I don’t always recognise it.

Just the other day I was chatting to a friend about the awesome specs of my new Zenbook, in particular marvelling at the solid-state drive. While reading specs for the SSD we noticed it has an unusually high operating shock tolerance of 1500g. I’m a little embarrassed to recount that I spent far too long trying to work out what this might feel like relative to, say, a punch in the face.

(The answer was an SSD falling from desk-height would be nothing compared to a face-punch from an elite boxer.)

So how deep should a programmer go? I think the answer lies in the old joke about Pete and Jim, the two friends who disturb a tiger; Pete reaches into his pack and pulls out a pair of running shoes. Jim looks at him. “Do you really think those shoes are going to make you run faster than that tiger?”

“I don’t have to run faster than the tiger. I just have to run faster than you.”

Pete is motivated by self-preservation, but a similar approach can work for programmers motivated presumably by self-improvement. The programmer needs to outrun his peers, or in other words

The expert programmer understands at least one level deeper.

This is not particularly profound advice, and the same is found in other specialisms. When Chess grandmaster Samuel Reshevsky was asked how many moves ahead he sees, he replied “always one more than my opponent!”

Don’t misunderstand – I’m not advocating one-upmanship. In an ideal world the programmer will understand all levels of abstraction, but in the real world almost no one has time for that.  Striving to understand one level deeper than your peers will set you apart as the go-to programmer.




Posted in Interviewing, Programming, Software Development

Reflections on leaving MegaCorp

To the person who next sits at this desk;

I have worked at MegaCorp on and off since 2002. This time I’m leaving and I won’t be back.

If I had a magic wand to fix I wish you If you (the person finding this note)

More than a code-review tool we need better code reviews

The tools are amazing. You can link them to your source repository and insert review comments almost anywhere. You can have a complete conversation about specific lines of code and perform many kinds of wizardry. But the tools don’t force developers to review thoughtfully and respectfully.


More than a wiki we need developers to learn to write well

Where writing well is measured by clarity and relevance, not by words per hour.

More than project meetings we need everyone to communicate well

If we communicated well we wouldn’t need most of the meetings we have. If you read this and wonder when this communication would happen if not in meetings, you are part of the problem.

More than accurate estimates we need to manage risks better

Sorry, it just isn’t sensible to derive a “contingency” number from a risk, add it to the original estimate and then proceed as if the risk has been dealt with.

More than risk-averse change-management processes we need a better appreciation of our existing technical debt

You see that look on your principal developer’s face? He’s just been told to fix a memory leak in [legacy product X]. That means he knows the code behind it is a huge pile of impenetrable crap. When you see that look, all bets are off. He’s bailing out the Costa Concordia with a teaspoon.

More than extra resources we need better qualified and experienced people

Have we learned nothing from 40+ years of throwing more and more people at failing projects?



Posted in Programming, Software Management

Writing unit tests for legacy code – an open letter to developers I work with

This is an email I sent today to developers who work with me, it is exactly as I wrote it except for project and developer names which I’ve redacted.

Dear Developers,

S asked me a difficult question today, and I think the answer (which took me a few minutes to arrive at) is worth sharing with all developers, mainly because many of you will surely face the exact same problem especially in [maintenance and enhancement] projects.

By now I think it is crystal clear that one of our non-negotiable requirements for coding is that we write unit tests.

For new projects it is easy – in fact for new projects you can write the tests before you write the actual code (ie TDD).

For legacy projects, by which I mean all the [maintenance and enhancement] projects, it is often a different story…

S had an offshore developer fix a bug in the xyz user interface. It was incorrectly prompting the user to save details when they hadn’t changed anything. (The details are a bit more complicated).

The fix was to add two lines of code.

Unfortunately these two lines were added in the middle of a method that was already 266 lines long. This method also had a large number of dependencies, meaning that to test the method it would be necessary to spend much, much more time writing the test than it took to find and make the fix in the first place.

S’s question was “should we insist on writing a test for this simple fix, when writing the test is going to take much longer?”

It was tempting to answer “no, don’t bother, it will be covered by system testing anyway”, but I am convinced this is absolutely the wrong answer.

I think the answer I gave S will apply to many situations you will encounter in [enhancement and maintenance] projects, S&M etc, perhaps even in new code  for stories previously written in “new” projects

The answer is

Always assume you should write the unit tests, and that you might need to re-factor the code to make it possible to write the unit test.

If you cannot write the test in a reasonable amount of time, you need to estimate how long it might take, and escalate the problem to your scrum team.

If the scrum team can’t help, keep escalating (eg to L, me, R etc) until you’ve exhausted possibilities.

If after that you still can’t do it in a reasonable time, it becomes a problem for the project – a scheduling, perhaps a resource problem.

It should never be a question of whether you (the developer) should sacrifice quality for the sake of expediency, because the answer is always that you shouldn’t.

The business (by which I mean the senior management team) can make that kind of decision, but not you – your job is to produce quality code, and unit testing is a not-negotiable part of that obligation.

Hope I’ve been clear, please let me know if not.  I’m especially interested to hear if you feel you have special circumstances that lead you to short-circuit the unit testing approach.



Posted in Programming, Software Development, Software Management

It’s a great time to be a programmer in the UK

According to the Office for National Statistics the ratio of vacancies to jobs in ICT reached a low point July 2009 and has been climbing steadily ever since.

Contrast this with vacancies in all industries, which dipped to around 1.8 vacancies per 100 jobs and stayed there, and it looks like it’s a great time to be a programmer in the UK.





Posted in Hiring, Software Management

Why software projects are always late (part 1)

Software projects are always late because they are not easy, and because people who should know better do stupid things.

It’s a question I’ve been asked many times over the past two decades and, let me tell you, quite often the question skates uncomfortably close to an accusation.

Well folks, after one frowning face too many it’s time to take a good hard look at some of the many reasons why projects go down the drain.

Projects in trouble tend to stay in trouble

Let’s start with something the software industry has known for almost forty years. If a project is in trouble, throwing money and people at it will almost always make things worse.

“Adding manpower to a late software project makes it later”

— Fred Brooks

It doesn’t matter how many times a project team is exhorted to redouble their efforts and work smarter, it doesn’t matter how many carrots are dangled or sticks brandished, it doesn’t matter how many incentives are waved around or (more commonly) how many pizzas are ordered for working late into the night, the basic facts about project deadlines and project scope won’t be changed by adding more people.

The one thing that does change when new people arrive is project complexity – it goes up. Apart from anything else, the number of lines of communication is multiplied. It’s hard enough getting ideas clear with one person. Adding a horde of newbies to a stressed out team won’t make that any easier, not even a little bit.

Of course, sometimes replacing a burned-out team of developers after they’ve been worked into a coma will temporarily increase velocity, but good luck sustaining that approach.

Unhelpful Metaphors

Software projects suffer from poor metaphors. Actually let me rephrase that. The metaphors are just bloody awful. Developing software is not like building a brick wall, you can’t count the bricks and extrapolate a finish date. It isn’t a factory situation.

If you think it is a factory situation, it leads you eventually to wonder why all the programmers can’t be replaced with code-generating machines. Don’t laugh, this problem is still taken seriously.

Nor is it a house, a sudoku puzzle, or a chicken dinner. It’s software, it’s code, and there’s nothing quite like it.

Perhaps software development is more like craft, and programmers should be more like craftsmen (proud of their work, taking time to do it right) but while this metaphor is more appealing than a brick wall, it’s still imperfect. For instance, everyone agrees it’s a poor craftsman who blames his tools, yet no sane programmer will spend much energy defending the monstrosity that was Visual SourceSafe. Take away the Visual Studio IDE and the productivity of many good programmers would fall through the floor. Apart from a few notable exceptions, programmers need the best tools.

Gardening is probably my favourite metaphor, with programmers as gardeners who tend the code base, who pull weeds (refactor and debug), plant seeds (add features), and train the growing plants (refine features as they get feedback). This metaphor works because plants are unpredictable, just like software requirements

But gardeners are lucky – they have hundreds of years of research and experimentation to build on. Programmers have a few decades of hackers mucking about pretending to have everything under control – our state of the art is Agile, which let’s face it is a blatant acknowledgement that things always go wrong, that things take longer than expected, and that no one really knows what they want from software.

Perhaps Donald Knuth has the right metaphor, perhaps programming is indeed Art of some kind. True, Art is not the impression you get reading his four (five?) mighty volumes, but I’m too humble to argue with the professor.

Poor management

Yes, programmers, this was probably the first thing you thought of, right? Management doesn’t understand programming, management interferes, management makes all the worst decisions and we would all be better of if they just left us the hell alone. Yes?

The realists know that someone has to bridge the divide between development and sales, and unless you’re in a tiny development shop, that divide is going to be bridged by a manager.

It’s great if the manager is also technical, but it’s more likely that management rises from finance or marketing or the boss’s family.

Aside: It’s my opinion that the best and most useful thing an “enterprise” project manager can do is run interference. The worst thing a project manager can do is interfere. (Don’t get me wrong, I’m only talking about interfering in matters they know nothing about.)


Ignoring risks

Ignoring software project risks really belongs in the same category as poor management, but I’m calling it out because it isn’t just managers who ignore risks. Have you ever worked on one of those projects where a comprehensive list of risks is produced (at great effort) and then filed with all the other risk logs, never to be seen again?

Treating all projects the same, regardless of risk, is plain stupid. There must be some kind of cognitive dissonance thing going on here, because it happens all the time.

That’s enough purging for one post. Next I’m going to look at outsourcing, wishful thinking, excessive optimism, and promising an end date before setting the scope of the project.


Update: Part 2 is here.






Posted in Software Management

Should I include GCSE results in my CV?

A quick follow-up to the post “The ideal CV of a software professional” to answer a question I’m asked surprisingly often in various forms:

I’m 28 years old. Should I include my GCSE results in my CV?

My first reaction as a hiring manager is no, don’t bother. What you did at school ten years ago is insignificant compared to what you’ve done since then.

However – a lot of people with experience do include their GCSE results in their CV, and there is probably a class of manager who wants to see them.

I would only say it’s unlikely you will be hired on the strength of your GSCE results.


Posted in CV

The ideal CV of a software professional

Tip #1: Make sure you include relevant keywords, but don’t overdo it.

Let’s not fool ourselves, the main purpose of a CV is to get you past the filtering process of recruitment agencies, HR departments, and other hurdles that stand between you and the hiring manager.

Many of these intermediaries will have no technical knowledge, and they will be looking for keywords. Yes, this means that you could in theory replace the agent with a one-line perl script, but that’s the way it is. (Oh, and apparently it’s your fault.)

The hiring manager will understand that contributing to the ECMAScript specification probably means you know JavaScript (and JScript for that matter) but the typical recruiter will not.

Don’t be like the Meta-tag-stuffers of ten years ago, because it will scream FAKE to the hiring manager. You’ve got to find a balance.

Tip #2: Write as well as you can

It’s perfectly fine if English is not your first language (my first language is Kiwi – Kia Ora Cuz!) but if what you write doesn’t make sense to the reader you will be at a severe disadvantage to say the least. What you write must be clear, and it must flow. Proof-read (or better, have a friend proof-read) and ruthlessly re-write or delete disconnected, incomplete, ambiguous or vague sentences.

When stuck for how to write something I find it helpful to pretend you’re telling it to a friend. Write down the words you used, and then revise it into shape. It’s good to show a bit of personality, but don’t ramble, and keep in mind that what might seem funny to you as you write your CV will not look so clever to everyone that might read it. Much better to let your personality show at the interview after you’ve established a rapport. If in doubt, always err on the side of playing it straight in your CV.

Some managers don’t care so much about spelling and grammar, but others (like me) care a great deal. I think my distaste stems from years of reviewing poor code, code which is often full of spelling errors. It doesn’t take a lot of extra effort to use the spelling checker in your word-processor or browser. Don’t ignore the red squiggly lines!

Finally, most people find that writing well takes a bit of effort and discipline, so be realistic and allow yourself plenty of time to write and revise.

Tip #3: Make sure your CV shows where you got your experience

So, let’s assume you’re applying for a job as a web developer, and the job advert is asking for “a minimum of five years experience with ASP.Net”. If you claim to have that experience, I’m going to want to see where you got it. If I don’t see it, I won’t necessarily assume you’re lying, but you will be at a disadvantage when I have another CV that clearly shows where the five years experience was obtained.

If you claim to be good at anything, it should be reflected in the experience you’ve written about. You need to be explicit, not least because job titles have become meaningless. Being an “analyst” could mean just about anything these days, and doesn’t necessarily imply analysis skill of any kind. A “web developer” could mean someone with programming skills, or someone with Photoshop skills. A “programmer” should mean someone who produces code (though this is a debateable point) but I’ve seen it used for jobs that amount to configuring systems via drop-down lists. You need to describe the technology, and how you used it.

Tip #4: Ignore anyone who tells you “strictly one or two pages”

Ignore all the advice about keeping a CV short. That’s for marketing and sales jobs, and for people with short attention spans. If you have lots of experience it should be reflected.

As a hiring manager I want to see it, and more importantly you should be proud of it, not ashamed.

Tip #5: There’s nothing wrong with emphasising certain skills to match a job advert

It took me a long time to internalise the idea that its OK to emphasise different skills depending on the job in question. Frankly, I used to think it was deceptive. But think of it from the hiring manager’s point of view – if a CV clearly calls out skills that match the advert, that will appeal more than a CV that contains the same skills but buries them among other, less relevant skills.

Highlighting relevant skills helps everyone.

If you have experience in a variety of areas, consider writing more than one CV, each one to emphasise different areas like database or analysis or customer-facing or whatever so that you can quickly respond to adverts when you find them.

Tip #6: Don’t leave unexplained gaps between jobs

Don’t leave gaps in your CV, especially big ones, it risks making you look like a layabout. If you were volunteering in Africa, great. If you took time off to retrain, or to pursue a start-up idea, or to raise your kids, that’s all good. The only thing you have to prove is that you can do the job. Yes, in some cases it might mean you have to accept a lesser salary, but a savvy employer will see that as an opportunity rather than a problem, and savvy exployers also know that life-experience counts for something.

Tip #7: Everyone enjoys “reading, music, and cinema”, it doesn’t need saying.

I reckon that 80% of CVs contain a personal interests section, and 80% of these contain the exact same things; reading, music, and cinema.

My take is that you really don’t need this section in a CV, and if all it contains is the same old stuff then you really, really don’t need it.

On the other hand, if you happen to be a world-class skittles champion, maybe that’s something to show off.

But generally, instead of trying to show how “well-rounded” you are in this odd corner of the CV, try to let that show through in the rest of your CV and in the interview.

If you get stuck, let me know.




Posted in CV

Interviewer or Inquisitor?

This post is by way of apology to everyone I’ve interviewed over the years who might have witnessed me learning on the job.

If I played one of these characters when we met, I’m sorry.

The Alpha Geek

To the Alpha Geek, the interview is nothing more than a violent competition to be won at all costs. Anything you can do, the Alpha Geek has done better. Anything you say will be shouted down in a torrent of acronyms and geek speak.

“Last night I recoded Dijkstra’s graph-search algorithm using OCaml and a toothpick.


The Invigilator

This character secretly doesn’t want to have anything to do with interviewing, nor do they wish to get into any complicated discussions about, well, anything.

Instead of asking questions about your experience and building some kind of rapport the Invigilator will set you an exam and leave you to it.

You will sit and work under exam conditions, possibly for many hours.

When you’ve handed in your assignment, that’s it. The invigilator will tell you a week later whether you passed or failed, but no correspondence will be entered into.

The Quiz Master

The Quiz Master will turn up with a suit and a stack of cards. The interview will consist of three rounds of general knowledge and trivia. Ten points for a correct answer but you will lose ten points for every incorrect answer.

“For ten points, write down the actual number represented by Int32.MaxValue”

Grand Inquisitor

You’d better be sure that the dates of your past employment match up, because nothing will escape the Grand Inquisitor’s notice.

“There are 14 days missing between these two jobs. Please explain.”


Dorothy wishes she was somewhere else, like Kansas. The interview will be short.

“Uh, hi! Sorry, can’t stay long. So, uh, tell me something about yourself. Oh really… OK, got any questions for me?”

The Flower Child

The Flower Child relates to everything you say on a profound level, and will probably hug you when the interview is finished.

“That’s really, really great. Can we hug now?”

Water Torturer

After ten hours of interviews, you’ve met the whole department, sat through hundreds of questions, coded up a complete replacement for their internal workflow system, and now you get the Water Torturer’s feedback, which will be non-committal, vage, probably metaphorical (“uh, you’re a great pilot, we just need to fly a different model plane”). You will be invited back for a second round of interviews. And a third, etc. You may perish before the interviewing ends.


Posted in Hiring, Interviewing

7 tips to hire the development team you really want

1. Get help. Seriously. I don’t care if you’re an expert in your field. Unless you’re also an expert in software development you’re not going to be able to tell a coder from a clown.

Not sure? Here’s a test. Can you tell at a glance that this code is bad?

for ( int i=0 ; i < this.MyControl.TabPages.Count ; i++ )
   this.MyControl.TabPages.Remove ( this.MyControl.TabPages[i] );

(Here’s why)

2. Hire the most important team members first, they can help you find the rest of the team. Start with the lead developer. Whatever you do, don’t hire a “project manager” who can’t code, regardless of their claims or apparent experience. If they can’t write code, they aren’t qualified.

3. Don’t forget – good programmers are not in thrall to IT recruiters. You just need to know where to look. Look on stackoverflow. Search twitter for developers bitching about the quirks in relevant technologies. Get your new dev lead to tell you where online she hangs out and look there.

4. Ignore offers of help from salespeople. It doesn’t matter if they’ve been fully trained in terminology. Their opinions should count only in proportion to how well they could design and build a system themselves.

5. Go for quality over quantity. This sounds obvious until I put it more pointedly – if you have thoughts of hiring more than one developer, consider combining two jobs into one and offering a salary twice as big. Top-notch programmers are worth it. This doesn’t apply if you’re intent on producing unmaintainable crap.

6. Avoid using any of these terms when advertising the job:

  • “fast moving” or “fast paced”
  • rock star
  • exciting
  • Don’t say SQL SERVER or PERL when you mean SQL Server or Perl.

In fact, probably best to get your new team lead to write the job description, or at least review it before it gets published. Don’t let sales people publish it for you, they will screw up and (to cite one example I witnessed last month) drop the # from C#, leaving your advert looking stupid. “.Net and C developers wanted!”

7. Don’t let yourself get desperate. Kind of self-evident, but worth remembering. If you get desperate you’ll end up hiring poor/mediocre programmers who will produce mountains of crap (oh believe me, I’ve seen this happen over and over). Plus, the programmers you should have hired will see the mounting piles of unmaintainable crap code and run a mile.


Posted in Hiring

Cranking the code-o-matic

Microsoft recently announced their latest version of Visual Studio, a product pitched on the idea that anyone can produce useful software provided they have the right tools.

They call it Visual Studio LightSwitch, and right from the outset they make it clear that coding will be optional.

Microsoft’s insistence that CODING is OPTIONAL will probably make it an instant hit with every business owner who is frustrated by the cost and complexity of software development. Either that, or it could be another Microsoft Bob.

A road much travelled

This is a road that has been travelled many times. You probably know the history. One of the earliest attempts to bring software development to non-programmers was with COBOL in 1960.

COBOL had the explicit design goal that non-programmers should be able to read COBOL code.

So here you are, have a squint at this.

While it’s certainly possible to pronounce most of the words in this code listing I don’t think anyone will equate that to a meaningful understanding.

We have witnessed more than fifty years worth of attempting to put software development into the hands of end-users but still the problem remains largely unsolved for anything beyond trivial or highly domain-specific solutions.

If it is clear that the goal of end-user-development is a very difficult problem, arguably unsolved, it is equally clear that the dream has never faded away. Why is that?

The dream endures, but why?

One reason could be a lack of trust.

Successful software needs constant maintenance. As a software product becomes harder and more costly to maintain (as most do) the boss who doesn’t quite trust his team will be tempted to look for alternatives that appear more trustworthy.

This basic lack of trust becomes a self-fulfilling prophecy as communication breaks down and the team and the boss begin to work in opposition. The boss is now a prime target for being sold the idea that software development can be very nearly automated. It’s not a logical process, it’s an emotional one.

In other cases the boss simply lacks the necessary understanding of software development. They might think that programming is actually an unskilled or semi-skilled profession, where the programmer simply memorises a quirky list of words and symbols and then types them in according to a specification written by someone in marketing.

All this raises the question of what it means to program a computer.

What is programming anyway?

Not sure about this:

This probably is:

What about editing a configuration file? Obviously this is not programming.

What if the config file contains functions?

htaccess can get interesting, does it count?

And despite having no visible code, I’m fairly sure SSIS/DTS packages count as programming.

Are we closer to finding the line that delineates programming?

Here’s how I define it

Programming is the act of translating an idea, a process, something out of your mind into terms a computer can understand and act upon.

It has often been said that a person does not really understand something until he teaches it to someone else. Actually a person does not really understand something until after teaching it to a computer, ie, express it as an algorithm.
—Donald Knuth

Now the whole question of whether the production of software “without coding” is irrelevant. It doesn’t matter.

What matters is whether you’ve produced an algorithm, a set of instructions that a computer can follow to achieve a useful result. That’s where the satisfaction, the mental effort, and the joy of programming lies.

Back to LightSwitch, if it encourages more people to try programming for the first time then I’m all for it. Realistically though, I stand by my first reaction:



Posted in Programming

Ed Guiness

I am the author of Ace the Programming Interview, published 2013 by John Wiley and Sons. In 2012 I founded, a volunteering organisation for programmers. I have been a professional programmer for more than 20 years, and a hiring manager since 2004.

Ask me anything.