Amazon (hardback, kindle)

InformIT (pdf, epub, mobi)

中文 (in Chinese)
日本語 (in Japanese)
한국말 (in Korean)
português

You can also see a list of all my publications and talks, including slides, on the Publications page.

Upcoming talks

Get the software


Why Software Development Methodologies Suck

Translations: 中文 | 한국말
There’s a lot of dogma in the religious wars around software development practices and methodologies. Are phase-gate methodologies effective at managing the risk of software development, or just risk management kabuki? Does TDD really make for higher quality software? Is pair programming a superior replacement for code review or just a way to inflate consulting rates? I’m going to argue that while scientific evidence to decide these claims is lacking, there are two general principles which can help us choose good practices while at the same time improving the value of the software we deliver: reduce cycle time and increase feedback.

Michael Feathers makes the following observation:

I think that, in the end, we just have to accept that developer skill is a far more significant variable than language choice or methodological nuances1. Frankly, I think we all know that, but we seem to suffer from the delusion that they are the primary knobs to tweak. Maybe it’s an extension of the deeply held view that from an economic viewpoint, it would be ideal if people were interchangeable.

The problem is, how do we get skilled developers? Since the concept of individual productivity in IT has never been satisfactorily defined, this is a particularly hard problem to solve. Lines of code – still a popular measure – suffers from the devastating flaw that a line of code is a liability, not an asset as is often thought. Measuring number of hours worked encourages heroic behavior – but experience shows that the “heroes” are usually the same people that cause projects to become late through taking unacceptable risks early on, and working long hours makes people stupid and leads to poor quality software. There is still no generally accepted set of professional standards or chartering system for IT professionals, and recruiting good people is very much an art rather than a science.

Psychologists have at least addressed the problem of why it is so difficult to acquire and measure skill in IT. As Daniel Kahneman says in Thinking Fast and Slow, there are “two basic conditions for acquiring a skill: an environment that is sufficiently regular to be predictable; [and] an opportunity to learn these regularities through prolonged practice.”

But traditional software projects are the opposite of a regular, predictable environment. The only good measure of success of a project – did the end result create the expected value over its lifetime? – is so distant from the critical decisions that caused that success or failure that it’s rare for anybody from the original team even to be present to get the feedback. It’s practically impossible to determine which of those decisions led to success or failure (in artificial intelligence, this is known as the credit-assignment problem).

These factors make it very hard for IT professionals to acquire the skills that lead to successful products and services. Instead, developers acquire the skills that allow them to most efficiently reach the goals they are incentivized by – usually declaring their work “dev complete” as rapidly as possible irrespective of whether the functionality is integrated and production-ready – and similar problems arise in other functional areas too.

The fact that software projects are complex systems rather than regular environments leads to another problem – the extreme difficulty of gathering data on which techniques, practices, and methodologies are actually effective, and the near impossibility of generalizing this data outside the context in which it was gathered.

In his excellent book The Leprechauns of Software Engineering Laurent Bossavit executes a devastating attack on software development folklore such as the “cost of change” (or “cost of defects”) “curve”, the claim that the variance in developer productivity is an order of magnitude, the idea of the cone of certainty, and many other cornerstones of methodological lore in software development. He shows that these theories – and many others – depend on very small sets of data that are gathered either from informal experiments run on computer science students, or projects which cannot possibly have been effectively controlled. The organization of the studies that form the basis of these claims is often methodologically unsound, the data poorly analyzed, and – most egregiously – the findings generalized well beyond their domain of applicability2.

As a result, it’s not possible to take seriously any of the general claims as to whether agile development practices are better than waterfall ones, or vice-versa. The intuitions of “thought leaders” are also a poor guide. As Kahneman says, “The confidence that people have in their intuitions is not a reliable guide to their validity… when evaluating expert intuition you should always consider whether there was an adequate opportunity to learn the cues, even in a regular environment.” As Ben Butler-Cole points out in his companion post, “why software development methodologies rock”, the very act of introducing a new methodology can generate some of the results the adopters of the methodology intend to bring about.

You might think that puts us in an impossible position when it comes to deciding how to run teams. But consider why software development is not a regular environment, and why it is so hard to run experiments, to acquire skills, and to measure which practices and decisions lead to success, and which to failure. The root cause in all these cases – the reason the environment is not regular – is that the feedback loop between making a change and understanding the result of that change is too long. The word “change” here should be understood very generally to mean change in requirements, change in methodology, change in development practices, change in business plan, or code or configuration change.

There are many benefits to reducing cycle time – it’s one of the most important principles that emerges when we apply Lean Thinking to software development. Short cycle times are certainly essential for creating great products: as Bret Victor says in his mind-blowing video Inventing on Principle, “so much of creation is discovery, and you can’t discover anything if you can’t see what you’re doing.”

But for me this is the clincher: It’s virtually impossible for us to practice continuous improvement, to learn how to get better as teams or as individuals, and to acquire the skills that enable the successful creation of great products and services – unless we focus on getting that feedback loop as short as possible so we can actually detect correlations, and discern cause and effect.

In fact, the benefits of having a short cycle time from idea to feedback are so important that they should form one of the most important criteria for your business model. If you have to decide between creating your product as a user-installed package or software-as-a-service, this consideration should push you strongly in the direction of software-as-a-service (I speak from experience here). If you’re building a system which involves hardware, work out how you can get prototypes out as quickly as possible, and how you can modularize both the hardware and the software so you can update them fast and independently. 3D printing is likely to make a huge impact in this area since it allows for the application of software development practices to the evolution of hardware systems. Working in cross-functional teams is more or less a requirement if you want to achieve a sufficiently short cycle time.

Software methodologies – even the “hire a bunch of awesome people and let them self-organize” methodology – suck because they so often lead to cargo-cult behaviour: we’re doing stand-ups, we have a prioritized backlog, we’re even practicing continuous integration for goodness’ sake – why is the stuff we make still shitty and late? Because you forgot the most important thing: building an organization which learns and adapts as fast as possible.


1 Although as Laurent Bossavit points out (private communication) “A developer’s skill is in part the method he/she knows and his/her reasons for preferring one language over another.”

2 I am not suggesting that we give up on running experiments to learn more about what works and what doesn’t in software development, and the contexts in which such claims are valid – quite the contrary, I’m saying we’re not trying nearly hard enough.

  • http://profiles.google.com/warren.postma Warren Postma

    People aren’t trying “hard enough”? You’ve just substituted one set of handwaving and lingo, for another.

    W

    • http://continuousdelivery.com/ Jez Humble

      Well with a technical subject there’s always lingo, so I’ll admit to that. But where do you think I am indulging in handwaving?

  • http://sgleadow.github.com Stewart Gleadow

    Nice post Jez. It reminds me of a post I read earlier in the year that distilled successful software delivery down into two key factors: do less, start earlier. Given those two factors, your short cycle times and fast increased feedback seem like the logical way to judge how much “less” is worthwhile.

    • wtpayne

      “Do less, start earlier”. I love it.

  • David

    Hiring smart people is hard. Letting the domain experts and the technical experts make decisions together is hard. Not shooting the bearer of bad news is hard. It requires a cultural change, and that is something which in my opinion & experience most organizations are unwilling and/or unable to do.

    But in my opinion & experience the failures usually can be traced back mostly to precisely those cultural factors.

    So organizations fall back on methodology because it’s something they think is easier to manage. And when they hear that the process needs to be adapted to their organization, they hear “we can do this without tackling any of the cultural issues”. Sorry folks but it doesn’t work that way.

  • Mateo Bengualid

    “I’m going to argue that while scientific evidence to decide these claims is lacking…”. Adhering to arbitrary principles won’t do any consistent good. If it’s lacking, that is the problem. It’s even recognized in the second quote: “I am not suggesting that we give up, [...] quite the
    contrary, I’m saying we’re not trying nearly hard enough.”.

    OTOH, Deming said that trying as hard as possible is useless: everyone is already trying hard!

    • http://continuousdelivery.com/ Jez Humble

      My point is that reducing cycle time and increasing feedback are necessary conditions for gathering scientific evidence. The problem is not that people aren’t trying hard – the problem is they’re trying hard to do the wrong things.

      • Michael Kimber

        Absolutely Spot on!!!

  • http://ripplingbrainwaves.blogspot.in/ ArtV

    I see your point, but as soon as you say, “Shorten the cycle time,” the audience will respond with the understandable question, “How?” Or in more depth, “How do we shorten the cycle time without devolving into chaotic cowboy programmers?” Suggested answers to those questions entail a software development methodology.

    • http://continuousdelivery.com/ Jez Humble

      I don’t think answering those questions means adopting a methodology. I think it means the members of the team have to think about how to answer that question for themselves. Holding regular retrospectives and running experiments in between them is a good way to do this. Having somebody else do the thinking for you and blindly adopting the bits of it that don’t require you to change your behaviour too much is not. Until they are backed up with real data, methodologies are best viewed as a post-hoc rationalization of the thinking a particular team did – not useless by any means, but potentially dangerous as a guide to what _your_ team should do. That opposition – waterfall or chaos – is just FUD IMO. Waterfall can’t compensate for a lack of discipline – it just allows you to consistently deliver mediocre software on a slow schedule.

      • wtpayne

        Very nice viewpoint, and very well put.

      • wtpayne

        …. Although there may well be something to be said for the ability to produce predictable results, even mediocre ones …

    • William Payne

      Tools. Automation. Discipline. More Tools. More Automation. … and, yes, some Methodology also. (But the Methodology is there to fill in the gaps left by the automation).

  • http://www.bakermatt.com/ Matthew Baker

    Totally agree. No ‘standard’ or ‘good’ set of tools is going to help an organization that has inherent communication problems, or has long feedback loops that arise from structural problems imposed by their processes.

    And while some have had issue with the ‘we’re not trying nearly hard enough’ statement, let me try to decode it. I take that to mean that we need to accept that software development projects are not ‘regular’, as you put it, and take real ownership of discovering what works for our context, our organization, and our culture.

    Or it could just mean that we need more controlled experiments, better standardization of data collection, or both. Though I feel that the former will yield much more than the latter, having objective data to compare yourself against can help self reflection. Michael Mah gave a presentation at ADP East back in 2010 that had probably the most data I’ve seen in a study comparing ‘agile’ and ‘waterfall’ that I’ve seen.

    • http://continuousdelivery.com/ Jez Humble

      Clearly some people have taken “we’re not trying nearly hard enough” personally. But I feel as responsible as anybody else – Laurent’s book was a big wake-up call for me. I think all your suggestions are good ones. But I also think that software projects can be made more regular through reducing cycle times.

      Thanks for the Michael Mah tip – I found his podcast here: http://agiletoolkit.libsyn.com/adp-east-2010-michael-mah-agile-teams-by-the-numbers-software-metrics-and-the-big-giant-database

      • http://www.bakermatt.com/ Matthew Baker

        Sounds like I need to read Laurent’s book. Can’t be afraid to confront confirmation bias.

  • Apo

    I think cycle time might be almost a valid way to measure software development. It’s actually measurable (or at least it’s possible to calculate a rough estimate) and it works nicely when applied to different areas of software development, from business to development to testing and maintenance.

    I guess the only problem with it is that figuring out what to measure and what to improve is usually hard and only the first part of the puzzle. And like any other measurement, it’s easy to mislead measurements.

    • Larry Fast

      Cycle time is also about test coverage. Your automated tests can get you so far. Then you’re bumping along the back roads of manual test. I like to define cycle time as the time needed to find your first high impact bug.

  • Unka Dave

    Ya’ll might want to consider picking up “A discipline for software engineering” by Humphrey; (PSP). Implement it. Talk about a short feedback loop; great text, great ideas, but difficult to do (at first)…the rewards are awesome.

    • http://profiles.google.com/rob5136 Rob Crawford

      The disappointing thing about PSP is how many interruptions you measure.

  • Pingback: Why Software Development Methodologies Suck « Hornet Dear Bernard

  • Shawn Michaels

    I’m so glad to have found your web page. My pal mentioned it to me before, yet never got around to checking it out until now. I must express, I’m floored. I really

    enjoyed reading through your posts and will absolutely be back to get more. BY – software development company

  • Paul Bukie

    Nice articles. I think much attention is given to issues of methodology instead of programmers skills which is the crux of every succesful software. Most methodologies takes up much time, thereby delaying the time frame for the project completion. Originators and proponents of such methodologies clamor for fame and superiority of their methods.

  • http://snrky.com Snarky

    There are still orgs that measure lines of code? Insanity. We recently had a reorg – usually the bane of developers – that came only three months after a previous reorg. But whereas the previous iteration attempted to solidify teams and hand projects to them, versus splitting them up every 6-12 months to restaff matrix-ed projects, and failed, most developers moving around frequently (yes, within only three months, they were moving from team to team), the new iteration actually seems to have done it and developers are actually looking forward to stable teams, stable processes, and a chance to enjoy “a regular, predictable environment” where they can improve from moment/month to moment/month. I’m not saying we’ve found a good methodology and that should get credit – I’m saying we finally settled on one that keeps things stable and allows good developers time to worry about code and not about where they’re going next and how it’ll be different.

  • beenatthisalongtime

    All good fodder – but I wonder whether the metrics (which I DO feel are important) will ever be as relevant as whether or not the methodology is comfortable for the business unit that needs the work done… The bliss comes when the customer is happy – - – or at least content and not screaming due to non-delivery

  • http://moseswallace.blog.com/2012/11/01/a-software-development-firm-in-india-your-perfect-choice-for-your-business-needs/ ChandlerWilliams

    why software development is not a regular environment, and why
    it is so hard to run experiments, to acquire skills, and to measure
    which practices and decisions lead to success, and which to failure.

  • http://bradleyingram.yolasite.com/ AlecStout

    Sounds like I need to read Laurent’s book. Can’t be afraid to confront confirmation bias.

  • http://www.facebook.com/tomas.riha.se Tomas Riha

    I find the balance between initiative and responsibility key. There has to be a balance and the higher the level of both the better the developer. Technical skill is secondary because if the individual has both he/she will take initiative and responsibility to ensure the technical quality even if they them selfs at that time can’t provide it.

    Another way I view a good developer is the amount of business value added per line of code. The best ones increase business value while decreasing lines of code.

  • http://reedvin.weebly.com/ Kenyon Edwards

    Find software development companies with stable history of managing different tasks.

  • Pingback: Process analysis, Programming, and Software Development « sourcecodematrix

  • Pingback: Why Software Development Methodologies Stink | LeanOrange