2009-09-16

The No-Cry Software Development Methodology

The programming section at the bookstore is jammed with prescriptive "methodology" books like these:

  • "Continuous Integration: Improving Software Quality and Reducing Risk"
  • "Rapid Development: Taming Wild Software Schedules"
  • "Implementing Lean Software Development: From Concept to Cash"
Likewise the parenting section:

  • "The No-Cry Potty Training Solution"
  • "The Attachment Connection: Parenting a Secure & Confident Child Using the Science of Attachment Theory"
  • "The Successful Child: What Parents Can Do to Help Kids Turn Out Well"
I want to talk about two such books that I didn't write:


At my work we got by for a very long time with an ad hoc release process, wherein a release plan -- the list of operational steps required to make the release -- was one of the unique products of each release. This worked for a long time. About six months ago, though, we started tackling hard problems in badly bitrotted components, and suddenly we had a string of awful releases. Every release, something went blooey, either because of a mistake in the release plan, or a mistake executing its unfamiliar contents, or a mistake in the code which required (often middle of the night) manual intervention and cowboy heroics by operations and engineering.

This led to my personal least-favorite type of meeting, the post-mortem (kill me!). So, to prevent future post-mortem meetings, we had secret technical cabal post-post-mortem meetings, in which we thrashed with the problem in a somewhat more productive manner. Ultimately (after a few failed experiments and a bunch more lousy releases) we settled on a simple change: we'd have exactly one release plan, re-used for every single release, and we'd code to match the release plan rather than write the release plan to match the code.

A few months later: the operational team has scripted every aspect of the release process. Human QA had formalized testing of the release process. Engineering had created automated tests to cover the basic required release behavior. And though we continued to hack away at the (historically) most brittle components in every release, we're back to weekly and almost completely uneventful releases.

A certain type of person seems to get to this point in their problem solving and suddenly think: "I should write a book about release methodology. I'll call it 'The No Post-Mortem Meeting Release Solution'! It'll sell like hotcakes, *everyone* hates post-mortem meetings!"


In my other life, as a parent, we recently went through a spell of awful fighting between the kids. Screaming, pushing, crying, name-calling, etc. Totally normal stuff for young kids, but still really unpleasant. After some thought and a lot of experimentation and so on, one of the tactics that worked really well was to start a cooking project when things were getting tense, and invite one or both of the kids to help. I started baking pies like crazy. And it really helps!

A few months later, it is not uncommon for the kids to be the ones to recognize their own boredom and suggest a cooking project. We eat pie every other day or so, which would make it well worthwhile even if the kids still fought incessantly... and while I can't say that fighting has been completely eliminated, it's better.

Again some people seem to get to this point and think: "Fantastic! I will now write and publish 'The Shoo-Fly Pie and Apply Pan Dowdy Cure For Sibling Rivalry'!"


In both cases, the new "methodology" arose when creative people, not overly constrained by conventions, traditions or rules, innovated specific tools to address specific problems.

So what about those books? They seem harmless enough -- someone came up with a solution to their specific problem and thought it might help others in similar situations. It strikes me as silliness to think that one can create a universal methodology for something as complex as potty training an individual toddler, or releasing any arbitrary piece of software. But I do think it's great to draw on the experiences of others when trying to solve a problem.

However, there is no value in a methodology if it does not solve a real problem. On the contrary: any tactic, strategy, methodology or philosophy that you adopt which is not solving a specific problem that *you* are having *right now* is a problem in and of itself. YAGNI! As with bloated code, a bloated process adds inertia and inflexibility. The more bloated it gets, the *harder* it will be to solve the unique problems you're going to face -- for which there is no prescription!

Caricature Rational Programmer (CRP): "I found this memory leak and it's gonna pooch the release that's supposed to go out tomorrow so I'm filing a CRM and a ticket and stuff and we're gonna slip by at least a month."
Me: "Fix the leak and save the release."
CRP: "But we're Rational Unified! And IBM says if we don't go through the change committee our software will be late, over-budget, defect-ridden and incomprehensible!"

Of course, any methodology can be abused this way (it's just that today is international pick on Rational Unified Process day). And I see this sometimes in parents, too:

Caricature Attachment Parent (CAP): "Co-sleeping is making my spouse and I miserable, the baby kicks like a ninja all night and we're not getting any sleep."
Me: "Put the baby in a crib; you'll sleep better after a few nights."
CAP: "But we're Attachment Parents! And Dr. Sears says unless you co-sleep your kid will be stupid and insecure!"

Yeah, they're caricatures. But here's the thing: anyone who wasn't there when the original problem was solved can never truly grok why they're doing things this way. They do it out of faith and tradition, perhaps fear -- which can all-too-easily become religion and dogma.