File this under “I think this should be obvious by now, but experience tells me it isn’t.” If it is obvious to you, great! Please share this information with others so it’s obvious to more people. If it’s new to you, great! I did something useful! Again, please share with others!

Have you recently been asked, or have you asked the question “Do you use Agile or Waterfall?” Or maybe “Do you prefer Agile or Waterfall?” Or have you made a comparison between Agile and Waterfall?

Usually, this comparison can be translated into:

Iterations, daily meetings, burndown charts, Scrum Masters, and all those other things specific to the Scrum process or lots of up-front planning with discreet stages of development and a prediction of releasing at some specific date in the future.

So what, you say? Well, unfortunately, the thing on the left isn’t necessarily Agile, and the thing on the right doesn’t exist.

Agile does not equal Scrum

On the “Agile” side of the comparison, everyone seems to have forgotten that Agile, as originally defined, is really just a few values:

• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan

Using an Agile development process means that a decision has been made to value the things on the left more than the things on the right. There are no guidelines on the correct ratios. That is left up to you.

Agile does not have to be the specific process called Scrum, which is where you get “sprints” and “burndown” charts and “Scrum Masters” and yada, yada, yada. Agile (again, as originally defined) does have some slightly more concrete principles where there is mention of preferring small timescales, but nothing gets written in stone. In fact, the last of those principles:

• At regular intervals, the team reflects on how 
  to become more effective, then tunes and adjusts 
  its behavior accordingly.

Makes it perfectly clear that to be Agile, nothing can be written in stone. The process itself always has to be up for grabs. If objecting to part or all of it is viewed as religious heresy, then by definition, I argue that it isn’t Agile. It is process for the sake of process.

Note that I’m not saying that you shouldn’t have a process. I’m not even saying that you shouldn’t use Scrum as your process. I do have some arguments to that effect, but those are for a different day. Right now, I’m simply saying that Agile is not a synonym for Scrum. If you’re really talking about the Scrum process, then you might as well use the word “Scrum.” If you use the word “Agile,” then I, personally, am going to interpret that according to the Agile Manifesto.

There never was a Waterfall, there is no Waterfall, and there never will be a Waterfall

Okay, now it’s Waterfall’s turn. (Funny to me seeing Waterfall even capitalized. You’ll be in on the joke in a minute.)

Okay, here’s the original Waterfall methodology as described by Dr. Winston W. Royce in 1970 in Managing the Development of Large Software Systems.

Looks familiar, huh? End of story, right? Nope. Two big problems. Go read the paper if you want. Go ahead. I’ll wait… Okay, done reading? If so, you found out two things:

1. The paper never mentions the word “waterfall” at all. That word won’t show up until some years later when Royce’s paper is referenced by another paper.
2. More importantly, right after that diagram, you’ll see an interesting note. “I believe in this concept, but the implementation described above is risky and invites failure.” Seriously. If you didn’t read it (who are we kidding), go back and look. Second page. First paragraph.

Now, depending on where you sit in the fake “Agile vs. Waterfall” debate, you’ll read that note very differently. If you lean towards “Waterfall,” you’ll read that the concept is fundamentally sound. If you lean towards “Agile,” you’ll read that “Waterfall” was always part of a strawman argument. Both sides have some merit. The paper does go on to describe an extensively altered process that Royce does support, and figure 2 is the ideal basis for that process. However, that altered process shares some of the same flexibility as “Agile” processes, albeit more restrained.

The pragmatic methodology

Am I being pedantic? I hope not. Okay, maybe a little. (But if I really wanted to be pedantic, I’d talk about big “A” Agile and little “a” agile. Not going to do that right now.) My point is: this argument keeps going, and everyone has forgotten, or never knew, what the words even mean. As the comparison is usually used, it might as well be asked like this:

Do you use a process that I like and think is right, or do you use a process that I don’t like and think is totally messed up?

Instead, I’d like to see people move beyond the comparison and just use the process that is appropriate for the particular project and particular team at a particular point in time. The question should become something like:

“What kind of process do you think we should use for this project?”

Hopefully the answer is more complete than “Agile” or “Waterfall”. If it isn’t, be ready to ask what those terms mean.

Giving some credit

If you were as surprised as I was about the Waterfall myth, I highly recommend The Leprechauns of Software Engineering by Laurent Bossavit. It’s definitely a red pill. Until I read the book, I thought the Waterfall diagram represented a real process. If you read the book, you will be amazed at all the other completely made-up nonsense we believe to be true about software development.

Warning: I’m not a mathematician. (In fact, I had to spell check that word.) If programming was pure math, I’d be screwed. I got really close to a math minor, but that last 400-level course hurt my brain, and I cried uncle. If I had known I wasn’t going to become a rock star, maybe I would have tried harder. I’m going to talk about a proof here, but only informally. Don’t ask me to break out the chalkboard and do actual math. You’ve been warned. So anyway…

I love the halting problem. The halting problem is basically: can I make an algorithm that can look at any program and tell if that program will either run forever or stop at some point? And the answer is, drum roll… no! You cannot look at any arbitrary program and run it through a magic program that gives you that answer. Aside from trivial programs or very special programs, you can only informally reason about what it will do.

The proof for this is pretty simple. You don’t really have to understand math (except to understand proof by contradiction), and you only have to understand a little bit of computation.

function doesHalt(fn) {
  // magic function to tell if fn will halt
}

function badFunction() {
  if (doesHalt(badFunction)) {
    while (true) {}
  } else {
    return;
  }
}

The above shows that doesHalt can’t possibly work in all cases, because badFunction has already broken it. We didn’t have to talk about IO or mutating state or anything. Just a little bit of bad logic. In case it’s not absolutely clear: doesHalt can’t work, because if badFunction passes the doesHalt test, it will loop forever, and if badFunction doesn’t halt, it will just return. So, our (very trivial) function does exactly the opposite of what it’s supposed to do. I’m making a mathematical/logic leap here, but imagine when you add in IO, random numbers, etc. I think it’s safe to say that the unsolvable problem gets even more unsolvable.

So, why do I love this?

Well, in my opinion, in a nutshell, it demonstrates why software “engineering” is a much different animal than most other kinds of engineering, and you fundamentally have to rely on informal reasoning. You can get good bridge builders to basically guarantee their bridge. And they can guarantee it with math. They can use mathematical formulas to prove that it will withstand a certain amount of weight and wind and that it won’t suddenly start vibrating like a wave. When programmers guarantee something, they’re likely full of crap. And if someone asks for a guarantee, they likely don’t understand how to build software.

Software is a wonderful thing, but as soon as you start making it, the genie is out of the bottle. You’ve signed up for chaos. You can do things like unit testing, but that only proves the ways your software works, not the thousands of ways it doesn’t work. You can do lots of static analysis, load testing, whatever, and probably make yourself feel better. At any point though, your program could become a runaway train.

So what can you do? I think you should follow the advice in Out of the Tar Pit. Keep things simple so you can apply informal reasoning. Let things get out of hand to the point where you can’t informally reason, and, well, good luck. I definitely won’t say that I’m good at following this advice, because simple is freaking hard. But it’s good advice, none-the-less.

This was going to be a post on how you could get rid of constructors and the new keyword in your JavaScript code, making it simpler and more flexible. Unfortunately, when I got to the section I was going to write about performance concerns, some of that original post kind of died. So, instead, you’ll get this slightly dissonant mess. I’ll tell you how you can get rid of constructors, but also why you might not want to get rid of them.

The status quo

Here’s the typical pattern in JavaScript used everywhere:

var Person = function (name) {
  this.name = name;
}

Person.prototype.speak = function () {
  console.log('Hi, my name is ' + this.name);
}

var joe = new Person('Joe');
joe.speak();

The status quo explained

The above code is a bit weird, but it works. JavaScript is a prototype-based language. In case you don’t know what that means, it means that it has no classes. It only has objects. JavaScript haters think it’s missing classes, but being a prototype-based language, it doesn’t need them. The biggest problem, in my opinion, is that the standard example above obscures this. The extra sugar actually gets in the way. I can only imagine that in an effort to appease Java developers, constructors and the new keyword were added to JavaScript. Unfortunately, haters gonna hate, so I don’t think it helped. I think JavaScript would have been better off embracing its prototype nature and letting the haters go off and build Java applets and JavaFX. Fortunately, with a modern JavaScript engine, you can uncover the pure prototype center of JavaScript.

In case you haven’t read a good JavaScript book written by a funny, grumpy man, I’ll explain the above code. As you might expect (by the naming convention), Person is a constructor. Every constructor (and in fact every function just in case it gets used as a constructor) has a prototype property hanging off of it. That property points to an object which will be the prototype of the object created when the constructor is called with new. The prototype acts kind of like a class, but it’s just an object like any other object. It’s just simple delegation. Any properties missing in the object created (i.e., instance) will delegate to the prototype object. Maybe a picture will help:

Another (simpler, but not ES5) way

Prior to ECMAScript 5, new (in conjunction with the prototype property) was the only way to connect an object to a prototype. Now there is Object.create. Let’s hold off on that for a minute though and talk about an even more direct way to connect an object to a prototype. In Firefox or Chrome (and I think IE >= 11) or node.js, try this:

var Person = {
  speak: function () {
    console.log('Hi, my name is ' + this.name);
  }
};

var joe = {
  name: 'Joe',
  __proto__: Person
};

joe.speak();

Okay, I think I’m supposed to tell you to never use that code again. Because it’s debated as to whether or not __proto__ should be mutable. Well, not so much debated. People just seem to really hate being able to modify __proto__. Regardless, you probably shouldn’t use it because it’s not in the ES5 spec. (But it looks like it might end up in the ES6 spec.) Anyway…

I think the above code makes the prototyping nature of JavaScript much more obvious. Person and joe are both just objects. The joe object just happens to have a magic __proto__ property that allows it to use behavior from the Person object. That is all. Nothing more. Move along. No additional complexity required. Compare this picture to the one above:

You might want to point out that the two examples are not equivalent. The first example with the constructor can run some initialization code, but the second example is just creating an object literal. Also, we’re exacerbating a major problem with the new keyword: we can’t control the implementation of object creation. If you expose the constructor directly, you’re bound to a very specific implementation for creating your objects. The second example is arguably even worse, because we’re expecting consumers to manually create their objects. Okay, let’s fix that.

Another (still not ES5) way

var Person = {
  speak: function () {
    console.log('Hi, my name is ' + this.name);
  }
};

var createPerson = function (name) {
  return {
    name: name,
    __proto__: Person
  };
}

var joe = createPerson('Joe');
joe.speak();

This is now more flexible than the original constructor-based example. Our “constructor” is really a factory function, so we can change the details of object creation at will. I would also argue that it’s simpler because it’s invoking fewer concepts. No special new keyword. No prototype object dangling off the constructor. Just objects, functions, and explicit prototype wiring. So really, move along. Take your __proto__ property, and you’re good to go.

But now, you might have two other complaints:

1. The code is longer.
2. You’re not supposed to use __proto__.

I would argue that if the code is simpler and more flexible, it makes up for the higher character count. The second problem is easy to fix with Object.create that we mentioned earlier.

Another (finally ES5) way

var Person = {
  speak: function () {
    console.log('Hi, my name is ' + this.name);
  }
};

var createPerson = function (name) {
  return Object.create(Person, {
    name: {
      value: name
    }
  });
}

var joe = createPerson('Joe');
joe.speak();

Now you might be complaining that it’s even longer. Oh, the humanity!

Object.create is the approved way to wire up an object to its prototype. The first parameter is the prototype object, and the second parameter is an object that contains property descriptors. Again, we’ve picked up some additional functionality here. By default, the property descriptor makes the property read-only and not enumerable. By contrast, the traditional approach of adding properties in the constructor gives us mutable properties. And Rich Hickey is sad when you create mutable properties! You don’t want to make Rich Hickey sad, do you?

At this point in my original blog post, I was going to tell you how to ignore Rich Hickey’s feelings and create an evil wrapper around Object.create, and I was going to show how to leverage prototypes to decrease the amount of typing required to use prototypes. Maybe I’ll do that in a later post. But for now, I erased all that and instead will skip to…

The bad news

I wrote some jsPerf tests to determine the performance difference of using the typical constructor/new pattern and using Object.create. Unfortunately, in Chrome, using a constructor is about 88 times faster than using Object.create. That kind of sucks if you’re trying to avoid using new.

I had planned on preaching that you should just abandon constructors and new except when you absolutely had to use them because some API or framework required them. But I now also have to say not to use them if you’re constructing a lot of objects. And that’s a bit more of a gray area. Because of that, I won’t try to preach anything. Chrome can (in my test on my aging iMac) create over 400,000 objects with Object.create. So, in a lot of cases, you’re not going to notice a difference. You can create thousands of objects in an imperceptible amount of time. So, if you like the pattern above, then use it. But if you’re doing something crazy like building an abstract syntax tree (something I do once in a while), you’ll have to optimize back to traditional constructors if you want to squeeze out some extra performance. For that reason, I won’t try to convince you to use it. Instead, I’ll let you decide what’s best. Personally, I’ll probably just use constructors to keep my code consistent. I will use (and recommend you use) a factory pattern though. Something like this:

var Person = function (name) {
  this.name = name;
}

Person.prototype.speak = function () {
  console.log('Hi, my name is ' + this.name);
}

Person.create = function (name) {
  return new Person(name);
}

var joe = Person.create('Joe');
joe.speak();

This doesn’t get rid of constructors but isolates them to a single spot. That allows you to easily change the way you create objects. (And, no, you won’t pay a performance penalty for this.)

So, wait, what are you saying?

1. Constructors are not necessary from a functional standpoint. You can get the same functionality by using a factory function to explicitly wire a prototype. Less features; more power.
2. Unfortunately, JavaScript compilers/interpreters optimize explicit wiring quite differently from implicit constructor wiring.
3. This only matters when you’re creating lots of objects.
4. Don’t start writing a long, preachy blog post without first doing some research!

From an article on Wired:

According to the guy who’s been in charge of Google’s servers for nearly 15 years now, the company has five-year data projections about how much capacity it will need, but he doesn’t put much stock in them. “We don’t believe our own numbers. The error bar is just too high,” says Urs Hölzle, Google’s Senior Vice President for Technical Infrastructure. “Five years ago, you basically didn’t have an iPhone, you didn’t have app markets. YouTube was much smaller. Facebook basically didn’t exist… It was just a radically different world. So who am I to say that I can know how the world five years from now is going to be radically different? I kind of know it is going to be, but I can’t tell you which ways.”

Again, the important part:

“We don’t believe our own numbers. The error bar is just too high,”

So can we stop making business and technical plans as if we can predict the future with a crystal ball? I’m pretty sure Google is more successful than your business by all metrics. If they can’t predict the future, then why do you think you can?

Of course, they do try to predict the future:

the company has five-year data projections about how much capacity it will need

Plans are useful. You can print them on a piece of paper and hand them out so everyone has an idea where they’re going. But you should also print something at the top like “This fantasy plan was printed on June 18, 2013. It will begin to deteriorate on June 19, 2013, or possibly sooner.”

Again, just to remind you: Google knows their plans are fantasy.

“We don’t believe our own numbers. The error bar is just too high,”

Please know that your plans are fantasy as well. If you claim to be agile, it’s right there in the manifesto:

Responding to change over following a plan

Yes, there is value in following a plan. But technology thrashes so much that you have to acknowledge that every plan is fantasy. Use the plan for planning. When it comes time to do the doing, be ready to face the cold fact that reality doesn’t give a crap about your plan.

I’ve used github issues as a tool for tracking stories and defects for almost two years now. I’ve always liked github issues because of its simplicity. Sure, it could use some more features, but its lack of features is often a benefit, in that I don’t waste time obsessing over configuring and perfecting the tool, as I have in the past. More importantly, when someone asks something like “Can you add a field so I can track my new favorite metric?”, I can happily say “No, I can’t,” and then get back to work. But one thing always bothered me: having to see everything as an “issue.” It just felt wrong. When I created a story, it always weirded me out that it was still fundamentally called an “issue.” That sounded a lot like “defect,” and I was putting stories in there. I didn’t like thinking of my stories as defects.

But recently, I found a solution to this problem. No, I didn’t find an option in github to change “issue” to “item” or whatever. I didn’t use greasemonkey/tampermonkey to override the text with a label. (Yes, I considered that.) No, the solution is just to consider that everything is a defect.

Think about it. Why are you writing code? You’re writing code because there is a problem. The problem is one of:

1. The software should do something that it currently doesn’t.
2. The software should stop doing something that it currently does.

It may not do something because it doesn’t yet exist. Or a feature wasn’t added yet. Or a feature was added, but it doesn’t work right. Maybe there was a fire in the data center. Or the UI is just too damned confusing.

It may need to stop doing something because we’re serving a new kind of customer, and that new kind of customer doesn’t want a particular behavior. Or a feature costs more in support costs than its value. Or in production, the feature has a quirky side effect because the database is configured slightly differently. Maybe there was a fire in a third-party data center, and now alarm emails are being sent out and taking down the email service. Or the UI is just too damned confusing.

Does it really matter why the software does or does not do something? I don’t really think so anymore. There’s a problem, and if we consider that problem a priority, we need to fix it. Calling it an issue, which is a synonym for problem works just fine for me.

Sure, you can try to create a formal definition. For example, you can draw the boundary at what was clearly defined. If I say to write me a function that generates a Fibonacci sequence, and you make one that gives me 14 when it should give me 13, then it’s clearly defective. Unfortunately, that’s a toy example. For real software, there will undoubtedly be more nuance. Every second you waste arguing over how to categorize your problems is time you could be spending solving them.

There is one case where it might be important to separate types of problems, and that’s in contractual obligations, either formal or informal. If one party contracts with another party to create a piece of software, there is a difference between missing features and buggy software. The difference is still subject to the same nuances though, and a verbal dance will be required. More importantly, many things have to go wrong to reach such a point, and I would argue that it’s just a distortion of economics or politics. At its core, there still is no fundamental value in the distinction. It’s just a means for one party to manipulate the other. The solution (I think) is better contracts and more transparent processes that allow either party leave at any time.

You might also want to call subproblems defects. For example, if I catch a problem in a particular story before it’s released, then maybe that’s a defect. But I think this is just a tracking distinction. I can just as well call that a task or a sub-story or a thingy. For measurement and tracking purposes, it is different from a story. But it’s still a problem. Maybe the original story was poorly worded. Maybe the developer is new and doesn’t understand the domain. Maybe the wireframe was wrong. Maybe somebody changed their mind. Maybe it turns out the UI is just too damned confusing. Again, it doesn’t really matter. There’s a new problem that’s standing in the way of solving the original problem. That sucks, but arguing about the type of problem is a waste of time and energy.

Another possible positive side effect of viewing it this way is that it removes the stigma from a developer working on a defect instead of implementing a feature. If we just view everything as a problem or puzzle to be solved, then it’s all good right? Nobody gets “stuck” working on defects because everybody is stuck working on “defects”. No need for gimmicks like bug fixing fridays or defect leaderboards. (Google for it, those ideas are out there.) Just prioritize all your problems and get to work.

Also, keep in mind that every piece of software has bugs. Lots of bugs. Lots of bugs that you will never, ever find. There’s a little thing called the halting problem that basically makes it impossible for you to know what your software will do in the future. (Except in special cases of special software written with special languages in special ways. Trust me, your software has bugs.) Oh, you have unit tests? Yeah, I have those too. But those only show places where your software works. Unit tests don’t show you places where your software doesn’t work. If a piece of software has a bug (in the forest, of course) and nobody finds it, is it a bug? If you haven’t found these bugs, then they’re definitely not priorities, right? If you knee-jerk fix things because they’re classified as defects, that’s…uh…dumb.

Of course, prioritizing your problems is its own problem, but removing the distinction helps a little. When you start looking at everything as a problem, you have an easy metric to figure out if something should even be considered as a priority. I think there’s a missing step when writing the agile boilerplate “As a [role], I want [my desire].” I think we should start by asking “What is the problem we’re trying to solve?” If you don’t know the problem, then I would argue you shouldn’t be working on the solution. The standard agile boilerplate tends to jump right into describing the solution.

I think the user story template puts us in the mindset of making wishes come true. And I think that often leads to feature bloat. And that leads to more complicated software that is confusing and breaks a lot. If we just step back and act as basic problem-solvers, we might, well… actually solve problems.

I recently shared that “every line of code is a liability” with a former colleague, and he later thanked me for the advice and the fish tacos (that I bought for lunch). I know it’s not new advice, but it was new to him, and it was new to me some months ago. So, I’ll make it the job of my first-ever blog entry to package up that advice and hopefully spread it around some more. And, perhaps, spin in some nuance.

A tale of two toys

First, an analogy. Not because I think my analogy is so clever, but to create some distance and remove any emotional attachment to code.

When I look at Lego pieces spread out on the playroom floor in my house, I am pleased. That’s because the liabilities seem diminished by the asset values.

Lego as liability

• Took money
• Takes up space
• Takes time to pick up, organize
• Takes time away from other activities

Lego as asset

• Provides entertainment multiple days per week
• Teaches engineering/architecture principles
• Creative outlet
• Encourages free play
• Encourages related activities like making stop-motion movies

Contrast this with the foosball table.

Foosball table as liability

• Took money
• Takes up a significant chunk of space
• Takes time away from other activities
• Is used so little, it collects dust

Foosball table as asset

• Encourages spirited, social play

My point is, don’t buy a Foosball table… or at least buy a cheap one off of craigslist… wait, that’s not my point at all.

This same comparison can be made of any toy or any thing for that matter. Inherently, you are guaranteed the liability half of the equation. If you get a dog, you will have to pick up poop. Buy lettuce, and some will go bad. The question is: does the asset side of the equation compensate? Is it enough that your dog greets you at the door each day? Will you make enough salads?

And so it is with code

Of course, the above seems obvious. But it’s not quite so obvious with code.

Those unfamiliar with the idea of code being a liability tend to think that any code that does something is valuable. Being busy writing code has to be better than not writing code. But code has hidden costs, just like toys.

Those who are familiar with this idea may subscribe to a kind of buddhist minimalism where they believe that the best code is none at all. Go make an empty repo on github and try to sell the product that runs on that code. Let me know when you’ve made a dollar. And if it’s legal.

Code as liability

• Takes time to create it
• Takes money to create it (if somebody is getting paid)
• Takes up cognitive space by adding complexity to the solution
• Takes time to maintain it
• Takes time to support it

Code as asset

• Coding practice
• Creative outlet
• Might make money
• Might save time by automating manual task
• Might entertain or educate

The above is not meant to be an exhaustive list, and things often get perverted. If a vendor is being paid to create the code and maintain it, the vendor can often treat those liabilities as assets. For the list above, I’m assuming non-perverted economics.

The point then, finally… Every line of code is guaranteed to be a liability, in several ways. But the chances of a given line of code being more than a coding exercise is pretty dismal. Code is just a necessary evil and holds little or no intrinsic value.

So, how do we reduce the risk of writing worthless code? I’m not sure I know the answer, but I think it’s similar to how you reduce the risk of buying worthless toys. You can’t analyze every possible decision up front (à la Big Design) and have your kids sit in a barren house. But you also can’t take a wad of cash to Toys R Us and randomly throw it around. Think ahead, but not so far that it paralyzes you. Make some mistakes, but throw out the junk. Absolutely throw out the junk. And then admire the good stuff. Spend your coding dollars (whether real or figurative) wisely. Buy fewer, higher-quality items. Try to make sure you’re solving bigger problems than the ones you’re creating.