The Null Object Pattern: When a slacker is just what you need

I had a challenge that was neatly solved by the Null Object pattern. I'd like to share it with you, so that I can explore the idea and provide a practical example.

Simplifying a bit, I have a Person object, and I need to fill it with details retrieved from an external system. When I started looking at the code, the call to look up the Person attributes took a Person as a passed-in parameter and modified it. That struck me as bad behavior ("Hey! I gave you that so you could use it, but I didn't expect you to change it. Sheesh."). I thought it would be more honest for the method to return information, which the controlling class could use to update the Person object if it chose to.

Let me name the players, to make this easier to follow. I have a class coordinating activities that, in our business context, is called a Translator. I created a Client that makes the actual calls to the external system. Before the refactoring, the Translator would call Client.Lookup(Person). The Client would create a message to the external system, get back a response, and use it to set attributes in the Person.

I changed Client.Lookup so that it does not change the Person, and instead returns a Response that contains the needed attributes. But if the external system did not have any info to return, should Lookup return null, throw an exception, ...?

Usually the most appropriate answer to this question is to throw an exception. If no info means you're in an invalid state or an unknown state, then it is not safe to continue, and the code should throw. In this case, though, we could continue. We didn't require the info coming back from the external system; it was just handy if available.

So I return null? But that means every time I call Client.Lookup, I have to check whether the Response is null before I use it. And so does anyone else who might be calling it in the future. It seems disingenuous for a method to say, "I'll give you a Response, but I might actually give you an empty bag. I hope you've guessed I might do that and planned accordingly."

   12 public void DoFancyBusinessSteps(Person person)

   13 {

   14     Response response = client.Lookup(person);

   15     if (response != null)

   16     {

   17         person.Address = response.Address;

   18         person.ExternalSystemId = response.ExternalSystemId;

   19     }

   20     //More stuff based on the Person...

   21 }


I'd rather return an object that is safe to use, regardless of what answer we got from the external system, and is helpful if we received useful info. This is the Null Object pattern.

I created an IResponse interface that exposes one method, Update(Person). Next I created two implementations of that interface, a Response and a NoDataResponse.

   12 public void DoFancyBusinessSteps(Person person)

   13 {

   14     IResponse response = client.Lookup(person);

   15     response.Update(person);

   16 

   17     //More stuff based on the Person...

   18 }


Response.Update uses its fields to set properties on the Person (with a method name that clearly states it is doing so). NoDataResponse.Update quietly does nothing. This allows the Translator to ask the Client to look up info about the Person, and ask the resulting Response to update the Person.

   21 public class NoDataResponse : IResponse

   22 {

   23     public void Update(Person person)

   24     {

   25     }

   26 }


I like it. As with all good tools, it's prudent not to over-use it. If quietly doing nothing would leave the Person object in a bad state, so that it blew up or corrupted data when you tried to use it later, then don't use the Null Object pattern. Throw an exception instead. The Null Object pattern is handy when you want to return an object that can do stuff in some conditions and will be harmless in other conditions.

Follow-up to the Retrospectives Workshop

My co-facilitator Suzy wrote up her reflections from the AgileAustin retrospectives workshop, where she shares more of the great insights that came from the participants.

Retrospectives: Collaborative Improvement

Suzy Bates and I facilitated an AgileAustin workshop this morning called "Cheaper Than an Off-Site: How to hold effective retrospectives that improve team collaboration and performance." It was absolutely a blast, from planning through delivery. A benefit I didn't anticipate (but should have) is that I learned a lot, myself. I'll share some of those epiphanies here.

If you were to ask me what one thing a team could do to improve its delivery of great software on a predictable schedule, I would say: hold effective retrospectives. A retrospective is a regularly recurring discussion where the team reflects on how they work together, and what they will change in order to get better. The end of a sprint, right after the demo, is a good time for this.

We presented the following as the key ideas:
  1. Safety
  2. Diversity of viewpoints
  3. Collective ownership
  4. Structured discussion


Being a facilitator for the workshop was a bunch of fun. Collaborating with Suzy was excellent—meshing our two approaches really enhanced the content, as I brought the philosophy and she brought the practical hands-on exercises, and we kept each other moving and making progress on our preparations. If you're tempted to lead a workshop but daunted, find a teammate.

The audience participation was even richer than I'd hoped. People enthusiastically contributed, leapt into the exercises with gusto, and shared some striking insights. I learned neat stuff. Here are some highlights:
  • Rotate who holds the role of retrospective leader (or sponsor). That brings fresh ideas and spreads the sense of shared ownership.

  • When a team member expresses dissatisfaction (for example, through a team satisfaction histogram), sometimes it is appropriate to acknowledge it without delving into why and root cause and solutions. If team safety means I can say how I feel and that's okay, then I should be able to say how I feel without getting interrogated about why I feel that way.

  • A bunch of "went well" items without any "needs fixing" items might indicate a lack of team safety. In today's exercise, our sprint teams barely knew each other, so they wanted to be polite, and so they were very positive in their reflections. When you're comfortable with your team and trust them, it becomes easier to talk about areas for improvement. I'd view a series of "everything was great" sessions as indicative of a lurking problem.

  • Affirmative Inquiry is a philosophy/strategy worth looking into. It replaces, for example, the negative "why is this broken" by changing it into a positive "what was successful in the past that we could apply here."

  • Book recommendation for Jim Highsmith's work (probably Agile Project Management?).


If you attended the workshop: Thanks! It rocked. If you weren't able to, keep an eye out for future AgileAustin workshops. They're announced on the AgileAustin email list, so sign up for that. If you don't live in Austin, well... nanny nanny boo boo.

Quantifying Benefits on Refactoring Work

When last we talked about estimating and prioritizing code maintenance, we'd left it as an exercise for the reader to devise a method to compare the relative size of the benefit from two different refactorings. In other words, you can work on A or B; which one is likely to give a greater benefit? I've had an idea for how to do this. It even includes some satisfying math.

Start with the premise that, when estimating the size of effort, it is sufficient—nay, preferable—to compare values in an abstract unit ("story points") that does not map directly to any real-world concept (such as "hours"). When you try to estimate effort in a real-world unit, people get distracted by (hung up on) the wrong details. Better to use an abstract unit that lets you compare the relative sizes of two efforts. Then you can prioritize them ("Let's do the smaller one."), and over time you develop a predictive measure of how many units your team can deliver. Can we create a similarly abstract yet useful unit for comparing benefits?

You undertake a refactoring because you want to make the code better. The benefit from this work comprises two pieces: how much you're going to improve an area of the codebase, and how important that area is. Your impact will be made up of how much "betterness" you can impart, and how much the betterness will matter.

You've probably had a similar experience: You find an area of the code that makes you frankly itch to improve it. You could dramatically increase its beauty. But it's not really used in very many places, and business needs hardly ever drive you to change it, so its beauty or lack thereof is actually rather insignificant. On the other hand, there's a class that makes you queasy every time you have to interact with it, but it's used everywhere, so changing it would be a huge, risky undertaking. It stays ugly, despite being so important.

There are a number of intuitive and emotional influences in those decisions, and they interact with each other in additive and multiplicative ways. This is a good place to apply some rigor, to get the emotions out of the way and compare options more objectively. You apply a similar rigor when you tackle a difficult decision by actually writing down the pros and cons in two columns on a piece of paper, so that you can see how the two sides stack up. So let's apply that to comparing the possible benefits from two refactorings. We only have time to do one, so we're trying to decide which one to do.

Consider a questionnaire, with pairs of questions, asking about what the code might be like after you're done.

  1. Future proofing:


    1. How much easier will it be to change it the next time?

    2. How often do we get asked to change this area?


  2. Regression proofing:


    1. How much better will our test suite be able to prevent defects in this area?

    2. How business-critical is it that we don't introduce defects in this area?


  3. Avoiding risk:


    1. How likely are we to succeed without creating problems?

    2. How tolerant is our business of risk in this area?


  4. Improving satisfaction and increasing velocity:


    1. How likely are we to reduce tech support incidents by this work?

    2. How many of our tech support incidents can be attributed to this area?


And so on, with questions that are specific to your own team. Collaborate with your team to create the questions, so that they represent the team's decisions. Note how a pair of questions covers how much improvement and how much that improvement will matter. Also see that for each question, a more emphatic answer is a good thing. For example, the avoiding-risk question asks how likely we are to succeed, not how risky the task is. Questions are phrased so that, the more you say "Yes, a lot," the more that's an endorsement in favor.

Now, depending on your bent, this next bit will remind you either of a prioritization matrix or a Cosmo quiz. Nevertheless, for each question in your questionnaire, answer a 1, 3, or 5, to represent "barely," "some," "a lot." Then multiply the coordinated a's and b's and add those up: (1a * 1b) + (2a * 2b) + (3a * 3b)... In this way, you represent the multiplicative relationship between How Much Better and How Much Does It Matter.

We need a unit for these scores. I'm thinking "bunits" (BYOO-nits) because they are a unit of benefit and of beauty. So if you have two refactoring tasks on the table during your sprint planning, and Refactoring A has an effort of 13 story points and a benefit of 17 bunits, while Refactoring B has an effort of 8 story points and 23 bunits, you can lean towards choosing Refactoring B. As with all matrices of this type, if that decision completely flouts your intuition, then discuss it with your team—maybe your intuition can be calmed, or maybe the questionnaire is failing to cover an important aspect of your work and needs some additional questions.

The real benefit of refactoring is proven only over time. Keep previous bunit estimates in mind and use those in your comparisons and trade-offs when selecting refactorings to undertake. Add this technique as another tool to help you understand the parameters of your decision, but never to override your own good sense.

This is a technique for quantifying the benefit of work that does not have a direct dollar ROI. Why quantify benefit, especially in a unit that has no analog in the real world? Two reasons. First, if you're going to prioritize your to-do list, every item needs some sense of effort and some sense of reward. It just makes sense to do easy things with lots of benefit before hard things that barely matter. Second, asking yourself questions like the above imparts rigor to your decision-making process. We are seekers of beauty, and want desperately to clean up whatever icky thing we looked at most recently. Surveying the choices and comparing relative benefits prods us to make sound business decisions, instead of scratching an itch.

The Trouble with Technical Debt

This article is not about the perils of accumulating technical debt, nor the challenges in paying it down. Instead, it is a call to action to the developer community that we change the way we talk about scheduling technical debt.

It's a constant sore spot and the source of many arguments between developers and product owners: How do you make time for refactoring? When do you pay down your technical debt? I say we developers need to change how we make our proposals.

We ask our product owners to prioritize the backlog: to attach a business value to each user story and put them in order. Then we provide a size estimate for each story, indicating its relative cost in time and resources, and voilà: a prioritized list of requirements with a cost and a benefit for each.

Then we propose investing some time in cleaning up a lurking problem. The work will improve our velocity in the future. By how much? A lot. And how long will it take? A while. So please fit this chunk of work into your carefully prioritized backlog.

You see what we have here, folks: an ArgumentNullException. The product owner has sorted a list of objects based on their cost and their benefit, and then we try to place an object into the list with an undefined cost and an undefined benefit. No amount of arguing is going to make that work.

The first counter-argument I receive from folks is that the business should trust us or, from the more progressive thinkers, that we need to earn the trust of the business. Um, perhaps. But imagine you were the person prioritizing your own list. Even if you trusted your own motives perfectly, how would you prioritize an item with unknown cost and unknown benefit amongst a list of items with known costs and benefits? And that's assuming perfect trust; I'm skeptical of our discipline to admit that substandard, icky code can be good enough. Developers (the good ones, anyway) have a strong desire for elegance and simplicity.

The second counter-approach I hear is timeboxing: allocate a fixed amount of time to refactoring and only spend that much time. That fills in a value in the "cost" column, but it does not address the undefined benefit. Also, can you think of a scenario where you've undertaken a significant refactoring, the clock ticks over, but you're not yet in a stable state? You either roll back the changes (and realize zero benefit), or muscle through (increasing the cost)? For the timeboxing approach to honestly reflect your cost, you must work with the discipline to make many small, beneficial changes, and stop when the clock runs out. Are you up for that?

Speaking of size, be sure to consider the testing cost. Technical debt accumulates in the areas folks are afraid to change. Those are usually business-important, application-spanning areas. Your refactoring will likely touch many features in the application, warranting thorough regression testing. How good is your automated test suite? Be responsible about how much testing you'll need, and be honest about how much time it will take.

Okay, agile developer community. If we're going to get technical debt work and refactoring into a sprint, we need to apply the same rigor that we ask from our product owners. Cut the work into a discrete unit. Write acceptance criteria so you'll know when you've finished. Make some kind of estimate of the benefit; talk with your teammates about how you might quantify such benefit. Using the acceptance criteria, estimate the complexity and size, same as if the work had been requested by your product owner. Observe and reflect on the predictive accuracy of your cost and benefit estimates to improve your estimates in the future. Start conservative to build experience and credibility over time.

Most product owners are happy to invest time in making the product more stable and making the team faster on future features. We need to give them estimated costs and estimated benefits so they can fit that work amongst other business priorities.

Organizing Ideas with Concept Maps

I love concept maps as a way of explaining a topic. (Here's an example, capturing what I learned at a KaizenConf session.) If you're a visual thinker, you'll definitely want to check this out. If you design UIs, this is also of interest.

The way I use concept maps most often is exploring and explaining a concept to myself. The act of drawing the map sorts my ideas visually, lets me hang new information off logical hooks, and gives me a picture to visualize when I want to recall the info later. (If you've talked with me about F# and watched where I gestured, you've seen that OO ended up on the left side of my map, and functional programming was right of center.)

CmapTools is a free software application that makes drawing maps intuitively easy—better than paper, because you can move concepts after you've fleshed out more of the landscape. And I have a strong preference for paper for brainstorming and thought-capturing, so that's saying something.

I like it not only as a tool, but also as an example of usability. It's so low-friction because you trigger actions (creating a concept, making a connection) right where you're already focused, in the work area, not in some menu at the top of the window. Click-and-drag from an existing concept to create a new concept and connect them; then type, click, type to enter the labels on the connection and the concept. I admire CmapTools for its non-noisy GUI—and of course I love it as a user because I can create a map fast enough to not lose the thread of my thoughts.

The example linked above is cool in another way: It's on a wiki where participants at KaizenConf are creating the conference proceedings. Once you go self-organizing, baby, you'll never look back.

Oh, I needed that.

So the thing about mind-expanding, discussion-rich conferences is they can leave you feeling a little overwhelmed with how much more you have yet to learn. At the end of the day, I wanted a cup of tea. The aphorism-inscribed flag on my teabag said:

"Keep up."

Sprint Heartbeat: Visual Task Tracking

Continuing to think about visually tracking the health of a sprint, I advocate tracking at the task level, rather than the story level. Recall that my goals for a successful solution include:
  • Radiate information.
  • Clearly communicate whether the sprint is on track and likely to conclude successfully.
  • Alert us to lurking risks, so that the team can react and adapt proactively.
  • Tell a story about the sprint from which we can learn during our retrospective.

Instead of moving user stories through phases (development, review, testing...), list their tasks. Tasks for a story could include testing and reviewing pieces, merging code into previous versions, meeting with a product owner to review the UI and clarify requirements, or convening a design session. You can add tasks to the list as they are discovered, which can signal that a story is ballooning or that a technical problem is thornier than originally estimated. If there are certain steps the team is trying to get more rigorous about (like writing unit tests before writing code), you can list them as explicit tasks until they become second nature.

Each day, estimate the number of hours remaining in each task. Not the hours spent, and not a comparison of estimates to actuals. This is not about "metricking estimation accuracy" or some other useless bean counting; it is about ascertaining right now, today, how many hours' worth of work is there left to do, and will that fit in the time we have left.

The hours remaining on a task might go up. (Ever peel back the wainscoting on a piece of code and find it's all termites underneath?) New tasks might get added during the sprint. The reason to track the hours remaining is so that you can adjust when you spot a potential problem—perhaps a team member is getting pulled away with other work, or is stuck in a mess of code that your expertise could alleviate, or maybe it's time to negotiate a trade-off with your product owner. To deal with any of these effectively, you need to know, if you'll pardon the vernacular: Where we at?

Sample agile task list, tracking hours remaining for each task

In the sample task list above, you can see that we're not only showing the hours remaining, but also tracking the movement of that number day after day. That allows us to make a graph, creating that visual heartbeat we're looking for.

Sample sprint burndown chart, comparing each day's hours remaining against an optimal trend line

At the end of the day (or completion of a task), update tomorrow’s column for the number of hours remaining for that task. If you finish a task on Day 4, then at the start of Day 5, there are zero hours remaining for that task. New tasks that crop up unexpectedly should show a zero until the day on which they were discovered, and then an hours-remaining estimate until finished.

The graph plots each day's sum of estimated hours remaining. It also shows an idealized burndown line, as if the team were completing [original hours estimate / number of days] worth of work each day. This makes the picture clearly communicate whether you're in fair weather or potential trouble.

If estimating hours remaining seems daunting, split the tasks into smaller pieces. A task should fit within a day, and the work you claim in the morning stand-up should fit within a day.

Time tracking is a sensitive topic, and suggesting it may not be well received by the team. Despite good intentions, many organizations use time accounting policies abusively. Superstitious beliefs, uncharacteristic vehemence, and irrational resistance are indicators of prior abuse. There is a tension between the two Agile tenants of transparency and self-organizing teams: I don’t want to hide data from you, but if I let you see the tools we’re using to track our progress, you’re liable to interfere, draw wrong conclusions, or base bonuses and promotions on them. These are not flaws in tracking sprint progress, though; they are deeper organizational issues, that are worth correcting and rehabilitating so that the team can take advantage of a burndown chart's benefits.

A graph tracking your team's completion of work gives you a picture of the state of the sprint, making it easy to see if you're on track to succeed, and alerting you to struggles early enough to react.

Visible Sprint Status: Maybe Not Corkboards?

I'm thinking about ways to track the activity—the status... progress... stuff—that happens within a development sprint.

You could conceive a user story as moving through a series of phases (e.g., development, code review, testing, accepted). You might set up your Scrum task board using this paradigm, where you move an actual piece of paper between different columns on a corkboard. Some project-management software packages I've evaluated encourage a workflow of phases.

I wonder if there's a better way. At the user-story level, the idea of "passing through phases" seems Waterfall-ish, which always snags my attention because so few human endeavors are that linear. Phases lead you into some non-Agile patterns of thought, and they obfuscate the current health of the sprint.

Mike Cohn's task boards (with pictures!) divide into rows as well as columns. A row corresponds to a user story, and the tasks that compose that story move through the columns, which correspond to statuses. I'd previously missed (or forgotten) this distinction, and I thought the cards on the board were stories. I finally noticed the difference while drafting up this blog post, because I went looking for corroboration about the disadvantages of moving user stories through phases on your task board. What follows is an analysis of those disadvantages: non-Agile patterns and obfuscated sprint status.

By non-Agile patterns, I mean that columns on a board draw boundaries (literal and metaphorical) between collaborators. Do you intend to tell your testers that they can't think about a story until it lands in their column? Are you keeping your product owner from looking at a feature until all the code is built and blessed (and you've run out of time in the sprint)? Are you throwing code over the wall at people who might as well be in a different department?

"Heck no!" you cry. But doesn't a task board with indivisible user stories, sitting inertly in discrete statuses, imply that's what you're doing? Well, it could. And the metaphor subtly seeps into your thoughts and behavior, constraining your team's responsiveness, creativity, and collaboration.

How does a story-oriented task board obfuscate the status of your sprint? After all, it is tangible, visible, large-as-life where everybody can see it. But what is it telling you? Here are some challenges I've observed.

The pieces of paper representing user stories are the same physical size, but the effort required for different stories can vary dramatically. You glance at the board for a gut check, but you'll draw the wrong conclusion from lots of little stories (unnecessary worry) or few big stories (misplaced complacency). You could ameliorate this by making the pieces of paper represent component tasks instead of whole stories, but does a mere task move into the testing phase? Does your product owner evaluate a task? Can your customers use one? No, they need complete user stories. Once you've made the papers represent similar-sized chunks of work, the moving-through-phases metaphor gets in the way again.

Phases are liable to proliferate, until you want more columns than is practical for an actual corkboard. For any given phase, you might want to distinguish between "Ready to be picked up" and "In progress." Even if you're tracking virtually in a project-management tool, having too many phases is still annoying; you spend your time clicking dropdown lists instead of creating software. If you're in this mode, though, one solution for distinguishing between "ready" and "in-progress" is to write your name (or set the "Assigned To" field) on an item when you claim it.

Some items visit a phase more than once—like when a tester discovers a bug and gives the story back to a developer. (See that hand-off? That's the non-Agile-ness rearing its head again.) So when you see an item in the development column, is that its first trip through, meaning a lot of work remains, or a subsequent trip and it's nearly finished? You can't tell by looking at the board.

A communicative tracking tool would show you where bottlenecks or wasteful idleness is occurring. Using phases obfuscates this as well, because it implies that your testers aren't even looking at a user story until the coding is complete. (Can you call a piece of coding complete if it hasn't been tested or shown to the product owner? I'd say not. So read "complete" as a different word that represents "passed from the Development phase to the Testing phase.") But testers at the beginning of a sprint are creating test plans and collaborating with developers and product owners on the acceptance criteria; their "Testing" column looks empty, but they are in fact very busy with useful work. Someone outside the team might look at the board, conclude that the testers are idle, and be tempted to distract them with other tasks.

What a list. It's strange to me to write it. At my old job, my team had to have a virtual task list so that we could share it across continents. I was always jealous of the kids who got to have real live note cards on real live corkboards. Oh, how wistful I was, to have such an elegantly simple solution.

But now that I've used a task board, I can see its limitations, and I wonder about alternatives. I still want to cling to the belief that a tangible solution is better than a software tool, but I might have to let that go. Software is much better, for example, at turning points of information into a communicative picture. It is also better at searching, archiving, sharing with out-of-town stakeholders, and scaling.

The right solution will radiate information. It will clearly communicate whether the sprint is on track and likely to conclude successfully. It will alert you to lurking risks, so that the team can react and adapt proactively. It will tell a story about the sprint from which you can learn during your retrospective. These are the requirements by which I'm evaluating tracking techniques. The right method will show the heartbeat of the sprint.

You're too kind. Tip yer waitress.

There's a "Pho King Coming Soon" sign near my house, which means there will soon be another Pho King restaurant. I wonder what their Pho King food will be like. I hope they have good Pho King service, and a nice Pho King ambiance.

Oh, why get excited? It's just Pho King noodles.

Yes, Jon and I can amuse ourselves for miles with this gag.