To estimate or to NOT estimate bugs, that is the question!
A practical way for you to handle bugs in your software team
This is a question which is as old as time itself, what do we do with bugs?
They are part of the software development process, things do go bang in production but if you ask the purists they will tell you “the best way to estimate bugs, is to not create them!”.
Mmmm yeah, we like to push boundaries and aspire for the ideal, but what do we actually do when bugs appear and we need to manage business expectations?
Before we kick off let, us just say, that not all bugs are created equal ….
A bug found during the development process:
Imagine you’re a developer and a bug has popped up as part of testing something you are working on. Now, this bug, in our opinion, should count as work that is part of your normal workflow. In other words, don’t go and create another bloody card for the work item you are on that is broken in some way. Just fix it.
We get it, you might be quick to say: “But if this happens too much, it means developers are producing low-quality code & we can’t see what was broken.” Maybe there is some truth to that. We lose transparency & we don’t want this to be happening every single time (where we lose visibility of what was broken with our work during the sprint). However, I would not want to spin up a separate work item when delivering value during the development process.
In plain English, would we create a separate test work item for the piece of work you’re on? Would we create a separate release work item as well? No, ideally not - so why would we do that for a bug which is part of the work?
A quick spoiler for the blog: when we are talking about estimation, it’s not for the work found during your development process.
A defect found in production:
If a bug is found in production, then that will come in as its own discrete piece of work. This is where we might talk about estimating it.
Why you should NOT estimate bugs
How in the world do we estimate things which are unknown? Something that seems small might take 2 intense weeks to fix. Something that seems large might be fixed in 30 minutes. We don’t know where this bug sits, so it’s very difficult to accurately estimate, right?
Yes, but hold on: isn’t this true of all work? The same unknown unknowns apply to work as well as bugs. True. What do we do with that work? Well, we might run a little investigation through a spike, and if it turns out to be larger, break the work up into smaller chunks, or ask team members to swarm and help. We can take this same approach to bugs. But there’s a more important reason why we deter people from thinking estimating bugs is something to normalise…
You’re putting bugs on a pedestal, damn it!
Bugs do not represent value. Bugs are mistakes we have made in the past. In a perfect world, we wouldn’t have bugs - and when we fix them, we technically aren’t delivering new customer value; we are fixing a mistake that shouldn’t have been there. So we don’t want to estimate bugs or count them as being a normal part of our work.
Since when does estimation equate to value?
If our estimations feed into the way we measure how much work or value we are delivering, this is treating our estimates as a proxy for value. This is a dangerous game, as we are getting close to putting too much importance on estimates. Estimates, or points, are just a tool and have a very low correlation with real value in the form of measurable customer impact.
Ok, ok, if you have to estimate bugs…here are some good reasons
Working on bugs takes up team capacity. One of the key benefits of estimating work is to help us try to match our incoming work with our capacity to do it. Fair enough, it’s a reality. Bugs count towards this capacity, so surely we should estimate them. But wait a second, we have a counter to our own point… we shouldn’t count bugs as part of our capacity; instead, we should always leave a certain percentage of our capacity free in our sprint to dedicate towards things like bugs, tech debt, upskilling, slack time, etc.
Ok, ok, I’ll stay on track and give you some more good reasons to estimate bugs. Delivering bug fixes can be customer value. This could be a bug or a fix that a customer has found, so delivering this fix should be something we estimate. Oh no, we can’t help ourselves, we have another counter. Again, are we equating estimation to value? To me, this indicates a misuse of estimation.
For Pete sake can you just tell me the answer: do we or don’t we estimate these bloody bugs!
For those of you who have worked with us or attended one of our skills workshops on data, you’ll know we aren’t massive fans of Story Points. I mean, we used to be - but over time we found them to be a very ineffective way to measure value, capacity, or help answer the key question that stakeholders and customers care about: when will it be done? This is another blog. (And yes, you can sign up for our "How to Estimate Without Story Points" skills workshop in July! Ping us a message.)
But briefly, the alternative we use instead of Story Points is also the method we use to handle bugs. For example, what do we do in sprint planning? We’re bigger fans of right-sizing, which goes like this:
Can we do this piece of work in 10 days or less?
If the answer is no: break it down into smaller pieces and try again.
If the answer is yes: great, bring it into the sprint.
If we start work on this and it turns out to be bigger than we expected, then we can break it down into smaller pieces now that we know more about it.
Should we follow this same method with bugs? Yes. Would this work count towards a team's capacity? Ideally, yes. If we call that estimating, then yes - it’s a good idea to estimate bugs.
Ultimately, we are fans of representing the work the team is doing as best we can. Whether it is new feature development, a bug fix, planned or unplanned work, etc. - if a piece of work is taking a significant amount of someone's time, we should ideally track and visualize it. This will help us more accurately assess the team's true capacity in the future.
If you’d like to learn more about how to estimate with our story points and how to use data to make better decisions in your teams and organizations, reach out and join us in the July Data Skills workshop.
Cheers!!