Shopzilla Technology Blog

The other day, I talked to Paul, a product owner, about a feature and said I wanted to remove it. He said, “no, we might as well just leave and not use it, it’s there, it’s free, maybe we’ll use it at some point, and there’s anyway a cost associated with removing it.” I agreed with what he said at the time, but something about what he said kept nagging me. It felt like I was missing something. I think I’ve understood what it was now, something about the cost of a feature – obvious in hindsight, but it took me a while to get there. What I realised was that existing features are not free. Or, phrased differently, finished features lead to development cost even if you don’t touch them.

The feature we were discussing was ‘presearch’, which allows us to predefine destinations for specific user searches that we feel don’t lead to optimal results. So if somebody searches for “sony tv”, that could have a predefined search result that points it to the Televisions category and where the search engine is asked to provide results for “sony televisions”, for instance. These predefined search results sit in a database table which has a couple of thousand entries in it, more for some countries, less for others. The reason why I felt we should remove it is that it requires manual maintenance of the presearch results and that just doesn’t scale really well. The result is that we don’t actively maintain the presearch values, so we don’t really know if they improve searches or make them worse.

Disregarding the benefits of the presearch feature for the moment: if we just let those couple of thousand predefined search results sit there in the database and leave the code in, that doesn’t cost us anything, right? Wrong. Here’s how it cost us this last week:
- One of my colleagues spent a day or so trying to figure out why a particular search he was making was getting very strange results when others seemed to work as expected – the reason: that one search happened to hit a presearch keyword.
- When making a code change to the way we treat searches, the change became more complicated in order to ensure that we didn’t break presearch queries.
- To be sure we understood how the features should interact, we needed to communicate with some people at the US office, which due to time differences and a couple of emails going back and forth took us a couple of calendar days.

Now, we don’t always touch code that is associated with presearch (or some other feature X whose business value might be in decline), so in our development team we don’t get this level of cost every week. But it happens often enough to be significant. And what’s more, in QA terms there is a probably larger cost associated with regression testing every existing feature in a large system.

So, my picture of what a feature’s lifetime cost looks like has changed like this:

Basically, there is an up-front development cost which includes designing the feature (business/product owner-type work), implementing it (programmer-type work), verifying it (QA-type work) and launching it (CM/system operations type work). Once it is included in the product and live, it continuously incurs what I for want of a better word will call an ‘ongoing cost’, not just in operating or maintaining that feature, but in development of other features. By development, I mean product definition, implementation and verification, so ongoing costs are made up of these components:
- Slower decision-making/added communication due to the larger amount of things to know and think about, both when defining features and implementing them. In the same vein, increased likelihood of mistakes in feature specifications or implementations due to the complexity of feature interaction.
- Slower implementation of new features due to the additional complexity of the system. It is harder to figure out the right way to fit things together if there are many of them.
- Regression testing costs that increase with every added feature.

OK, so if that type of diagram shows the lifetime cost of a feature, it becomes pretty apparent that for a long-lived system, the initial development cost of a feature might well be much less than the ongoing cost that follows it. Here are some things that reduce the ongoing cost:
- Automated testing: doing this well has an immediate effect on the cost of regression testing. However, doing it also has an impact on the cost of adding or changing a feature, since it is likely to break the automated test. Quality test code is important here, the problem isn’t solved by just having automated tests in place.
- Implementation quality: investing a little bit more at the development stage to ensure that the implementation is good can lead to significantly reduced maintenance costs. Insufficient investment in implementation quality is a part of what is known as technical debt. Having the right implementation means that it is easier to understand it, test it and change it without breaking it. It also reduces the interaction between features, which reduces the ongoing costs directly.
- Documentation: having described what the feature does and how it is supposed to interact with other parts of the system makes it easier for people to understand it when they need to add or update a related feature later. Of course, creating and above all maintaining good documentation is not free.

I used to think that old software systems gained cruft and lost flexibility over time only because of poorly implemented changes that would accumulate over time. That would mean that good engineering practices – being good at the three bullet points above – should be a sufficient solution to prevent system ageing. I now think that the simple facts that systems gain more features over time is enough, even if every feature is perfectly implemented, to create a significant slowdown. So that means that to keep a system healthy and modifiable/agile, you need to decommission features over its lifetime. It would be great to be able to somehow measure the ongoing costs and benefits of features so you could decommission the ones that have too high ongoing costs relative to their benefits.