Learn how to manage technical debt from a business perspective
Recently I get a lot of questions related to Technical Debt and Legacy products which triggered me to write this article, explaining how I managed technical debt a while ago. I want to share this information to provide you extra options, I do not claim this system is the best out there, it is just one of many but definitely worth a try.
What is it about?
Let me start by explaining how I think about technical debt and legacy products so that it is clear where we start from, in short it means to me:
Low quality solutions, products, modules or code from a technical perspective.
As you can see I do not make the difference between stuff from the past or the stuff from yesterday, both can build up the technical debt very rapidly. Yes, there are differences and we could spend pages trying to explain but this would not contribute to the content of this article.
Technical debt will result in longer and more costly maintenance activities, eventually ending up in a solution that is close to impossible to change or to extend. This latter statement points to a moment in time where a lot of companies start with their "NG" products, Next Generation to the rescue. After +/-5-6 years working on their NG-product, they have built (again) enough Technical Debt to ignite another NG... and so on and so on... Looking at Technical Debt as such shows us that we have to deal with a "Cost of Delay", the longer we do not address the issue the more it will cost us to deal with it. Up until some moment that it is actually cheaper to start over (NG's) instead of dealing with it. No matter what option you choose, if you do not manage your technical debt for a long time, you'll face a huge business cost for sure!
As long as you are working in a small company, having to deal with a single product or project, working in an Agile way, you might be able to handle this pretty well and you could disregard the rest of this article. But what if you have to deal with a development organization of 300+ people? A department that is maintaining and building a solution that consists out of numerous products, that on their own exists out of numerous modules, that again exists out of numerous components...?
Well, I have been in that situation a couple of times and tried different things, one that seemed to work best for the organization is one where we combined business strategy with technical debt in a single management system. First of all I worked together with the department head and found somebody inside his organization that had the right skills, mindset and motivation to become an overall solution architect for that business. Our solution architect was going to take up the responsibility to manage the department's technical debt. Together with this solution architect we generated a diagram that represented the entire solution decomposed to a meaningful level. Today this can easily be done with tools that automatically extract information from the existing code base(s) and requires little to no manual interaction.
Note: After seeing the diagram, our solution architect immediately wanted to restructure the model to a more appropriate architecture and instruct the teams to follow his model(s). It was key for me and the business to keep him focussed on the problem.
With the diagram at hand we identified which team(s) were involved in which module(s). Allowing the entire organization to validate the assumptions made by us, having an offline review round that lasted about 3 weeks. After those 3 weeks we had a good overview of the current situation and the actors involved. A good state to move on and find re-factoring items that mattered to the business as it is of no use to work on code that does not require change in the near future.
Note: Re-factoring just for the sake of re-factoring is something a lot of architects and software engineers are eager to do, keep them away from it and reason with them first.
Supported by the department director, we prepared a 1-day workshop to identify technical debt for each module in the diagram using color codes. We invited a minimum of 2 team members per team within the organization to an off-site location, having +/-80 attendees in a single room coming from different countries in Europe. The agenda: - Introduction of the assignment at hand - 1 hour for everybody to group modules (each module was uniquely identified on a single index card) in 3 groups: High, Medium or Low Technical Quality - 1 hour to split the Low into another 3 groups: Low-High, Low-Medium, Low-Low - 1 hour to split the Medium into another 3 groups (allowing people to move things from initial Low to Medium and the other way around) - 1 hour to split the High into another 3 groups - 2 hours to get through the items in the parking lot, about 15 items if I remember correctly - 30mins for everybody to review the results - Adding color codes and closing the meeting
Note: Decent facilitation is key when you setup such kind of workshops and do not try this without professional support. The above mentioned workshop was facilitated with the support of 4 professionals next to myself. The workshop required their full attention to make sure flow and energy was kept during the sessions. For example wisely using the Parking Lot for items that required more discussion to categorize.
Using the results from the above workshop we completed the solution diagram and rolled-up the color codes towards solution level, giving us a good view on the technical debt of the complete solution at any level.
At this moment it was time to evaluate the necessity to re-factor items from a business perspective. Business management was sitting outside the department and required the lobby work of the department manager to get invited and help them preparing for another workshop. We supported them structuring everything they had in mind where they needed something done by the development department. It went from business cases, business drivers to use cases and other simple problem statements. Everything was allowed and added on a list that the head of business, supported by his team, prioritized for us. This took us about 2 days of intensive workshops which I will not elaborate on in this article as this is not a part of managing technical debt but about managing a portfolio and requirements.
A third workshop was scheduled where business people allocated each item out of the above mentioned list to the solution diagram that was drawn on the floor of the meeting room. They allocated items to the level they were certain it belonged to. It did not matter if they allocated items to a product, module or component as this was in a lot of cases something that the team(s) would discover in a later phase. Important was that we could copy the color code from the solution diagram onto the todo item and evaluate the amount of cards (todo items) that were located in certain domains of the diagram and thus solution. This kind of visual representation immediately gave valuable input to the solution architect on which parts were in need of re-factoring. At the same time we got consensus from business that this was indeed very important to address, even more important than adding new features to them.
Maintaining this process
After this initial round the solution architect, together with the team(s) involved, re-factored the business critical domains and by the next time the solution architect extracted a new diagram from the existing code he discovered that not only the items he focussed on were designed better but that also all the new stuff that was added was better structured. He claims that this was a result of all the collaborative workshops they had instead of having everything arranged asynchronously and on the intranet. A bright guy that solution architect.
Today it takes the solution architect about 3d/month to maintain the process of visualizing the technical debt and having it linked to the business priorities. All the other time he has, he spends on coaching and educating teams within the domain of software architecture allowing the entire organization to grow. I hope this article will help other people around that are situated in a similar situation, giving them some idea's on how they could move on. Context will be different and as such the final solution will be different but then again... We are curious about your context and how this solution could be adapted to it. So... do not hesitate to contact us to brainstorm about it. We will be happy to support your quest to a better world for all stakeholders involved. Or you could join one of our free code- or legacy code retreat sessions that could help you reduce the risks of building up technical debt.
Total Cost of Managing Debt: - 1 day extracting the solution diagram - 1 day adding team(s) working on module level (lowest level leaves) - 3 weeks review period for the diagram by entire organization - 1 day preparing a meeting to identify Technical Debt on module level - 1 day to get the Technical Debt visible - 3 days to map incoming customer requests, projects, feature requests... (business strategy) to the diagram - 3d/month to maintain the diagram and support the business for a 300+ development organization. Kick-off 6 days over 6 weeks; follow-up 3 days for each month to avoid having to build that next Next Generation of the same thing!