The Shocking Truth about Firefighting in I.T. Organizations!

Why are there so many IT people out there who have the feeling they're actually a firefighter? Why do they have the feeling to be jumping from one burning house to another? Why do they have the feeling work is driven by fire alarms? Why is it that the loudest voice in the office, the biggest fire, gets things done?

A bunch of questions I asked myself too, and there are some organizational patterns that we should consider. Once you know them, you can beat them!

In today’s article we’ll discuss the “Contract Game” pattern.

What are the signals the “Contract Game” is active in your organization?

Let me try to help you identify wether the “Contracting Game” pattern is active in your organization by listing up some common signals I've observed within organizations from around the world.

The work to be done day in, day out, is driven by deadlines, and, most probably, deadlines are agreed upon a couple of months (or even years) upfront. Possibly by somebody who might even have left the company already or at least is, today, focused on something else (the next sale, the next project, the next budgeting round…). And when these deadlines come closer, the traffic light turns “red”! That’s when things really get done around here. When this feeling of “we should have had it yesterday” is there, that’s when suddenly stuff gets done. How? By bypassing all general procedures in place in the organization.

You feel like a deadline firefighter!

Every new release, customers are calling you because they are experiencing blocking issues. Causing an urge to stop the ongoing work, fix the issues, deliver the fix and get the customer going again. Most of the time bypassing all general procedures in place in the organization.

You feel like an issue firefighter!

And while you are firefighting the above, the organization is setting up a separate department to start building the “Next Generation” or, these days, “Green Field” product that will replace yours. As you are highly occupied with firefighting, the organization needs to hire new people for that. You feel bad because you assume they have the “fun” work now, but don’t worry, they’ll soon join you as a firefighter because the organization is designed for it.

Contract Game Pattern explained

Before we go into the game, let’s first explore the concept of a “contract” or, even better, “contract thinking”. In order to do so I’ll take you along on the journey of defining a contract between the #INeedStuff company and the #IBuildStuff company. Together we’ll explore how this contract is created, a physical contract, but the same is happening without physical contracts. Hence the term “contract thinking” and not just “contracts”.

#INeedStuff wants to improve a certain aspect of its business and has a certain amount of money bucketed for that. Let’s say 500k€. Of course, they want the most out of their money. Understandably! So they start thinking: If we have this part then it would be nice if we have that on top of it, and if that is there we could have this as well,… And this goes on for a while, expanding their “need” to have some leverage when going into contract negotiations. Trying to squeeze as much as possible out of their 500k€! And yes, as you could guess, as soon as they have spent time figuring out their solution they wanted it already yesterday.

The “need” has grown beyond the “needs” and pressure to deliver is active.

With their expanded solution, they knock on the door of, #IBuildStuff. The “contract” negotiations start.

Both, #INeedStuff as well as #IBuildStuff, know that IT projects generally run beyond budget, scope and deadlines. Which has been investigated by Oxford researchers Alexander Budzier and Bent Flyvbjerg who published an Academic Paper on the subject.

So what do you do? #INeedStuff will go into the contract negotiations with the mindset “I Need More, more, more…”, while #IBuildStuff will go into the contract negotiations with the mindset “I Want Less, less, less…”.

Take a moment for yourself and try to answer this question before you continue the reading: “When do both parties agree to sign the contract?”

You might have come up with:

  • When they get what they want and know they can build it.
  • When they feel comfortable that they can commit to it.
  • When there is a buffer for unknown added.
  • When…

Now let’s ask you another question: “When is the contract really used after signing it?”

You might have come up with:

  • For finances and payments.
  • To evaluate what’s in and what’s not.

But in short, it's when things go wrong. Really, in most cases the contract is used when things go wrong to evaluate who’s fault it is and who’s going to give in to the other.

Considering this behavior, we could conclude that both parties will sign the contract when they have maximized their opportunity to shift blame to the other party.

Contracts get signed when both parties have maximized their opportunity to shift blame to the other party.

That’s great, we have a contract, stating budget, but considering the above, also dates and scope as it is obvious you will need that in a contract to maximize your opportunity to shift blame to the other party.

At that moment in time #IBuildThings assigns a Project Manager to make sure things get delivered accordingly. The Project Manager takes the contract and makes a commitment to get it done. A nice project plan is made up: phase 1 – functional analysis and design, phase 2 – technical analysis, phase 3 – development, phase 4 – testing, phase 5 – deployment and support.

Phase 1 – Functional analysis & design

New people are assigned to clear out the need of #INeedStuff in more detail, adding some UX design. They collaborate with #INeedStuff and describe their findings in documents, powerpoints, wireframes, design guidelines and so on.

As stated before, we know things in IT projects will go wrong, so while describing the needs in more detail they remember all those times where they had to defend themselves because “This was really what the customer described to them!”. What happens now? They are smart and figure out that the customer says “something simple” but really wants “something more elaborated” and thus they describe the “something more elaborated”.

The “need” has grown beyond the “needs” and pressure to deliver is active.

As you can imagine, this takes more time than was fixed in the original project plan but no worries, we know we have great software developers in #IBuildStuff, so we can squeeze that time box a bit as well as the testing time box. Nothing to worry about, project still “green”.

Phase 2 – Technical analysis

New people are brought on board; they consume the information from phase 1 and evaluate the technical needs. What components are needed, what kind of infrastructure is needed, what will we create, what will we buy…

As stated before, we know things in IT projects will go wrong, so while describing the technical needs in more detail they remember those times they had to defend themselves that “this infrastructure was more than good enough for the given needs!”. What happens now? They are smart and add all the things they can imagine are needed: failovers, load balancing, recovery,… 

add all the things you can imagine are needed: failovers, load balancing, recovery,… 

As you can imagine this takes more time than estimated in the original project plan but no worries, we still have great software engineers and they will nail things from the start so let's squeeze the development and definitely the testing time box. Nothing to worry about, project still “green”.

Phase 3 – Development

New people are brought on board to build the things described in phases 1 and 2. They take in the documentation, which is now a lot more than they expected from the initial contract and project plan. But hey, they still have a couple of months to go. Nothing to worry about, project still “green”.

At the same time, developers are fixing defects from previous projects, maybe even sneaking in a couple of small extra features customers are screaming about. This takes up a lot more time, time that isn't there as those projects were closed. This time is taken from the ongoing projects and the closer to the end of the development period the more the urgency grows to get things built for these ongoing ones. A burning platform that is so hot that developers forget about craftsmanship and just hack things together. Magically things get built in time!

Developers are true magicians and always hit the deadline.

But by hacking things together we initiate a new wave of urgencies that will take up time from the next project to come. A vicious cycle! But hey, project deadline is met, so still “green”. See prevous article on Technical Debt.

Phase 4 – Testing

New people are brought on board to test the delivery against the things described in phases 1 and 2. As mentioned in previous phases, test period has been limited to the absolute minimum, most probably just enough to test the delivery 1 time.

Testers take in the documentation, create test plans and go for it. As you know, lots of things don’t seem to work as expected and lots of defects are registered in the system. Making it very clear, impossible even, to hide anymore, that the project status is actually very very “red”.

No wonder, we all went to school and know the effect of whispering a sentencein the first person's ear, passing it via several kid's ears and having it written down at the end. We all know that is not working well but still in organizations we keep this alive.

The “red” status makes sure the developers focus on defect fixing, even though their time was planned for another project already. Therefore disturbing the plan of the next project, making sure we keep working as firefighters.

Phase 5 – Deployment and Support

New people are brought on board to deploy the delivery (with known defects) onto the infrastructure defined in phase 2. From now on things go from a CAPEX perspective into an OPEX perspective, in some cases this might be a totally different service contract. In other words they are on their own!

The developers from phase 3 are highly occupied firefighting several issues already and as operational support is a different organization (internal or external), they will not help out anymore. Ensuring that the new people supporting the operations will find ways around the defects they encounter, even build/develop their own extensions on top of it. The origin of #devops maybe.

As you can see in the above scenario, everything has been put in place to stick to the contract and the contract was the optimization goal within the organizational structures that are in place. This is causing all people involved to be highly occupied in firefighting, either by fixing issues or by finding ways to blame the other party: the contract game pattern.

Of course, the #IBuildStuff organization is aware of their internal problems and for each problem they identify they (need to) hire new people and invent new roles, because people already in the system are too busy to reflect on “how” things are done. There is no time for Change! As the organization is adding more roles to the system to deal with problems, their profitability is going down. A systemic signal of what is called “Organizational Debt”, and just like “Technical Debt” it has a delayed effect, causing people to react too late rather than in time.

Organizations build (invisible) debt in 2 dimensions, Technical and Organizational, and will pay the interests!

Above is a scenario describing a physical contract but the same thing happens without a physical contract. In many organizations people take commitments (mental contracts) by saying things like: I promise you…, I commit…, I will… and many more.

Even in Agile organizations I see people treating Release and Sprint Plans as a full blown commitment (a contract). A clear sign of this is a Sprint without a Sprint Goal or having a fake Sprint Goal, that is not driving the Sprint but is merely a side remark. Another clear sign is the focus on velocity and estimates.

I guess you get the picture!

It is not a people issue; it is a system design issue!

What to do about the Contract Game Pattern?

Be aware that the contracting game is part of the organizational design or even just your personal environment. Do not make (mental) contracts anymore but work with empirical process control that allows and incorporates learning, like Scrum for example.

Do not commit but be committed.

Design your organization in a way that you have Feature Teams that are organized by customer value. You then put those Feature Teams directly in contact with the users and engage in a kind of problem versus solution fit experiment, a bit like Lean Startup. You shift focus from clarity of scope, to clarity of the problem to fix or the impact to generate. The first step to do this, is by defining a way to validate how you know the problem is fixed or the impact is good enough (preferably using trends over time). After that, you might want to add budget as a constraint to the possible solution.

From that moment on, you brainstorm on experiments in collaborative workshops with the customer. You setup a process to validate hypotheses on what is a possible solution, or on what will generate enough impact. You do this by defining backlog items, building them in a short iteration (limit costs), check the impact and adopt your future hypotheses (or even remove the one you just tried). This is what we call scientific learning, or in most process oriented models a variation of the PDCA (Plan Do Check Act), or in a Scrum Sprint: Sprint planning – build – Sprint Review – Sprint Retrospective.

You focus on the metrics defining the value and you follow up on budgets spent accordingly. You evaluate how much more budget you want to spend in generating a certain value/impact, or you decide to pivot to something else.

You steer your budgets based on generated value, not on scope delivered.

Next to that you allow the development team to pull in the work as they think is feasible for a Sprint. Allowing them to build things right! Not hacking code together, software craftsmanship is key to avoid initiating a re-enforcing, negative loop to destruction. Having a focus on value and craftsmanship generates options in scope. Maybe an easy to build solution to a problem is good enough for now. Maybe 80% impact generated is good enough for now. See also the 25/25 rule on achieving more by doing less by B. Ronen, T.G. Lechler and E.A. Stohr.

And above all… do not add roles, extra organizational units or extra sourcing contracts to your organization! Ask the development teams to deal with problems and give them time to do so. This will avoid building up organizational debt. In Scrum, there is a Sprint Retrospective built in to help you manage and keep focus on avoiding organizational debt. This also means that the part of the organization that is building the product also must maintain the product! Indirectly this will encourage the builders to stay away from hacking code together, as they will need to clean up the mess and not a separate operations organization (us versus them).

Today we still see a lot of organizations suffering from: we have a problem here… let’s create a role for that, voila, problem solved (or assigned). As such building up a huge pile of organizational debt. Better would be: we have a problem here… team how can we fix it, what do we need? (see video recording of my talk at Agile Prague: How organizations go nuts)

Companies with a blablabla problem too easily create a blablabla role for it.

By not building up organizational debt you keep the organization simple. You reduce the overall need for documentation to handover information. As such you reduce a lot of waste within the organization's operational model, and increase its overall profit. Congratulations!

PS: The above mentioned pattern gets even worse when you do not have dedicated teams working together but rely on individual resource planning instead of team based planning. Or as I like to say:

No home, no glory.

Check out our events page to learn more about organizing for complexity, change and acquire overall business Agility! Especially our PSx (Professional Scrum), LeSS (Large-Scale Scrum) and Management 3.0 courses.