The Fourth Horseman of Test Automation - Isolation

A lot of time and thought has been spent on the subject of why applications dies. This is the last in our series on why test automation projects die.

When a test automator goes home at the end of the night, they want something fairly simple. They want to feel that the hard work, the struggle, the frustration, and the successes that they went through during the day mattered. The stakeholders of an automation project want it to have delivered value. That lines up nicely, win for all! Lets check the boxes. Meaningful positives? Check. No meaningless negatives? Check. Trusted reporting? Check. Timely? Check. Ready for tomorrow? Check.

Except... data is not enough. Salespeople will tell you that sales decisions are rarely about the data, and largely about

the feeling. To keep the project going, to maintain the continued "purchasing" of automation, it has to do something different than deliver value. It has to feel like it delivers value. Yes, that does mean that many automation projects continue without value because they feel good, but let's leave that aside for right now and focus on how valuable automation projects are felt.

The biggest thing that automation has to do to feel valuable is to be visible. When leadership wants to know the state of the project, is the automation output on their dashboard, or do they ask someone for an email summary? What about when the product team needs to make release decisions? When the project manager or scrum master wants to check status? Do the application developers check the automation to see if they are on track, or do they wait for a qa engineer to tap on their shoulder?

If there is a translation layer between the test automation and its stakeholders, then the value of the project will not be felt.

The importance of the automation is abstract to the stakeholders, just one tool among many that the test team uses. When challenges arise, it makes thoughts like "We can just buy something else" or "Well, the team can just muscle through and test manually" sound reasonable. The work both done to and done by the automation are invisible, and therefore easily dismissed.

The fourth horseman of a test automation apocalypse is isolation. Test automation projects that are isolated from their stakeholders are always at risk, without regard for the project's value.

For the technical team, though, feeling the value goes deeper than visibility. These should be the staunchest defenders and supporters of the automation work, but that is most true when we have the least isolation. The application developers need to do more than check the integration reports. They need to be able to run the tests in their environments. Have a bug that needs reproduced? Run this test. Make a change that could break something? Try this suite before you push. A development team that depends on their tests this way will riot at the suggestion of abandoning it, and rightly so.

Additionally, if automation is separate from application development, multiple sustainability problems crop up. First and foremost, designing a primary application that is testable does not stay a priority in the face of other pressures. That makes building and maintaining automation much more fragile and time consuming. Secondly, test automation gets excluded during the planning and estimation process. In such circumstances, automation immediately becomes a death march project. Its scope is fixed (to match the application development plan), its time is fixed (release schedule), and its resources are fixed. Pain and death are on the way.

So why wouldn't a team make their automation visible in this way? Well, immaturities, failures, and systemic problems around the automation (including not just those of the automators) become visible as well. That can be a lot of pressure and feel very risky. It takes courage to have advertise our successes and failures this way. However, the alternative is to go home knowing no one saw or valued the work done today, and maybe, they'll decide they can throw it away tomorrow.