More deterministic tests + better signal-to-noise with branch ‘cross-targeting’

Unlike app development, QA automation always deals with two codebases, always in parallel: the automation code, and the actual application under test. Let’s take a look at the relationship between these two — specifically, stable and unstable branches, and how “cross-targeting” these branches can improve your automation efficiency big time.

A Tale of Two Repositories

Maintaining a stable branch

In the mainstream app development world, every organization (hopefully) carries both a stable and unstable branch. The names might change — release or master for stable, integration or dev for unstable — but the theme is the same:

• Continuously maintain a version of our code which is more reliable / less buggy
• Carefully curate / manage changes to this branch
• If a bug is found in production, we can “hotfix” against stable quickly
• Bettter understood code, less fear of breaking changes

It’s a lesson learned in b̶l̶o̶o̶d̶ lost profits. Unfortunately, many or most QA automation efforts don’t implement such a system. Which leads to our next point:

If you don’t maintain a stable branch, then your main branch is _unstable_

Implementing an automation-stable branch

We create a new branch in our automation repo. Next, remove cruft, old tests, old libraries. Assign a test developer on cleanup duty for a week or two (three?).

• Merge your new automation features to unstable first
• Run all new functionality first in unstable
• Once a test has burned in successfully with other code, then you can promote that change to stable and let it play for the adults.

Cross-Targeting

At first blush, you might think that stable tests stable and vice versa. But that is actually exactly wrong. What we actually want is a cross pattern, like so:

With this setup, we are isolating instability to only one of the two applications. When we see bugs, we will have a high degree of confidence the problem lies on the unstable side.

Two different test vectors, two different priorities

With an “unstable” or development branch for automation, we’re now free to take more risks.

If we restrict messaging + alerts on autom-unstable to QA only, then we can make complex refactors and cleanups, break and fix and re-break, and all the noise from these failures is completely isolated.

Your cleanup of a core helper class, your upgrade to python requests, your buggy new test, just rained out the whole ballgame … and the bosses will never know. Fix it in unstable, try again, promote it later when it’s actually working.

Meanwhile, your stable branch is bug-free, finding issues and building confidence from executives and app developers:

By establishing an automation-stable branch and curating it with care, we can cross-target our repositories for a dramatic improvement in performance.

It’s a fair amount of work of course.

But consider: it’s also a lot of work chasing bugs around. You’re uncertain about the state of QA at a given time. You’re afraid to cleanup or refactor. That’s a lot of work too and that work sucks.

We can clean up our QA room, or leave it a mess. Either way it’s work … but one ultimately hurts a lot less.

Happy branching !