Skip to Content

Think before you automate

Tuukka Virtanen
May 3, 2024

In this blog post, I want to tell you a little fictional story, but a story that could be true, and probably is true in one organization or another.

The story begins with a tester, who has to test new user creation works in a financial system.
The tester’s task is to input all this user data into the system through the UI,
which has multiple different text inputs, drop-down menus and radio buttons.
Doing this task takes the tester about 10 minutes and then they can check
that the user was correctly created into the system.

Life goes on and the testing team is looking to improve their testing process.

A new improvement is suggested: What if instead of using the UI for the input,
the tester could write the needed data into Excel sheet cells that is then fed to the system.
There is no protest and the new workflow is implemented.

Now the tester’s task is to input all this user data into the correct Excel row cells.
Doing this task takes the tester about 5 minutes and then they can check
that the user was correctly created into the system.

Life still goes on and the testing team is looking to once again improve their testing.

A new improvement is suggested: What if we automate the filling of the Excel?
There is no protest and the Excel filling is automated.

Now the tester’s task is to upload an empty Excel sheet to the CI server and
it takes about 2.5 minutes for the automation job to run and deliver the filled Excel sheet
to be fed to the system.

Everything seems better, right? We have reduced the testing task time from 10 minutes down to 2.5 minutes.
But we have also introduced a test automation developer into the equation and have to take that into
consideration. So, now the testing task takes only 2.5 minutes per test, but maintaining the Excel sheet
automation and keeping it updated for the new versions, requires an hour of automation work per week.

Let’s zoom out for a second.

What if we automated the browser UI in the first step of this journey, instead of using the Excel?
Automating the browser user input is implemented and now it takes 2.5 minutes for the job to run in CI server and create a new user.

So, now we have two different roads that seem to lead to the same outcome – that it still takes 2.5 minutes to test the new user creation. But there are differences. In the first example, the test automation developer is now spending about 1 hour per week to maintain the Excel automation. In the second example, the test automation developer also needs to maintain the browser UI automation, which can be argued, changes more often than inputting the data to the system through Excel files. But it also assures that UI input is working as expected, which is not tested in the Excel option.

So, in the end, we come back to the same age-old question, regardless of how we got there:
What do we want to test?

Do we want to test the UI as part of the user creation?
Or should it be left for other types of tests that are run throughout the development pipeline?
Or do we want to just test how many different user combinations we can have in the system?

Luckily, this example was just a story.

But how do we make sure that this story has a happy ending?

If you first find the answer to that age-old question mentioned earlier, everything else (including the automation), will follow, but not the other way around.

About the author

Consultant | Finland
Test automation consultant with technical experience in test automation and quality assurance. TMap Next certified Test Engineer with knowledge in test planning and execution and test design techniques. Master of Science in Information Management. Indie game development as a side project. Creative and visual thinker.

Leave a Reply

Your email address will not be published. Required fields are marked *

Slide to submit