Magic is in the air – Gartner’s Magic Quadrant for Software Test Automation

Last week, Gartner published the always eagerly awaited Magic Quadrant for Software Test Automation. And over the last few days, it has been circulated and quoted a lot of times. But what does this all mean? Are the non-Leaders tools not worth it? Do we need to follow the leader(s)? Are there only 10 automation tool vendors?

Let’s try to make a bit more sense of this. In the report, Gartner clearly states why they used only 10 vendors and all the criteria it used to include them, which is fine but the interesting bit is in the “Market Overview” section where you can find the following quote: “Open-source and cloud solutions have strongly disrupted the market from a pricing perspective.”. This is great as open-source (being free) has, of course, a huge advantage in the value for money, but that is not the whole story. As in the same paragraph, there is a warning: “They can carry a high cost related to building and maintaining a working and the productivity achieved may not always be at the desired levels.”. The latter is not to be discounted as those are too often the hidden costs of using Open-Source tools.

Seeing Tricentis, Eggplant and SmartBear in the “Leaders” quadrant is not surprising as all three have been working on expanding their tools with new technologies and capabilities in 2019. What is surprising is that Micro Focus has dropped out of the “Leaders” quadrant. Having looked back in history it seems that the only reason Micro Focus was in that quadrant (for Test Automation tooling) is likely due to the purchase of HPE in 2017 who had then leading test automation software. This brings up an interesting thought… Where did today’s leaders come from and where have the previous leaders gone? I have taken a look at the Magic quadrants from Gartner over the last seven years and tracked the movement of each company.

Looking at the seven-year timeline of these vendors it shows SmartBear and Worksoft both started in the “Visionaries” before breaking out into the “Leaders” quadrant. Both of them share the achievement of having featured in every quadrant over the past seven years, only in 2019 SmartBear is regarded as a leader while Worksoft has become a niche player. Eggplant (which started as TestPlant) took four years as a visionary to break through to the “Leaders” quadrant. It is only Tricentis that has held on to the “Leaders” title for the last five years and every year still gradually improving its position, they seem to be in a stable position.

The biggest vendors, IBM and Micro Focus (includes HPE) now reside in the “Challengers” quadrant, both having large portfolios of test tools but are currently not regarded as leaders in the test automation market. Of course, they remain a force to be reckoned with but interestingly that the three biggest leaders currently are all relatively small companies. Micro Focus has moved across three quadrants over the last seven years. Looking at the trajectory these companies have over the last few years, it promises to be an interesting 2020!

What it does mean for day to day running? As detailed in the Gartner’s report, each vendor has its strengths and weaknesses, it also does not include any of the open-source tools. So, if you are looking at Test Automation tooling in 2020, it is important to make a balanced decision, be true to yourself and your organisation. Ask for help, we have plenty of contacts with these vendors and we have an overwhelming experience with open-source tools… remember a test tool is not just for Christmas… It is a strategic implementation!!