Skip to Content

Magic is in the air – Gartner’s Magic Quadrant for Software Test Automation

Marco Venzelaar
December 26, 2019

Last week, Gartner published the always eagerly awaited Magic Quadrant for Software Test Automation. And over the last few days, it has been circulated and quoted a lot of times. But what does this all mean? Are the non-Leaders tools not worth it? Do we need to follow the leader(s)? Are there only 10 automation tool vendors?

Let’s try to make a bit more sense of this. In the report, Gartner clearly states why they used only 10 vendors and all the criteria it used to include them, which is fine but the interesting bit is in the “Market Overview” section where you can find the following quote: “Open-source and cloud solutions have strongly disrupted the market from a pricing perspective.”. This is great as open-source (being free) has, of course, a huge advantage in the value for money, but that is not the whole story. As in the same paragraph, there is a warning: “They can carry a high cost related to building and maintaining a working and the productivity achieved may not always be at the desired levels.”. The latter is not to be discounted as those are too often the hidden costs of using Open-Source tools.

Seeing Tricentis, Eggplant and SmartBear in the “Leaders” quadrant is not surprising as all three have been working on expanding their tools with new technologies and capabilities in 2019. What is surprising is that Micro Focus has dropped out of the “Leaders” quadrant. Having looked back in history it seems that the only reason Micro Focus was in that quadrant (for Test Automation tooling) is likely due to the purchase of HPE in 2017 who had then leading test automation software. This brings up an interesting thought… Where did today’s leaders come from and where have the previous leaders gone? I have taken a look at the Magic quadrants from Gartner over the last seven years and tracked the movement of each company.

Looking at the seven-year timeline of these vendors it shows SmartBear and Worksoft both started in the “Visionaries” before breaking out into the “Leaders” quadrant. Both of them share the achievement of having featured in every quadrant over the past seven years, only in 2019 SmartBear is regarded as a leader while Worksoft has become a niche player. Eggplant (which started as TestPlant) took four years as a visionary to break through to the “Leaders” quadrant. It is only Tricentis that has held on to the “Leaders” title for the last five years and every year still gradually improving its position, they seem to be in a stable position.

The biggest vendors, IBM and Micro Focus (includes HPE) now reside in the “Challengers” quadrant, both having large portfolios of test tools but are currently not regarded as leaders in the test automation market. Of course, they remain a force to be reckoned with but interestingly that the three biggest leaders currently are all relatively small companies. Micro Focus has moved across three quadrants over the last seven years. Looking at the trajectory these companies have over the last few years, it promises to be an interesting 2020!

What it does mean for day to day running? As detailed in the Gartner’s report, each vendor has its strengths and weaknesses, it also does not include any of the open-source tools. So, if you are looking at Test Automation tooling in 2020, it is important to make a balanced decision, be true to yourself and your organisation. Ask for help, we have plenty of contacts with these vendors and we have an overwhelming experience with open-source tools… remember a test tool is not just for Christmas… It is a strategic implementation!!

About the author

Managing Consultant | Test Automation | UK
Marco started his career in Quality Assurance with ISO certifications including environmental and safety regulations. His attention to detail on processes continued into his career as an expert testing consultant where he integrated this with his passion for test automation, performance testing and now applying this to Cognitive QA. Marco builds lasting relationships with clients and tool vendors,

    Comments

    3 thoughts on “Magic is in the air – Gartner’s Magic Quadrant for Software Test Automation

    1. We are a local gov’t with several COTS apps we need to establish a patching schedule for. We do NO coding for them but do have contractors create/update integrations periodically. In order to implement patching with our small staff/budget we need an automated test tool. All the core apps are internal running on our servers, they do connect to numerous external third party apps. What testing tools should we look at?

      1. There are a lot of options here for you. While thinking of budget constraints might mean immediately looking at open source tools, not all of those offer no-code solutions. Then again an experienced Automator will be able to create a low/no-code solution around it.
        There are a number of low-cost tooling out there that might be of use, but you will have to look at how much they are compatible with your COTS applications. Plan a short exercise where you shortlist test tools against your applications and do a small proof of concepts. Do keep in mind the longevity of the solution, using internal staff or a dependency on external contractors will be part of that equation.

    Leave a Reply

    Your email address will not be published. Required fields are marked *