we solve

We can help your business

Request a Free Demo / trial


10 October, 2023

Why Do People Have Bad Experiences With Test Tools?

why do people have bad experiences with test tools

I speak to many people with a negative view of test tools, having been burned by them. I hear things like:

  • They never do quite what we want
  • When we need them, they break
  • We spend more time maintaining scripts than executing them
  • There’s little or no support
  • They suck up so much time from expensive contractors

As a long-time champion of test tools, this saddens me, but unfortunately, it doesn’t surprise me.

When confronted with this news, I must dig deeper to discover why. Are they doing something wrong? What are their processes? And most importantly… which tools are they using?

Who is Likely to Experience Test Tool Issues?

As a general rule, people with a negative view of test tools fall into one of three camps:

  1. People who were burned by a bad experience (usually with automation) a long time ago
  2. People who have used early-stage tools, seduced by the promise and disappointed by the reality that they were not mature or feature-rich
  3. Or, more likely, people who have only been exposed to open source tools

There is no reason for a poor experience with test tools these days. Modern professional test tools are stable, reliable, and affordable.

So, why do so many testers use test tools that are not stable and reliable?

Why Do You Experience Reliability and Stability Issues with Open Source Test Tools?

  1. Setup Challenges: Open source tools often lack streamlined installation processes. You may have to install several plug-ins just to get a basic tool. Users may scour forums and documentation to set up and configure the tools correctly. This can be incredibly challenging for those without extensive technical knowledge or experience. Incorrect setup will leave you facing countless problems further down the line.
  2. Constant Maintenance: Open source tools typically depend on a community of contributors for infrequent and irregular updates, leading to potential compatibility issues with newer technologies or operating systems. Often, these updates don’t come in time, and you’ll need to spend effort troubleshooting issues, understanding updates, or looking for deprecated functionalities. These tools just seem to fall over, you rationalise it because they’re free, but over time, you’ll end up spending a fortune just to keep them running.
  3. Limited Support: Since these tools are community-driven, professional and timely support is basically non-existent. When you run into issues, you rely on community forums, which might not offer a prompt or practical solution or give bad advice. Conflicting advice from people trying to help, but in reality, burning your time whilst you sift through and try what they suggest.
  4. Functionality: they don’t do everything you need, so you end up enhancing and augmenting them, or using multiple separate tools to plug the gap.

It’s Easy to See Why Businesses Are Drawn to Open Source

Why do businesses opt for open-source tools when professional & supported solutions exist?

Generally, there are a couple of reasons, often working in tandem:

  • Open source tools typically have no upfront costs, so they are perceived as “free”.
  • Consultancies, contractors and testers frequently recommend them.

Unfortunately, both of these reasons are flawed.

As I’ll explore throughout this insight, open source tools are laden with hidden costs that negate the license costs of professional tools.

As for the testers, they generally recommend these sub-standard solutions because:

  • They enjoy the challenge of getting them to work and like being that go-to person
  • Less money spent on tools means more money for people. This is why consultancies and contractors love them.
  • You’ll become reliant on those testers to keep your unstable and unreliable tools running.

Professional Test Tools Provide the Stability and Reliability You Need

Opting for a professional licensed test tool can seem more costly than open source. However, if you opt for the right tools, you’ll get reliability and stability, and they’re a more economical solution when you factor in the long-term benefits.

Here are five reasons why you should move to professional test tools:

  1. Reliability and Stability: Professional licensed tools like LoadRunner and UFT are built by professional software houses and undergo rigorous testing. These tools aren’t hobbies or side projects, they are built by large enterprises for professional situations.
  2. Professional Support: Licensed tools come with dedicated support. This means that if you ever run into an issue, there’s a team ready to assist, ensuring minimal disruption to your testing processes. If it has a bug, they will fix it whilst you get on with something else.
  3. Regular Updates: Professional developers regularly update these tools – it’s their job, and you are paying for it. You’re less likely to encounter compatibility issues or bugs that hinder your testing. They will also respond to customer requests and add new functionality for you, included in the price you have paid.
  4. Built-in Features: Many licensed tools offer built-in functionalities that must be manually added in open source alternatives. This saves time and reduces the complexities of setting up and running your tests.
  5. Security: Licensed tools often prioritise security, ensuring that your testing environment is not vulnerable to potential threats. The importance of this should not be underestimated –  You don’t want to introduce security vulnerabilities into your environment.

How to Get Stable and Reliable Test Tools

It’s easy to see why people think test tools are inherently flaky – because they’ve only been exposed to open source or low cost solutions. However, your test tools don’t need to be a constant pain in the backside.

Licensed test tools like the LoadRunner and UFT suites provide professional stability and reliability.

Sure, the upfront cost of open source tools is negligible, the hidden costs of setup, maintenance, potential downtimes, and delayed projects can add up.

On the other hand, with licensed tools, you pay more upfront but save in terms of reduced man-hours, achieve faster project delivery, and gain the peace of mind that comes with reliability and professional support.

Licensed tools are the pragmatic choice for anyone seeking stability, reliability, affordability and long-term success.

If you are still not convinced, challenge me to show you the value and return on investment you could get with professional tools.

Related Products

LoadRunner CloudUFT One
Stephen Davis
by Stephen Davis

Stephen Davis is the founder of Calleo Software, a OpenText (formerly Micro Focus) Gold Partner. His passion is to help test professionals improve the efficiency and effectiveness of software testing.

To view Stephen's LinkedIn profile and connect 

Stephen Davis LinkedIn profile

10th October 2023
cut testing costs

Cut Software Testing Costs With These Essential Tactics

Balancing rigorous testing with cost-effectiveness is a challenge. You can test too much, take too much time, and spend too much money on testing. Luckily, there are proven ways to refine your testing practices, increase efficiency, and cut your software testing costs.

performance testing is like herding cats

Herding Cats: Performance Testing Strategies for 2024

Performance testing is more challenging than ever. Technology is more complex, customer expectations are higher, and an increasing number of people and departments need to be involved to make performance testing effective. This article shines a light on strategies to help you execute successful performance test cycles in 2024.

skimping on performance test tools

Why Skimping on Performance Tools Is a False Economy

Performance tool cost is likely to be less than 10% of the performance testing budget, yet many still think that slashing this figure saves money. Common sense tells us that if you do not spend on quality tools, you will spend more on people. In this insight, I look at why investment in professional performance test tools is so important.

The Human Cost of Software Failure

Horizon: The Human Cost of Software  Failure

The Post Office Horizon scandal destroyed lives and livelihoods and is rooted in defective software. This insight looks at the scandal from a testing perspective and serves as a cautionary tale for all involved in deploying large-scale IT systems.

LoadRunner Cloud Migration

Remote Testing with LoadRunner Might Be Breaching Your Contract

Did you know that most LoadRunner Professional licenses are only valid for a specific location? This means that (unless you’ve bought the country or worldwide usage add-on) working from home, other offices or geographic locations is prohibited and doing so could incur financial penalties.

Software Testing Steps For an Easier Life

Simplify Software Testing With These 3 Easy Steps

Software test professionals like us are under constant pressure to get more done in less time. These three software testing steps will make your life easier, help you achieve your goals, and allow you to focus on your own progression.

2023 top testing articles

The Most Popular Testing Articles of 2023

in 2023 we produced 50+ testing insights to help software professionals test better than ever before and consider testing from a different perspective – today we look back at the 3 most popular of these insights.

rudolph the red nosed tester

Rudolph the Red-Nosed Tester

It was December 23rd at the North Pole, and Santa’s software team were enjoying their Christmas go-live party. They’d pushed the new CRM (Christmas Resource Management) system over the line just in time, and they were very pleased with themselves about it. Everyone apart from Rudolph, the tester.



Related Products

LoadRunner CloudUFT One

Related Articles


To get other software testing insights, like this, direct to you inbox join the Calleo mailing list.

You can, of course, unsubscribe 

at any time!

By signing up you consent to receiving regular emails from Calleo with updates, tips and ideas on software testing along with the occasional promotion for software testing products. You can, of course, unsubscribe at any time. Click here for the privacy policy.

Sign up to receive the latest, Software Testing Insights, news and to join the Calleo mailing list.

You can, of course, unsubscribe at any time!

By signing up you consent to receiving regular emails from Calleo with updates, tips and ideas on software testing along with the occasional promotion for software testing products. You can, of course, unsubscribe at any time. Click here for the privacy policy.