we solve

We can help your business

Request a Free Demo / trial


Insights | From a different perspective | Functional Testing | How to | Mobile Testing | Performance Testing | Test Management Tools
18 September, 2017

7 Mistakes to Avoid When Buying Software to Improve Your Testing

It’s only natural that you have high expectations when you buy software to improve your testing.

You want to improve your ability to test and to make life easier. You may be expecting to get greater traceability, visibility, test coverage, more automation, to save time – or all of these.

However, too frequently, those benefits never pan out.

I’ve seen software bought with good intentions languishing on a shelf within a matter of weeks or months. Companies use their new software, but get even more frustrated than before because it doesn’t do what they want. Projects are delayed and more work is created.

Why does this happen? What makes it so hard to get the benefits that were so tangible when you bought your software?

Today I want to share with you some of most common and avoidable mistakes you can make when buying testing software.

Need help building the business case – see this helpful guide

It is not an easy process. There are many factors to consider, and getting them wrong will potentially cost you tens or hundreds of thousands of pounds.

In my 20+ years in the testing industry, I’ve seen companies make every possible mistake, giving me a good insight into what can go wrong and how to avoid it.

Selecting the right testing software for your business is possible. It just takes some planning – and avoiding the following pitfalls….

1. Unclear Requirements

Don’t even think of buying testing software without having a clear understanding of your requirements – what you really need the software to accomplish, what applications you will need to test today and in the foreseeable future.

Without these details, you can’t make an informed, cost-effective decision, and could end up with software that doesn’t do what you want, and that you regret.

One company that came to me was considering £250k worth of tools based on a recommendation by an IT Consultancy. They had no clear requirements or understanding of what they needed – and what the consultancy had recommended was based on information that was at least 3 years out of date, so not the best solution for them. 

This is why I take people through an in-depth fact find to determine what they’re trying to achieve. Don’t skip this step, and don’t rush through it either. Here are some questions to think about right at the beginning of the process:

  • What testing do you need to do and why?
  • What is the application under test and does it have any unusual characteristics?
  • Who is going to be using the software, and how?
  • Are there a sufficient number of people trained to use the software? If not, where will you get them from?
  • How long do you need the software for?
  • What are your success criteria? What does ‘good’ look like?

This is just the starting point and will lead to other questions. Building a picture of the requirements will lay the foundation for selecting the software and writing the business case.

2. Not getting stakeholder buy-in – all of them

Share your vision for the tools with others with a vested interest, as you may need their support to get the tools embedded and delivering real value to your company.

Have you discussed your plans with your stakeholders to ensure that they understand your vision? Were they excited by the improvements it could make to the testing process?

If not, you may have a challenge getting your plans off the ground – or worse still, people who will actively work against you. It happens!

Ensure that the Head of Development is also included and confirms buy-in. Dare I say that some developers view testing as a delay to deploying their perfect software and consider tools that bridge the void between development and test as something to be avoided.

3. Analysis paralysis

Spending so much time thinking about it, gathering details on alternative solutions or evaluating too many alternatives can lead to the decision never being made.

I have come across situations where months after the initial contact the company is no closer to a decision, as there is always one more tool to evaluate. The cost of this is wasted effort and the opportunity cost of not having tools to use must be huge. 

There are ways to avoid this. Clear requirements and limiting the evaluation to those tools that are truly relevant will go a long way to shortening the process.

4. How many licences do we need?

One of the biggest mistakes I’ve seen involved a major financial institution. They wanted to test their website to 25,000 users. But while they did have 25,000 users per hour, they discovered that website visitors never even went above 2,000 concurrent users – i.e. people using the website at the same time.

They spent over £500k on the software for this test. (Not with me, I hasten to add.)

Understanding the distinction between ‘users per hour’ and ‘concurrent users’ could have saved a mountain of cash…..

Have a clear understanding of the numbers, particularly if the software is based on a per-user model. Understanding “concurrent users” is key, however. In 50% of conversations I find that it is not well understood.

Get this wrong and you will either have:

  • Too few licenses, which can cause project delays and the embarrassment of having to buy more
  • Too Many which is a waste of money – this can be even more embarrassing.

5. Not keeping up with technology

When you decide to buy a new mobile, you check what’s new on the market. Make sure you do the same with your testing software.  

Be careful about using outdated knowledge. Don’t assume that because you once conducted tests with particular software a few years ago, that you know its capabilities today. One of my clients ruled out HPE because they thought it didn’t meet their needs. They based that decision on four-year-old information, which was completely out-of-date.

In the interim, HPE had modernised their software and introduced new rental and SaaS options. They have also introduced a number of new tools for Agile test management, a really great tool for monitoring mobile applications and new function and performance automation tools.

While this industry may not change as quickly as the mobile world, it still evolves. Make sure you have the most recent vendor information and software specifications. You don’t want to miss out on the latest innovations.

6. Free can come with a cost

Budget is always a major concern. However, be careful when considering free software.

The downfalls frequently outweigh the savings. Who will support the software if you find a defect? Will the software be enhanced in the future to support new requirements or advances in technology?

You may discover, too late, that the tool you’ve selected doesn’t offer the analysis capabilities you need, may not integrate into your other software and is generally not fit-for-purpose.

One client came to me after using a performance tool because it was free. However, it didn’t scale beyond 100 users. They needed it for 1,000 users.

If you start with a free or cheap tool, and then have to switch because of a limitation, you may not be able to save or reuse those initial test results.

Manually parsing together results to create the complete picture…. or having to start your testing all over again… or developing a capability your freeware lacks… can be more costly than picking a quality tool from the start. It can also delay your project considerably.

Consider your overall costs before you start, instead of trying to cut corners. This will save you time, money and headaches later.

7. Picking the wrong funding option

Getting your expenditure approved is not always easy.

There are different funding options nowadays, though, which can make your testing software considerably cheaper. Most testing software nowadays can be bought as SaaS, and categorised as operating expenditure rather than capital expenditure.

For many companies this is much easier to get signed off!

Added bonus: SaaS software is also much quicker to deploy because you don’t need to install any hardware, and easier to scale because you are paying-as-you-go.

Your path to success

It is so tempting in today’s fast-paced business world to cut corners, shave timelines and use free or cheap tools to save a few pounds.

Selecting quality testing software will improve your bottom line and simplify your testing. Avoid mistakes by ensuring you:

  • Go through a rigorous process to uncover your requirements
  • Get stakeholder buy-in
  • Obtain current knowledge of solutions available
  • Have a clear plan for what you will evaluate based on the requirements you have built
  • Understand how the solution will work within your organisation
  • Select the most appropriate purchase model
  • Remember that cheap can cost you more long-term

If you need help building the business case, this guide will help you.

How To Turn a No into a Yes

If selecting the right software is something you’d like help with, please do get in touch – we’d be delighted to help!

Stephen Davis
by Stephen Davis

Stephen Davis is the founder of Calleo Software, a OpenText (formerly Micro Focus) Gold Partner. His passion is to help test professionals improve the efficiency and effectiveness of software testing.

To view Stephen's LinkedIn profile and connect 

Stephen Davis LinkedIn profile

18th September 2017
the next generation of test tools

The Evolution of Test Tools: Passing the Baton

Test tools have long been essential to software quality and testing efficiency, but over the years, they have transformed significantly, becoming more powerful, cost-effective, and reliable than ever before. One test tool suite has been at the forefront of this evolution, tracing its lineage back to the 1990s.

Service Virtualization

Seriously, Why Aren’t You Using Service Virtualization?

Service Virtualization (SV) remains one of the most criminally underused tools in software testing. It’s astonishing to me just how many organisations still ignore its benefits – test earlier and save money. If you’re not using SV, you are missing out—plain and simple.

UFT Digital Lab

UFT Digital Lab 24.2: What’s New?

If you need to test multiple devices, OS versions, or browser types, you need UFT Digital Lab from OpenText (previously known as Mobile Center and UFT Mobile). It has a comprehensive suite of features specifically designed for mobile, tablet and web testing across myriad platforms.

TruClient performance testing

TruClient: Quick, Easy & Powerful Performance Testing

TruClient significantly speeds up the script creation process and allows you to capture true end-to-end application performance, including protocol level and front-end performance, e.g. how long screens or pages load.

ALM QC 241

What’s New in ALM/QC 24.1: Release Highlights

If you’re involved with waterfall projects, you may well be familiar with the test management tool ALM/Quality Center (ALM/QC), but you might be surprised at its capabilities. The latest release, Version 24.1, is a modern test management powerhouse.

VSM Building Blocks

VSM is Easy with ValueEdge from OpenText

Value Stream Management (VSM) can revolutionise your business, driving customer value and removing unnecessary costs and investments. However, it does require a rethink of internal processes and culture, which can be a daunting prospect. Fortunately, ValueEdge from OpenText is an end-to-end suite of powerful, user-friendly, intuitive tools that make VSM easy!

Unlock Value with Software Testing

Unlock Value with VSM: The Crucial Role of Software Testing

Delivering high-quality solutions is an essential part of Value Stream Management (VSM). After all, substandard solutions don’t deliver value to customers or shareholders. That’s why software testing is a central component in VSM, and it’s now easier than ever with the powerful and user-friendly OpenText ValueEdge testing tools.

How to implement VSM

How to Implement Agile & Test Management Within VSM

ValueEdge Agile and ValueEdge Quality work together to enable smooth software development within VSM landscapes. By integrating the two tools, you increase the efficiency and effectiveness of Agile and DevOps testing processes, driving quality and value.


VSM: Reduce Project Costs The Right Way

It’s hard for businesses to understand where their investments are going and whether they will translate to business and customer value, especially when it comes to software development…. This is where VSM comes in.

AI the unseen software development partner

AI: The Unseen Partner in Modern Software Engineering

Within software projects, AI (and ML) is increasingly used in areas including code generation, test automation, and project management, although many of these uses are often not in an officially sanctioned capacity. Where ChatGPT is a bit of a Jack-of-all-trades, enterprise AI solutions are far more targeted, and far more robust.



Related Articles

To get other software testing insights, like this, direct to you inbox join the Calleo mailing list.

You can, of course, unsubscribe 

at any time!

By signing up you consent to receiving regular emails from Calleo with updates, tips and ideas on software testing along with the occasional promotion for software testing products. You can, of course, unsubscribe at any time. Click here for the privacy policy.

Sign up to receive the latest, Software Testing Insights, news and to join the Calleo mailing list.

You can, of course, unsubscribe at any time!

By signing up you consent to receiving regular emails from Calleo with updates, tips and ideas on software testing along with the occasional promotion for software testing products. You can, of course, unsubscribe at any time. Click here for the privacy policy.