It’s only natural that you have high expectations when you buy software to improve your testing.
You want to improve your ability to test and to make life easier. You may be expecting to get greater traceability, visibility, test coverage, more automation, to save time – or all of these.
However, too frequently, those benefits never pan out.
I’ve seen software bought with good intentions languishing on a shelf within a matter of weeks or months. Companies use their new software, but get even more frustrated than before because it doesn’t do what they want. Projects are delayed and more work is created.
Why does this happen? What makes it so hard to get the benefits that were so tangible when you bought your software?
Today I want to share with you some of most common and avoidable mistakes you can make when buying testing software.
Need help building the business case – see this helpful guide
It is not an easy process. There are many factors to consider, and getting them wrong will potentially cost you tens or hundreds of thousands of pounds.
In my 20+ years in the testing industry, I’ve seen companies make every possible mistake, giving me a good insight into what can go wrong and how to avoid it.
Selecting the right testing software for your business is possible. It just takes some planning – and avoiding the following pitfalls….
1. Unclear Requirements
Don’t even think of buying testing software without having a clear understanding of your requirements – what you really need the software to accomplish, what applications you will need to test today and in the foreseeable future.
Without these details, you can’t make an informed, cost-effective decision, and could end up with software that doesn’t do what you want, and that you regret.
One company that came to me was considering £250k worth of tools based on a recommendation by an IT Consultancy. They had no clear requirements or understanding of what they needed – and what the consultancy had recommended was based on information that was at least 3 years out of date, so not the best solution for them.
This is why I take people through an in-depth fact find to determine what they’re trying to achieve. Don’t skip this step, and don’t rush through it either. Here are some questions to think about right at the beginning of the process:
- What testing do you need to do and why?
- What is the application under test and does it have any unusual characteristics?
- Who is going to be using the software, and how?
- Are there a sufficient number of people trained to use the software? If not, where will you get them from?
- How long do you need the software for?
- What are your success criteria? What does ‘good’ look like?
This is just the starting point and will lead to other questions. Building a picture of the requirements will lay the foundation for selecting the software and writing the business case.
2. Not getting stakeholder buy-in – all of them
Share your vision for the tools with others with a vested interest, as you may need their support to get the tools embedded and delivering real value to your company.
Have you discussed your plans with your stakeholders to ensure that they understand your vision? Were they excited by the improvements it could make to the testing process?
If not, you may have a challenge getting your plans off the ground – or worse still, people who will actively work against you. It happens!
Ensure that the Head of Development is also included and confirms buy-in. Dare I say that some developers view testing as a delay to deploying their perfect software and consider tools that bridge the void between development and test as something to be avoided.
3. Analysis paralysis
Spending so much time thinking about it, gathering details on alternative solutions or evaluating too many alternatives can lead to the decision never being made.
I have come across situations where months after the initial contact the company is no closer to a decision, as there is always one more tool to evaluate. The cost of this is wasted effort and the opportunity cost of not having tools to use must be huge.
There are ways to avoid this. Clear requirements and limiting the evaluation to those tools that are truly relevant will go a long way to shortening the process.
4. How many licences do we need?
One of the biggest mistakes I’ve seen involved a major financial institution. They wanted to test their website to 25,000 users. But while they did have 25,000 users per hour, they discovered that website visitors never even went above 2,000 concurrent users – i.e. people using the website at the same time.
They spent over £500k on the software for this test. (Not with me, I hasten to add.)
Understanding the distinction between ‘users per hour’ and ‘concurrent users’ could have saved a mountain of cash…..
Have a clear understanding of the numbers, particularly if the software is based on a per-user model. Understanding “concurrent users” is key, however. In 50% of conversations I find that it is not well understood.
Get this wrong and you will either have:
- Too few licenses, which can cause project delays and the embarrassment of having to buy more
- Too Many which is a waste of money – this can be even more embarrassing.
5. Not keeping up with technology
When you decide to buy a new mobile, you check what’s new on the market. Make sure you do the same with your testing software.
Be careful about using outdated knowledge. Don’t assume that because you once conducted tests with particular software a few years ago, that you know its capabilities today. One of my clients ruled out HPE because they thought it didn’t meet their needs. They based that decision on four-year-old information, which was completely out-of-date.
In the interim, HPE had modernised their software and introduced new rental and SaaS options. They have also introduced a number of new tools for Agile test management, a really great tool for monitoring mobile applications and new function and performance automation tools.
While this industry may not change as quickly as the mobile world, it still evolves. Make sure you have the most recent vendor information and software specifications. You don’t want to miss out on the latest innovations.
6. Free can come with a cost
Budget is always a major concern. However, be careful when considering free software.
The downfalls frequently outweigh the savings. Who will support the software if you find a defect? Will the software be enhanced in the future to support new requirements or advances in technology?
You may discover, too late, that the tool you’ve selected doesn’t offer the analysis capabilities you need, may not integrate into your other software and is generally not fit-for-purpose.
One client came to me after using a performance tool because it was free. However, it didn’t scale beyond 100 users. They needed it for 1,000 users.
If you start with a free or cheap tool, and then have to switch because of a limitation, you may not be able to save or reuse those initial test results.
Manually parsing together results to create the complete picture…. or having to start your testing all over again… or developing a capability your freeware lacks… can be more costly than picking a quality tool from the start. It can also delay your project considerably.
Consider your overall costs before you start, instead of trying to cut corners. This will save you time, money and headaches later.
7. Picking the wrong funding option
Getting your expenditure approved is not always easy.
There are different funding options nowadays, though, which can make your testing software considerably cheaper. Most testing software nowadays can be bought as SaaS, and categorised as operating expenditure rather than capital expenditure.
For many companies this is much easier to get signed off!
Added bonus: SaaS software is also much quicker to deploy because you don’t need to install any hardware, and easier to scale because you are paying-as-you-go.
Your path to success
It is so tempting in today’s fast-paced business world to cut corners, shave timelines and use free or cheap tools to save a few pounds.
Selecting quality testing software will improve your bottom line and simplify your testing. Avoid mistakes by ensuring you:
- Go through a rigorous process to uncover your requirements
- Get stakeholder buy-in
- Obtain current knowledge of solutions available
- Have a clear plan for what you will evaluate based on the requirements you have built
- Understand how the solution will work within your organisation
- Select the most appropriate purchase model
- Remember that cheap can cost you more long-term
If you need help building the business case, this guide will help you.
If selecting the right software is something you’d like help with, please do get in touch – we’d be delighted to help!
