Products

Problems
we solve

We can help your business

Request a Free Demo / trial

Insights

Insights | From a different perspective
12 February, 2024

Horizon: The Human Cost of Software  Failure

The Human Cost of Software Failure

The Horizon scandal at the UK Post Office is a sobering reminder of the potentially devastating human cost when technology fails and those in charge fail to act responsibly.

  • 3,500 sub-postmasters blamed for financial loss
  • 700 found guilty of crimes (93 convictions have been overturned so far)
  • 236 sent to prison

To summarise the scandal, the Post Office wrongly prosecuted people for stealing. This was due to a number of issues in a software solution called Horizon, which Fujitsu developed. Prosecutions in which Horizon evidence is thought to feature started in 1999 (the year the system went live) and continued until 2015. Many innocent people were sent to jail, and many more had their lives destroyed.

This insight looks at this dire affair from a testing perspective and serves as a cautionary tale for all involved in deploying large-scale IT systems.

What Happened to Their Software QA Processes?

At the heart of the scandal, there must have been some sort of QA failure during the development or commissioning of the Horizon system; it is abundantly clear that the system went live with critical defects.

Software QA processes are about more than just testing; they rely on stakeholders taking test results seriously and mitigating risks appropriately.

This begs the question: was there inadequate testing, or did Fujitsu and/or the Post Office ignore the results?

Sadly, neither would surprise me. We’ve all seen these scenarios play out in more projects than we can remember.

Please note: I do not mean inadequate testers; I mean that testers are often not allowed to perform adequate testing, with under-resourced teams being asked to do too much in too little time.

Were Test Results Ignored?

It’s uncomfortably common for project stakeholders to ride roughshod over the testing team when it comes to phase exit meetings. How often have you seen the leadership team go live with known critical defects despite the test team’s protestations?

Paul Patterson, Fujitsu’s European boss, knew of at least 29 bugs in the system over a 20-year years. Mr Patterson is on record that the Post Office knew about these issues and concealed the evidence in court cases against the sub-postmasters.

So, let’s unpack what he said:

  • 29 bugs? According to this article from Computer Weekly, a senior software developer who worked at Fujitsu alleged that the system should never have been launched and bosses at Fujitsu knew this. He alleged that everybody knew it had problems, stating, “It had gone through the test labs God knows how many times, and testers were raising bugs by the thousand.”
  • 20 years – we have all seen systems go live with bugs, but you find and fix them. 20 years though?! Then, you use the unreliable system to provide evidence to convict people. Why are we not hearing about people from Fujitsu and the Post Office facing criminal charges?

Did They Do Enough Testing?

It’s often standard practice for testing windows, and therefore coverage, to be reduced when things get tight – after all, the project must hit those arbitrary deadlines, even if the software is delivered late to test.

It goes without saying that the less testing you do, the higher the possibility of defects leaking through to live solutions. …but again, for those bugs to remain for over 20 years is outrageous.

The ITV drama Mr Bates vs The Post Office, broadcast in January 2024, gave one of the reasons why the problems occurred: data loss between the Post Office computer and the central systems.  Was this tested adequately? If so, did they simulate packet loss and unstable communications? I doubt we will ever know.

Was There Any User Acceptance Testing?

User Acceptance Testing (UAT) is a critical phase where potential end-users of the system test the software to ensure it meets their needs.

I have always believed that UAT is more about acceptance than testing, allowing users to walk through their processes and sign off on the solution. The real testing should have already been done.

The scandal has highlighted the possibility that the Post Office may have neglected this crucial step. If they did not involve real-world users, the Post Office ignored the standard protocols of software implementation and the voices of those who would be most affected by the system’s failures.

Catastrophic Communication Breakdown or Conspiracy?

What staggers me and many others about the whole debacle is the failure of the Post Office and Fujitsu to listen to warnings from users, insiders and the investigators they paid to look into this.

What could be more important than triaging and fixing the critical defects being repeatedly reported? Surely, there should be mechanisms to safeguard against such failures.

Whether this resulted from a culture of dismissing feedback or a more profound systemic issue remains a topic of debate.

What The Hell Was The Support Desk Doing?

It has been said that the support desk repeatedly told each sub-postmaster that they were the only ones experiencing these issues.

This clearly was not true, as the Post Office blamed 3,500 sub-postmasters for financial losses caused by Horizon.

With this many issues, it should have been easy to pinpoint the errors; the information should have been acted on, with the defects fixed and any testing coverage gaps plugged, with new tests added to the regression pack.

The Human Cost of Systemic Failure

Pinpointing the ultimate responsibility for this tragedy is beyond my remit, but there is no doubt that there was an inadequate process. This system was not ready for deployment.

However, for years, The Post Office senior management said there was not a problem, and Fujitsu remained quiet until called to account by a government Select Committee. I am sure you can draw your own conclusions.

You don’t need me to tell you that the consequences have been dire; lives and livelihoods have been destroyed.

  • 700 sub-postmasters were found guilty of crimes
  • 236 sub-postmasters were sent to prison
  • 4 sub-postmasters took their own life
  • 18 have died without seeing justice or full compensation

All because of a low-quality computer system. Just let that sink in for a minute.

Yes, the way the software issues were handled was atrocious, but I firmly believe that a rigorous QA process could and should have prevented this from happening in the first place and prevented the immense human cost.

Plus, let’s not forget that this continued for 20 years!

What Lessons Can We Learn?

The Criminal Cases Review Commission (CCRC ) said the Horizon scandal was “…the most widespread miscarriage of justice the CCRC has ever seen and represents the biggest single series of wrongful convictions in British legal history.”

The Horizon scandal underscores the importance of rigorous testing, effective communication, and genuine collaboration.

It is a stark reminder that accountability and transparency are not just corporate buzzwords but essential to public trust.

As our industry moves forward, we must continue to push for a culture that prioritises end-users and acknowledges the real-world implications of software quality and the critical importance of testing. I do think that, in general, we are heading in the right direction on this, but this just highlights that we are not there yet and have some way to go.

It’s not rocket science:

  • Give testing enough time
  • Test against real-world conditions
  • Take test results seriously
  • Do not go live with low-quality solutions
  • Communicate system issues with the end users
  • Do not cover up and do not lie.

Incidentally, if you haven’t already, I recommend watching the ITV drama ‘Mr Bates vs The Post Office’ and the associated documentary. They’re both available on ITVX in the UK. Be warned, the behaviour of The Post Office and Fujitsu and their treatment of the sub-postmasters is likely to make you very angry.

Stephen Davis
by Stephen Davis

Stephen Davis is the founder of Calleo Software, a OpenText (formerly Micro Focus) Gold Partner. His passion is to help test professionals improve the efficiency and effectiveness of software testing.

To view Stephen's LinkedIn profile and connect 

Stephen Davis LinkedIn profile

12th February 2024
test management tools are the foundation

Build a Foundation for Testing Success: Choosing a Test Management Tool

Test management tools give unparalleled views of software development progress, provide quality assurance and peace of mind, and can generate positive returns on investment – more than just paying for themselves. This insight discusses some of the contenders and gives recommendations.

risk v reward

Risk v Reward: Are Test Management Tools Worth It?

Are test management tools worth the money? It’s easy and common to assume there are more impactful ways to spend project funds than test tools. But does this downplay the important role a professional test management tool can play in success?

The evolution of test management tools

Test Management Tools: Past, Present, and Future

Understanding where things have come from can often help inform where they are going. The story of test management tools goes back at least three decades and this insight offers a precis of their past, present, and future…

5 automation trends

Software Test Automation: 5 Important Trends for 2024

Software test automation has evolved massively over the last few years; gone are the days of flaky tools, gargantuan setup effort, and scripts that require constant human intervention. The integration of cutting-edge technologies and methodologies has redefined the role of test automation within the software development lifecycle.

Automate Everything With One Tool

Software Testing Simplified: Automate Everything With One Tool

With so many software test automation tools to choose from, companies often cherry-pick a suite of low-cost options to test their full landscape. Unfortunately this is highly problematic, adding unnecessary complexity, increasing costs and undoing any of the purported benefits.

UFT One Automation Heavy Lifting

Test Automation: 6 Reasons UFT One is The Only Tool You Need

In the fast-paced world of software development, functional testing is critical to ensure your solutions perform as expected. Unlike manual testing, which is time-consuming and prone to human error, test automation streamlines the testing process, offering a faster, more accurate, cost-effective solution. There is an array of automation tools available, but OpenText UFT One is the standout choice, offering a complete solution for all your needs.

cut testing costs

Cut Software Testing Costs With These Essential Tactics

Balancing rigorous testing with cost-effectiveness is a challenge. You can test too much, take too much time, and spend too much money on testing. Luckily, there are proven ways to refine your testing practices, increase efficiency, and cut your software testing costs.

performance testing is like herding cats

Herding Cats: Performance Testing Strategies for 2024

Performance testing is more challenging than ever. Technology is more complex, customer expectations are higher, and an increasing number of people and departments need to be involved to make performance testing effective. This article shines a light on strategies to help you execute successful performance test cycles in 2024.

skimping on performance test tools

Why Skimping on Performance Tools Is a False Economy

Performance tool cost is likely to be less than 10% of the performance testing budget, yet many still think that slashing this figure saves money. Common sense tells us that if you do not spend on quality tools, you will spend more on people. In this insight, I look at why investment in professional performance test tools is so important.

Insights

Search

Related Articles

InsightsTrending

To get other software testing insights, like this, direct to you inbox join the Calleo mailing list.

You can, of course, unsubscribe 

at any time!

By signing up you consent to receiving regular emails from Calleo with updates, tips and ideas on software testing along with the occasional promotion for software testing products. You can, of course, unsubscribe at any time. Click here for the privacy policy.

Sign up to receive the latest, Software Testing Insights, news and to join the Calleo mailing list.

You can, of course, unsubscribe at any time!

By signing up you consent to receiving regular emails from Calleo with updates, tips and ideas on software testing along with the occasional promotion for software testing products. You can, of course, unsubscribe at any time. Click here for the privacy policy.