By Tony Collins
Dozens of families gathered in the ballroom of a Hilton hotel to hear independent investigators announce the most likely cause of an air crash that killed 132 air passengers.
Some wondered whether official investigations into air crashes always ended up protecting powerful corporate interests. For several years the manufacturer Boeing had denied that a technical malfunction was the cause of the crash. It blamed the pilots.
This was the longest inquiry in the history of the National Transportation Safety Board, an investigative organisation funded by the US government. Congress has mandated the Board’s independence and objectivity.
At first, each Boeing 737 incident was treated as a single unique event. In the absence of any clear evidence of a technical malfunction, suspicion fell on the pilots.
The 737 is, after all, the best-selling commercial jet airliner in history. It has an extraordinary safety record.
Then evidence began to mount that various 737 incidents might have been linked.
After thousands of tests over several years, air crash investigators made a discovery – that a particular technical malfunction could, after all, have caused the incidents.
It was an intermittent malfunction – and one that occurred in a rare set of circumstances. It left no trace. It might have caused a succession of seemingly-unique major incidents.
Now the final verdict on the likely cause of USAir Flight 427’s destruction was imminent. As families sat in silence at the Hilton Hotel, Springfield, Virginia, five board members of the National Transportation Safety Board voted – in public – on whether they accepted the findings of their staff investigators who’d pointed to the likely cause being a technical malfunction, not the pilots.
The vote was unanimous; and some relatives wept. The probable cause was not the pilots. It was “most likely” to have been a technical malfunction.
Boeing accepted the final report into the crash of Flight 427. “We respect the Board’s opinion,” said Boeing after the vote. It made rudder-related design changes that eventually cost more than $100m.
Human or machine?
What do various incidents involving Boeing 737s have to do with a campaign for justice for 198 former sub-postmasters and their families?
At issue in both cases is whether human or machine was to blame for a plethora of incidents.
Former sub-postmasters, who used to run local post offices across the UK, say that technical malfunction, or a combination of human error and unusual, unexpected equipment behaviour, was the cause of their distress, misfortune, jailing or bankruptcy.
The Post Office blamed them for losses shown on its “Horizon” system and required that they pay the shortfall in question. This led to financial ruin for some of them. The Post Office insisted its equipment was not at fault. It pointed to the lack of evidence of any technical malfunction.
But investigations into rare crashes of 737s show that it’s possible for a major corporation to be mistaken when it clears its own equipment and blames the equipment’s human operators.
The 737 investigations found that “no evidence of a technical malfunction” did not necessarily mean “no technical malfunction”.
The UK government reached a similar conclusion at the end of a campaign by families to set aside an RAF finding of gross negligence against two pilots, Flight Lieutenants Jonathan Tapper and Rick Cook, who died when a Chinook helicopter, ZD576, crashed on the Mull of Kintyre in June 1994.
For 16 years the RAF and Ministry of Defence insisted that there was no evidence of a relevant technical malfunction on the last flight of Chinook ZD576. They blamed the pilots for the crash. But leaked MoD technical papers established that the Chinook’s engine computer systems could fail in unpredictable ways – sometimes intermittently – and leave no evidence.
In the end – after a 17-year campaign for justice by the pilots’ families – the UK government set aside the RAF’s finding against Tapper and Cook, mainly because of doubts over whether the pilots or technical malfunction, or a combination of both, caused the crash.
Arguably, the Chinook and 737 controversies established the principle that, despite the absence of firm evidence of a technical malfunction, a major incident could still be caused by one, or a series of them.
This may be an important consideration in Post Office cases because, in some criminal trials of sub-postmasters, the absence of evidence of a technical malfunction that caused the losses shown on Horizon has counted against the defendants.
It counted against former sub-postmaster Lee Castleton who disputed in a civil action the Post Office’s claim that he owed amounts totalling £27,000. These sums were shown on Horizon as losses.
The judge in the case said, “It is inescapable that the Horizon system was working properly in all material respects.” Castleton lost the case and was left with costs of £321,000. The following year he filed for bankruptcy.
In a separate case, a criminal hearing where former sub-postmaster Seema Misra was the defendant, a jury agreed with the Post Office’s case that the Horizon system was tried and tested, had been in use at thousands of Post Offices for several years, and was fundamentally reliable and robust.
Misra was jailed for the theft of £75,000 in a case based on the Post Office’s computer evidence. She said she hadn’t taken a penny.
When sub-postmasters could not prove the existence of a fault on Horizon that explained the losses, the conclusion was that they were personally responsible for the shortfall.
About 30 of the 198 individual complaints against Horizon are from former sub-postmasters who received criminal convictions over the losses.
Boeing and the Post Office
With its turnover of about $94bn [£76bn), Boeing is nearly ten times the size of the Post Office. The Post Office has a turnover of less than £1bn. Boeing has vast facilities and specialist teams to investigate crashes full-time. Still, its judgments on the probable cause or causes of major incidents are not infallible.
A number of 737 incidents have shown that, even with relevant incident data available, it may take years of assiduous and expensive independent investigations to get to the likely truth.
In the case of the 737 incidents, the suspect component at the centre of investigations, a power control unit, was based on an old design (certified in the 1960s) – and straightforward in its operation relative to the Horizon system.
In comparison, the Horizon system has hundreds of thousands of lines of code and is complex, taking into account its many upgrades over more than a decade and its interactions with different hardware, networks, interfaces and a central data centre. Adding to this complexity are user uncertainties over procedures for dealing with problems.
But one of the most striking single aspects of any comparison between 737 crashes and the Horizon controversy is that it took professional full-time independent investigators in the US several years and thousands of tests on one suspect component only, before they were able to establish not that the component in question had been the cause of two fatal crashes and a succession of other incidents but that it had been the “probable” cause.
More than $1m was spent investigating the power control unit and still there was no firm evidence that the suspect component was the cause.
The Post Office has, arguably, required a higher standard of proof from local sub-postmasters.
By insisting that there was no evidence of a malfunction that resulted in losses, the Post Office put the onus on sub-postmasters to prove otherwise. Establishing that Horizon was the “probable” or “likely” cause – the standard of proof required in commercial aircraft accidents – was not good enough in cases of sub-postmaster complaints.
In response to the complaints of former sub-postmasters, the Post Office has made a number of similar statements:
“There is no evidence that faults with the computer system caused money to go missing at these Post Office branches. There is evidence that user actions, including dishonest conduct, were responsible for missing money.”
Another Post Office statement said,
“To date, and after two and half years of investigation and independent review, the facts are that Post Office has found no evidence, nor has any been advanced by either an Applicant [former sub-postmaster] or Second Sight [the independent investigators of sub-postmaster complaints], which suggests that Horizon does not accurately record and store branch transaction data or that it is not working as it should.”
Boeing made similar points in its submission to the National Transportation Safety Board on the crash of Flight 427. Boeing pointed to a lack of evidence of technical malfunction while pointing to evidence of the actions of human operators (pilots).
Boeing said,
“There is no evidence to support a conclusion that an uncommanded full rudder deflection occurred (the rudder moving in the opposite direction to that commanded by the pilots).
“While there is not conclusive evidence of a crew-commanded, sustained left-rudder input, such a possibility is plausible and must be seriously considered, especially given the lack of evidence of an airplane-induced rudder deflection.”
Indeed Boeing’s conclusion in its submission to investigators of 737 incidents was similar to the Post Office’s position that there was “no systemic problem” with Horizon.
Boeing said,
“There is no data to indicate that the Eastwind Flight 517 event, the United Flight 585 accident, and USAir Flight 427 accident were caused by a common airplane malfunction.” [Boeing had argued that each incident was different – a similar argument to the Post Office which said each complaint by sub-postmasters about the Horizon system was “demonstrably different and influenced by its own particular facts”.]
In a separate submission to the National Transportation Safety Board, the manufacturer of the 737’s suspect power control unit, Parker Hannifin, made a point similar to Boeing’s.
“In sum, after years of one of the most critical examinations in aviation history, there is no evidence that the main rudder PCU [power control unit] from Flight 427 malfunctioned or was other than fully operational.”
Last word
But the National Transportation Safety Board, as a statutory authority, had the last word.
Its conclusion did not coincide with the view of Boeing or Parker Hannifin.
It said the most likely cause of the crash of Flight 427 was that the rudder moved in the opposite direction to that commanded by the flight crew. The final investigation report said,
“Probable Cause
“The National Transportation Safety Board determines that the probable cause of the USAir flight 427 accident was a loss of control of the airplane resulting from the movement of the rudder surface to its blowdown limit [full aerodynamic limit].
“The rudder surface most likely deflected in a direction opposite to that commanded by the pilots as a result of a jam of the main rudder power control unit servo valve secondary slide to the servo valve housing offset from its neutral position and over-travel of the primary slide.”
Could both sides be right?
On the face of it, the Post Office and former sub-postmasters have contradictory arguments, just as Boeing’s assertions and the investigators’ finding of likely technical malfunction may seem contradictory.
It’s possible, though, that these arguments are not as contradictory as they seem.
It is conceivable the Post Office was correct when it said there was no conclusive evidence of a technical malfunction; and it’s equally conceivable the former sub-postmasters were correct when they said a technical malfunction was partly or entirely to blame for the losses.
Possible similarities and differences
Campaign4Change has looked closely at some of the similarities and differences between 737 rudder incidents and the Post Office cases.
The Post Office and Boeing investigated each incident as a separate matter. Both organisations found no systemic problems. But, unlike Boeing, the Post Office always had the upper hand in its investigations: it was able to require that sub-postmasters pay, in many cases, tens of thousands of pounds that were shown as losses on Horizon.
There’s a risk of trivialising the consequences of 737 crashes when making comparisons with the Horizon controversy. It can be argued, though, that both involved major incidents that ruined lives; and both cases raise the question of whether any large corporation, once it has taken a position that its equipment was not to blame for a single major incident – let alone a number of incidents – will ever change its mind unless forced to.
One particular difference between the UK and US investigations into major incidents is that the US regulatory system allows Boeing to make a submission to the investigations board – which it did, contesting the board’s draft finding that blamed technical malfunction for 737 incidents and crashes – but Boeing had to abide by the independent board’s final decision.
The Post Office did not have to abide by the findings of its independent investigators Second Sight and was able to end Second Sight’s contract. The Post Office said it had given Second Sight “notice regarding its contract“.
Another difference: in the US, the regulatory system allowed the National Transportation Safety Board to require information from the various equipment manufacturers; and the Board’s investigators could obtain information independently of the manufacturers, usually with their cooperation but not necessarily.
In comparison, the Post Office determined what information it passed to Second Sight and the families. On this point Second Sight had its concerns.
In one of its reports for the Post Office, Second Sight said,
“We have experienced significant difficulty in obtaining access to a number of documents we believe are necessary for the purposes of our investigation, notwithstanding Post Office’s commitment to make requested documents available to us.”
The Post Office says it made available to Second Sight thousands of documents but not those that were the subject of legal privilege .
There’s a further difference between the US and UK investigations. In the US, the National Transportation Safety Board did its own investigations or supervised those carried out by equipment manufacturers. It even had the power to exclude equipment owners from participating in the inquiry.
In 2010 American Airlines was excluded from participating in an investigation into an incident involving one of its 757 aircraft because its technicians downloaded and accessed information from the plane’s black box [digital flight data recorder] before it was examined by independent investigators.
US regulations require that the National Transportation Safety Board is the first to see, download or access information from the black boxes.
A Board press release criticised American Airlines. It said,
“Although a thorough examination by our investigators determined that no information from the DFDR [digital flight data recorder] was missing or altered in any way, the breach of protocol by American Airlines personnel violates the Safety Board’s standards of conduct for any organization granted party status in an NTSB investigation.
“Because maintaining and enforcing strict investigative protocols and procedures is vital to the integrity of our investigative processes, we have revoked the party status of American Airlines and excused them from further participation in this incident investigation.”
When the Post Office investigated Horizon systems in the light of losses shown on the systems, it had the authority to retain full control of system information throughout.
As well as being the owner of the system, the Post Office was responsible for commissioning the investigations into the actions of the sub-postmasters. It was also the prosecuting authority and supplier of the material facts involved.
Other possible considerations
- In the US, there was no procedure for pilots to follow if they had a rudder hardover (where the rudder moves to its fullest extent and jams against a mechanical stop). The principle was that pilots were not trained to cope with problems that theoretically couldn’t occur. Were sub-postmasters faced with malfunctions that were considered impossible and so hadn’t been trained to cope with them?
- Human operators may make the ultimate mistake but they might have been reacting to malfunctions, problems with design, inaccurate information or confusing interfaces. [The Post Office had 1.5 million Horizon helpline calls in a three-year period which is a possible sign that many local post office staff did not fully understand the system or how it worked.]
- The US pilots’ trade union ALPA [Airline Pilots Association] was formed partly because of a perception that the government’s automatic response to major incidents was to blame pilots.
- After major incidents, the Post Office and Boeing have pointed to the extraordinary record of reliability of their equipment, the implication being that a systemic problem is highly unlikely. The 737 had (and still has) an extraordinary safety record: 264 million flight hours and an uncommonly low crash rate. Airlines have ordered at least 11,550 of them, more than any other commercial aircraft in history. It’s in use in 111 countries. Its reliability record is the best in the world. On average more than 2,000 737s are in the air at any one time. It has carried 17 billion passengers – about twice the world’s total population. It has flown about 120 billion miles, the equivalent of 640 round trips from the earth to the sun. The Post Office says of Horizon: “Horizon is robust and effective in dealing with the six million transactions put through the system every day by our postmasters and employees at 11,500 Post Office branches. It is independently audited and meets or exceeds industry accreditations. There have been 500,000 users of the system since it was introduced.”
- The design of the 737 rudder system had been considered fail-safe. It was thought it would work properly even when problems occurred. The system had built-in “redundancy”. Every lever inside the lower power control unit had a second lever that moved in concert, in case one should break. There were two hydraulic systems in case one should fail. There was a standby actuator in case the main power control unit stopped working. Even so, after thousands of tests, investigators found it could fail in very rare circumstances.
- The Post Office has listed the many procedures and processes in place for subpostmasters to handle problems or technical failures. The Post Office said, “Horizon is capable of handling power and telecommunications problems. In Post Office branches, postmasters are responsible for power supplies and the cabled telecommunications lines. Interruptions in power supplies and telecommunication lines are a risk faced by all IT systems. There are, however, recovery systems built into Horizon to prevent losses occurring where there is a power or telecommunication failure. There is no evidence to suggest that either of these events would cause losses in branches where the recovery process has been correctly followed by branch staff. There is however evidence of branch staff failing to follow the recovery process properly. This would cause discrepancies in a branch accounts and could be a cause of losses. It is however the result of human error by Applicants [former sub-postmasters] or their staff, and not a failing of the Post Office or Horizon.”
- US air crash investigators were able to glean much from listening to voices in the cockpit shortly before incidents occurred. No such luxury existed in the investigation of Post Office Horizon losses. The Post Office cannot have known what was in the minds of the sub-postmasters at the time: whether they had criminal intent or were utterly baffled by what was appearing on their screens.
- The National Transportation Safety Board after its initial investigation into the fatal crash of United Airlines 585 at Colorado Springs in 1991, reached a conclusion that the probable cause was “undetermined reasons”. Would the Post Office consider such a possibility in the case of Horizon losses?
- After the unexplained crash of Flight 585, the National Transportation Safety Board kept tabs on 737 rudder problems even without evidence they were the likely cause of any serious incidents. Does this mark a different investigative approach to the Post Office which appears to have had a mindset that its equipment could not be to blame for losses?
- The fact that five leading members of the National Transportation Safety Board voted publicly on the probable cause, or causes, of a major incident limited the potential for an institutional mindset to develop. The Board often modified or rejected the findings of its investigators.
- Tests could not be carried out on 737 equipment until all parties agreed on how each piece would be tested. Agreement involved the Federal Aviation Authority as regulator, Boeing, the pilots’ union ALPA and the machinists’ union. In contrast the Post Office was in complete control of its investigations into Horizon losses.
- The existence of the National Transportation Safety Board is a check against parties protecting their own corporate interests, namely the reputation of their equipment, after a major incident. What similar check exists to prevent the Post Office from seeking to protect its corporate interests – namely the reputation of its equipment – after a number of major incidents?
- Would the conclusions of the investigations into the 737 incidents have been different if Boeing had been the authority in charge of the final report?
A useful book on the crash of Flight 427 is by Bill Adair, which is an inside account of the 737 rudder incidents. He had access to all the main parties involved.
Also useful is the final report of the National Transportation Safety Board into the crash of Flight 427. It contains Boeing’s submission.
In January 2017, the High Court granted Justice for Subpostmasters Alliance, which represents the accused former sub-postmasters, a Group Litigation Order against the Post Office. There are 198 sub-postmasters on the High Court claim form and several hundred more are likely to join as claimants.
If the case goes to appeal, it could continue for years.
Or the Post Office could choose to settle rather than spend public money fighting a case which could be seen as a self-vindicating exercise – one that prolongs the misery for the subpostmasters and their families.
Campaign4change emailed the Post Office a list of detailed questions, based on this article. A Post Office spokeswoman replied that, “given that there is currently litigation it’s not appropriate for Post Office to comment”.
Last year, after a BBC Panorama documentary on the complaints of sub-postmasters and the Horizon system, the Post Office issued the following statement:
BBC Panorama – Our response
The Post Office wholly rejects extremely serious allegations repeated in BBC’s Panorama programme of 17 August 2015. The allegations are based on partial, selective and misleading information.
- The Post Office does not prosecute people for making innocent mistakes and never has
- There is no evidence that faults with the computer system caused money to go missing at these Post Office branches
- There is evidence that user actions, including dishonest conduct, were responsible for missing money
We are sorry if a small number of people feel they have not been treated fairly in the past but we have gone to enormous lengths to re-investigate their cases, doing everything and more than we committed to do.
All of the allegations presented in the programme have been exhaustively investigated and tested by the Post Office and various specialists over the past three years or more. The unsubstantiated claims and theories that continue to be levelled against the Post Office are at odds with the facts and are constructed from highly partial, selective and inaccurate information.
This is about individual cases and the Post Office will not discuss those in public for very good reason. The Criminal Cases Review Commission (CCRC) is reviewing a small number of cases involving criminal convictions. It will be provided with all available information including confidential legal material not available to others and we believe the CCRC should be allowed to complete its reviews without external comment. We also gave a commitment of confidentiality to people who put forward cases to us for re-investigation.
The Horizon computer system is robust and effective in dealing with the six million transactions put through the system every day by our postmasters and employees at 11,500 Post Office branches. It is independently audited and meets or exceeds industry accreditations.
Background facts
Prosecutions
The Post Office has always taken its duty to act fairly, proportionately and with the public interest in mind extremely seriously. The Prosecutions it brings are scrutinised by defence lawyers before they advise their clients and are, ultimately, ruled upon by the courts.
If money is missing from a Post Office branch and the fact that cash is missing has been dishonestly disguised by falsifying figures in the branch accounts, the Post Office is entitled to take action and does so based on the facts and circumstances of that specific case. Though rare, where there is evidence of criminal conduct, a decision may be made to prosecute.
Prosecutions are brought to determine whether there was criminal conduct in a branch, not for the Post Office’s financial considerations.
Post Office prosecutors are all experienced criminal lawyers, many of whom have significant experience in prosecuting for both Post Office and the Crown Prosecution Service. In the rare instances that prosecutions are undertaken, the Post Office follows the Code for Crown Prosecutors (the same code as the Crown Prosecution Service). The Code requires a prosecution to have sufficient evidence and be in the public interest, both of which are kept under review right up to and including any trial. It means there must be sufficient evidence for each charge – if a theft charge is brought, there must be sufficient evidence for a realistic prospect of a conviction for theft.
A charge upon which there is no evidence will inevitably fail. It is the duty of the defence lawyers to identify to the court where there is insufficient evidence to sustain a charge. If the court agrees then the Judge must dismiss that charge.
The Post Office takes extremely seriously any allegation that there may have been a miscarriage of justice. We have seen no evidence to support this allegation. The Post Office has a continuing duty after a prosecution has concluded to disclose any information that subsequently comes to light which might undermine its prosecution or support the case of the defendant and continues to act in compliance with that duty.
The Horizon Computer System
Horizon is robust and effective in dealing with the six million transactions put through the system every day by our postmasters and employees at 11,500 Post Office branches. It is independently audited and meets or exceeds industry accreditations. There have been 500,000 users of the system since it was introduced.
Nevertheless, rigorous re-investigations were undertaken into claims made by 136 mainly former postmasters that the system caused losses in their branches.
There is overwhelming evidence that the losses complained of were caused by user actions, including in some cases deliberate dishonest conduct. The investigations have not identified any transaction caused by a technical fault in Horizon which resulted in a postmaster wrongly being held responsible for a loss of money.
There is also no evidence of transactions recorded by branches being altered through ‘remote access’ to the system. Transactions as they are recorded by branches cannot be edited and the Panorama programme did not show anything that contradicts this.
Resolution of cases
The Post Office was approached in 2012 by a small number of largely former Postmasters and MPs with the concern that faults in the Horizon computer system had caused losses at their Post Office branches.
In response the Post Office set up an independent inquiry and, when that found nothing wrong with the system, established a scheme to enable people to put forward individual complaints, providing financial support to those making claims so that they could obtain independent professional advice.
There were 150 cases put forward, 43 of which involved criminal convictions.
A number of the cases are now resolved, through mediation or otherwise, and the remainder of cases where the courts have not previously ruled have been put forward for mediation.
Mediation is overseen by the Centre for Effective Dispute Resolution (CEDR), an established leading and entirely independent organisation. Those who have been offered mediation can still exercise their available rights if mediation is not successful – mediation itself doesn’t stop that.
Mediation cannot overturn a previous court ruling – only the courts can do so.
Campaign4Change’s questions to the Post Office
Based on this article, Campaign4Change put some questions to the Post Office:
- If an organisation the size of Boeing can be mistaken when it clears its own equipment and blames the human operators (pilots), it is possible that the Post Office was mistaken when it cleared its own equipment and blamed the sub-postmasters? [Boeing, which is much bigger than the Post Office, has vast test facilities and matching resources for investigations.]
- One outcome of the US investigations was that “no firm evidence of a technical malfunction” did not necessarily mean there was no technical malfunction. The 737 rudder system malfunction was found eventually to have been intermittent. It left no trace. [We know from the crash of a Chinook helicopter on the Mull of Kintyre in June 1994 that it’s possible for computer systems to fail to work properly – sometimes with an intermittent fault – and leave no trace.) Does the Post Office accept that mechanical or digital equipment can suffer from an intermittent fault that leaves no trace?
- Any comment please on the point that “no evidence of a technical malfunction” does not necessarily mean “no technical malfunction”?
- Any comment please on the point that large corporations, once they have cleared their equipment from blame after a single major incident – or further similar incidents – are unlikely ever to change their minds unless forced to?
- One of the most striking single aspects of any comparison between 737 crashes and the Horizon controversy is that it took professional full-time independent investigators in the US several years, millions of dollars and thousands of tests on one suspect component only, before they were able to establish not that the component in question had been the cause of two fatal crashes and a succession of other incidents but that it was the “probable” cause. There was no evidence that the suspect component was the cause. Has the Post Office required a higher standard of proof from sub-postmasters by requiring “evidence” to suggest that a Horizon malfunction or malfunctions caused the incidents in question?
- Boeing had to abide by the findings of the National Transportation Safety Board even though the Board did not agree with Boeing’s conclusions. The Post Office did not have to abide by the findings of its independent investigators Second Sight and was able to end Second Sight’s contract. Any comment please?
- In the US, the regulatory system allowed the National Transportation to require information from the various equipment manufacturers; and it could obtain information independently of the manufacturers, usually with their cooperation but not necessarily. In comparison, the Post Office determined what information it passed to Second Sight and the families. On this point Second Sight had its concerns. In one of its reports for the Post Office, Second Sight said, “We have experienced significant difficulty in obtaining access to a number of documents we believe are necessary for the purposes of our investigation, notwithstanding Post Office’s commitment to make requested documents available to us.” Any comment please?
- The National Transportation Safety Board had the power (which it exercised) to exclude organisations that owned the equipment in question from participating in the inquiry. When the Post Office investigated Horizon systems in the light of losses shown on the systems, the Post Office, although owner and operator of the equipment in question, had the authority to retain full control of system information throughout. Any comment please?
- The design of the 737 rudder system had been considered fail-safe and was certified on this basis. It had built-in “redundancy”. Even so, after thousands of tests, investigators found it could fail in very rare circumstances. The Post Office has explained at some length its Horizon failure back-up processes and procedures. Nevertheless could these prove fallible in very rare circumstances, in ways not yet fully understood?
- Boeing said it was open to any theory even if it meant Boeing was at fault. Is this the Post Office’s position?
- After the crash of United Airlines Flight 585 at Colorado Springs in 1991, the National Transportation Safety Board kept tabs on 737 rudder problems even without evidence they were the likely cause of any serious incidents. Does this mark a different investigative approach to the Post Office which appears to have had a mindset that its equipment could not be to blame for losses?
- The NTSB after its initial investigation into the fatal crash of United Airlines 585 reached a conclusion that the probable cause was “undetermined reasons”. Would the Post Office consider such a possibility in the case of Horizon losses?
- Tests could not be carried out on 737 equipment until all parties agreed on how each piece would be tested. Agreement involved the Federal Aviation Authority as regulator, Boeing, the pilots’ union ALPA and the machinists’ union. In contrast the Post Office was in complete control of its investigations into Horizon losses. Any comment please?
- The existence of the National Transportation Safety Board is a check against parties protecting their own corporate interests, namely the reputation of their equipment, after a major incident. What similar check exists to prevent the Post Office from seeking to protect its corporate interests – namely the reputation of its equipment – after a number of major incidents?
The Post Office’s reply (as mentioned earlier) was that “given that there is currently litigation it’s not appropriate for Post Office to comment”.
Postmasters tell their story – Computer Weekly investigation in 2009
Sub-postmasters and Horizon – timeline of events, 2009 to 2016 – Computer Weekly
Campaign4Change 
The other significant points are,
1 the vast majority of the convicted SPMRs were charged with false accounting (having covered up insurmountable losses over months or years) NOT theft – there have been very few cases where there has been any proven trail of cash missing and being used/deposited/spent by SPMRs, so in many ways its a bit like losing QE money – it only ever existed in the computer
2 Crown POs, owned and run by the PO itself MUST (almost by definition) experience overs and shortages on a daily basis. POL has never disclosed the quantum of these gains/losses, but have disclosed the existence of a central “bucket” account which accrues these gains and losses. It is by no means inconceivable that local PO losses are ending up in this account over time, so a system caused branch shortage may be a PO HQ gain
3 Even when an error is evident and has been reported, it can take 12 weeks or more for it to be corrected, straddling 3 or 4 PO month end processes
4 From personal experience, I can attest that the calibre of staff manning the Hell Desk (as it is fondly known) can on occasion give minutely detailed instructions that, instead of correcting the problem, actually make it worse, and only when the original error has been doubled or tripled does it get escalated to someone who knows what they are doing
5 The fact that the Second Sight audit was stopped just at the point where they were asking probing questions is indicative of something not right
6 Finally, HMG/POL have now reduced SPMR incomes to such a level that, regardless of the Horizon outcome, swathes of offices are likely to close over the next 12 – 18 months. In our own case, we effetively subsidise the PO from our convenience store earnings, but with ever increasing wage costs, coupled with PO rate cuts, this cannot continue.
The average PO transaction earns us 15p, but costs about 17p in staff time. Unsustainable
LikeLiked by 1 person
Hello, Tony,
Thank you for sticking with this deeply troubling story and presenting it logically.
I recall the Chinook scandal. All of those VIPs in one helicopter flying in less than favourable weather conditions. There were many aspects of that case, and the speedy way the pilots were conveniently declared guilty, that did not enhance the reputation of the RAF.
Passenger airlines are of course accountable – to the most discerning jury of them all – fee paying clients. Eventually, the airlines would have to investigate.
What worries me is that the Post Office and their personnel do not have the same cachet. All of our institutions in this country seem to be run on “top-down, bullying, the powerful dictating” systems.
What perennially concerns me is the implication of absolutism. All systems, human and non-human, are subject to failures. However, rare, the Horizon system must also be vulnerable.
As far as the non-appearance of evidence being interpreted as there being no evidence, anyone who has even a glancing interest in our law courts will know of cases where evidence – was not sought, was held back, was misinterpreted, was altered, was ‘lost.’
Thank you again.
Kindest regards, Zara.
P.S. When it comes to Court cases, always handy to remember that, if you are genuinely innocent, there is no bottom to the bottom of the barrel as far as the prosecution goes.
LikeLike
Thank you Zara. You’ve clearly summed up the most troubling aspects of the Horizon controversy. It’s difficult to understand how so many sub-postmasters with no history of criminality could have their ruined on the basis of computer evidence that losses existed. Tony Collins
LikeLike
Hi Tony. An excellent piece of investigative journalism. Well done. I get involved in a lot of expert witness work for the courts, so can see the approach that has been taken by them. In fairness to the Post-Office, there may well have been some poor behaviour by some of the sub-postmasters, but this seems a very high proportion at one time – all commensurate with the implementation of a new IT system. Intermittent faults do exist in software and are notoriously difficult to evidence. I don’t know the background to the forensics on this case, but in reading your article, I did start to wonder (a) how many prosecutions / cases / allegations were there in the five years leading up to the implementation of the Horizon system to see if that provides in indication as to whether the number of allegations / prosecutions / cases were higher or lower for the subsequent, say, 5 year period after the implementation of Horizon? (b) What the original internal business case for the implementation of Horizon was? i.e. were there already moves afoot to reduce suspected poor financial behaviour from sub-postmaster (in the eyes of the Post-Office)? If so, this would have been documented in the business case and would give some indicators? (c) The development of Horizon. If the initial requirements capture, design, piloting and test methods did not reflect accurate use cases, then it is likely the design and coding were inherently flawed at the outset. Whilst this may have been rectified in later releases (I note that the statement from the Post-Office in respect of Horizon working is in the present tense, not affirming the past tense), how was it tested? Was it tested by representatives from sub-postmasters? To what degree was it concurrently tested prior to implementation with each branch, against the ‘human-error’ each branch was already having? In other words, were the branches that had the financial/stock discrepancies prior to the implementation of Horizon, the same ones that had them after the implementation of Horizon or where they different branches? If they were different branches, had those individuals had any past record of other behaviour that would lead to suspicions of financial misconduct? In terms of legally privileged information that Second Sight were unable to be furnished with, I wonder whether that included internal evidence of the software developers own bug reports/issues logs/development & design notes for remediation? It shouldn’t have, because those details cannot be protected by legal privilege – did the defence lawyers think to ask for them? The above details are all speculation, but we see many cases fail because the legal teams and technical experts simply don’t ask for the right information that would lead to appropriate identification of the real causes. Anyway – rant over. Well done again, and I hope that the facts and truth of the situation are revealed quickly. Best regards. Allan
LikeLiked by 1 person
Thank you Allan. You’ve asked a series of good questions. Not a rant at all.
You make the point that there might have been poor behaviour by some of the sub-postmasters. There may have been some mistakes by sub-postmasters – the Horizon helpdesk had 1.5 million calls in three years from trained professional users – but does this give the Post Office the right to take action that ruins their lives?
The Post Office has a contract with sub-postmasters that makes the sub-postmasters responsible for losses caused through their “own negligence, carelessness or error”. The sub-postmasters are also responsible for the actions of their staff. But it’s one thing for the Post Office to prove mistakes by sub-postmasters and quite another thing for sub-postmasters to prove that the Horizon system could act unpredictably in rare circumstances, as in the Boeing 737 crashes, especially given that the Post Office may not itself know all the potential weaknesses in the system, procedures or operation.
Boeing learnt about weaknesses it had not fully understood in the 737 rudder system only after years of costly testing, computer simulations and investigations that pulled together a vast array of diverse information.
A further point about the onus being on sub-postmasters to prove faults or unpredictable behaviour with the way Horizon works is that they rely on information disclosed to them by the Post Office. It’s up to the Post Office to decide what information it is legally obliged to disclose. The Post Office says it has disclosed all relevant information but there has been contention on this point.
You raised points about whether prosecutions and allegations were higher or lower in the years after Horizon’s implementation. You also ask about the quality of the original design and implementation and whether it took full account of the way it would be used.
Horizon’s predecessor “Pathway” was a troubled project. It was cancelled with wasted costs of more than £150m. Horizon rose from Pathway’s ashes and dates back more than a decade. Indeed the contract between sub-postmasters and the Post Office dates back to 1994. How the system behaved after implementation is unclear but it has certainly served most sub-postmasters well, especially given the number of changes and upgrades to Horizon to take in additional services offered by local Post Offices. That said, the question arises as to whether it works well all the time – in other words whether all bugs and procedural eventualities are fully understood. As you know, there have been serious incidents in the private and public sectors caused by software upgrades that are not fully understood before implementation; and the Boeing 737 crashes highlight the possibility of an intermittent malfunction that leaves no trace.
None of the sub-postmasters, to my knowledge, had any past record of other behaviour that would lead to suspicions of financial misconduct. Courts convicted of theft on the basis on the Post Office’s computer evidence, not any proof that sub-postmasters benefited from a single penny of the losses shown on Horizon.
Thanks again Allan for raising many good questions – including the one about the technical details the Post Office cannot protect by legal privilege. Tony Collins.
LikeLiked by 1 person
Hi Tony. Thanks for the further info. My point about poor behaviour was only in the context that out of 11,500 (ish) sub-postmasters, there may have been a bad egg or two who saw potential opportunities in the system not being able to reconcile. It was not suggesting (or not meant to suggest) that there were bad eggs as an inference. Not knowing much about the case other than press reports, I was just trying to take a balanced view.
The point about the technical development details of the software was raised because of what you said about the Boeing investigation; i.e. it was not beyond reasonable doubt that there were mechanical issues, given what the experts had investigated and found. The problem is that I don’t have any line of sight over what was asked by the Horizon users (sub-postmasters’) defending solicitors or technical experts – nor line of sight into the process of criminal/negligence investigation of the Postmasters.
But in my experience, particularly where criminal investigations though the ‘misuse’ [I’ll come back to that phrase in a moment] of systems is concerned, having clarity over the user/software requirements identification, design, development, testing, piloting, and most importantly, the on-going test results process from the software development team prior to each release of the software of Horizon, would certainly have given an indication into not only the method they were using for the processes of development/implementation/testing, but also the development team’s competence in identifying and resolving issues promptly.
From that, the description details of the issues/call logs from the 1.5m calls, would tie together whether any financial reconciliation issues arose, how/whether they were picked up by the development team and what was done to resolve them – by cross reference to the bug fixing above. In this way, it would have given a much clearer line of sight into whether any ‘reasonable doubt’ existed as to the integrity of the data in Horizon thus questioning to what degree the criminal prosecutions were also open to reasonable doubt.
In addition, where the postmasters have a sub-clause about their own mistakes – this can only be applied where it is reasonable. If the system is chosen by a user and it turns out to be pretty un-user friendly and difficult to operate, there is recourse under the auspices of (a) the system not being of satisfactory quality [bugs] (b) and not fit for purpose – including if it is really difficult to operate. Where a system is ‘forced upon’ a user (say a condition of being a sub-postmaster), irrespective of it being a contractual clause, with the right solicitors/experts asking the ‘right’ questions of evidence, it would be difficult to enforce a ‘self-harm’ clause if the system is (a) very onerous to operate and (b) as a result of being onerous to operate, transactional mistakes are made. On top of this, where system failures occur (power or bugs) then you can test for data integrity (missing/ non-reconciled transactions) by a series of highly conditioned transactional stress testing methods. Again, line of sight into the development/testing process and the user call logs would help to evaluate this.
As I say, I don’t have line of sight into what was actually done by the solicitors/experts and whether this information was (the ‘right’ questions were) requested.
It just all seems very odd. Again, I know very little about the specifics, but the defence approach for the sub-postmasters seems a little disjointed at first blush.
Keep up the good work. Best wishes. Allan
LikeLiked by 1 person
Thank you Allan for your further comment. You made a valid point about a bad egg or two and I’m glad you mentioned it. It’s rare but not unknown for a sub-postmaster to steal and the Post Office has a duty to be vigilant and prosecute in such a case. It seems to me, though, that the sub-postmasters the media have mentioned in this campaign have been pillars of the local community. Indeed it’s not unknown for villagers to crowd-fund payments to the Post Office of “losses” shown on Horizon, such was the confidence of the local community in the integrity of their village postmasters and postmistresses. It also seems to me that the large number of complaints over the Horizon losses suggests a problem – so far unexplained – that links these complaints.
Boeing regarded each 737 crash as different to others. Eventually independent investigators discovered they were linked.
Thank you also for raising valid questions about the software quality, testing and validation. It is being reported that the Post Office is not supplying the log of Horizon faults. I’ll post a piece on this at some point.
I’m pleased the accused sub-postmasters have excellent representation in their campaign for justice. What I cannot understand, on the basis of the facts that have emerged so far, is why the Post Office is prolonging the misery for so many.
The families of Flight Lieutenants Rick Cook and Jonathan Tapper had to wait 17 years to clear the reputations of the two pilots whom the RAF found to be grossly negligent after the crash of Chinook ZD576. Surely there’s a lesson there for the Post Office? It has the ability to start negotiations to settle this matter now.
I am grateful Allan that you’ve applied your professional expertise to this case. Tony Collins.
LikeLike