Category Archives: Campaign4Change

FT reports on a death after Post Office Horizon IT system shortfall

By Tony Collins

The FT reported yesterday on a class action against the Post Office over the “faulty” Horizon IT system.

In an article or more than 1,000 words, it said that 522 former sub-postmasters are involved in the legal action.

A procedural hearing with a managing judge will take place in October 2017, which should lead to a timetable for final resolution by the court.

The FT reported on two families (previously unpublicised cases) whose lives have been devastated by shortfalls shown on the Post Office’s Horizon branch accounting system.  In one case, the Post Office dismissed Deirdre Connolly, a former sub-postmistress, after an apparent shortfall of £15,600. The alleged deficit was found during an unannounced branch audit.

The FT said that, out of fear, she made up the apparent loss with help from relatives. The Post Office did not prosecute. Her son later attempted suicide, which she attributed to his witnessing the stress she was under.

The FT also reported on a successful businessman, Phil Cowan, whose business ventures included a post office in Edinburgh run by his wife and her friend. He said that a £30,000 deficit shown on the branch electronic ledger account was a factor in his wife’s death from an accidental overdose of anti-depressants, alcohol and cold medicine. She was 47.

He attributed the shortfall to a technical glitch.

Cowan told the FT,

“This situation I know for a fact had a huge contribution to her passing away. It had a massive effect on her.”

In 2015 the Daily Mail reported on Martin Griffiths, a sub-postmaster from Chester, who stepped in front of a bus one morning in September 2013.

An inquest heard that Griffiths, 59, was being pursued by the Post Office over an alleged shortfall of tens of thousands of pounds.

The Post Office reached a settlement with his widow and required the terms of it to be kept confidential.

Court case

The legal action between the Post Office and the sub-postmasters could be said to be a simple one, at least from the PO’s perspective. Sub-postmasters signed a contract that held them responsible for losses shown on the branch accounting system (whether or not there was any evidence they gained from the shortfalls).

The Post Office’s lawyers will argue that there is no evidence that Horizon or any of its related elements such as network and communications equipment was to blame for the losses. Under its contract with sub-postmasters, the Post Office is entitled to pursue the former sub-postmasters for the losses.

It is this contract that is the main point of legal relevance, rather than claims by sub-postmasters that the losses were not real, that they didn’t steal any money and have had their lives, and their family’s lives, ruined by the Post Office’s actions against them.

For the sub-postmasters, lawyers will argue that errors were caused by software bugs and inadequate training and support. The FT article referred to a “pattern of bullying and intimidation by the Post Office dating back to shortly after Horizon was rolled out”.

After shortfalls were discovered, people were held and their homes searched,  Alan Bates of the Justice for Sub-postmasters Alliance told the FT. Freeths solicitors are handling the Alliance’s case.

Comment

The Post Office’s enforcement of its contract with sub-postmasters after discrepancies were found on Horizon raises the question of whether the law in this case has little – or perhaps nothing – to do with right and wrong.

The Post Office may have a contractual right to pursue former sub-postmasters for shortfalls shown on Horizon.

But does the Post Office’s conformance with the law – its contractual right to take action – make the action right?

In Vermont, it’s unlawful for women to be fitted with false teeth without the written permission of their husbands. It would be perfectly legal for Vermont’s lawyers to prosecute offenders. But being lawful to prosecute doesn’t make it right to do so.

It was perfectly lawful for the state to prosecute Alan Turing in 1952 (and Oscar Wilde in 1895) for homosexual acts. That the prosecutions were lawful (and were possible factors in their premature deaths) didn’t make the prosecutions right.

If NASA made its space missions conditional on a requirement that astronauts sign a contract that made them responsible for anything that went wrong, they would probably sign – because of their overwhelming desire to go into space. But if something went wrong would it be right for NASA to enforce the contact (assuming the astronauts survived?).

It can be lawful to enforce a contract but wrong to do so.

Post Office happy?

The Post Office (which is still publicly owned) sounds relaxed about going to court. The FT quoted the Post Office as saying,

“We welcome [the group litigation order] as offering the best opportunity for the matters in dispute to be heard and resolved.

“We will be continuing to address the allegations through the court’s processes and will not otherwise comment on litigation whilst it is ongoing.”

Even at this late stage, it’s not too late for the Post Office’s directors to ponder on the matter of right and wrong rather than go ahead with a court case merely because they can.

They have the power to exacerbate the devastation for hundreds of families. They also have the power to withdraw from the court case, settle and reduce the risk of any further personal tragedies.

This is where the distinction between enforcing a legal right and doing the right thing couldn’t be clearer.

Post Office faces class action over “faulty” IT system – FT

Shedding new light on the Post Office Horizon IT controversy?

IT staff pay price of helping to solve BBC’s gender pay gap?

By Tony Collins

The Register reported yesterday that hundreds of IT roles at the BBC will be offshored to cheaper wage locations under a £560m contract renewal with its incumbent outsourcing supplier Atos.

Atos executives said in a conference call heard by The Register that a new “Aurora” IT contract model will involve a significant amount of offshoring and new tooling. The new contract is due to start in October.

Employees may go into an “availability pool” and some may be redeployed within Atos.

Only a few are expected to stay on the BBC account. Most of the roles may be exported to Atos centres in Poland and India. About 400 Atos staff are affected.

Last week James Purnell, BBC director of Radio and Education, told  Newsnight that the £75m expected to be saved from the IT programme with Atos could help address the gender pay gap.

Thank you to David Orr for alerting me to The Register’s article.

 

 

Is Barnet Council up to the job of managing its suppliers – including Capita?

By Tony Collins

Tonight (27 July 2017) Barnet Council’s audit committee meets to discuss the interim year-end findings of BDO, its external auditor.

BDO identifies a “significant risk” in relation to the council’s contract management and monitoring. There are “numerous issues”, says BDO.

Barnet is well known in the local government community for having adopted a “commissioning council” concept. This means it has outsourced the vast majority of its services, leaving officers and the ruling Conservative group to set policy and monitor suppliers.

Capita is a main supplier. Its responsibilities include cemeteries, ICT and collecting council tax.

BDO’s report for tonight’s council meeting says that, with the council’s services now being delivered through various outsourcing arrangements, “it is important to establish strong contract management and monitoring controls”.

It adds that such controls “allow the Council to ascertain whether or not it is receiving value for money from the use of its contractors, and to take remedial action where issues are identified”.

On this point – contract management and monitoring –  BDO says,

“During the course of 2016/17 we have noted a number of internal audit reports which have raised significant findings in this area.

“In addition, further concerns have been identified through our own audit work. As such, we have recognised a significant risk to our use of resources [value for money] opinion.”

BDO’s findings are interim. It cannot finalise its final statutory report until many questions are answered and errors, financial misstatements and lapses in disclosure are corrected in Barnet’s draft financial accounts.

The auditors comment in their report on the “number and value of errors found” and the “level of misstatement in the current year accounts”.

These are some of BDO’s findings so far:

  • Large advance payments (about £44m in prepayments) as part of the Customer Service Group contracts with Capita. Not all of the payments were set out in the payments profile of the original contract. Significant payments were made at the start of the contract (and in subsequent years) to cover capital investment and transformational expenditure. The financial profile of the contract anticipates the advance payments being used by 2023. One advance payment of £19.1m in December 2016 covers service charge payments relating to the first three quarters of 2017/18. The council receives a £0.5m discount for paying in advance. The council also paid for some projects in advance. BDO finds that there was proper council scrutiny of the decision to make the payments.
  • Barnet overspent on services in 2016/2017 by £8.3m.
  • There’s a budget gap prior to identified savings of £53.9m over the three years to 2020.
  • There’s a substantial depletion in the council’s financial reserves.
  • Will claimed savings materialise? “Savings targets remain significant and achievement of these will be inherently challenging, as evidenced by the overspend in 2016/17.”
  • Net spending on the Customer and Support Group contracts with Capita increased to £34.4m in 2016/17 from £26.9m the previous year.
  • More than 100 officials at Barnet receive at least £60,000 a year and twelve at least £100,000.
  • Some councillors have failed to make formal declarations. A “poor response rate as compared to other authorities” says BDO’s report.

Comment:

You’d think a “commissioning council” – one that outsources the delivery of most of its services – would, above all, have a firm grip on what its main suppliers are doing and what they’re charging for.

In fact BDO’s report for tonight council meeting rates the council’s contract management and monitoring at “red”. BDO has identified “numerous” issues.

It’s easy for Barnet Council to issue press releases on the tens of millions it claims to have saved on its contracts with Capita.

But BDO possesses the facts and figures; and it questions the council’s “use of resources” – in other words “value for money”.

At the outset of its joint venture with IBM, officials at Somerset County Council spoke of planned savings of £180m over 10 years. In fact the deal ended up losing at least £69m.

Barnet blogger “Mr Reasonable” who has long kept a close eye on payments made by Barnet to Capita doubts that the council is up to the job of properly scrutinising Capita. We agree.

It was clear to many in 2013 when Barnet signed contracts with Capita that the council was unlikely to find the money to acquire adequate contract monitoring expertise and resources, given that its suppliers were required to deliver such a wide range of complex services.

Barnet Council’s most adept scrutineers, rather than local councillors, have proved to be its dogged local bloggers who include Derek Dishman (Mr Mustard), John Dix (Mr Reasonable), Theresa Musgrove (Mrs Angry) and Roger Tichborne (The Barnet Eye).

Had ruling councillors taken local blogger warnings more seriously, would they have specifically avoided becoming a “commissioning council”?

Public sector not reporting multiple cyber attacks

By Tony Collins

Successful cyber attacks on parts of the NHS and some councils and universities have not been reported to the police – even where criminals have locked information and demanded ransom payments, an investigation by The Yorkshire Post found.

The National Crime Agency, which is the UK’s lead agency against organised crime, human, weapon and drug trafficking and cyber crime, has said that “under-reporting of cyber crime remains a key barrier to our understanding of its true scale and cost”.

Its comments were aimed at the directors in the private sector. But it’s clear that the public sector is not setting an example.

The Yorkshire Post says that the Mid Yorkshire Hospitals NHS Trust had two ransomware attacks last year in which data was encrypted on some departmental drives with demands for payment made to unlock it. While no payment was made and the information recovered from back-up systems, neither incident was reported to police.

Barnsley Council had 13 successful ransomware attacks since April 2016 and none was reported to the police. No ransoms were paid, data was restored from back-up systems and accounts were disabled and changed to “render any captured credentials of little use”.

Three of Yorkshire’s universities had almost 300 successful attacks in the 
last three years. None was reported to police.

The 
University of York had 237 incidents which included nine distributed denial of service attacks and a further seven incidents in which servers were “compromised” by hackers.

A spokesman for the university said: “We did not consider that any incident caused sufficient loss, either monetary or of data, to justify reporting to the police.”

The University of Huddersfield had 54 successful attempts and nothing was reported to the police “due to low level impact”.

Ensuring the buck stops nowhere?

In a National Audit Office blog, the NAO’s cyber director Tom McDonald and digital transformation specialists Yvonne Gallagher (who’s a former CIO in two government departments) and Max Tse pointed to a lack of accountability in the public sector for deterring cyber attacks and managing the risks.

In health, for example, the Department of Health’s delegates to NHS England, which funds over 200 local clinical commissioning groups to purchase care from local health trusts.

Social care is the responsibility of the larger local authorities who are accountable to their local electors.

NHS Digital has some overview of data and IT systems for the health and social care sectors (through its management of national NHS IT systems, such as the NHS Spine or N3 Network) and it has a dedicated Data Security Centre, but it has no authority over councils and trusts to ensure even simple security measures are implemented locally, such as software updates and patches.

The National Audit Office found that, across government, “there has been little coherence between the several lines of governance and senior oversight of cyber and information security”.

It added,

“A number of organisations and a plethora of working-level groups have been involved in cyber security and supporting digital transformation across government. The government itself has described these arrangements as an ‘alphabet soup’.”

There’s also a shortage of IT security skills in the public sector, which is exacerbated by the high number of so-called “transformation” projects and programmes and a reliance on legacy systems such as Windows XP which proved vulnerable in the WannaCry attack, said the National Audit Office.

Comment

The government could make it mandatory for Whitehall, councils, the NHS and other parts of the public sector – including the police – to report incidents to the National Crime Agency.

It’s unlikely to happen though.

There’s a woeful lack of reporting and accountability in the public sector on IT-related matters.  WannaCry and hundreds of other “successful” incidents in public sector in the past year will not make any difference.

That the public sector will work to reduce the ill effects of cyber attacks is a given. It’s also inevitable that it’ll work hard at ensuring, in line with culture and convention, that, when there are “successful” incidents, the buck stops nowhere.

Thank you to Zara Pradyer for alerting me to the Yorkshire Post article.

Whitehall renews facade of openness on major IT projects

By Tony Collins

Headlines yesterday on the state of major government IT projects were mixed.

Government Computing said,

“IPA: Whitehall major projects show ‘slow and steady’ delivery improvement”

Computer Weekly said,

“Government IT projects improving – but several still in doubt”

The Register said,

“One-quarter of UK.gov IT projects at high risk of failure – Digital borders, digital tax and raft of MoJ projects singled out”

The headlines were prompted by the Infrastructure and Projects Authority’s annual report which was published yesterday.

The report listed the RAG – red/amber/green – status of each of 143 major projects in the government’s  £455bn major projects portfolio. Thirty-nine of these are ICT projects, worth a total of £18.6bn.

Publication of the projects’ red/amber/green status – called the “Delivery Confidence Assessment” – seemed a sign that the government was being open over the state of its major IT and other projects.

A reversal of decades of secrecy over the progress or otherwise of major IT projects and programmes?

In a foreword to the Infrastructure and Project Authority’s report, two ministers referred twice to the government’s commitment to openness and accountability.

MP Caroline Nokes, Cabinet Office minister, and MP Andrew Jones, a Treasury minister, said in their joint foreword,

“The government is also committed to transparency, and to being responsive and accountable to the public we serve.

“Accordingly, we have collected and published this data consistently over the past five years, enabling us to track the progress of projects on the GMPP [Government Major Projects Portfolio] over time.

“We will continue to be responsive and accountable to the public.”

But the report says nothing about the current state of major IT projects. The delivery confidence assessments are dated September 2016. They are 10 months out of date.

This is because senior civil servants – some of whom may be the “dinosaurs” that former minister Francis Maude referred to last month – have refused to allow politicians to publish the red/amber/gtreen status of major projects (including the Universal Credit programme and the smart meters rollout) unless the information, when published, is at least six months old.

[Perhaps one reason is to give departmental and agency press officers an opportunity to respond to journalists’ questions by saying that the red, red/amber of amber status of a particular major project is out of date.]

Amber – but why?

An amber rating means that “successful delivery appears feasible but significant issues already exist” though any problems “appear resolvable”.

In September 2016 the Universal Credit programme was at amber but we don’t know why. Neither the IPA or the Department for Work and Pensions mention any of the “issues”.

The £11bn smart meters rollout is also at amber and again we don’t know why. Neither the IPA nor the Department for Business, Energy and Industrial Strategy mention any of the “issues”. Permanent secretaries are allowed to keep under wraps the IPA’s reasons for the red/amber/green assessments.

Even FOI requests for basic project information have been refused.  Computer Weekly said,

“Costs for the Verify programme were also withheld from the IPA report, again citing exemptions under FOI.”

Comment

The senior civil servants who, in practice, set the rules for what the Infrastructure and Projects Authority can and cannot publish on major government projects and programmes are likely to be the “dinosaurs” that former Cabinet Office minister Francis Maude referred to last month.

Maude said that Whtehall reforms require that new ministers “face down the obstruction and prevarication from the self-interested dinosaur tendency in the mandarinate.”

Clearly that hasn’t happened yet.

The real information about Universal Credit’s progress and problems will come not from the Infrastructure and Projects Authority – or the Department for Work and Pensions – but from local authoritities, housing associations, landlord organistions, charities and consumer groups such the Citizen’s Advice Bureau (which has called for Universal Credit to be halted), the local press, the National Audit Office and Parliamentary committees such as the Public Accounts Committee and Work and Pensions Committee.

On the smart meter rollout, the real information will come not from the Infrastructure and Projects Authority – or the Department for Business, Energy and Industrial Strategy – but from business journalist Paul Lewis, consumer advocate Martin Lewis, business organistions such as the Institute of Directors,  experts such as Nick Hunn, the Energy and Climate Change Committee and even energy companies such as EDF.

Much of this “real” information will almost certainly be denied by Whitehall press officers. They’ll be briefed by senior officials to give business journalists only selected “good news” facts on a project’s progress and costs.

All of this means that the Infrastructure and Projects Authority may have good advice for departments and agencies on how to avoid project failures – and its tact and deference will be welcomed by permanent secretaries – but it’s likely the IPA will be all but useless in providing early warnings to Parliament and the public of incipient project disasters.

Ministers and some senior civil servants talk regularly about the government’s commitment to openness and accountability. When it will start applying to major government IT projects?

 

UK.gov watchdog didn’t red flag any IT projects. And that alone should be a red flag to everyone

 

 

 

 

Why are councils hiding exit costs of outsourcing deals – embarrassment perhaps?

Tony Collins

Excerpt from Taunton Deane council’s confidential “pink pages”.
The last sentence contains a warning that IBM-owned SWO – Southwest One – may try to “maximise revenues” on exiting its joint venture with the council.

Somerset County Council has refused a Freedom of Information request for the costs of exiting its joint venture with IBM.

But a secret report written last year by officers at Taunton Deane Borough Council – which was a party to the IBM-owned joint venture company Southwest One  – warned that the supplier could attempt to “maximise revenues on exit”.

It said,

“… from experience anything slightly ambiguous within the contract is likely to be challenged by SWO [Southwest One] in order to push it into the chargeable category as they attempt to maximise revenues on exit”.

A separate section of the confidential report said,

“disaggregating from the SWO [Southwest One] contract will be complex and expensive …”

Taunton Deane Borough Council did not tell councillors what the exit turned out to be. The figures are also being kept secret by Somerset County Council which signed the “transformative” SWO joint venture deal with IBM in 2007.

Both councils have now brought back services in-house.

Secrecy over the exit costs is in contrast to Somerset’s willingness to talk in public about the potential savings when local television news covered the setting up of Southwest One in 2007.

The silence will fuel some local suspicions that exit costs have proved considerable and will have contributed to the justifications for Somerset’s large council tax rise this year.

£69m losses?

David Orr, a former Somerset County Council IT employee, has followed closely the costs of the joint venture, and particularly its SAP-based “transformation.

It was his FOI request for details of the exit costs that the council refused.

Orr says that Somerset has lost money as a result of the Southwest One deal. Instead of saving £180m, the joint venture has cost the council £69m, he says.

FOI

Under the Freedom of Information Act, Orr asked Somerset for the “total contract termination costs” including legal, consultancy, negotiation, asset valuations, audit and extra staffing.

He also asked whether IBM was paid compensation for early termination of the Southwest One contract. In replying, the council said,

“The Authority exited from a significant contract with Southwest One early, and the services delivered through this contract were brought back in-house in November 2016.

“The Authority expects the costs to fall significantly now it has regained control of those services.

“Somerset County Council made payment under the ‘Termination for Convenience’ provisions of the original contract. We do hold further information but will not be releasing it at this point as we believe to do so would damage the commercial interests of the County Council, in that it would prejudice the our negotiating position in future contract termination agreements in that it would give contractors details on what terms the Council was willing to settle …”

Orr will appeal. He says the Information Commissioner has already established a principle with Suffolk Coastal District Council that the termination costs of a contract with a third party should be disclosed. The commissioner told Suffolk Coastal council that, in opting out of FOI,

“there is no exemption for embarrassment”

Hidden costs

Taunton’s pink pages paper said that the Southwest One contract’s Exit Management Plan provided for a smooth transfer of services and data, and for access to staff to assess skills and do due diligence.

In practice, though, there were many exit-related complications and costs – potential and actual. The paper warned that Taunton would need to find the money for:

  • Exit programme and project management costs
  • Early termination fees
  • Contingency
  • ICT infrastructure disaggregation
  • Service transition and accommodation costs
  • Disaggregating SAP from Southwest One. Also the council would need to exit its SAP-based shared services with Somerset County Council because the estimated costs were lower when run on a non shared services basis. SAP covered finance, procurement, HR, payroll, website and customer relationship management.
  • Costs involved in a “soft” or “hard” (adversarial) exit.
  • Estimating council exit costs when IBM was keeping secret its own Southwest One running costs.
  • Staff transfer issues.

Comment

So much for open government. It tends to apply when disclosures will not embarrass local government officials.

In 2007 Somerset County Council enjoyed local TV, radio and newspaper coverage of the new joint venture with IBM. Officials spoke proudly on camera of the benefits for local taxpayers, particularly the huge savings.

Now, ten years later, the losses are stacking up. Former Somerset IT employee and FOI campaigner Dave Orr puts the losses at £69m. And local officials are keeping secret the further exit costs.

Suffolk Coastal District Council lost an FOI case to withhold details of how much it paid in compensation to a third party contractor to terminate a contract. But at least it had published its other exit costs.

Somerset is more secretive. It is withholding details of the sums it paid to IBM in compensation for ending the joint venture early; it also refuses to publish its other exit costs.

Trust?

Can anything said by councils such as Somerset or Barnet in support of major outsourcing/joint venture deals be trusted if the claimed savings figures are not audited and the other side of the story – the hidden costs – are, well, hidden?

In local elections, residents choose councillors but they have no say over the appointment of the permanent officials. It’s the officials who decide when to refuse FOI requests; and they usually decide whether the council will tell only one side of the story when public statements are made on outsourcing/joint ventures.

Across the UK, local councils employed 3,400 press and communications staff –  about double the total number in central government – in part to promote the authorities’ services and activities.

What’s the point if they publicise only one side of the story – the benefits and not the costs?

Somerset’s decision to refuse Orr’s reasonable FOI request makes, in its own small way, a mockery of open government.

It also gives just cause for Somerset residents to be sceptical about any council statement on the benefits of its services and activities.

Will MPs call BA to account over IT power problems?

By Tony Collins

Experts are questioning BA’s explanation of the power problems that disrupted the travel plans and arrangements for 75,000 people at the weekend.

BA says it is “reviewing” what went wrong at the weekend but is under no regulatory duty to publish the findings.

There is little pressure from shareholders to hold BA to account. The share price of BA’s parent International Airlines Group is higher today than a month ago.

Sceptical

Yesterday the BBC’s business editor Simon Jack accused IAG of dodging tough questions it will “surely have to answer” and the FT quoted IT and electricity experts who are sceptical of the airline’s explanations.

But MPs on the Transport select committee – a new one will be formed after the general election next week – could decide, if pressed by their constituents, to have an inquiry into BA’s power problems.

If so, they could question BA’s chief Alex Cruz or Willie Walsh, the chief executive of IAG.

In 1997 the committee held an inquiry into the escalating costs and problems on IT contracts at the Swanwick air traffic control centre in Hampshire. MPs decided to publish the contents of an independent report into the problems by technology consultancy Arthur D Little.

Any 2017 inquiry by the committee could hold BA to account in a way that would not otherwise be possible. Lessons from the failures may be useful to the public and private sectors.

UPS failure

Meanwhile what went wrong and why seems confused.

The Telegraph says the BA review is focusing on the uninterruptible power supply (UPS) to Boadicea House, one of two data centres close to Heathrow airport.

The UPS in question delivers power through the mains, diesel and batteries.

On Saturday morning, shortly after 8.30am, power to Boadicea House through its UPS was shut down. The reasons are unclear.

If power had returned to the servers in Boadicea House slowly this would have allowed the airline’s other Heathrow data centre, at Comet House, to take up some of the slack, said the Telegraph.

But, on Saturday morning, just minutes after the UPS went down, power was resumed in what one Telegraph source described as “uncontrolled fashion.”

This caused “catastrophic physical damage” to BA’s servers, which contain everything from customer and crew information to operational details and flight paths.

The Telegraph said that if power had been restored more gradually, BA would have been able to cope with the outage, and return services far more quickly than was the case.

The FT said yesterday that the UPS malfunctioned, cutting off the power supply. But it said that “some people working in the field have questioned” the explanation. They said it is very rare for UPS systems to fail. Even if they do, it should not affect the continued supply of mains electricity to the data centres they serve.

Not a technology problem?

BA has said there was an “immediate loss of power” from the UPS. When power returned, a surge physically damaged its IT servers. It had to replace the damaged equipment.

Willie Walsh said the meltdown was not a technology problem. The FT quoted him as saying, “You give me any IT system in the world and I’ll show you how good it is when it doesn’t have any electrical power going to it.”

Walsh insisted there was “no data loss, no data corruption”. He said the IT systems “functioned how they are supposed to function.”

But the FT quoted Jonathan Glover, co-founder of PSI, a company that helps businesses protect their equipment against sudden, unexpected power surges, who said the failure of a UPS “was relatively unlikely as they are robust and well-proven pieces of equipment”.

He added that, even if the UPS system did fail, it should not make a difference to the power supply to the airline’s IT system. The answers given don’t make a lot of sense, he said.

Alan Woodward, visiting professor at the department of computer science at the University of Surrey, agreed. He told the FT,

“It is like on your laptop and if you just pull the plug out of the back, it shouldn’t affect your laptop. It keeps running until the battery runs down. Even if you unplug the battery [of a laptop], it doesn’t like it from a data perspective, but plug it back in again, you don’t suddenly get a big power surge.”

Woodward said one possible explanation was that a voltage regulator contained within the UPS might have malfunctioned but when they fail the power usually stops, he added.

Another expert on UPS technology said that even if the system had failed, it would simply have been bypassed and normal electricity supply should have continued.

Why would the failure of the UPS affect BA’s back-up data centre?  The answer is unknown. BA would not comment on whether their two Heathrow-based data centres relied on the same UPS.

Ryanair on Tuesday pointed out that it had IT systems in three locations around Europe and if one went down, there were backups at each of its data centres. Ryanair’s data centres are not close to each other.

Two electricity companies whose low-voltage networks cover Heathrow airport and the surrounding area have denied there were any issues on their networks on Saturday morning.

Transient voltage surge arresters can shield against power surges from the local electricity network and malfunctions in a company’s own equipment but it is unclear whether BA had these fitted and if it did whether they worked.

The FT quoted an expert as saying that BA either had inadequate defences or  didn’t have the right level of industrial-level surge protection. BA has not commented on what protection measures it had.

Will BA publish its review?

BA may be reluctant to reveal the results of its review for various reasons. Parts of its IT appear in the UK could be run by non-BA staff. The failures could raise questions about the corporate oversight of any non BA specialists, possibly at board level.

It is also possible that an internal review could highlight fundamental managerial weaknesses – such as unclear or confused IT responsibilities in the UK or at IAG – after the outsourcing of IT skills to India last year.

Damian Brewer, an analyst at RBC Capital Markets, told the Telegraph that if BA’s early diagnosis of the cause of the crisis is correct, bosses’ failure to prepare for such an incident in the light of other carriers’ problems “suggests fundamental management and planning weakness”.

“It seems highly questionable why similar incidents with major US carriers in the last year have failed to see IAG move to ensure its airlines had plans in place to mitigate this risk, already seen elsewhere, and also to have contingency plans in place,” he said.

“At present, it appears that BA management have seemingly not taken account of IT risk precedent already seen and already known at other carriers.”

In what BA has said publicly about the IT problems, much of it has focused on what didn’t happen (a cyber attack) and on the people who were not responsible (Tata in India or energy companies). It told the BBC  the problems were “definitely not a consequence of underinvestment or cost-cutting.”

“All the parties involved around this particular event have not been involved with any type of outsourcing in any foreign country,” said Cruz.  “They have all been local issues around a local data centre who [sic] has been managed and fixed by local resources.”

Comment

Without an inquiry by the newly-formed Transport Committee, BA will find it easy to keep the lid on the results of its inquiry into the failures.  This would be a pity given the lessons that could be learned.

It’s ironic that the aviation industry has an exemplary reputation for reporting even minor problems that relate to safety. There is a duty to report even a ruffled carpet in an aircraft aisle that could trip up passengers or crew.

But there is no duty to account for an IT failure that disrupted the lives of 75,000 people across the world because it was not a safety issue. Provided the company pays satisfactory compensation, the fiasco will probably be out of the public eye in a few months.

But MPs, on behalf of their constituents,  could hold BA to account.

Anyone who wants to ask MPs to hold an inquiry into the BA failures could write to:

Transport Committee
House of Commons
London
SW1A 0AA

Telephone: 020 7219 3266
transcom@parliament.uk
Twitter: @CommonsTrans

The Committee’s clerk is Gordon Clarke: clarkeg@parliament.uk

Thank you to Dave Orr for his regular updates on the BA problems

BA’s IT: Will Transport Committee MPs ask the tough questions? – Government Computing

Full details of meltdown revealed (says Daily Telegraph)

BA board to demand IT chaos inquiry – Simon Jack, BBC

Aftermath of the cyber attack – will ministers learn the wrong lessons?

By Tony Collins

At least 16 NHS trusts out of 47 that were hit by the ransomware attack continue to face problems, according to BBC research.

And, as some patients continued to have their cancer treatments postponed, Tory, Labour and Lib-dem politicians told of their plans to spend more money on NHS IT.

But will any new money promised by government focus on basic weaknesses – such as the lack of interoperability and the structural complexities that made the health service vulnerable to cyber attack?

Last year when the health secretary Jeremy Hunt announced £4bn for NHS IT, his focus was on new technologies such as smartphone apps to order repeat prescriptions rather than any urgent need to upgrade MRI, CT and other medical devices that rely on Windows XP.

Similarly the government-commissioned Wachter review “Making IT Work: Harnessing the Power of HealthInformation Technology to Improve Care in England made no mention of Windows XP or any operating system – perhaps because ministers were much more likely to welcome a review of NHS IT that focused on innovation and new technologies.

Cancer treatments postponed

The Government’s position is that the NHS was not specifically targeted in the cyber attack and that the Tories are putting £2bn into cyber security over the next year.

Theresa May said yesterday,

“It was clear warnings were given to hospital trusts but this is not something that was focused on attacking the NHS. 150 countries are affected. Europol says there are 200,000 victims across the world. Cyber security is an issue we need to address.

“That’s why the government, when we came into government in 2010, put money into cyber security. It’s why we are putting £2bn into cyber security over the coming year.”

Similarly Jeremy Hunt, health secretary, told the BBC that the attack affected international sites that have “some of the most modern IT systems”.

But the BBC’s World at One gave an example of how the NHS’s IT problems were affecting the lives of patients.

It cited the case of Claire Hobday whose radiography appointment for breast cancer at Lincoln County Hospital was cancelled on Friday (12 May 2017) and she still doesn’t know when she’ll receive treatment. Hobday said,

“I turned up by hospital transport for my second radiotherapy session, and I, along with many other patients – at least 20 other people were waiting – and they said the computers weren’t working.

“I do have to say the staff were very good and very quickly let us all know that they were having trouble with the computers. They didn’t want to misinform us, so they were going to come and talk to us all individually and hoped they would be able to rectify it.

“Within half an hour or so they came out and said, ‘We’re really sorry but it’s not going to get sorted. We’ll send you all home and give you a call on Sunday’ which didn’t happen.

“But they did ring me this morning (15 May 2017) to say it’s not happening today and if transport turns up please don’t get in it, and it’s very unlikely it will happen tomorrow.

“It is just a bit upsetting that other authorities have managed to sort it but Lincolnshire don’t seem to have been able to do that.”

United Lincolnshire Hospitals Trust told World at One it will be back in touch with patients once the IT system is restored.

Roy Grimshaw was in the middle of an MRI scan – after dye was injected into his blood stream –  when the scan was stopped and he was asked to go back into the waiting room in his gown, with tubes attached to him, while staff investigated a computer problem. After half an hour he was told the NHS couldn’t continue the scan.

Budgets “not an issue”?

GP practices continue to be affected. Keiran Sharrock, GP and medical director of Lincolnshire local medical committee, said yesterday (15 Mat 2017) that systems were switched off in “many” practices.

“We still have no access to medical records of our patients. We are asking patients to only contact the surgery if they have an urgent or emergency problem that needs dealing with today. We have had to cancel routine follow-up appointments for chronic illnesses or long-term conditions.”

Martha Kearney – BBC World at One presenter –  asked Sharrock about NHS Digital’s claim that trusts were sent details of a security patch that would have protected against the latest ransomware attack.

“I don’t think in general practice we received that information or warning. It would have been useful to have had it,” replied Sharrock.

Kearney – What about claims that budget is an aspect of this?

Sharrock: “Within general practice that doesn’t seem to be the reason this happened. Most general practices have people who can work on their IT and if we’d been given the patch and told it needed to be installed, most practices would have done that straight away.”

GCHQ

World at One also spoke to Ciaran Martin, Director General for Government and Industry Cyber Security.  He is a member of the GCHQ board and its senior information risk owner.  He used to be Constitution Director at the Cabinet Office and was lead negotiator for the Prime Minister in the run-up to the Edinburgh Agreement in 2012 on a referendum on independence for Scotland.

Kearney: Did your organisation issue any warnings to the health service?

Martin: “We issue warnings and advice on how to upgrade defences constantly. It’s generally public on our website and it’s made very widely available for all organisations. We are a national organisation protecting all critical sectors and indeed individuals and smaller organisations as well.”

Huge sums spent on paying ransoms?

Kearney asked Martin, “How much money are you able to estimate is being spent on ransoms as a result of these cyber attacks?” She added,

“I did hear one astonishing claim that in the first quarter of 2016 more money was spent in the USA on responding to ransomware than [was involved] in armed robberies for the whole of that year?”

Martin: “First let me make clear that we don’t condone the payment of ransoms and we strongly advise bodies not to pay and indeed in this case the Department of Health and the NHS have been very clear that affected bodies are not to pay ransoms. Across the globe there is, sadly, a market in ransomware. It is often the private sector in shapes and sizes that is targeted.”

Martha Kearney said the UK may be a target because it has a reputation for being willing to pay ransoms.

Martin, “We are no more or less a target for ransomware than anywhere else. It’s a global business; and it is a business. It is all about return on investment for the attacker.

“What’s important about that is that it’s all about upgrading defences because you can make the return on investment lower by making it harder to get in.”

If an attacker gets in the aim must be to make it harder to get anything useful, in which case the “margin on investment goes down”. He added,

“That’s absolutely vital to addressing this problem.”

Are governments at fault?

Martin,

“Vulnerabilities will always exist in software. Regardless of who finds the underlying software defect, it’s incumbent on the entire cyber security ecosystem – individual users, enterprises, governments or whoever – to work together to mitigate the harm.”

He added that there are “all sorts of vulnerabilities out there” including with open source software.

Windows XP

Computer Weekly reports – convincingly – that the government did not cancel an IT support contract for XP.

Officials decided to end a volume pricing deal with Microsoft which left NHS organisations to continue with XP support if they chose to do so. This was clearly communicated to affected departments.

Government technology specialists, reports Computer Weekly, did not want a volume pricing deal with Microsoft to be  “comfort blanket” for organisations that – for their own local reasons – were avoiding an upgrade from XP.

Computer Weekly also reported that civil servants at the Government Digital Service expressed concerns about the lack of technical standards in the NHS to the then health minister George Freeman.

Freeman was a Department of Health minister until July 2016. In their meeting with Freeman, GDS officials  emphasised the need for a central body to set technical standards across the NHS, with the authority to ensure trusts and other organisations followed best practice, and with the transparency to highlight those who chose not to.

A source told Computer Weekly that Jeremy Hunt was also briefed on the security risks that a lack of IT standards would create in a heavily-federated NHS but it was not considered a priority at that top political level.

“Hunt never grasped the problem,” said the source.

There are doubts, though, that Hunt could have forced trusts to implement national IT security standards even if he’d wanted to. NHS trusts are largely autonomous and GDS has no authority to mandate technical standards. It can only advise.

How our trust avoided being hit

A comment by an NHS IT lead on Digital Health’s website gives an insight into how his trust avoided being hit by the latest cyber attack.  He said his trust had a “focus on perimeter security” and then worked back to the desktop.

“This is then followed up by lots of IG security pop ups and finally upgrading (painfully) windows XP to windows 7…” He added,

“NHS Digital have to take a lead on this and enforce standards for us locally to be able to use.”

He also suggests that NHS Digital sign a Microsoft Enrollment for Windows Azure [EWA] agreement as it is costly arranging such a deal locally.

 “NHS Digital must for me, step in and provide another MS EWA as I am sure the disruption and political fall-out will cost more. Introduce an NHS MS EWA, introduce standards for software suppliers to comply with latest OS and then use CQC to rate organisations that do not upgrade.”

Another comment on the Digital Health website says that even those organisations that could afford the deployment costs of moving from XP to Windows 7 were left with the “professional” version, which “Microsoft has mercilessly withdrawn core management features from (e.g. group policy features)”.

The comment said,

“There are a lot of mercenary enterprises taking advantage of the NHS’s inability to mandate and coordinate the required policies on suppliers which would at least give the under-funded and under-appreciated IT functions the ability to provide the service they so desperately want to.”

A third comment said that security and configuration management in the NHS is “pretty poor”. He added, “I don’t know why some hospitals continue to invest in home-brew email systems when there is a national solution ready and paid for.

“In this recent attack most the organisations hit seem to use local email systems.”

He also criticised NHS organisations that:

  • Do not properly segment their networks
  • Allow workstations to openly and freely connect to each other in a trusted zone.
  • Do not have a proper patch / update management regime
  • Do not firewall legacy systems
  • Don’t have basic ACLs [access control lists)

Three lessons?

  • Give GDS the ability to mandate no matter how many Sir Humphreys would be upset at every challenge to their authority. Government would work better if consensus and complacency at the top of the civil service were regarded as vices, while constructive, effective and forceful criticism was regarded as a virtue.
  • Give the NHS money to spend on the basic essentials rather than nice-to-haves such as a paperless NHS, trust-wide wi-fi, smartphone apps, telehealth and new websites. The essentials include interoperability – so that, at the least, all trusts can send test results and other medical information electronically to GPs –  and the upgrading of medical devices that rely on old operating systems.
  •  Plan for making the NHS less dependent on monolithic Microsoft support charges.

On the first day of the attacks, Microsoft released an updated patch for older Windows systems “given the potential impact to customers and their businesses”.

Patches are available for: Windows Server 2003 SP2 x64Windows Server 2003 SP2 x86, Windows XP SP2 x64Windows XP SP3 x86Windows XP Embedded SP3 x86Windows 8 x86, and Windows 8 x64.

Reuters reported last night that the share prices of cyber security companies “surged as investors bet on governments and corporations spending to upgrade their defences”.

Network company Cisco Systems also closed up (2.3%), perhaps because of a belief that it would benefit from more network spending driven by security needs.

Security company Avast said the countries worst affected by WannaCry – also known as Wannacypt – were Russia, Taiwan, Ukraine and India.

Comment

In a small room on the periphery of an IT conference on board a cruise ship , nearly all of the senior security people talked openly about how their board directors had paid ransoms to release their systems after denial of service attacks.

Some of the companies – most of them household names – had paid ransoms more than once.

Until then, I’d thought that some software suppliers tended to exaggerate IT security threats to help market their solutions and services.

But I was surprised at the high percentage of large companies in that small room that had paid ransoms. I no longer doubted that the threats – and the damage – were real and pervasive.

The discussions were not “off-the-record” but I didn’t report their comments at the time because that would doubtless have had job, and possibly even career ramifications, if I had quoted the security specialists by name.

Clearly ransomware is, as the GCHQ expert Kieran Martin put it, a global business but, as ransoms are paid secretly – there’s not a whisper in corporate annual accounts – the threat has not been taken seriously enough in some parts of the NHS.

The government’s main defence is that the NHS was not targeted specifically and that many private organisations were also affected.

But the NHS has responsibility for lives.

There may be a silver lining if a new government focuses NHS IT priorities on the basics – particularly the structural defects that make the health service an easy target for attackers.

What the NHS doesn’t need is a new set of politicians and senior civil servants who can’t help massaging their egos and trying to immortalise their legacy by announcing a patchwork of technological marvels that are fun to work on, and spend money on, but which gloss over the fact that much of the NHS is, with some notable exceptions, technologically backward.

Microsoft stockpiled patches – The Register

UK government, NHS and Windows XP support – what really happened – Computer Weekly

NHS letter on patches to counter cyber attack

Multiple sites hit by ransomware attack – Digital Health (31 comments)

Lessons from the WannaCrypt – Wannacry – cyber attack according to Microsoft

 

MPs suggest Cabinet Office is losing its grip on departments – but does it care?

By Tony Collins

The Register has an excellent piece by Kat Hall on how the Cabinet Office is losing its grip on Government departments.

Citing the annual report of the all-party Public Accounts Committee, Hall says there are issues where “departments repeatedly don’t do what they have been told or asked to do by the centre”.

An analysis by The Register found that

“government departments are winning significantly more exemptions to splash the cash on expensive IT projects since the departure of former Cabinet Office minister Francis “Mad Frankie” Maude last year”.

Chair of the Public Accounts Committee Meg Hillier said: “After my second year as Chair I am increasingly concerned about the long-term accountability of senior civil servants.

“The game of musical chairs starts as one Permanent Secretary moves on and they all change jobs in the system. And few are in post long enough to have a vested interest in the long-term aims of their department or a project.

“And there is the age-old tension between a department and central Whitehall through the Cabinet Office.”

Universal Credit and HMRC’s plans to overhaul its Aspire IT contract – the biggest in Europe – were outlined as being two areas of concern. As was the Home Office’s Emergency Services Network.

“The Home Office seemed to downplay the risks to the contract and its being caught unawares by the contractor does not reassure us that the Department is on top of the contract or this project. This could cost the taxpayer dear,” it said.

Comment:

It’s hard to argue with a comment on Hall’s piece by @JagPatel3 who suggests that some in Whitehall are as preoccupied with spin as with the efficient delivery of public services.

“… Government is preoccupied with presentation, manipulation of words and the dark art of spinning – instead of working on its programme of reform to deliver public services efficiently, to satisfy the wants, needs and expectations of the electorate.

“The political imperative of needing to put a positive slant on everything the Government does or will do, irrespective of whether it is true or not, is the reason why spin has become the centrepiece of this Government’s communications strategy.

“And because Government has got a monopoly on inside information (enabling it to maintain extremely tight control), it uses spin to divert attention away from the key issues that really matter to citizens …

“the eagerness with which senior Civil Servants have complied with their political masters’ desire to see policy announcements framed around presentation and spin, at the expense of substance, would explain why their skills set has been narrowed down to this single, dark art.”

The commentator also says that the “intense focus of attention on presentation alone has resulted in a massive gap opening up between the leadership and lower ranks of the Civil Service, who have to deal with the reality of delivering public services on the ground, on a day-to-day basis, which has in itself, led to alienation and disaffection”.

A good summary. Many ordinary civil servants are doing the hard work of delivering public services while a few of their masters are preoccupied with keeping what they do secret and justifying or defending all else that is published in National Audit Office reports, other third-party reports or leaked emails.

It’s hardly surprising the Cabinet Office is losing control of departments. Since Maude’s departure it doesn’t want control. It has become clear that it wants, in a hassle-free way,  to continue with Sir Humphrey’s non-integrated approach to government.

The Cabinet Office is just another Whitehall department. Why would it want to be an “enforcer?”

After a major IT failure, how did Barts NHS trust manage its image?

By Tony Collins

It sounded serious. Under the headline

“Cancer patients in limbo as five hospitals suffer ‘major’ IT crash”

the Daily Telegraph said,

“Hundreds of cancer patients have been denied treatment at one of England’s biggest hospital trusts due to a major IT failure that ground basic services to a halt.

“Doctors at five large London hospitals have reported 11 days of “chaos” after the systems used to prescribe chemotherapy doses and share x-ray and MRI images broke down on April 20.

“Barts Health NHS Trust said at least 136 operations had been cancelled due to the crash, as well as “hundreds” of cancer treatment sessions.

“The computer failure also means frantic staff have been unable to process blood tests for all but the most critical cases…

“A doctor at the Royal London Hospital told the Daily Telegraph: ‘We have been forced to leave sick patients on the ward while we go down 16 floors to catch a glance at an x-ray image, then come back and make treatment decisions based on a hazy recollection of it…

“An email sent by managers to staff last week said the crisis had forced cancer teams to rebuild patient records ‘from scratch’.

A medic at Whipps Cross hospital was quoted as saying that a lot of people were stuck in hospital needlessly which increased the likelihood of infection.

The trust runs Mile End Hospital, Newham University Hospital, The Royal London Hospital, St Bartholomew’s Hospital and Whipps Cross University Hospital as well as other NHS sites.

The Barts trust website says it delivers “high quality compassionate care to the 2.5 million people of east London and beyond”.

It has a turnover of £1.25 bn and a workforce of 15,000, making it the largest NHS trust in the country.

According to Health Service Journal, an internal email from Barts’ chief clinical information officer Tim Peachy said the IT failure was primarily a result of an “unexpected failure of a small number of physical disks on which data is stored”.

At one point the trust was manually processing blood test results and X-rays, and arranging for porters at its hospitals to hand deliver paperwork to clinicians.

Barts’ reputation 

In the light of the failure and disclosures in Health Service Journal, Barts confirmed the IT problems in statements to the media. It also contacted patients who were affected by the problems. A Barts statement this week said,

“A major computer equipment failure on Thursday 20 April resulted in a number of IT applications being unavailable to staff.

“‘Unfortunately, it has been necessary to cancel 136 operations, representing about 2.5% of our usual weekly in-patient activity. Several hundred chemotherapy appointments have been cancelled, however we have now recovered the chemotherapy prescribing database.

“Clinical teams have completed a patient-by-patient review to ensure that the appropriate course of action is taken for each of them, endeavouring to keep the disruption to an absolute minimum.

“We apologise to those affected and will reschedule their appointment for as soon as we are able.

“A number of applications have been affected to varying degrees. We have made significant progress in many areas including pathology (blood testing), with image viewing now also restored across the Trust. There are still some other areas where it will take time before we are on track again.”

It added,

“We continue to work urgently to maintain the operational resilience of our services, using tried and tested contingency plans to keep our patients safe.”

Despite the seriousness of the problems, the effect on patients and the uncertainties that media coverage might have created in the minds of those intending to go to Barts’ hospitals, the trust made no mention of the difficulties on its website – where it has a “latest news” section –  or on Twitter.

Barts uses Twitter for good news announcements, comments and congratulations, sometimes with dozens of daily tweets.

But why no mention of the IT problems?

On this point, a Barts spokeswoman said,

“We do not rely on social media to update patients. As a proportionately small number of people will be impacted on by the IT situation we are communicating directly to those affected including at outpatients clinics and via phone, letter as well as through communications with our healthcare partners including GPs.”

Comment

In its media statements Barts has been more open than some NHS organisations.

The usual NHS cycle after a major IT-related failure is a statement saying teething problems have been resolved, or are being resolved, followed by a succession of similar statements over the next few days, weeks or months when it becomes clear the problems haven’t been resolved.

This is what happened with e-Referral Service and Capita’s problems handling GP support services.

That hasn’t happened at Barts. But despite its openness with the media, it’s odd  the trust has published many congratulatory tweets in the past two weeks without a mention of any IT-related problems. They are not even alluded to.

It’s also odd that on its website the Barts “Latest News” section has no mention of the difficulties. But the website does have various good news announcements, including a reference to a positive Care Quality Commission report in April 2007.

Trusts do not have to account to patients, Parliament or anybody for IT-related problems. They are under no obligation to apologise to patients whose stays in hospital are unnecessarily prolonged, or whose appointments, operations and blood tests are cancelled or delayed because of IT-related difficulties.

Back-up systems? 

They also have no obligation to give the public any reason for the failure or explain why there was no back-up system that ensured patients were unaffected.

But amid so many positive announcements, statements and comments to the public on its website and on Twitter, should Barts have left out the other side of the story?

The NHS is an organisation that’s attuned to promulgating good news. It’s rare for a trust board paper and or a trust website to have anything but a good news feel to it.

But telling the public one side of the story does not encourage the public to believe officialdom when it says: “Trust us. We know what we’re talking about.”

Thank you to Zara Pradyer for letting me know about the Daily Telegraph article.