Category Archives: NHS

When “life and death” NHS IT goes down

By Tony Collins

Almost unnoticed outside the NHS an email was circulated by health officials last weekend about a national “severity 1” incident involving the Electronic Prescription Service, running on BT’s data Spine .

“The EPS [electronic prescriptions service] database is currently experiencing severe degradation of performance. … BT engineers [are] currently investigating with the database application support team,” said the email.

A severity 1 or 2 incident, which involves a temporary loss of, or disruption to, the Spine or other national NHS system,  is not unusual, according to a succession of emails forwarded to Campaign4Change.

The Department of Health defines a severity 1 incident as a  failure that has the potential to:

— have a significant adverse impact on the provision of the service to a large number of users; or

— have a significant adverse impact on the delivery of patient care to a large number of patients; or

— cause significant financial loss and/or disruption to NHS Connecting for Health [now the Health and Social Care Information Centre], or the NHS; or

— result in any material loss or corruption of health data, or in the provision of incorrect data to an end user.

The Health and Social Care Information Centre, which manages BT’s Spine and other former NPfIT contracts, reports that the spine availability is 99.9% or 100%. But the HSCIC’s emails tell a story of service outage or disruption that is almost routine.

If the spine and other national services  are really available 99.9% of the time, is that good enough for the NHS, especially when ministers and officials are increasingly expecting clinicians and nurses to depend on electronic patient records and electronic prescriptions?  In short, are national NHS IT systems up to the job?

NHS staff access the spine tens of millions of times every month, often to trace patients before accessing their electronic records.  The spine is pivotal to the use of patient records held on Rio and Cerner Millennium systems in London. It is critical to the operation of Choose and Book, the Summary Care Record, Electronic Prescription Service pharmacy systems, GP2GP, iPM/Lorenzo, and the Personal Demographics Service.

According to a Department of Health letter sent to the Public Accounts Committee, payments to BT for the Spine totalled £1.08bn by March 2013.

BT says on its website that its 10-year NHS Spine contract involves developing systems and software to support more than 899,000 registered NHS users. The HSCIC says the Spine is used and supported 24 hours a day, 365 days a day.

“There is a huge amount of industrial-strength robustness, availability, disaster recovery, that you cannot get someplace else,” said a BT executive when he appeared before MPs in May 2011.

Life and death  

Sir David Nicholson spoke of the importance of the spine and other national NHS systems at a hearing of the Public Accounts Committee in 2011. He said they were

“providing services that literally mean life and death to patients today … So the Spine, and all those things, provides really, really important services for our patients…”

When Croydon Health Services NHS Trust went live with a Cerner Millennium patient records system at the end of September a “significant network downtime” – of BT’s N3 network – had an effect on patients.

A trust board paper, dated 25 November 2013 says:

“CRS Millennium (Cerner) Deployment -Network downtime – Week 1.  In particular, the significant network downtime in week 1 (BT N3 problem) led to no electronic access to Pathology and Radiology which resulted in longer waits for patients in the Emergency Department (ED) leading to a large number of breaches. This was a BT N3 problem which has been rectified with BT …”

Below are some of the emails passed to Campaign4Change in the past four months. Written by the Health and Social Care Information, the emails alert NHS users to outages or disruption to GP or national NHS IT systems.

Some HSCIC messages of disruption to service

October 2013

Severity 2
HSCIC
National
CQRS has not received a number of participation status messages.
Also affecting: GPES
USER IMPACT:
CQRS Users are not able to manually submit specific information, this will impact the users’ business process for entry of achievement data.
ACTION BEING TAKEN:
Following a configuration change by the GPES Business Unit a specific code has now been added to the GPET-Q Database. We are currently awaiting confirmation that the addition of the relevant code has been successful. Discussions are taking place regarding the re-submission of status messages. HSCIC conference calls are on-going.

[A severity 2 service failure is a failure [that] has the potential to:

– have a significant adverse impact on the provision of the service to a small or moderate number of service users; or

– have a moderate adverse impact on the delivery of patient care to a significant number of service users; or

– have a significant adverse impact on the delivery of patient care to a small or moderate number of patients; or

– have a moderate adverse impact on the delivery of patient care to a high number of patients; or

– cause a financial loss and/or disruption … which is more than trivial but less severe than the significant financial loss described in the definition of a Severity 1 service failure.]

**

Severity 2
HSCIC
BT Spine
National
Intermittent performance issues on TSPINE.
T-Spine
RESOLUTION:
BT Spine have confirmed that the incident has been resolved and users are able to perform routine business processes without delays.

November 2013

Severity 1
BT Spine
HSCIC
National
Users are unable to log into PDS.
USER IMPACT:
All sites are currently unable to access PDS, this is causing a delay to normal services.
ACTION BEING TAKEN:
BT Spine are working to restore service.

**

Severity 2
BT Spine
HSCIC
National EPS users.
Slow performance on reliable and unreliable messages for EPS.
USER IMPACT:
This is causing delays to routine business processes as some users may be experiencing slow performance with the EPS service.
BT investigating.

**

Severity 2
BT Spine
HSCIC
National
Slow performance on EPS Messaging.
USER IMPACT:
This is causing delays to routine business processes as some users may be experiencing slow performance with the EPS service.
ACTION BEING TAKEN:
BT moved the database to an alternate node following application server restarts. This temporarily restored normal message response times however performance has started to degrade again. BT Investigation continues.

**
Severity 1
Atos
HSCIC
National
Multiple users were unable to log in to the Choose & Book application.
ATOS made some network configuration changes overnight 19th/20th November which restored service. After a period of monitoring throughout the day yesterday the service has remained stable and at expected levels. Further activities and investigation will be carried out by several resolver teams which will be scheduled through change management.

**
Severity 2
BT Spine
HSCIC
National
Slow performance on EPS Messaging.
No further issues of slow response times with EPS messaging have occurred today. BT Spine to continue root cause investigation.

**

Severity 2
Cegedim RX
HSCIC
National
Cegedim RX – Users are experiencing slow performance in EPS 1 and EPS 2.
USER IMPACT:
Users are experiencing slow performance and delays to routine business processes when using EPS 1 and EPS 2.
ACTION BEING TAKEN:
Following a restart of application services, traffic has improved for all new EPS messages. However there is a backlog of EPS messages which may cause delays to routine business processes. Cegedim RX to continue to investigate.

**

December 2012

Severity 1
BT Spine
HSCIC
National
Performance issues have been detected with the transaction messaging system (TMS).
Also affecting: Choose and Book, GP2GP
USER IMPACT:
This may cause delays to routine business processes. This may have an effect on all Spine related systems. This includes PDS, Choose and Book, PSIS, SCR, ACF Services.
ACTION BEING TAKEN:
This has been resolved but BT are currently monitoring performance. Further investigation is required by BT into the root cause.

**

Severity 2
GDIT – CQRS
HSCIC
National
DTS has not processed a CQRS payment file.
CQRS
Also affecting: GPES
USER IMPACT:
This is causing delays to routine business processes.
ACTION BEING TAKEN:
GDIT are currently developing a fix which will be rolled out tomorrow evening, pending successful testing.

January 2014

Severity 1
BT Spine
HSCIC
National
TMS reliable messaging unavailable.
USER IMPACT:
TMS reliable messaging unavailable and users having to implement manual workarounds.
ACTION BEING TAKEN:
Issues experienced due to a planned change overrunning, BT Spine continue to implement the transition activity in order to restore service.

**

Severity 2
BT Spine
HSCIC
National
Users have experienced intermittent issues with the creation and cancellation of smartcards in CMS [Card Management Service for managing smartcards].
CMS
USER IMPACT:
This is intermittently causing delays to routine business processes as some users have been unable to create, cancel, cut or print cards in CMS.
ACTION BEING TAKEN
Users may experience issues with the creation and cancellation of cards in CMS. BT have identified a fix for the issue which is currently undergoing testing prior to deployment into the live environment.

**

Severity 2
BT Spine
HSCIC
National
The maternity browser was unavailable within NN4B.
RESOLUTION:
BT identified a problematic server which was recycled to restore system functionality.

**

Spine scheduled outage for essential maintenance activity.

During critical work to migrate to a new storage solution on Spine an issue was experienced on the Transaction Messaging Service (TMS) in September of this year. The issue resulted in BT failing over the TMS database from its usual site on Live B to Live A to restore service. The failover was completed well within the Service Level Agreement and no detrimental long term impacts to the service were incurred.

On the 15th January 2014 between approximately 22:00-23:30, HSCIC, in conjunction with BT, are planning to relocate the TMS database back to Live B, this is for several critical reasons:

  1. The issues experienced, which prompted the failover, are fully resolved and will not be experienced again as the storage migration work is now complete.
  2. The Spine service is designed to operate with all databases running on Live B so this work supports the optimum configuration for the service.
  3. Most critically the transition for all data on Spine to Spine2 has been designed to operate from a standby site with no live databases on it. Therefore to support the Spine2 transition this work is absolutely essential.

In order to facilitate a safe relocation of the database a 1.5 hour outage is required to TMS. The impact of this to Spine is significant and results in effectively an outage for Spine and its interfaces to connecting systems for that period. The time and date is aimed at the lowest times of utilisation for Spine, to minimise impact to end users, as well as not impacting critical batch processing and Choose & Book slot polls.

 

Date & Time

Change Start Change Finish Services Affected Outage Duration
15/01/2013 22:00 15/01/2013 23:30 Transaction Messaging Service (TMS) 1.5 hours
Service  Impact Description
Choose and Book The Choose and Book service will be available but functionality will be limited until the TMS database has switched over.Users of the web application will experience limited retrievals during the outage window.The system will not be able to create shared-secret for patients who have not been referred via Choose and Book before.Service Providers will be unable to:

  • Perform clinic re-structures and re-arrange appointments for patients for directly bookable services
  • Send DNA messages to Choose and Book.

For directly bookable services the following functionality will be unavailable:

  • Booking appointments
  • Rearranging appointments
  • Creating new patient accounts

Choose & Book systems will need to queue the messages and resend to Spine once the TMS service is enabled.

Due to the timing of the outage slot polls will not be affected.

Summary Care Record application (SCRa) The SCRa application will be available but functionality will be limited until the TMS database has switched over. Simple traces can be completed on PDS data but users will be unable to perform any PSIS updates (e.g. GP summary updates)
DSA The DSA application will be available but functionality will be limited until the TMS database has switched over.Simple traces can be completed on PDS data but users will be unable to perform any PSIS updates (e.g. GP summary updates).
Electronic Prescription Service (EPS)Pharmacy Systems Reliable messaging will be unavailable for the duration of the switchover work as the TMS service will be suspended dual site. All messages received from EPS systems will be rejected and not go into retry.EPS systems will need to queue the messages and resend to Spine once the TMS service is enabled.
EPS Batch The PPA response for any “claim” messages will not be sent to PPA/PPD. However, EPS will send those response(s) again when the retry jobs are re-activated after the switchover exercise is over. Response for any “claim” messages will not be received until after the switchover. Retry jobs will resend the responses once the TMS service is enabled.
Existing Service Providers (ESPs) There will be varying impacts depending on the product, release version and Spine compliant modules of the solution.ESP systems will need to queue the messages and resend to Spine once the TMS service is enabled.
GP2GP GP2GP will be unavailable until the TMS database has switched over.GP2GP systems will need to queue the messages and resend to Spine once the TMS service is enabled.
GP Extraction Service (GPES) GPES functionality will be unavailable until the TMS database has switched over.Messages will be queued on Spine and processed once the TMS service is restored.
GP Systems Functionality for Choose & Book, EPS and GP2GP, SCR will be limited until the TMS database has switched over.For Choose & Book directly bookable services the following functionality will be unavailable:

  • Booking appointments
  • Rearranging appointments
  • Creating new patient accounts

Systems will need to queue the messages and resend to Spine once the TMS service is enabled.

iPM/Lorenzo The real-time connection to Spine will be unavailable during the TMS outage. However both systems can be disconnected from Spine and operate without synchronised PDS data.iPM/Lorenzo will need to queue the messages and resend to Spine once the TMS service is enabled.
Millennium An outage of PDS reliable messaging will impact Millennium users.Users will be unable to:

  • trace patients
  • register new patients on PDS
  • book or reschedule appointments

Millennium will need to queue the messages and resend to Spine once the TMS service is enabled.

NN4B Trusts will need to be aware that during the outage NHS numbers cannot be generated, new-births cannot be registered and blood-spot labels cannot be generated and should plan accordingly.All birth notifications will be queued and processed once the TMS service is enabled.
Personal Demographics Service (PDS) Simple traces can be completed on PDS data.PDS reliable messaging will be unavailable until the TMS database has switched over.
RiO Users will be unable to:

  • trace patients
  • register new patients
  • book or reschedule appointments

The RiO system will need to queue the messages and resend to Spine once the TMS service is enabled.

TMS Event Service (TES) The majority of TES functionality will be unavailable during the outage.Trusts will need to be aware EPS, Death notifications, and Patient Care Provision Notifications (change of pharmacy) will be queued and sent to the receiving systems once the TMS service is restored.Any impacted notifications will be queued and sent to the receiving systems once TMS is restored.
TMS Batch (DBS, CHRIS, ONS) DBS will be unavailable until the TMS database has switched over (DBS processing will be suspended for the duration of the exercise).As the TMS switchover will be scheduled to start at 22:00, CHRIS batch should complete before the outage starts (CHRIS batch runs at 20:00 nightly).ONS processing will start at 18:00 nightly. If it doesn’t complete before 22:00, the messages will be queued and processed once the TMS service is restored.

**

Severity 2
BT Spine
HSCIC
National
Users are unable to grant worklist items within UIM.
USER IMPACT:
This is causing delays to routine business processes as users are unable to complete their worklist items within the UIM application.
ACTION BEING TAKEN:
BT investigating.

**

Severity 1
BT Spine
HSCIC
National
The EPS database is currently experiencing severe degradation of performance.
USER IMPACT:
Delays to routine business processes.
ACTION BEING TAKEN:
BT engineers currently investigating with the database application support team.

Comment

David Nicholson is right. The NHS has become dependent on systems such as the Spine. But can doctors ever trust any aspect of the safety of patients to systems that are not available 24×7 as they need to be in a national health service?

It appears that BT and other suppliers have not been in breach of service level agreements, and the HSCIC has a good relationship with the companies.  But does the HSCIC have too great an interest in not finding fault with its suppliers or the contracts, for finding fault  could draw attention to any defects in a service for which the HSCIC is responsible?

Have national NHS IT suppliers a strong enough commercial or reputational interest  in avoiding  a disruption or loss of service, so long as they keep within their service level agreements? 

If nobody sees anything wrong with the reliability of existing national NHS IT services improvements are unlikely. Diane Vaughan’s book on the culture and organisation of NASA shows that experts in a big organisation can do everything right according to the rules  and procedures – and still have a disastrous outcome.

Top 5 posts on this site in last 12 months

Below are the top 5 most viewed posts of 2013.  Of other posts the most viewed includes “What exactly is HMRC paying Capgemini billions for?” and “Somerset County Council settles IBM dispute – who wins?“.

1) Big IT suppliers and their Whitehall “hostages

Mark Thompson is a senior lecturer in information systems at Cambridge Judge Business School, ICT futures advisor to the Cabinet Office and strategy director at consultancy Methods.

Last month he said in a Guardian comment that central government departments are “increasingly being held hostage by a handful of huge, often overseas, suppliers of customised all-or-nothing IT systems”.

Some senior officials are happy to be held captive.

“Unfortunately, hostage and hostage taker have become closely aligned in Stockholm-syndrome fashion.

“Many people in the public sector now design, procure, manage and evaluate these IT systems and ignore the exploitative nature of the relationship,” said Thompson.

The Stockholm syndrome is a psychological phenomenon in which hostages bond with their captors, sometimes to the point of defending them.

This month the Foreign and Commonwealth Office issued  a pre-tender notice for Oracle ERP systems. Worth between £250m and £750m, the framework will be open to all central government departments, arms length bodies and agencies and will replace the current “Prism” contract with Capgemini.

It’s an old-style centralised framework that, says Chris Chant, former Executive Director at the Cabinet Office who was its head of G-Cloud, will have Oracle popping champagne corks.

2) Natwest/RBS – what went wrong?

Outsourcing to India and losing IBM mainframe skills in the process? The failure of CA-7 batch scheduling software which had a knock-on effect on multiple feeder systems?

As RBS continues to try and clear the backlog from last week’s crash during a software upgrade, many in the IT industry are asking how it could have happened.

3) Another Universal Credit leader stands down

Universal Credit’s Programme Director, Hilary Reynolds, has stood down after only four months in post. The Department for Work and Pensions says she has been replaced by the interim head of Universal Credit David Pitchford.

Last month the DWP said Pitchford was temporarily leading Universal Credit following the death of Philip Langsdale at Christmas. In November 2012 the DWP confirmed that the then Programme Director for UC, Malcolm Whitehouse, was stepping down – to be replaced by Hilary Reynolds. Steve Dover,  the DWP’s Corporate Director, Universal Credit Programme Business, has also been replaced.

4) The “best implementation of Cerner Millennium yet”?

Edward Donald, the chief executive of Reading-based Royal Berkshire NHS Foundation Trust, is reported in the trust’s latest published board papers as saying that a Cerner go-live has been relatively successful.

“The Chief Executive emphasised that, despite these challenges, the ‘go-live’ at the Trust had been more successful than in other Cerner Millennium sites.”

A similar, stronger message appeared was in a separate board paper which was released under FOI.  Royal Berkshire’s EPR [electronic patient record] Executive Governance Committee minutes said:

“… the Committee noted that the Trust’s launch had been considered to be the best implementation of Cerner Millennium yet and that despite staff misgivings, the project was progressing well. This positive message should also be disseminated…”

Royal Berkshire went live in June 2012 with an implementation of Cerner outside the NPfIT.  In mid-2009, the trust signed with University of Pittsburgh Medical Centre to deliver Millennium.

Not everything has gone well – which raises questions, if this was the best Cerner implementation yet,  of what others were like.

5) Universal Credit – the ace up Duncan Smith’s sleeve?

Some people, including those in the know, suspect  Universal Credit will be a failed IT-based project, among them Francis Maude. As Cabinet Office minister Maude is ultimately responsible for the Major Projects Authority which has the job, among other things, of averting major project failures.

But Iain Duncan Smith, the DWP secretary of state, has an ace up his sleeve: the initial go-live of Universal Credit is so limited in scope that claims could be managed by hand, at least in part.

The DWP’s FAQs suggest that Universal Credit will handle, in its first phase due to start in October 2013, only new claims  – and only those from the unemployed.  Under such a light load the system is unlikely to fail, as any particularly complicated claims could managed clerically.

 

MP calls for candour after Cerner NPfIT go-live at Croydon

By Tony Collins

Richard Bacon, a long-standing member of the House of Commons’ Public Accounts Committee, has called on Croydon Health Services NHS Trust to be more open about problems it faces after deploying a Cerner Millennium patient records system at the end of September.

The installation was carried out by BT under the London Programme for IT – a branch of the NPfIT.  The Health and Social Care Information Centre, which has taken on BT and CSC contracts under the NPfIT, was the trust’s partner for the Cerner deployment.

Bacon has closely followed the NPfIT and written a chapter on it in his book, “Conundrum: Why every government gets things wrong and what we can do about it” which he co-wrote with Christopher Hope, the Telegraph’s senior political correspondent.

According to fragments of information in Croydon Health Services’ latest board papers, dated 25 November 2013, the trust has faced a series of problems after the NPfIT Cerner go-live.

They included:

–  N3 Network downtime and waiting time breaches.

 Excessive waits for patients in A&E

 Going over budget.

– Significant loss of income.

 A bid to recover Cerner costs.

– A need for HSCIC support for delays. 

-A need for extra investment in Cerner to “stabilise the operational position”

The trust has not published any specific report on the implementation’s problemsNow Bacon says it is “unacceptable for any trust not to disclose the problems it faces – and possibly patients face – after a major IT implementation such as Cerner”.

He adds:

“If these implementations go wrong they can affect the safety of patients.  We know this from some NPfIT deployments at other  trusts. For Croydon to say that board members have been kept informed of the potential risks of the Cerner implementation through the “Corporate Risk and Board Assurance Framework”  is not reassuring.

“This is putting a matter of importance in the small print. Indeed, for officials to brief board members on the potential risks, rather than actual events, is also of concern.

“Patients need to know that Croydon takes a duty of candour seriously. If the Trust cannot be open about its IT-related problems, how can we be sure it will be open about anything else to do with patient safety?”

Patient records go-live “success” – or a new NPfIT failure 

Patient records go-live “success” – or a new NPfIT failure?

By Tony Collins

John Goulston says the go-live of a new patient records system at his trust is a “success”.

He should know. He’s Chief Executive of Croydon Health Services NHS Trust. He’s also chair of the trust’s Informatics Programme Board which has taken charge of bringing Cerner Millennium to Croydon’s community health services and the local University Hospital, formerly the Mayday.

He was formerly Programme Director of the London Programme for IT at NHS London – a branch of the NPfIT.

In a report two weeks ago Goulston said the trust deployed the “largest number of clinical applications in a single implementation in the NHS”. Croydon went live with Cerner Millennium on 30 September and 1 October 2013.

Said Goulston in his report:

“Administrative functions do not engage clinicians; providing them with a suite of clinical functionality has been justified as each weekday approx. 1,000 staff are logged on and using the system. CHS [Croydon Health Services] has in Phase 1 deployed, in addition to patient administration, the largest number of clinical applications in a single implementation in the NHS England.”

BT helped install Millennium at Croydon under the National Programme for IT.  The trust’s spokesman says the Department of Health provided central funding, and the trust paid for implementation “overheads”.  The Health and Social Care Information Centre was the trust’s partner for the go-live.

The Centre is the successor for Connecting for Health. It has taken on CfH’s officials who continue to help run the NPfIT contracts with BT and  CSC.

Goulston said that Cerner and BT have paid tribute to the trust which installed Millennium in A&E, outpatients, secretarial support and cancer services, and elsewhere.

“Our partners Cerner, BT and Ideal have commented that the Trust has undertaken one of the most efficient roll-outs of the system they have worked on, with more users adopting the system more quickly and efficiently than other trusts … the success we have achieved to date is the result of the efforts of every single system user and all staff members,” said Goulston.

Best Cerner implementation yet?

Optimistic remarks about their launch of Cerner Millennium were also made in 2012 by executives at the Royal Berkshire NHS Foundation Trust.  Their optimism proved ill-judged.

Of the Millennium go-live at Royal Berkshire, trust executives said that it “had been considered to be the best implementation of Cerner Millennium yet and that despite staff misgivings, the project was progressing well”.   This positive message should be disseminated, they said.

Months later they told the Reading Chronicle of patient safety issues and a financial crisis arising from the Millennium implementation.

A Royal Berkshire governors Rebecca Corre was quoted as saying: “There is a patient safety issue when staff write down observations and then there is an hour before they can get it onto the computer. If it is an experienced nurse, they may pick up a problem, but others may not.”

Ed Donald, Chief Executive of Royal Berkshire was quoted as saying:

“Unfortunately, implementing the EPR [electronic patient record] system has at times been a difficult process and we acknowledge that we did not fully appreciate the challenges and resources required in a number of areas.”

Are executives and managers at Croydon Health Services NHS Trust  now similarly afflicted with an unjustified optimism about the success of their Cerner go-live?  

Past consequences of NPfIT go-lives hidden?

The Department of Health has claimed benefits for the NPfIT of £3.7bn to March 2012 but there have been trust-wide failures: thousands of patients have had their appointments, care or treatment delayed by difficulties arising from past implementations of patient record systems under the NPfIT.  For thousands of patients waiting time standards have been exceeded or “breached” because of disruption arising from troubled go-lives.

In nearly every case trusts made it difficult for the facts to come out publicly. Vague or unexplained fragments of information about the consequences of the NPfIT implementation appeared  in different board papers over several months. The facts only emerged after a journalistic investigation that required scrutiny of many board papers and follow-up questions to the trust’s press office.

So Campaign4Change investigated Croydon Health’s implementation of Cerner Millennium to see if the Francis report’s call for a “duty of candour” over mistakes and problems in the NHS have made any difference to the traditional fragmentation of facts after NPfIT go-lives of patient record systems.

The Francis report called for “openness, transparency and candour“.  Trusts were told not to hide sub-standard practices under the carpet. The health secretary Jeremy Hunt said it can be “disastrous” when bad news does not emerge quickly and the public are kept in the dark about poor care.

To my questions about the Cerner Millennium implementation Croydon trust’s spokesman always responded promptly and tried to be helpful. But it appears that trust executives have given him limited information about consequences of the go-live, and have preferred to indulge the “good news” NHS culture that Jeremy Hunt warned about.

On being asked what problems the trust has faced since the go-live the spokesman gave various answers that made no mention of the problems.

“All of our staff received training on the system, and we are continuing to offer our teams support as it is embedded.”

What of the problems arising from the implementation, and has the board been fully informed?

“Millennium has featured regularly on the Corporate Risk Register presented to each Part 1 Board meeting.   In addition, implementation has received detailed confidential consideration at Part 2 of Board meetings, (which is why you won’t find it in our public board papers).”

Given Francis’s call for duty of candour,  should the trust be more open about its problems?

“The initial roll out for CRS Millennium was introduced over three days at the Trust, with a phased approach.  We did this to ensure the system was working in each department, before introducing it in another area.

“We are monitoring waiting time performance and records management so we can identify any issues if they emerge. The system is still being introduced in some services and when this is completed we will be able to assess the overall programme,” said the spokesman.

Does Croydon’s unwillingness to give in its statements to me any details of problems indicate that the culture of a lack of transparency in the NHS will be hard to change, no matter how many times Jeremy Hunt talks about the need for candour when things go wrong?

The spokesman:

“I’d like to be clear about the Trust’s approach:

  • The Trust board has been cited on the roll out of CRS Millennium and any potential risks throughout the process.  As I previously noted, the board received an update in September.  The board meeting, which will take place on Monday of next week, will receive a further update from the Chief Executive.  The papers from this meeting will be published on our website and the meeting takes place in public;
  • A meeting chaired by the Chief Operating Officer has reviewed any operational matters arising on a daily basis.  This is an internal meeting for clinicians and managers which has informed the implementation process;
  • Patients and visitors to the hospital have been kept fully appraised of the introduction of the system and were made aware that they may experience some delays to the check-in process while staff became familiar with the new computer system;

“These actions would suggest that the Trust has been transparent in its approach.  You are welcome to review the board papers when they are published.”

Serious problems now emerge

Croydon did indeed publish its board papers on 25 November 2013 – which is to its credit because not all NHS trusts publish timely board papers.

But it’s mostly in the small print of various board papers that details emerge of Millennium-related problems. The shortcomings are mentioned as individual items rather than in a single, detailed Cerner Millennium deployment report.  This leaves one to question whether trust directors have an overview of the seriousness of the difficulties arising from its implementation of a new patient records system.

These are some excerpts from deep inside Croydon’s latest board papers:

Breaches in waiting time standards

– “CRS Millennium (Cerner) Deployment -Network downtime – Week 1.  In particular, the significant network downtime in week 1 (BT N3 problem) led to no electronic access to Pathology and Radiology which resulted in longer waits for patients in the Emergency Department (ED) leading to a large number of breaches. This was a BT N3 problem which has been rectified with BT providing CHS with the required scale of N3 access (>600 concurrent users and >1,600 users on any day – which is the largest network usage of any trust in England).”

– • “Hospital Based Pathways: The deployment of CRS Millennium was a particular challenge in the month across the multiple service areas within the Directorate of A&E, Surgery and Maternity.

• “Cancer & Core Functions: With the implementation of CRS Millennium, the open pathways part of RTT [referral to treatment – patient waiting times) may fail the standard – validation will be completed after the narrative for this report… “

Excessive waits in A&E

– “The main drivers adversely affecting the performance in the month [October 2013) for A&E were the deployment of CRS Millennium and the commencement of winter pressures due to the seasonality change.  A&E  4-Hour Total Time in Department Target: 95.00%. Actual: 91.57%.”

Over budget

“The Trust position as at October is an adverse variance of £4.1m. This is a significant deterioration on the Month 6 position. The movement is mainly due to a significant reduction in income mainly as a result of operating issues caused by the Cerner deployment (£0.9m)…  Actual £14.8 (£14.8)m; Budget £10.7m; Variance £4.1m.”

“Cerner Millennium: Plan YTD [year-to-date] £245,000; Actual YTD  £621,000;

Significant loss in income

“… A new patient administration system was deployed in the Trust on the 30th September and 1st October (Cerner Millennium). The deployment has resulted in significant loss in income in September and October £ 1.1m. Trust performance on Activity Planning Assumptions and Key Performance Indicators is substantially worse than plan …”

Extra costs

“Medical £412k and admin £148k agency levels continue to be high due to cover for vacancies, annual leave, sickness and release of staff for Cerner training. The Trust has also incurred additional costs associated with the Cerner deployment (£600k) including overtime payments to administration staff and training costs.”

Bid to recover Cerner costs?

“… The Trust is currently forecasting a deficit position of £17.8m, which is £3.3m off the plan submitted to the NHS Trust Development Authority. This is a £3m movement from the month 6 forecast and is as a result of operational issues caused by the Cerner deployment. The current projected impact is an additional costs £1.7m and a loss in activity £1.1m . An application is to be made to recover the additional cost/losses relating to the Cerner deployment [of £2.9m] …”

HSCIC support for delays

“Cerner Millennium – Revised implementation date to Sept 2013 (achieved) ,with resultant additional costs including additional PC requirements of £146k, specialist support services £300k, procurement costs £91k, data cleansing costs £200k.

“Health& Social Care Information Centre (HSCIC) has confirmed support for the delayed implementation will be provided, accounting treatment of support to be confirmed with Department of Health.”

More money to stabilise operational position?

“As a result of operational issues caused by the Cerner deployment , Income is significantly reduced in October. The forecast assumes that the Trust will resume normal operating levels from November and that an element of the income lost will be will be recovered in the latter part of the year. A business case is being submitted to the Trust Board for additional investment in Cerner to stabilise the operational position.

“If there are further operational issues due to the Cerner deployment then this will significantly impact on the year end forecast…”

Over-optimism?

Principal risk -reporting output from Cerner is not accurate or timely. Officer in charge: CEO. Before go-live risk scores: June 2013 – 16; July – 16; Aug  – 10; Sept – 10. After go-live risk score (for Oct): 20 [high risk of likelihood and consequences]

Principal risk – operational readiness following the implementation of Cerner. Officer in charge: COO.  Before go-live risk score 15. Post go-live: 20. Risk rating before go-live – Green. After go-live – Red.

Red risks

Corporate Risk Assurance Framework

Nine risks are reported as Red [two of which relate directly to Millennium]:

“… Reporting output from Cerner is not accurate or timely. Data migration was successful. However reliance on external provider as internal knowledge has not yet been fully gained. A data quality dashboard with exception reporting is in place.

“… Operational readiness following the implementation of Cerner CRS Millennium impact conveyed to Trust Development Authority e.g. ED [Emergency Department] reporting and cost overruns

Risk scores

– Failure of CRS millennium to deliver anticipated benefits – 12. Officer in charge: CEO

– Reporting output from Cerner is not accurate or timely – 20. Officer in charge: CEO

– Operational readiness following the implementation of Cerner – 20. Officer in charge: COO

Croydon’s trust’s response to problems

Said John Goulston, Croydon’s CEO, in his latest [November 2013] report to the board of directors:

“The issues being encountered now with CRS Millennium are not due to any lack of integration testing with legacy applications or testing of workflow. They can be attributed to changing from a 25 year old Patient Administration System (Patient Centre) which did not require working in real time, was simple and intuitive to use, easily configurable and flexible to our needs.

“CRS Millennium’s patient administration functions are almost the complete opposite and the language used is new for our staff i.e. conversations, encounters etc. For our staff it has been a big ask for them to step into and up to such a complex application.”

He added: “The benefits of the new system are that each patient will have a single accurate electronic record that can be viewed and kept up-to-date by hospital and community clinical staff. This will eventually mean less time searching for patient notes, missing documentation and duplicating patient information…

“As with any massive change, there are still some challenges to tackle in making the system work effectively for every single user, in a diverse and complex organisation.

“However the success we have achieved to date is the result of the efforts of every single system user and all staff members. I would like to thank all our staff for their hard work in getting the Trust to this important stage.”

The trust spokesman gave me this statement on the problems:

“The Trust board has been given regular reports on the roll out of CRS Millennium and any potential risks throughout the process, not least through its regular reviews of the Corporate Risk and Board Assurance Frameworks.  As I previously noted, the board received a specific update in September.

“As you already know, November’s board meeting received a further update from the Chief Executive.  The papers from this meeting were published and the meeting takes place in public;  Those attending are invited to put forward questions.

“A meeting chaired by the Chief Operating Officer continues to review operational matters.  This is an internal meeting for clinicians and managers which has informed the implementation process;

“Patients and visitors to the hospital have been kept fully appraised of the introduction of the system and were made aware that they may experience some delays to the check-in process while staff became familiar with the new computer system;

“As you highlight from the board report, Cerner & BT noted that ‘the Trust has undertaken one of the most efficient roll-outs of the system they have worked on’   The papers also note some operational challenges as the system was rolled out.  These have been addressed as part of the daily meetings I reference above – these are mainly concerned with users familiarising themselves with the system and have been addressed through the support and training staff received.

“In terms of the costs, the introduction of CRS Millennium has been supported by central funding from the Department of Health with the Trust paying the implementation overheads.   These costs are a matter of public record and the Trust publishes annual Accounts as part of its Annual Report.”

Comment

When you go into hospital it’s reassuring to know the directors will be well informed and open about problems that could affect you.

The approach of Croydon Health Services NHS Trust to openness about its problems is not reassuring. It is no better or worse than other trusts that have implemented Cerner’s Millennium. In fact the timely publication of its board papers means it is more open than some.

But it should not require a time-consuming journalistic investigation to establish the consequences for patients of an NPfIT go-live. It has required just such an investigation after the go-live of Millennium at Croydon.

Board directors will not have the time to dig for, and piece together, information about internal problems that could delay patient appointments, treatment and care. They need the unpalatable facts in one place. Croydon Health Services has failed to make it easy for patients or board directors to see what has gone wrong.

NPfIT deployments at other trusts have led, cumulatively, to thousands of patients having appointments that were disrupted, or who had to wait longer to be seen than necessary, or whose records were not available, or who were seen with another patient’s records.

In shying away from telling the whole truth trusts take their cue from the top: the Department of Health has always made it hard to establish facts about anything to do with the NPfIT.  Said the Public Accounts Committee in its report The National Programme for IT in the NHS: an update on the delivery of detailed care records systems in July 2011:

 “It is unacceptable that the Department [of Health] has neglected its duty to provide timely and reliable information to make possible Parliament’s scrutiny of this project.

“Basic information provided by the Department to the National Audit Office was late, inconsistent and contradictory.”

Unanswered questions

Croydon has questions to answer, such as how many breaches of waiting time standards it has had, and may still be having, due to problems arising from the go-live. Other unanswered questions:

– What does a “a large number of breaches” in the Emergency Department mean? Have each the patients affected been told?

– Why are the risks related to the implementation much higher after go-live than before, given that the trust has had years to prepare for the go-live, and the many lessons it could have learned from other trusts?

– Exactly what problems are still affecting patients?

In a post-Francis NHS, Jeremy Hunt has demanded openness about mistakes and problems. There is an agreed need for change – but how can Hunt change an NHS culture – indeed a public sector culture – in which senior executives, in troubled IT implementations, will always emphasise the good news over the bad, perhaps hoping the bad will always remain hidden?

Who polices police IT reports?

By Tony Collins

The police, and civil and public servants in central government, the NHS and local authorities criticise journalists for biased reporting – taking selected facts out of context.

They’re sometimes right.  Journalists working for national newspapers can draft an article that is diligently balanced only to find, by the time it’s published, it leaves out facts which would have complicated, blunted, or contradicted the main points.

It’s one thing for this to happen in the world of journalism. You don’t expect public bodies to report on their own affairs with a partiality that rivals out-of-context reporting by some newspapers.

But it appears to be happening so regularly that one-sided self-reporting on organisational performance may be becoming the norm in the public sector.

In the NHS subjective, positive reporting in board papers – where managers tell directors what they think they want to hear – could help to explain why Cerner patient record implementations have, for years, gone badly wrong for the same reasons.

In recent months reports without balance have been published on the performance of Avon and Somerset Police’s IT outsourcing contract with IBM. 

Somerset County Council, Taunton Deane Borough Council and Avon and Somerset Police  are minority shareholders in a private company, Southwest One,  which is owned by IBM.

Confusingly, Taunton Deane Borough Council issued positive reports about its successful partnership with Southwest One – and then it decided to take some services back in-house.

Now it has emerged – only as a result of FOI requests by Somerset resident and campaigner Dave Orr – that two independent organisations, the National Audit Office, and HM Inspectorate of Constabulary, have commented positively on Avon and Somerset Constabulary’s partnership with Southwest One, based entirely on the unaudited opinions of the police force itself.

SAP

From his FOI requests Orr learned that the Avon and Somerset’s outsourcing deal with Southwest One has not gone entirely as expected. The National Audit Office’s FOI team has released notes of a joint visit by the NAO and HM Inspectorate of Constabulary to Avon and Somerset police in December 2012.  The visit was to find out about how well Southwest One was delivering services to the police force.  

The NAO’s notes are positive in parts. They say that performance has improved considerably since the implementation of the contract.

“Implementation of SAP improving the accounts close-down process, initial issues being resolved and a good quality of service being provided regularly.”

But there is another side to the story that is not reflected in the published accounts of Avon and Somerset’s relationship with Southwest One. The NAO’s [unpublished] field notes say:

“Fewer than expected benefits have been realised from IT due to the considerably different security requirements of the Police compared to the Councils.

“It also took a long time for SAP to be implemented. There has yet to be a duty management system implemented by SWOne which is part of the contract… SAP would have benefited from some pre-launch testing or piloting.”

A letter to Orr from the Home Office appears to confirm that Avon and Somerset Police’s participation in Southwest One is an unequivocal success.

“The private sector can help to deliver police support services better and at lower cost. Every pound saved means more money for the front line, putting officers on the streets…

“In its report “Policing in Austerity: rising to the challenge [2013] Her Majesty’s Inspectorate of Constabulary identified the Southwest One partnership as being a key element in achieving savings for Avon and Somerset Constabulary while ensuring better procurement, streamlining business support processes, and ensuring better use of police officer time.

“The report also noted that the Southwest One collaboration was the first of its kind for policing in England and Wales and that to date, no other force has delivered this level of partnership with local authorities.”

A little of the other side of the story comes in the last sentence of the Home Office letter to Orr which says: “We understand that Avon and Somerset Constabulary continues to work closely with IBM to resolve any technical difficulties and improve the services provided by Southwest One.”

Indeed a table on page 155 of HMIC ‘s 2013 report Policing in austerity: rising to the challenge indicates that Avon and Somerset Constabulary has one of the worst records of any police force when it comes to savings delivered between 2010/11 and 2012/13. [Table: Key indicators of the challenge – quartile analysis.]

Southwest One began a 10-year contract providing services to Avon and Somerset Police in 2008. The services included enquiry offices, district HR, estates, financial services, site administration, facilities, corporate human resources, information services, purchasing and supply, and reprographics. The contract involves 554 seconded staff.

Comment

Police forces, councils, the NHS and central government departments need  a few Richard Feymans to report on their organisation’s performance. Feynman was a gifted scientist, MIT graduate and noble prize winner who was chosen as a commissioner to report on the cause , or causes, of the Challenger Space Shuttle “O” rings accident on 28 January 1986.

He reported with such independence of mind and diligence that his hard-hitting findings were not considered acceptable to be included in the main report of the Presidential Commission of inquiry into the accident.  Feynman had to be content with having his findings published as an appendix to the Commission’s report – and an edited appendix at that.  

He suggested in his book “What do you care what other people think?” that his appendix was the only genuinely balanced part of the official inquiry report. 

“For a successful technology, reality must take precedence over public relations, for Nature cannot be fooled,” said Feynman.

One of his questions was whether “organisation weaknesses that contributed to the [Shuttle] accident [was] confined to the solid rocket booster sector, or were they a more general characteristic of NASA.”

One of Feynman’s conclusions:

“It would appear that, for whatever purpose – be it for internal or external consumption – the management of NASA exaggerates the reliability of its product to the point of fantasy.”

If such exaggeration happens at NASA it can happen in UK police force IT reports, and in board papers on the performance of councils and NHS trusts.

When journalists get it wrong it’s usually to their eternal regret. In the public sector positive unbalanced reporting is so “normal” that hardly anyone involved realises it’s a deviant practice. The US author Diane Vaughan coined a phrase for such corporate behaviour.  She called it the normalisation of deviance.  

It’s surely time for public bodies to move away from the norm and start reporting on their performance, and the performance of their outsourcing other private sector contracts, with balance, objectivity and independence of mind.   

If managers knew that reports on the progress of their contracts would be audited for impartiality and competence over organisational self-interest, perhaps they would have a greater incentive to avoid badly thought through outsourcing deals and IT implementations.

Is this why some council and NHS scandals stay hidden for years?

NAO report “Private sector partnering in the police service”

Dave Orr’s HMIC FOI requests and answers

NAO’s FOI responses on Avon and Somerset Police

Is this a reason some council and NHS scandals stay hidden for years?

By Tony Collins

Six years into Southwest One’s joint venture between IBM and three public authorities, the outsourced service is not a big success.  

Somerset County Council, one of the joint venture’s partners, has been in dispute with IBM, the major shareholder in Southwest One. It cost the county council £5.9m to settle, including £800,000 in costs when bringing back staff who had been outsourced.

The joint venture’s SAP-based “transformation” led to complaints of poor quality of service by some of Somerset’s finance users; the venture has consistently made losses and on the matter of savings, Somerset Cabinet member for resources, David Huxtable, said there have been some but added:

“It was a very complex contract and lots of the savings were predicated on an ever-increasing amount of money being put into public services and we know in the last four years that has gone into reverse.”

Now IBM has sold its low-margin customer care outsourcing unit which could affect the future of Southwest One.

Yet a smaller partner in Southwest One, Taunton Deane Borough Council, describes the relationship as a “success”. Reports to Taunton Deane’s councillors on Southwest One are remarkably positive.

Dave Orr an IT specialist who used to be work for Somerset County Council and has kept a close eye on Southwest One since it was formed in 2007 has drawn my attention to Taunton’s latest reports on the joint venture. When read quickly Taunton’s reports are upbeat, almost breathless with praise for the joint venture.

Below are excerpts from two reports that have been written for today’s meeting of Taunton Deane’s Corporate Scrutiny Committee.  The first is a “Procurement Transformation Update”, report to the council by Southwest One’s Chief Procurement Officer.

There are no hints of any difficulties on the contract except a comment that cutting spending will make it harder to achieve procurement savings. From Southwest One’s report to Taunton:

“Executive Summary

“As at 31/07/13, in excess of £1.8m [report’s emphasis] savings have been delivered to the Council through signed-off procurement-related initiatives brought about by Southwest One’s Strategic Procurement Service. This is up from £1.59m when last reported in January 2013.

“A further £1.364m of savings are scheduled to be delivered from these signed-off initiatives during the life of the current Southwest One contract, which expires in 2017.

“Multiple projects continue to be progressed by Southwest One Strategic Procurement Service which are expected to significantly add to the pipeline of savings. These Include initiatives for a new pool & spa at Blackbrook; Waste; Insurance; various small scale initiatives within the DLO/HPS areas .”

A second report for Taunton’s Corporate Scrutiny Committee “SouthwestOne Partnership Update report” is written jointly by a team at Taunton council and the CEO of Southwest One. Again it’s upbeat and summarises Southwest One’s performance over the last six months.

“Service delivery for TDBC, viewed in the round, is broadly on track. The majority of services perform well or extremely well (eg Customer Services). We do have concerns in some areas and we are working closely with the services in question to remedy the issues. “

The report says that the shared service model in conjunction with larger authorities provides Taunton with “much needed resilience” (report’s underlining) in service delivery, although “this has been impacted to a certain extent by changes made recently to the contract by the other partners”.

Additionally, “our secondee staff to SWO benefit from ‘assured employment’, which was offered by IBM”.

A survey of staff in June 2013 “saw marked improvements in staff morale and communication”.

Sickness absence for the financial year to the end of March 2013 was slightly down to about 9 days per full-time employee though up a little more recently.

Appendices – now for the problems

It’s only when councillors come to the report’s appendices that they will see some detail of the problems. But how many councillors will scrutinise a report’s appendices? From the Taunton report’s appendices:

“There are service and capacity issues. The helpdesk move caused significant problems, leading to an increased number of issues being raised with the Client Team from TDBC [Taunton Deane Borough Council] staff. We are closely monitoring the plan SWO [Southwest One] have put in place to fix these issues.

“Project delivery capacity and project scheduling continues to cause concern, with improved governance within TDBC highlighting this problem more acutely. Our issue tracker is currently tracking 11 escalated issues with SWOne, 6 of them with a Red RAG status.

“ SRM (Supplier Relationship Management) performance in SAP continues to be well below the required level despite the amount of focus it is receiving from SWOne. Work on a revised governance process for the SAP system is underway and looks likely to deliver a more controlled SAP Change process.”

The most serious problem – and it is not mentioned until the penultimate page of the report’s appendices – is that savings will be nowhere near the original target of £10m.

“This is red, tracked against the original [savings] £10m target. To date £2.8m has been signed-off and it is not yet clear how the lower target of £5.7m will be achieved as there are fewer savings opportunities and initiatives emanating from SWO…”

From the small print of Taunton Deane’s report it is possible to work out that the cost of the council’s SAP implementation was supposed to have been paid off by savings but hasn’t. Indeed a debt of nearly £1m is still incurring interest.

Comment

Perhaps it’s unfair to pick on Taunton Deane’s reports to councillors. The positive tone is little different to dozens, perhaps hundreds, of NHS, council and central government board reports I have read over two decades.

If you’re a director of a public authority your job is probably made harder if you’re getting self-vindicating internal reports on the organisation’s progress. It would be more helpful if management reports were neutral and objective, framed by unvarnished facts.

When you hire a roofing company and it reports back on the finished job, you want to know about the tiles that leave a gap or are loose, not the ones that fit nicely.

NHS trust reports can often be particularly one-sided, often of the type that say:

“We had 3 fatalities on the main staircase last month because of a ruptured floor lining but the overall accident rate in that part of the building is down over the last 3 years and our falls rate overall is 3% below the average for the NHS as a whole.

“Our contractor confirms that the floor lining is within KPI requirements and a repair will be effected shortly.”

It appears that those who write board reports for public authorities feel an obligation to motivate and inspire, to leave the reader feeling good, to clothe bad news in layers of good news, omit it altogether or put it in the appendix hardly anyone reads.

Is this one reason so many outsourcing and NHS scandals stay hidden for years? 

NPfIT central costs rise by tens of millions – even after “dismantling”

By Tony Collins

On 22 September 2011 the Department of Health announced the dismantling of the NPfIT. As the press release was being issued some officials at the department were aware that they were continuing to spend tens of millions on central administrative costs of the programme.

Today’s report of the Public Accounts Committee has a figure for the central costs of the NPfIT until the end of March 2012 of about £890m. Before the DH announced the dismantling of the programme, in March 2011, the DH put the central costs at £817m.

So there has been a rise in central admin costs of about £70m since the NPfIT was supposedly dismantled.

The administrative costs are separate from spending on the contracts with BT or CSC. The admin costs don’t include the delivery of a single laptop to the NHS under the NPfIT. They are simply the central costs of administering the programme – including day rates for consultants – such as day rates of £1,700 to help senior officials prepare for appearances before MPs on the Public Accounts Committee.

The central costs have never been explained, not even by the National Audit Office which has published several reports on the NPfIT.  It is known that some central costs are explained by items of questionable benefit such as the commissioning of DVD films that marketed the NPfIT.

Some of the cost categories have emerged as a result of an FOI request (below).  Officials made regular visits to various parts of the globe to promote the success of the NPfIT. It’s thought that the DH has spent more than £100m on consultants for the programme.

Millions of pounds have been spent with public relations companies. The DH spent about £30,000 on press cuttings in two years alone.  Released central costs for just two years of the NPfIT between 2005 and 2007 include:

  • £1.23m with Expotel Hotel Reservations
  • £1.87 Harry Weeks Business Travel
  • BT conferencing – £1.15m
  • Intercall video conferencing – £274,973
  • MWB (Serviced Offices) – £15.8m
  • Regus – offices and meeting rooms – £3.17m
  • Spring International Express (courier and other services) – £192, 662
  • Cision UK (press cuttings) – £30,000
  • Fishburn Hedges (includes public relations) – £559,310
  • Good Relations (public relations] – £1.55m
  • Porter Novelli (public relations and information) – £943,000
  • ASE Consulting – £31.7m
  • Capgemini – £15m
  • Deloitte MCS – £42.8m
  • Atos Consulting – £32.3m
  • Gartner – £3.8m
  • QI Consulting – £14.5m
  • Tribal Consulting – £6.9m

Comment

Central administrative costs of nearly £900m on a single IT programme are breathtaking. That makes the National Programme for IT in the NHS one of the world’s largest public sector IT projects – before a penny has been spent on deliveries of hardware or software to the NHS.

It’s almost as surprising that not even the National Audit Office has been able to obtain a breakdown. Has central spending been properly controlled? Perhaps not, given that the DH, even this year, spent up to £1,700 a day on consultants to brief a senior official for a hearing of the Public Accounts Committee in June 2013.

Maybe the taxpayer should be grateful that the consultants were hired for only 52 days between February and June 2013 to prepare for the Committee’s hearing, and that the DH managed to renegotiate the day rate down from £1,714 to £1,000 a day between April and June.

Maybe the taxpayer should be grateful that the total cost of the consultancy for preparing for the PAC hearing was only £73,563.

But the £73,563 was spent after the DH estimated its central administrative costs on the NPfIT at nearly £900m – which are costs up to 31 March 2012.

It’s also remarkable that some at the DH still consider the NPfIT a success. This was the NAO’s conclusions on the NPfIT in its May 2011 report on the NPfIT Care Records Service:

“Central to achieving the Programme’s aim of improving services and the quality of patient care, was the successful delivery of an electronic patient record for each NHS patient. Although some care records systems are in place, progress against plans has fallen far below expectations and the Department has not delivered care records systems across the NHS, or with anywhere near the completeness of functionality that will enable it to achieve the original aspirations of the Programme.

“The Department has also significantly reduced the scope of the Programme without a proportionate reduction in costs, and is in negotiations to reduce it further still. So we are seeing a steady reduction in value delivered not matched by a reduction in costs.

“On this basis we conclude that the £2.7 billion spent on care records systems so far does not represent value for money, and we do not find grounds for confidence that the remaining planned spend of £4.3 billion will be different.”

But this was the Department of Health’s view on NPfIT Care Records Service value for money:

“The Department considers, however, that the money spent to date has not been  wasted and will potentially deliver value for money… The Department believes that the flexibility provided by the future delivery model for the programme will deliver functionality that best fits the needs of the clinical and managerial community. The future architecture of the programme allows many sources of information to be connected together as opposed to assuming that all relevant information will be stored in a single system. This approach has been proven in other sectors and is fully consistent with the Government’s recently published ICT strategy.”

This contradiction between the DH’s view of the NPfIT, and the NAO’s, indicates, perhaps, that the DH continues to live in a world not entirely attached to reality.

From April 2013, the DH’s central team and some local programme teams responsible for the NPfIT moved to the Health and Social Care Information Centre which has taken over the local service provider contracts with BT and CSC. Will it be able to control central spending on the very-much-alive NPfIT?

Update:

The central costs could rise much further – possibly by more than £100m – if the eventual settlement of the legal case between the DH and Fujitsu works out badly for the taxpayer. Legal costs on the case so far are about £31m.

Trust spends £16.6m on consultants for Cerner EPR

By Tony Collins

Reading-based Royal Berkshire NHS Foundation Trust says in an FOI response that its spending on “computer consultants since the inception of the EPR system is £16.6m”.

The Trust’s total spend on the Cerner Millennium system was said to have been £30m by October 2012.

NHS IT suppliers have told me that the typical cost of a Trust-wide EPR [electronic patient record] system, including support for five years, is about £6m-£8m, which suggests that the Royal Berkshire has spent £22m more than necessary on new patient record IT.

Jonathan Isaby, Taxpayers’ Alliance political director, said: “This is an astonishing amount of taxpayers’ money to have squandered on a system which is evidently failing to deliver results.

“Every pound lost to this project is a pound less available for frontline medical care. Those who were responsible for the failure must be held to account for their actions as this kind of waste cannot go unchecked.”

 The £16.6m consultancy figure was uncovered this week through a Freedom of Information request made by The Reading Chronicle. It had asked for the spend on consultants working on the Cerner Millennium EPR [which went live later than expected in June 2012].

The Trust replied: “Further to your request for information the costs spent on computer consultants since the inception of the EPR system is £16.6m.”

The Chronicle says that the system is “meant to retrieve patient details in seconds, linking them to the availability of surgeons, beds or therapies, but has forced staff to spend up to 15 minutes navigating through multiple screens to book one routine appointment, leading to backlogs on wards and outpatient clinics”.

Royal Berkshire’s chief executive Edward Donald had said the Cerner Millennium go live was successful.  A trust board paper said:

 “The Chief Executive emphasised that, despite these challenges, the ‘go-live’ at the Trust had been more successful than in other Cerner Millennium sites.”

A similar, stronger message had appeared was in a separate board paper which was released under FOI.  Royal Berkshire’s EPR [electronic patient record] Executive Governance Committee minutes said:

“… the Committee noted that the Trust’s launch had been considered to be the best implementation of Cerner Millennium yet and that despite staff misgivings, the project was progressing well. This positive message should also be disseminated…”

Comment

Royal Berkshire went outside the NPfIT. But its costs are even higher than the breathtakingly high costs to the taxpayer of NPfIT Cerner and Lorenzo implementations.

As senior officials at the Department of Health have been so careless with public funds over NHS IT – and have spent millions on the same sets of consultants – they are in no position to admonish Royal Berkshire.

So who can criticise Royal Berkshire and should its chief executive be held accountable?

When it’s official policy to spend tens of millions on EPRs that may or may not make things better for hospitals and patients – and could make things much worse – how can accountability play any part in the purchase of the systems and consultants?

The enormously costly Cerner and Lorenzo EPR implementations go on – in an NHS IT world that is largely without credible supervision, control, accountability or regulation.

Cash squandered on IT help

Trust loses £18m on IT system

The best implementation of Cerner Millennium yet?

Firecontrol disaster and NPfIT – two of a kind?

By Tony Collins

Today’s report of the Public Account Committee on the Firecontrol project could, in many ways, be a report on the consequences of the failure of the National Programme for IT in the NHS in a few years time.

The Firecontrol project was built along similar lines to the NPfIT but on a smaller scale.

With Firecontrol, Whitehall officials wanted to persuade England’s semi-autonomous 46 local fire authorities to take a centrally-bought  IT system while simplifying and unifying their local working practices to adapt to the new technology.

NPfIT followed the same principle on a bigger scale: Whitehall officials wanted to persuade thousands of semi-autonomous NHS organisations to adopt centrally-bought technologies. But persuasion didn’t work, in either the fire services or the NHS.

More similarities

The Department for Communities and Local Government told
the PAC that the Firecontrol control was “over-specified” – that it was unnecessary to have back-up to an incident from a fire authority from the other side of the country.

Many in the NHS said that NPfIT was over-specified. The gold-plated trimmings, and elaborate attempts at standardisation,  made the patient record systems unnecessarily complicated and costly – and too difficult to deliver in practice.

As with the NPfIT, the Firecontrol system was delayed and local staff  had little or no confidence it would ever work, just as the NHS had little or no faith that NPfIT systems would ever work.

Both projects failed. Firecontrol wasted at least £482m. The Department of Communities and Local Government cancelled it in 2010. The Department of Health announced in 2011 that the NPfIT was being dismantled but the contracts with CSC and BT could not be cancelled and the programme is dragging on.

Now the NHS is buying its own local systems that may or may not be interoperable. [Particularly for the long-term sick, especially those who have to go to different specialist centres, it’s important that full and up-to-date medical records go wherever the patients are treated and don’t at the moment, which increases the risks of mistakes.]

Today’s Firecontrol report expresses concern about a new – local – approach to fire services IT. Will the local fire authorities now end up with a multitude of risky local systems, some of which don’t work properly, and are all incompatible, in other words don’t talk to each other?

This may be exactly the concern of a post-2015 government about NHS IT. With the NPfIT slowly dying NHS trusts are buying their own systems. The coalition wants them to interoperate, but will they?  

Could a post-2015 government introduce a new (and probably disastrous) national NHS IT project – son of NPfIT – and justify it by drawing attention to how very different it is to the original NPfIT eg that this time the programme has the buy-in of clinicians?

The warning signs are there, in the PAC’s report on Firecontrol. The report says there are delays on some local IT projects being implemented in fire authorities, and the systems may not be interoperable. The PAC has 

” serious concerns that there are insufficient skills across all fire authorities to ensure that 22 separate local projects can be procured and delivered efficiently in so far as they involve new IT systems”.

National to local – but one extreme to the other?

The PAC report continues

“There are risks to value for money from multiple local projects. Each of the 22 local projects is now procuring the services and systems they need separately.

“Local teams need to have the right skills to get good deals from suppliers and to monitor contracts effectively. We were sceptical that all the teams had the appropriate procurement and IT skills to secure good value for money.

“National support and coordination can help ensure systems are compatible and fire and rescue authorities learn from each other, but the Department has largely devolved these roles to the individual fire and rescue authorities.

“There is a risk that the Department has swung from an overly prescriptive national approach to one that provides insufficient national oversight and coordination and fails to meet national needs or achieve economies of scale. 

Comment

PAC reports are meant to be critical but perhaps the report on Firecontrol could have been a little more positive about the new local approach that has the overwhelming support of the individual fire and rescue authorities.  

Indeed the PAC quotes fire service officials as saying that the local approach is “producing more capability than was expected from the original FiReControl project”. And at a fraction of the cost of Firecontrol.

But the PAC’s Firecontrol Update Report expresses concern that

– projected savings from the local approach are now less than originally predicted

– seven of the 22 projects are running late and two of these projects have slipped by 12 months

– “We have repeatedly seen failures in project management and are concerned that the skills needed for IT procurement may not be present within the individual fire and rescue authorities, some of which have small management teams,” says the PAC.

On the other hand …

The shortfall in projected savings is small – £124m against £126m and all the local programmes are expected to be delivered by March 2015, only three months later than originally planned.

And, as the PAC says, the Department for Communities and Local Government has told MPs that a central peer review team is in place to help share good practice – mainly made up of members of fire and rescue authorities themselves.

In addition, part of the £82m of grant funding to local fire services has been used by some authorities to buy in procurement expertise.

Whether it is absolutely necessary – and worth the expense – for IT in fire services to link up is open to question, perhaps only necessary in a national emergency.

In the NHS it is absolutely necessary for the medical records of the chronically sick to link up – but that does not justify a son-of-NPfIT programme. Linking can be done cheaply by using existing records and having, say, regional servers pull together records from individual hospitals and other sites.

Perhaps the key lesson from the Firecontrol and the NPfIT projects is that large private companies can force their staff to use unified IT systems whereas Whitehall cannot force semi-autonomous public sector organisations to use whatever IT is bought centrally.

It’s right that the fire services are buying local IT and it’s right that the NHS is now too. If the will is there to do it cheaply, linking up the IT in the NHS can be done without huge central administrative edifices.

Lessons from FireControl (and NPfIT?) 

The National Audit Office identifies these main lessons from the failure of Firecontrol:

– Imposing a single national approach on locally accountable fire and rescue authorities that were reluctant to change how they operated

–  Launching the programme too quickly without applying basic project approval checks and balances

– Over optimism on the deliverability of the IT solution.

– Issues with project management including consultants who made up half of the management team and were not effectively managed

MP Margaret Hodge, chair of the Public Accounts Committee, today sums up the state of Firecontrol

“The original FiReControl project was one of the worst cases of project failure we have seen and wasted at least £482 million of taxpayers’ money.

“Three years after the project was cancelled, the DCLG still hasn’t decided what it is going to do with many of the specially designed, high-specification facilities and buildings which had been built. Four of the nine regional control centres are still empty and look likely to remain so.

“The Department has now provided fire and rescue authorities with an additional £82 million to implement a new approach based on 22 separate and locally-led projects.

“The new programme has already slipped by three months and projected savings are now less than originally predicted. Seven of the 22 projects are reportedly running late and two have been delayed by 12 months. We are therefore sceptical that projected savings, benefits and timescales will be achieved.

“Relying on multiple local projects risks value for money. We are not confident that local teams have the right IT and procurement skills to get good deals from suppliers and to monitor contracts effectively.

“There is a risk that the DCLG has swung from an overly prescriptive national approach to one that does not provide enough national oversight and coordination and fails to meet national needs or achieve economies of scale.

 “We want the Department to explain to us how individual fire and rescue authorities with varied degrees of local engagement and collaboration can provide the needed level of interoperability and resilience.

“Devolving decision-making and delivery to local bodies does not remove the duty on the Department to account for value for money. It needs to ensure that national objectives, such as the collaboration needed between fire authorities to deal with national disasters and challenges, are achieved.”

Why weren’t NPfIT projects cancelled?

 NPfIT contracts included commitments that the Department of Health and the NHS allegedly did not keep, which weakened their legal position; and some DH officials did not really want to cancel the NPfIT contracts (indeed senior officials at NHS England seem to be trying to keep NPfIT projects alive through the Health and Social Care Information Centre which is responsible for the local service provider contracts with BT and CSC).

PAC report on Firecontrol

What Firecontrol and the NPfIT have in common (2011)

Hospitals accuse Capita of failings

By Tony Collins

A nine-page letter written on behalf of eight health trusts is said to criticise Capita for “persistent minor failings” in managing payroll and other work formerly carried out by their human resources departments, says the Liverpool Post which has a copy of the letter.

The failings listed in the letter are said to include:

– overpaying staff, with trusts having problems recovering the monies paid out;

– breaching data protection by sending staff personal details to other employees;

– paying someone due to start work two months’ salary, despite their dropping out of the recruitment process;

 – delays in pre- employment checks, leading to highly valuable candidates withdrawing their application for a job;

– losing sensitive and confidential information

The Post says the letter threatens terminating the contract. “Health trusts stressed, unless they sort the problems out, they will not only deduct the cost incurred to them out of Capita’s payment but continued failure will result in them terminating its contract,” said the paper.

The letter was said to have been written by Debbie Fryer, director of human resources at Aintree UniversityHospitals, Fazakerley, on behalf of several trusts within the North Mersey Framework that have contracted out their payroll and human resources work to Capita.

It represents Fazakerley Hospital, Alder Hey Children’s Hospital, the mental health trust Mersey Care NHS Trust, Liverpool Community Health NHS Trust, Liverpool Women’s Hospital, Royal Liverpool and Broadgreen hospitals, Wirral’s specialist Clatterbridge Cancer Centre and the specialist brain hospital The Walton Centre.

In 2011 the Capita Group announced  that it had been appointed as preferred supplier by a NHS North Mersey collaboration to deliver HR, payroll and recruitment services for up to 12 NHS trusts in Mersey.

The seven-year contract was worth up to £27m, with an option to extend for a further three years.  The contract was  expected to involve the TUPE transfer of up to 150 employees to Capita and the set up of new shared service centre based in Liverpool.

Capita said at the time it was first time NHS trusts had come together in the way they did to collectively outsource their HR, payroll and recruitment functions. 

The Liverpool Post says the letter expresses concern that Capita displayed a “laissez faire” attitude to personal data which had the potential to be “extremely damaging” to the trusts’ reputations and employee morale.

Trusts were said to have had difficulties recovering sums overpaid to employees, particularly former employees. Examples of lost documentation were said to be “almost too numerous to mention”, with documents seemingly disappearing into a “black hole”.

Ms Fryer is said to have been alarmed at some of the content of a report on Capita by auditors Grant Thornton in May. The letter sought concrete proposals on how Capita was going to resolve the situation.

A spokesman for Capita told the Post: “Capita is under contract with 10 trusts in the north west of England as a part of a framework agreement to deliver transactional HR services, including payroll and recruitment.

“As a part of this contract, Capita has been consolidating each trust’s individual HR and recruitment processes moving these to one common process applicable to all trusts under the framework.

“The simpler, improved process will make HR services easier and quicker for staff to use, lightening the administrative burden so trusts can focus on patient care.

“In order to implement these valuable changes, Capita and the trusts are currently undergoing a period of transformation as individual, often paper-based, services move to this common process.

“During this period, some challenges have arisen for both the trusts and Capita. However, Capita is working closely with the trusts involved to overcome those issues identified in order to deliver an enhanced service for trusts and their staff.”

Liverpool Post article