Category Archives: IT projects

Hospital group wins $106m settlement in Cerner dispute

Posted on 19/02/2014 | 1 comment

By Tony Collins

 A US health organisation Trinity Medical Centre has won a $106m settlement in  a legal dispute with Cerner, which is one the main suppliers of patient record systems to NHS trusts in London and the south.

Under the NPfIT BT has installed Cerner at trusts that include the Royal Free, London, Barnet and Chase Farm Hospitals NHS Trust, Weston Area Health NHS Trust, Barts Health NHS Trust, North Bristol NHS Trust and more recently at Croydon Health Services NHS Trust.

The Wall St Journal says a clinical patient accounting program Trinity bought from Cerner in 2008 was defective and didn’t deliver the promised benefits, which Cerner disputed. Trinity sought about $240m in damages; Cerner estimated $4m.

The companies agreed to submit the dispute to arbitration which began in October 2013.

Cerner said it “strongly disagrees” with the award and believes the claim was based on unique circumstances. It called the award the only material judgment against Cerner in its 34-year history.

US lawyer Michael Dagley says his firm won a $106m settlement for North Dakota-based Trinity Medical Centre in an arbitration case against Cerner.

The firm says that Trinity alleged in 2012 that patient accounting software and other services purchased from Cerner were defective, producing thousands of billing errors.

“We think it’s tremendously significant because it represents the first major victory that we’re aware of by a health care provider against a software vendor,” Dagley said in a statement.

“Providers are under pressure to automate and vendors are under pressure to offer integrated products. Providers want one vendor for all their IT needs, so the vendors have this incentive to deliver software to the market as quickly as possible, and that can lead to products being introduced that are immature and defective, which in health care, can cause tremendous damage.”

Last year Cerner said it believed the chance of a material loss related to the matter was remote and it had 147 hospitals and 735 clinics using the patient accounting program.

Despite the settlement Cerner’s share price has held up well.

Trinity Medical Centre is a non-profit organisation with about 2,700 employees.

1 Comment

Posted in BT, BT Health, Campaign4Change, Cerner, e-health, health, health records, IT projects, legal disputes, NHS, npfit, operational issues

When “life and death” NHS IT goes down

Posted on 29/01/2014 | 1 comment

By Tony Collins

Almost unnoticed outside the NHS an email was circulated by health officials last weekend about a national “severity 1” incident involving the Electronic Prescription Service, running on BT’s data Spine .

“The EPS [electronic prescriptions service] database is currently experiencing severe degradation of performance. … BT engineers [are] currently investigating with the database application support team,” said the email.

A severity 1 or 2 incident, which involves a temporary loss of, or disruption to, the Spine or other national NHS system,  is not unusual, according to a succession of emails forwarded to Campaign4Change.

The Department of Health defines a severity 1 incident as a  failure that has the potential to:

— have a significant adverse impact on the provision of the service to a large number of users; or

— have a significant adverse impact on the delivery of patient care to a large number of patients; or

— cause significant financial loss and/or disruption to NHS Connecting for Health [now the Health and Social Care Information Centre], or the NHS; or

— result in any material loss or corruption of health data, or in the provision of incorrect data to an end user.

The Health and Social Care Information Centre, which manages BT’s Spine and other former NPfIT contracts, reports that the spine availability is 99.9% or 100%. But the HSCIC’s emails tell a story of service outage or disruption that is almost routine.

If the spine and other national services  are really available 99.9% of the time, is that good enough for the NHS, especially when ministers and officials are increasingly expecting clinicians and nurses to depend on electronic patient records and electronic prescriptions?  In short, are national NHS IT systems up to the job?

NHS staff access the spine tens of millions of times every month, often to trace patients before accessing their electronic records.  The spine is pivotal to the use of patient records held on Rio and Cerner Millennium systems in London. It is critical to the operation of Choose and Book, the Summary Care Record, Electronic Prescription Service pharmacy systems, GP2GP, iPM/Lorenzo, and the Personal Demographics Service.

According to a Department of Health letter sent to the Public Accounts Committee, payments to BT for the Spine totalled £1.08bn by March 2013.

BT says on its website that its 10-year NHS Spine contract involves developing systems and software to support more than 899,000 registered NHS users. The HSCIC says the Spine is used and supported 24 hours a day, 365 days a day.

“There is a huge amount of industrial-strength robustness, availability, disaster recovery, that you cannot get someplace else,” said a BT executive when he appeared before MPs in May 2011.

Life and death  

Sir David Nicholson spoke of the importance of the spine and other national NHS systems at a hearing of the Public Accounts Committee in 2011. He said they were

“providing services that literally mean life and death to patients today … So the Spine, and all those things, provides really, really important services for our patients…”

When Croydon Health Services NHS Trust went live with a Cerner Millennium patient records system at the end of September a “significant network downtime” – of BT’s N3 network – had an effect on patients.

A trust board paper, dated 25 November 2013 says:

“CRS Millennium (Cerner) Deployment -Network downtime – Week 1.  In particular, the significant network downtime in week 1 (BT N3 problem) led to no electronic access to Pathology and Radiology which resulted in longer waits for patients in the Emergency Department (ED) leading to a large number of breaches. This was a BT N3 problem which has been rectified with BT …”

Below are some of the emails passed to Campaign4Change in the past four months. Written by the Health and Social Care Information, the emails alert NHS users to outages or disruption to GP or national NHS IT systems.

Some HSCIC messages of disruption to service

October 2013

Severity 2
HSCIC
National
CQRS has not received a number of participation status messages.
Also affecting: GPES
USER IMPACT:
CQRS Users are not able to manually submit specific information, this will impact the users’ business process for entry of achievement data.
ACTION BEING TAKEN:
Following a configuration change by the GPES Business Unit a specific code has now been added to the GPET-Q Database. We are currently awaiting confirmation that the addition of the relevant code has been successful. Discussions are taking place regarding the re-submission of status messages. HSCIC conference calls are on-going.

[A severity 2 service failure is a failure [that] has the potential to:

– have a significant adverse impact on the provision of the service to a small or moderate number of service users; or

– have a moderate adverse impact on the delivery of patient care to a significant number of service users; or

– have a significant adverse impact on the delivery of patient care to a small or moderate number of patients; or

– have a moderate adverse impact on the delivery of patient care to a high number of patients; or

– cause a financial loss and/or disruption … which is more than trivial but less severe than the significant financial loss described in the definition of a Severity 1 service failure.]

**

Severity 2
HSCIC
BT Spine
National
Intermittent performance issues on TSPINE.
T-Spine
RESOLUTION:
BT Spine have confirmed that the incident has been resolved and users are able to perform routine business processes without delays.

November 2013

Severity 1
BT Spine
HSCIC
National
Users are unable to log into PDS.
USER IMPACT:
All sites are currently unable to access PDS, this is causing a delay to normal services.
ACTION BEING TAKEN:
BT Spine are working to restore service.

**

Severity 2
BT Spine
HSCIC
National EPS users.
Slow performance on reliable and unreliable messages for EPS.
USER IMPACT:
This is causing delays to routine business processes as some users may be experiencing slow performance with the EPS service.
BT investigating.

**

Severity 2
BT Spine
HSCIC
National
Slow performance on EPS Messaging.
USER IMPACT:
This is causing delays to routine business processes as some users may be experiencing slow performance with the EPS service.
ACTION BEING TAKEN:
BT moved the database to an alternate node following application server restarts. This temporarily restored normal message response times however performance has started to degrade again. BT Investigation continues.

**
Severity 1
Atos
HSCIC
National
Multiple users were unable to log in to the Choose & Book application.
ATOS made some network configuration changes overnight 19th/20th November which restored service. After a period of monitoring throughout the day yesterday the service has remained stable and at expected levels. Further activities and investigation will be carried out by several resolver teams which will be scheduled through change management.

**
Severity 2
BT Spine
HSCIC
National
Slow performance on EPS Messaging.
No further issues of slow response times with EPS messaging have occurred today. BT Spine to continue root cause investigation.

**

Severity 2
Cegedim RX
HSCIC
National
Cegedim RX – Users are experiencing slow performance in EPS 1 and EPS 2.
USER IMPACT:
Users are experiencing slow performance and delays to routine business processes when using EPS 1 and EPS 2.
ACTION BEING TAKEN:
Following a restart of application services, traffic has improved for all new EPS messages. However there is a backlog of EPS messages which may cause delays to routine business processes. Cegedim RX to continue to investigate.

**

December 2012

Severity 1
BT Spine
HSCIC
National
Performance issues have been detected with the transaction messaging system (TMS).
Also affecting: Choose and Book, GP2GP
USER IMPACT:
This may cause delays to routine business processes. This may have an effect on all Spine related systems. This includes PDS, Choose and Book, PSIS, SCR, ACF Services.
ACTION BEING TAKEN:
This has been resolved but BT are currently monitoring performance. Further investigation is required by BT into the root cause.

**

Severity 2
GDIT – CQRS
HSCIC
National
DTS has not processed a CQRS payment file.
CQRS
Also affecting: GPES
USER IMPACT:
This is causing delays to routine business processes.
ACTION BEING TAKEN:
GDIT are currently developing a fix which will be rolled out tomorrow evening, pending successful testing.

January 2014

Severity 1
BT Spine
HSCIC
National
TMS reliable messaging unavailable.
USER IMPACT:
TMS reliable messaging unavailable and users having to implement manual workarounds.
ACTION BEING TAKEN:
Issues experienced due to a planned change overrunning, BT Spine continue to implement the transition activity in order to restore service.

**

Severity 2
BT Spine
HSCIC
National
Users have experienced intermittent issues with the creation and cancellation of smartcards in CMS [Card Management Service for managing smartcards].
CMS
USER IMPACT:
This is intermittently causing delays to routine business processes as some users have been unable to create, cancel, cut or print cards in CMS.
ACTION BEING TAKEN
Users may experience issues with the creation and cancellation of cards in CMS. BT have identified a fix for the issue which is currently undergoing testing prior to deployment into the live environment.

**

Severity 2
BT Spine
HSCIC
National
The maternity browser was unavailable within NN4B.
RESOLUTION:
BT identified a problematic server which was recycled to restore system functionality.

**

Spine scheduled outage for essential maintenance activity.

During critical work to migrate to a new storage solution on Spine an issue was experienced on the Transaction Messaging Service (TMS) in September of this year. The issue resulted in BT failing over the TMS database from its usual site on Live B to Live A to restore service. The failover was completed well within the Service Level Agreement and no detrimental long term impacts to the service were incurred.

On the 15th January 2014 between approximately 22:00-23:30, HSCIC, in conjunction with BT, are planning to relocate the TMS database back to Live B, this is for several critical reasons:

  1. The issues experienced, which prompted the failover, are fully resolved and will not be experienced again as the storage migration work is now complete.
  2. The Spine service is designed to operate with all databases running on Live B so this work supports the optimum configuration for the service.
  3. Most critically the transition for all data on Spine to Spine2 has been designed to operate from a standby site with no live databases on it. Therefore to support the Spine2 transition this work is absolutely essential.

In order to facilitate a safe relocation of the database a 1.5 hour outage is required to TMS. The impact of this to Spine is significant and results in effectively an outage for Spine and its interfaces to connecting systems for that period. The time and date is aimed at the lowest times of utilisation for Spine, to minimise impact to end users, as well as not impacting critical batch processing and Choose & Book slot polls.

 

Date & Time

Change Start Change Finish Services Affected Outage Duration
15/01/2013 22:00 15/01/2013 23:30 Transaction Messaging Service (TMS) 1.5 hours
Service  Impact Description
Choose and Book The Choose and Book service will be available but functionality will be limited until the TMS database has switched over.Users of the web application will experience limited retrievals during the outage window.The system will not be able to create shared-secret for patients who have not been referred via Choose and Book before.Service Providers will be unable to:

  • Perform clinic re-structures and re-arrange appointments for patients for directly bookable services
  • Send DNA messages to Choose and Book.

For directly bookable services the following functionality will be unavailable:

  • Booking appointments
  • Rearranging appointments
  • Creating new patient accounts

Choose & Book systems will need to queue the messages and resend to Spine once the TMS service is enabled.

Due to the timing of the outage slot polls will not be affected.
Summary Care Record application (SCRa) The SCRa application will be available but functionality will be limited until the TMS database has switched over. Simple traces can be completed on PDS data but users will be unable to perform any PSIS updates (e.g. GP summary updates)
DSA The DSA application will be available but functionality will be limited until the TMS database has switched over.Simple traces can be completed on PDS data but users will be unable to perform any PSIS updates (e.g. GP summary updates).
Electronic Prescription Service (EPS)Pharmacy Systems Reliable messaging will be unavailable for the duration of the switchover work as the TMS service will be suspended dual site. All messages received from EPS systems will be rejected and not go into retry.EPS systems will need to queue the messages and resend to Spine once the TMS service is enabled.
EPS Batch The PPA response for any “claim” messages will not be sent to PPA/PPD. However, EPS will send those response(s) again when the retry jobs are re-activated after the switchover exercise is over. Response for any “claim” messages will not be received until after the switchover. Retry jobs will resend the responses once the TMS service is enabled.
Existing Service Providers (ESPs) There will be varying impacts depending on the product, release version and Spine compliant modules of the solution.ESP systems will need to queue the messages and resend to Spine once the TMS service is enabled.
GP2GP GP2GP will be unavailable until the TMS database has switched over.GP2GP systems will need to queue the messages and resend to Spine once the TMS service is enabled.
GP Extraction Service (GPES) GPES functionality will be unavailable until the TMS database has switched over.Messages will be queued on Spine and processed once the TMS service is restored.
GP Systems Functionality for Choose & Book, EPS and GP2GP, SCR will be limited until the TMS database has switched over.For Choose & Book directly bookable services the following functionality will be unavailable:

  • Booking appointments
  • Rearranging appointments
  • Creating new patient accounts

Systems will need to queue the messages and resend to Spine once the TMS service is enabled.
iPM/Lorenzo The real-time connection to Spine will be unavailable during the TMS outage. However both systems can be disconnected from Spine and operate without synchronised PDS data.iPM/Lorenzo will need to queue the messages and resend to Spine once the TMS service is enabled.
Millennium An outage of PDS reliable messaging will impact Millennium users.Users will be unable to:

  • trace patients
  • register new patients on PDS
  • book or reschedule appointments

Millennium will need to queue the messages and resend to Spine once the TMS service is enabled.
NN4B Trusts will need to be aware that during the outage NHS numbers cannot be generated, new-births cannot be registered and blood-spot labels cannot be generated and should plan accordingly.All birth notifications will be queued and processed once the TMS service is enabled.
Personal Demographics Service (PDS) Simple traces can be completed on PDS data.PDS reliable messaging will be unavailable until the TMS database has switched over.
RiO Users will be unable to:

  • trace patients
  • register new patients
  • book or reschedule appointments

The RiO system will need to queue the messages and resend to Spine once the TMS service is enabled.
TMS Event Service (TES) The majority of TES functionality will be unavailable during the outage.Trusts will need to be aware EPS, Death notifications, and Patient Care Provision Notifications (change of pharmacy) will be queued and sent to the receiving systems once the TMS service is restored.Any impacted notifications will be queued and sent to the receiving systems once TMS is restored.
TMS Batch (DBS, CHRIS, ONS) DBS will be unavailable until the TMS database has switched over (DBS processing will be suspended for the duration of the exercise).As the TMS switchover will be scheduled to start at 22:00, CHRIS batch should complete before the outage starts (CHRIS batch runs at 20:00 nightly).ONS processing will start at 18:00 nightly. If it doesn’t complete before 22:00, the messages will be queued and processed once the TMS service is restored.

**

Severity 2
BT Spine
HSCIC
National
Users are unable to grant worklist items within UIM.
USER IMPACT:
This is causing delays to routine business processes as users are unable to complete their worklist items within the UIM application.
ACTION BEING TAKEN:
BT investigating.

**

Severity 1
BT Spine
HSCIC
National
The EPS database is currently experiencing severe degradation of performance.
USER IMPACT:
Delays to routine business processes.
ACTION BEING TAKEN:
BT engineers currently investigating with the database application support team.

Comment

David Nicholson is right. The NHS has become dependent on systems such as the Spine. But can doctors ever trust any aspect of the safety of patients to systems that are not available 24×7 as they need to be in a national health service?

It appears that BT and other suppliers have not been in breach of service level agreements, and the HSCIC has a good relationship with the companies.  But does the HSCIC have too great an interest in not finding fault with its suppliers or the contracts, for finding fault  could draw attention to any defects in a service for which the HSCIC is responsible?

Have national NHS IT suppliers a strong enough commercial or reputational interest  in avoiding  a disruption or loss of service, so long as they keep within their service level agreements? 

If nobody sees anything wrong with the reliability of existing national NHS IT services improvements are unlikely. Diane Vaughan’s book on the culture and organisation of NASA shows that experts in a big organisation can do everything right according to the rules  and procedures – and still have a disastrous outcome.

1 Comment

Posted in Atos, BT, Campaign4Change, culture, Department of Health, e-health, Government IT, IT projects, LPfIT, NHS, npfit, Summary Care Record, supplier relationships

Tagged

BBC’s DMI project – another fine mess that was predictable

Posted on 28/01/2014 | 3 comments

By Tony Collins

A National Audit Office memorandum published today on the BBC’s failed £125.9m Digital Media Initiative is a reminder – as in most failed big IT-enabled projects – that the causes have nothing to do with software and everything to do with management and people.

The NAO’s memorandum tells an all too familiar story with government IT (and the BBC is a public sector organisation):

– Over-optimism about the ability to implement

– Over-optimism about the ability to achieve the benefits

– Unclear requirements

– No thorough independent assessment of the technical design to see whether the DMI was technically sound

– The successful completion of the most straightforward of technology releases for the DMI, but these proved an unreliable indicator of progress.

– Technical problems and releases not meeting user expectations which contributed to repeated extensions to the timetable for completing the system, eroding user confidence and undermining the business case.

– Poor internal reporting. “The governance arrangements for the DMI were inadequate for its scale, complexity and risk. The BBC did not appoint a senior responsible owner to act as a single point of accountability and align all elements of the DMI. Reporting arrangements were not fit for purpose,” said the NAO.

– In the same way as the DWP failed with Universal Credit to take full account of recommendations in review reports, the BBC “did not adequately address issues identified by external reviewers during the course of the programme”.  The BBC had been aware that business requirements for the DMI were not adequately defined.

The BBC estimates that it spent £125.9m on the DMI. It offset £27.5m of spending on the DMI against transfers of assets, cash and service credits that formed part of its financial settlement with DMI’s previous developer Siemens. This left a net cost of £98.4m.

The BBC cancelled the DMI without examining the technical feasibility or cost of completing it, said the NAO.

The Corporation has written off the value of assets created by the programme, but is exploring how it can develop or redeploy parts of the system to support its future archiving and production needs.

Diane Coyle, Vice Chairman BBC Trust, said:

“We are grateful to the NAO for carrying out this report, which reinforces the conclusions of the PwC review commissioned by the Trust. It is essential that the BBC learns from the losses incurred in the DMI project and applies the lessons to running technology projects in future.

“The NAO’s findings, alongside PwC’s recommendations will help us make sure this happens. As we announced last December, we are working with the Executive to strengthen project management and reporting arrangements within a clearer governance system.  This will ensure that serious problems can be spotted and addressed at an earlier stage.”

Amyas Morse, head of the National Audit Office, said today:

“The BBC Executive did not have sufficient grip on its Digital Media Initiative programme. Nor did it commission a thorough independent assessment of the whole system to see whether it was technically sound.

“If the BBC had better governance and reporting for the programme, it would have recognized the difficulties much earlier than May 2012.”

Comment

The DMI project is exemplar of all that tends to go wrong in big government IT-enabled projects. Strong independent oversight and independent reviews that were published would have provided the accountability to counterbalance over-optimism.  But these things never seems to happen.

There are also questions about why the BBC took on the project from Siemens  and turned what could have been a success into a financial disaster.

NAO memorandum on the BBC’s Digital Media Initiative

3 Comments

Posted in change management, DWP, Government IT, IT projects, IT-related failures, managing change, procurement, project management, public sector, PWC

Tagged

BBC World at One’s focus on Government IT

Posted on 28/01/2014 | 5 comments

By Tony Collins

The lead item on BBC R4’s World at One on Friday was about Government IT contracts.

On the programme were the government’s Chief Procurement Officer Bill Crothers, Cabinet Office minister Francis Maude, the chairman of the Public Accounts Committee Margaret Hodge, the UK IT Association, and me.

Some of the points made:

–  Bill Crothers gave an example of what he called “abuse” by some big IT suppliers. He said a young man who works for him lost his power cable. The supplier quoted £65 for a replacement. The price should have been £5 or £6.  When Crothers queried it, the supplier justified its price on grounds of security. Crothers could not believe that a power lead had security implications so he questioned the price again and received several pages of explanation from the supplier, which he did not read. Eventually the supplier “was good enough to reduce the price to £37”.

– HMRC was charged £30,000 for changing some text on its website.

– Francis Maude said a DWP team and a further 12 people from the Cabinet Office’s Government Digital Service had built – in only three months – a prototype of a digital solution to support the introduction of Universal Credit. The system cost just over £1m, he said. [Separately big IT suppliers at DWP have been paid £303m up to March 2013 for Universal Credit work.] Maude declined to predict the outcome of the “twin-track” work on the UC project.

– Some big legacy systems may soon need replacing – those that pay about £60bn a year in state pensions and collect nearly £100bn a year in VAT. “Those are going to be big projects,” said Margaret Hodge. “I don’t think we have seen the end of big projects, or the end of disasters.”

World at One in detail

Presenter Shaun Ley and BBC political correspondent Ross Hawkins focused on government IT because of an announcement by the Cabinet Office that it is drawing the line on “bloated and wasteful IT contracts”. The Cabinet Office was pitching its announcement as marking a “massive change,” said Hawkins.

Ley said Francis Maude announced the safeguards  in an attempt to ensure that IT contracts don’t become multi-billion pound failures. He said that the abandoned NPfIT had cost close to £10bn.

Hawkins quoted the UK IT Association as saying that  government did not know how to do deals with smaller suppliers. On the government’s relationship with big suppliers UKITA said the government was like a “battered wife or husband who doesn’t seem to know how to leave.”

Appalling

Hawkins said Crothers has the air of a man going to war. Crothers’ conclusion on the way things are at the moment:

“This is about the oligopoly, the cluster of big suppliers that have had it took good for too long. It’s reflective of monopolistic or oligopolistic behaviour.  It is not acting as if they are in hungry and in a competitive market.  That’s appalling.”

Universal Credit

Hawkins asked Francis Maude how confident he was that what was being put in place on Universal Credit would work.

“I hope it will work,” said Maude. “The digital solution was created by a team within DWP with a dozen or so GDS [Government Digital Service] staff assisting.

“They created a working prototype for a digital solution within 3 months at a cost of only a bit over £1m. That certainly can be basis of a successful long-term solution.”

Hawkins [to Maude] “I asked you whether you were confident the approach with DWP would work and you said you hoped it would. That suggests to me that maybe you are not (confident).”

Maude: “N0-one knows with these things. Anyone who says you are certain everything is going to succeed … the way we do things now is build something quickly, test it, prove it, test it with users, and so you can’t have certainty about any of these outcomes.”

Outsourcing failures

Hawkins said “We have had story after embarrassing story about outsourcing failures [such as the] government being charged for tagging dead people … now ministers  have an interest in coming out on the front foot and just for once being on the attack and having a whack at the IT companies.

“You don’t need to be a political genius to work out why they would like to do that rather than be endlessly explaining themselves after embarrassing stories in the papers.”

Ley (to me): “Is this the best way to deal with the problems government has experienced? The journalist Tony Collins has written widely  about project failures in IT in both the public and private sectors.”

I replied that big companies have sometimes charged a lot to make small software changes.  The Cabinet Office’s “red lines” were a good idea though they were a formalising of restrictions that had been in place some time.

The Cabinet Office doesn’t have the power to make changes happen because departments are accountable to Parliament for their spend and so don’t want much interference from the Cabinet Office. But the Cabinet Office is right to try and reduce the amounts spent on big projects.

Ley: “What will be the effect of breaking up contracts?”

I said I hoped the Cabinet Office’s restrictions would bring about a change in culture in departments against the assumption that big is beautiful. Big projects should be split into components which would give SMEs a greater involvement and could reduce the risks of projects failing.

More project disasters?

Hodge gave her reaction to the Cabinet Office’s restrictions in the context of the Universal Credit project.

“Francis Maude and Cabinet Office have been trying really hard to get some sense into the way that project has developed. But sadly the news we have had lately suggests to me that they have failed. It is about £400m so far on IT.

“What went wrong there was that the department [DWP] thought it [UC] was a big IT project instead of thinking:  we are going to be changing our business; we are going to get 6 benefits rolled into one. They [the DWP] have not written off that money [£303m] which is what my committee thinks they should have done, because they want to save face. Down the line I think we’ll see some disasters there.

“There are a lot of projects around  government, what are called legacy projects, where old systems need to be replaced . They are big projects – pensions in DWP where £60bn is given out a year;  VAT receipts  in HMRC where nearly £100bn is collected. Those are going to be big projects. I don’t think we have seen the end of big projects, or the end of disasters.”

Ley: “What about breaking them up into smaller projects? Won’t that reduce potential risks?”

Hodge: “The important thing is what Tony Collins was saying to you. What we find is that the skills don’t exist within departments, either to commission the IT properly or to manage the suppliers once they have the IT in place.

“We are about to examine the army recruitment contract – I think that is what we’ll find.  The MoD hasn’t got the skills to manage it.

Ley: “Do you welcome the ending of automatic contract extensions?”

“I warmly welcome that. This is a small step in the right direction. Having an expert as we have in Bill Crothers in the Cabinet Office is really important. What we haven’t got are skills in the departments. It is not like a business. If it was, Bill Crothers would probably run IT across the whole of government. Our departments run in silos. They haven’t got the skills. They have this demand for big, big programmes in the future and I don’t think we have seen, sadly, the end of IT disasters.”

Update

Thank you to Dave Orr for drawing my attention to an excellent piece on the World at One item by procurement expert Peter Smith who concludes:

“… There is a big issue – large suppliers have not covered themselves in glory, but small suppliers just can’t develop huge systems for DWP or MOD.

“The large suppliers must have a role, but we have to manage these contracts better. And the answer can’t just be a small hit squad in Cabinet Office. This needs real capability development across government, which we haven’t really seen as yet in a coordinated fashion.”

BBC World at One – Government IT contracts

Bill Crothers on BBC Radio 4 – suppliers get another good kicking

5 Comments

Posted in DWP, Government digital Service, Government IT, HMRC, IT projects, IT-related failures, managing outsourced services, MoD, npfit, outsourcing, procurement, project management, public sector, public services, reducing cost strategically, supplier relationships, Universal Credit

Tagged , ,

Are Govt IT-based project disasters over? Ask the Army

Posted on 16/01/2014 | 3 comments

By Tony Collins

When senior civil servants know an IT-based project is in trouble and they’re unsure how bad things are, they sometimes offer their minister an all-encompassing euphemism to publicly describe the status of the scheme – teething.

Which may be why the defence secretary Philip Hammond told the House of Commons in November 2013 that the IT project to support army recruiting was having “teething” problems.

Now Hammond knows more, he says the problems are “big”. He no longer uses the “t” word. Speaking about the £440m 10-year Recruitment Partnering Project in the House of Commons this week Hammond said:

“Yes, there are big problems with the IT and I have told the House on repeated occasions that we have IT challenges…”

Only a few days ago Cabinet Office minister Francis Maude suggested that Government IT was no longer a byword for disaster, though he accepted there were still challenges.

In a speech on how he expected the UK to become the G8’s most digital government by next year (whatever that means) Maude said: “… it’s great news that DVLA is about to launch online driving records which can be used by anyone with a driving licence as well as by the insurance industry.

“Back in 2010 our digital offering was limited at best and government IT was a by-word for disaster … There are still challenges but with the help of the Government Digital Service I am determined that the UK will be the G8’s most digital government by next year.”

A few days later The Times reported on a leaked Gartner report on the army Recruitment Partnering Project. The report expressed concerns about the entire plan, including a poor project management team and delays that were allowed to spiral out of control.

It claimed that the Army’s recruitment division had failed to challenge MoD policy in 2011 that had apparently favoured the less suitable of the two competing bidders chasing the contract.

Hammond is said to be mulling over a £50m payout for Capita to build a new infrastructure for the recruiting system instead of trying to integrate it with systems supplied by the “Atlas” consortium under the Defence Information Infrastructure project. Hammond told the House of Commons this week:

“… there have been initial difficulties with that recruiting process as we transition to the new recruiting arrangements with Capita.

“In particular, we have encountered difficulties with the IT systems supporting the application and enlistment process. The decision to use the legacy Atlas IT platform was deemed at the time to be the quickest and most cost-effective way of delivering the new recruitment programme.

“An option to revert to a Capita hosting solution was included in the contracts as a back-up solution.

“I was made aware in the summer of last year that the Army was encountering problems with the integration of the Capita system into the Atlas platform. Since then we have put in place a number of workarounds and mitigation measures for the old IT platform to simplify the application process, and we have reintroduced military personnel to provide manual intervention to support the process.

“Having visited the Army’s recruitment centre in Upavon [Wiltshire] on 30 October, it became clear to me that, despite the Army putting in place measures to mitigate those problems in the near term, further long-term action was needed to fix the situation.

“It was agreed in principle at that point that the Atlas system was not capable of timely delivery of the Capita-run programme and that we would need to take up the option of reverting to Capita building a new IT platform specifically to run its system, which will be ready early next year.

“… we have already taken action to bring in a range of new initiatives that will make it progressively easier and quicker for applicants … the introduction this month of a new front-end web application for Army recruitment; a simplified online application form; more streamlined medical clearance processes …

“With an improved Army recruitment website, streamlined medicals and an increase in the number of recruiting staff, recruits should see a much-improved experience by the end of this month.

“.. we are looking at further ways of improving the management of the recruiting process in the intervening period before the introduction of the advanced IT system now being developed in partnership with Capita, which is expected to be deployed in February 2015…”

Vernon Croaker, Labour’s defence spokesman, said the recruitment project was an IT fiasco. He wondered why Hammond had initially described the problems as teething.

“Today we have learned [from newspapers] that the problems are even worse than anyone thought and still have not been fixed.

“Will the Defence Secretary tell the House which Minister signed off the deal and who has been responsible for monitoring it?

“… Will the Secretary of State also confirm that £15.5m has been spent building the existing flawed computer system behind the project? Finally, is it correct that this continuing disaster is costing taxpayers £1 million every month?…”

Croaker quoted a minister Andrew Robathan as telling MPs on 10 April 2013 that the “Recruiting Partnering Project with Capita…will lead to a significant increase in recruiting performance”.

Croaker said: “Is there any Member of this House, any member of our armed forces or, indeed, any member of the British public who still believes that?”

In March 2012 Capita announced that the Recruitment Partnering Project was valued at about £44m a year for 10 years and was expected to deliver benefits in excess of £300m to the armed forces. It would “release military recruiters back to the front line” said Capita.

Comment. Francis Maude is probably right: there don’t seem to be as many big IT-based project failures as in previous decades. But then the truth isn’t known because progress reports on big IT-related schemes are not published.

Indeed little would be known about the Capita Recruitment Partnering Project is not for the leaked report to The Times. Without the leak, public information on the state of the project would be confined to Hammond’s “teething problems” comment to MPs last November.

Internal and external reports on the state of the Universal Credit IT project continue to be kept secret.  It’s not even clear whether ministers are properly briefed on their big IT projects. Hammond almost certainly wasn’t last year. IDS was left to commission his own “red team” review of Universal Credit IT.

Perhaps the “good news” reporting culture in Whitehall explains why the NHS IT scheme, the NPfIT, continued to die painfully slowly for 7 years before senior officials and ministers started to question whether all was well.

Hammond is still getting wrong information. He described “Atlas” systems in the House of Commons as the “legacy IT platform”.

The Atlas contract for the Defence Information Infrastructure was awarded in 2005 for 10 years. It doesn’t even expire until next year. It may be convenient for officials to suggest that the reason Capita has been unable to link new recruitment systems into the DII network is because DII is old – legacy IT.  But the multi-billion pound Atlas DII project cannot be accurately described as “legacy” yet.

If ministers don’t get the truth about their big IT projects until serous problems are so obvious they can no longer be denied, how can Parliament and taxpayers expect to get the truth?

Lessons from NASA?

NASA put in place processes, procedures and rules to ensure engineers were open and deliberately adversarial in challenging assumptions. Even so it has had difficulties getting engineers to express  their views freely.

Diane Vaughan in her excellent book “The Challenger Launch Decision” referred to large organisations that proceeded as if nothing was wrong “in the face of evidence that something was wrong”.  She said NASA made a series of seemingly harmless decisions that “incrementally moved the space agency towards a catastrophic outcome”.

After the loss of Challenger NASA made many changes. But an investigation into the subsequent tragedy of the Columbia space shuttle indicated that little had actually changed – even though few of the top people who had been exposed to the lessons of Challenger were still in position.

If NASA couldn’t change when lives depended on it, is it likely the UK civil service will ever change?  A political heavyweight,  Francis Maude has tried and failed to get departments to be more open about progress or otherwise on their big IT-based projects.  Permanent secretaries now allow the out-of-date “traffic light” status of some projects to be published in the annual report of the Major Projects Authority. That is not openness.

The failure so far of the Recruitment Partnering Project, the routine suppression of information on technology-based scheme such as this, and the circumscribed “good news” briefings to ministers, suggest that government IT-based project failures are here to stay, despite the best intentions of the Cabinet Office, GDS and the Major Projects Authority.

Thank you to campaigner Dave Orr for his email on the recruitment project

3 Comments

Posted in Accenture, Agile, BT, Campaign4Change, Capita, change management, Defence, DWP, excessive secrecy, GDS, Government digital Service, Government IT, HP, IBM, IT projects, IT-related failures, Major Projects Authority, MoD, npfit, openness and transparency, public sector, public services, Universal Credit

Tagged , ,

Top 5 posts on this site in last 12 months

Posted on 02/01/2014 | 1 comment

Below are the top 5 most viewed posts of 2013.  Of other posts the most viewed includes “What exactly is HMRC paying Capgemini billions for?” and “Somerset County Council settles IBM dispute – who wins?“.

1) Big IT suppliers and their Whitehall “hostages”

Mark Thompson is a senior lecturer in information systems at Cambridge Judge Business School, ICT futures advisor to the Cabinet Office and strategy director at consultancy Methods.

Last month he said in a Guardian comment that central government departments are “increasingly being held hostage by a handful of huge, often overseas, suppliers of customised all-or-nothing IT systems”.

Some senior officials are happy to be held captive.

“Unfortunately, hostage and hostage taker have become closely aligned in Stockholm-syndrome fashion.

“Many people in the public sector now design, procure, manage and evaluate these IT systems and ignore the exploitative nature of the relationship,” said Thompson.

The Stockholm syndrome is a psychological phenomenon in which hostages bond with their captors, sometimes to the point of defending them.

This month the Foreign and Commonwealth Office issued  a pre-tender notice for Oracle ERP systems. Worth between £250m and £750m, the framework will be open to all central government departments, arms length bodies and agencies and will replace the current “Prism” contract with Capgemini.

It’s an old-style centralised framework that, says Chris Chant, former Executive Director at the Cabinet Office who was its head of G-Cloud, will have Oracle popping champagne corks.

2) Natwest/RBS – what went wrong?

Outsourcing to India and losing IBM mainframe skills in the process? The failure of CA-7 batch scheduling software which had a knock-on effect on multiple feeder systems?

As RBS continues to try and clear the backlog from last week’s crash during a software upgrade, many in the IT industry are asking how it could have happened.

3) Another Universal Credit leader stands down

Universal Credit’s Programme Director, Hilary Reynolds, has stood down after only four months in post. The Department for Work and Pensions says she has been replaced by the interim head of Universal Credit David Pitchford.

Last month the DWP said Pitchford was temporarily leading Universal Credit following the death of Philip Langsdale at Christmas. In November 2012 the DWP confirmed that the then Programme Director for UC, Malcolm Whitehouse, was stepping down – to be replaced by Hilary Reynolds. Steve Dover,  the DWP’s Corporate Director, Universal Credit Programme Business, has also been replaced.

4) The “best implementation of Cerner Millennium yet”?

Edward Donald, the chief executive of Reading-based Royal Berkshire NHS Foundation Trust, is reported in the trust’s latest published board papers as saying that a Cerner go-live has been relatively successful.

“The Chief Executive emphasised that, despite these challenges, the ‘go-live’ at the Trust had been more successful than in other Cerner Millennium sites.”

A similar, stronger message appeared was in a separate board paper which was released under FOI.  Royal Berkshire’s EPR [electronic patient record] Executive Governance Committee minutes said:

“… the Committee noted that the Trust’s launch had been considered to be the best implementation of Cerner Millennium yet and that despite staff misgivings, the project was progressing well. This positive message should also be disseminated…”

Royal Berkshire went live in June 2012 with an implementation of Cerner outside the NPfIT.  In mid-2009, the trust signed with University of Pittsburgh Medical Centre to deliver Millennium.

Not everything has gone well – which raises questions, if this was the best Cerner implementation yet,  of what others were like.

5) Universal Credit – the ace up Duncan Smith’s sleeve?

Some people, including those in the know, suspect  Universal Credit will be a failed IT-based project, among them Francis Maude. As Cabinet Office minister Maude is ultimately responsible for the Major Projects Authority which has the job, among other things, of averting major project failures.

But Iain Duncan Smith, the DWP secretary of state, has an ace up his sleeve: the initial go-live of Universal Credit is so limited in scope that claims could be managed by hand, at least in part.

The DWP’s FAQs suggest that Universal Credit will handle, in its first phase due to start in October 2013, only new claims  – and only those from the unemployed.  Under such a light load the system is unlikely to fail, as any particularly complicated claims could managed clerically.

 

1 Comment

Posted in Accenture, Agile, BT, Campaign4Change, Capgemini, Cerner, Cloud, Cloud Computing, CSC, DWP, excessive secrecy, FOI, Freedom of Information, G-Cloud, Government IT, health records, HP, IBM, IT projects, IT-related failures, legal disputes, local democracy, local government, LPfIT, Major Projects Authority, NHS, npfit, outsourcing, public-private partnerships, Real-Time Information, SAP, Universal Credit

Tagged , ,

Will truth ever be told when things go wrong?

Posted on 28/11/2013 | 5 comments

By Tony Collins

Cabinet Office minister Francis Maude has criticised civil servants who don’t always tell ministers what is going on in their departments. He used the Universal Credit project as an example.

He told the Financial Times: “There were a lot of failures in DWP and it isn’t good that it took a review commissioned . . . by the secretary of state to disclose what was going on.”

He added:

“You’ll find a lot of ministers don’t know a lot of things going on in the department because there’s no way you’ll find out.”

Maude’s comments touch on a common factor in IT-related project disasters in government – that ministers get mostly “good news” from their officials, and learn little or nothing about the seriousness of problems until a debacle is only too apparent to be denied.

But can ministers or the boards of large private companies ever expect their senior staff to be the bearers of bad news?

The Performing Right Society did not find out the truth about its failing IT-based project until it appointed a new head of IT who had no emotional equity in what had gone on before. [Crash – chapter 1)

The National Audit Office report “Universal Credit: early progress” referred to a “good news” culture at the Department for Work and Pensions that “limited open discussion of risks and stifled challenge”.

Ministers in charge of the Rural Payments Agency’s Single Payment Scheme said they were kept in the dark about the seriousness of IT-related problems. “When delays occurred, many stakeholders only found out at the last minute,” said a report of the Public Accounts Committee.

“Conspiracy of optimism”

The PAC report of March 2007 is worth a further mention:

“Lord Bach [minister in charge of the Single Payment Scheme] told us that he felt very let down by the advice he had received from the RPA [Rural Payments Agency], upon whom he said the Department relied very heavily in these circumstances, and the “conspiracy of optimism” on the part of the Agency.”

Lord Bach told MPs that he kept being told by officials that all was well.

“I frankly have to say that I do not think that that was satisfactory from senior civil servants whose job is to tell ministers the truth.”

Let down by civil servants – Universal Credit

Now the FT reports that Francis Maude has “entered the controversy over the implementation of the government’s universal credit scheme”. Maude told the FT he believed that Iain Duncan Smith, the work and pensions secretary, had been let down by his civil servants.

Maude said senior civil servants in charge of projects should tell ministers bluntly if they felt they were being misdirected and insist on a formal “letter of direction” to show that they had raised their objections. If they did not, they should be accountable for failings on their watch.

Maude did not comment directly on whether Robert Devereux, the top official in Mr Duncan Smith’s department, should take the rap for the much-criticised implementation of universal credit, but said: “I think everybody has to take responsibility for what they were part of”.

SROs accountable to MPs?

He suggested that civil servants who are in charge of big projects, known as senior responsible owners (SROs), should account directly to parliament, which would “toughen the relationship with ministers” and give officials a greater incentive to challenge developments they believed were wrong.

He said: “If you have an SRO who knows that he or she is going to be hauled up in front of select committees and interrogated . . . then I think you’re much more likely to have what is a very healthy thing in our system which is push-back. . . There’s a great phrase ‘speaking truth unto power’ and it’s very important – it doesn’t happen enough.

He added: “I’ve never had a civil servant come to me and say ‘Would you like us to stop doing this?’ The answer might easily be, ‘yes’.”

Comment:

Do ministers and boards of large private companies always have to commission their own independent reports to find out if their organisation’s biggest IT-based projects are failing? Probably.

The problem is not one of lying. Civil servants tend not to lie. Neither do senior executives when reporting to their boards. But the sin of omission – the art of not telling the truth while not lying – is well practiced in public life.

A succession of IT-based project disasters in the US, Australia and the UK show that truth is the first casualty of any large failing IT-based project.

Barnet Council and Capita

It’s isn’t just IT-based projects that bring out the sin of omission. Outsourcing deals do too. Barnet Council’s outsourcing deal to Capita is mired in controversy over truth.

Why did Barnet’s officials give Capita £16m after saying that the council had no spare cash, and that Capita would make the necessary upfront IT investments?

Officials have given a long-winded explanation which is a little like the drawn-out, incomprehensible explanation a six year-old may give in the playground when teacher asks why he took his friend’s bar of chocolate.

Liverpool LDL, BT and excessive mark-ups?

Liverpool Direct Ltd, a joint venture between Liverpool council and BT, is also mired in a controversy over truth. According to the Liverpool Daily Post, Local Government Minister Brandon Lewis has questioned whether LDL is proving value for money. There are allegations of excessive mark-ups on IT and services supplied by BT to the council.

It seems that BT makes a mark-up on what it supplies to LDL and LDL makes a further mark-up on what it supplies to the council.

But a council spokesperson said: ““The mark up incorporates a calculation of the cost of setting up a particular piece of hardware or software by LDL. The important figure is the profit after tax per item which is much lower, and on some items, LDL actually makes a loss.”

The minister said Liverpool Council needed to open up its books if it wants to insist it gets value for money from the BT deal. Will Liverpool Council open up?

Hardly.

Politicise parts of the civil service?

There is a strong argument for politicising the top echelons of the civil service so that ministers are not so reliant on officials who are thought to be neutral but evidence shows can be biased towards good news and suppressing the bad.

Ministers and boards of large companies do not need various versions of the truth when things go wrong. They need their own version.

As Richard Nixon said when accepting the presidential nomination in 1968 [pre-Watergate]:

“Let us begin by committing ourselves to the truth—to see it like it is, and tell it like it is—to find the truth, to speak the truth, and to live the truth.”

Doubtless Nixon believed it when he said it. Just as countless officials and executives in public and private life believe they are speaking the truth when they ministers and boards on their big IT-based projects. It may be the truth. But how much of it are they telling?

Update:

In a tweet BrianSJ3 makes a great suggestion: Genchi Genbutsu – “go and see for yourself” he says.

5 Comments

Posted in BT, Campaign4Change, Capita, DWP, excessive secrecy, Government IT, IT projects, IT-related failures, local democracy, local government, public-private partnerships, Universal Credit

Tagged , ,

More IT-based megaprojects derail amid claims all is well

Posted on 30/10/2013 | 2 comments

By Tony Collins

If one thing unites all failing IT-based megaprojects in the public sector it is the defensive shield of denial that suppliers and their clients hold up when confronted by bad news.

It has happened in the US and UK this week. On the Universal Credit  project, the minister in charge of the scheme, Lord Freud, accepted none of the criticisms in a National Audit Office report “Universal Credit: early progress”.   In a debate in the House of Lords Lord Freud quoted from two tiny parts of the NAO report that could be interpreted as positive comments.

“Spending so far is a small proportion of the total budget … and it is still entirely feasible that [universal credit] goes on to achieve considerable benefits for society,” said Lord Freud, quoting the NAO report.

But he mentioned none of the criticisms in the 55-page NAO report which concluded:

“At this early stage of the Universal Credit programme the Department has not achieved value for money. The Department has delayed rolling out Universal Credit to claimants, has had weak control of the programme, and has been unable to assess the value of the systems it spent over £300 million to develop.

“These problems represent a significant setback to Universal Credit and raise wider concerns about the Department’s ability to deal with weak programme management, over-optimistic timescales, and a lack of openness about progress.”

And a shield of denial went up in the US this week where newspapers on the east and west coast published stories on failing public sector IT-based megaprojects.  The LA [Los Angeles] Times said:

As many as 300,000 jobless affected by state software snags

“California lawmakers want to know why Deloitte’s unemployment benefits system arrived with major bugs and at almost double the cost estimate. The firm says the system is working.”

The LA Times continued:

“Problems are growing worse for the state’s Employment Development Department after a new computer system backfired, leaving some Californians without much-needed benefit cheques for weeks.”

The Department said the problems affected 80,000 claims but the LA Times obtained internal emails that showed the software glitches stopped payment to as many as 300,000 claimants.

Now lawmakers are setting up a hearing to determine what went wrong with a system that cost taxpayers $110m, almost double the original estimate.

Some blame the Department’s slow response to the problems. Others point the finger at a Deloitte Consulting.

The LA Times says that Deloitte has a “history of delivering projects over budget and with problematic results”. Deloitte also has been blamed, in part, for similar troubles with upgrades to unemployment software in Massachusetts, Pennsylvania and Florida, says the paper.

“We keep hiring the same company, and they keep having the same issues,” said Senator Anthony Cannella.  “At some point, it’s on us for hiring the same company. It’s faulty logic, and we’ve got to get better.”

In 2003 California planned to spend $58m upgrading its 30-year-old unemployment benefits system. By the time the state awarded Deloitte the contract in 2010  the cost estimate had grown by more than $30m.

The Department handed out $6.6bn to about 1 million unemployed Californians in 2012. The software was expected to ease the agency’s ability to verify who was eligible to receive benefits.

Problems began when the Department transferred old unemployment data to the new system. The software flagged claims for review — requiring state workers to manually process them.

The LA Times says that officials thought initially the workload would be manageable, but internal emails showed the agency was quickly overwhelmed. Phone lines were jammed. For weeks, the Department’s employees have been working overtime to clear the backlog.

A poor contract?

In a contract amendment signed two months ago California agreed to pay Deloitte $3.5m for five months of maintenance and operations costs. Those costs should have been anticipated in the contract said Michael Krigsman, a software consultant who is an expert on why big IT-based contracts go awry. He told the LA Times:

“It’s a striking oversight that maintenance was not anticipated at the beginning of the contract when the state was at a much stronger negotiation position.”

By the time the middle of a project is reached, the state has no choice but to stick with Deloitte to work out bugs that arise when the system goes live, he said.

System works

Loree Levy, a spokeswoman for the Department, said the system is working, processing 80% of claims on time. As for the troubles, she said, “There is a period of transition or adjustment with any large infrastructure upgrade like this one.”

Deloitte spokeswoman Courtney Flaherty said the new California system is working and that problems are not the result of a “breakdown or flaw in the software Deloitte developed”.

System not working?

While there seems to be no project disaster in the eyes of the Department and Deloitte Consulting, some of the unemployed see things differently. One wrote:

“I am a contract worker who had to fight for my unemployment benefits. I won my case and yet they still cannot pay me… It’s been more than 3 weeks since I won my appeal and as of this moment, I am owed 13 weeks of back payments. To add insult to injury, they cannot send me current weeks to certify and they refuse to even try to help me to get back into the online system.

“I blame Deloitte, but it is California that carries the heaviest burden of fault… We’re nearing November and they still haven’t fixed an issue that began over Labor Day? Nonsense!

“This is untenable for everyone affected …We are owed reparations as well as our money at this point. It’s a funny word, affected. That means families and individuals are going hungry but can’t get food stamps or welfare. It means evictions and repossessed cars. It means destroyed credit, late fees, years of turmoil and shame for people already dealing with unemployment. Shame on you California.”

Another wrote:

“ … Not communicating is NOT an answer. Unemployed individuals caught up in the nightmare were told to be patient.  Rents and other expenses were still accumulating.  But [when you] add on additional fees: late fees, restoral fees, interest fees, etc…….you get the picture.

“Dear Governor Brown,

“Please reimburse me for all additional fees I’ve had to absorb to survive this fiasco.  You are going to make me payback any overpayments, but ignore the cost to the unemployed taxpayer.  This is  appears to unfair.  Perhaps Deloitte should pay us back from their contracted funds before they receive their final payment.  I am saving all of my receipts to deduct from my 2013 tax return.

“BTW Gov Brown – I am still waiting on additional payments as of today and DMV registration for my vehicle was due on 10/20/13.  Are you going to waive the penalty for late payment? Am I the only one with this question?”

Scrutiny

California’s state Assembly has set a date of 6 November 2013 for a hearing into the Department’s system upgrade.

“We’re going to look at EDD, the contractors and others to see how the system broke down so we can avoid this in the future,” said Henry Perea, chair of the Assembly’s Insurance Committee, which has oversight over the jobless benefits program.

On its website Deloitte says:

“Deloitte continues to help EDD [Employment Development Department] transform the level of service it provides to unemployed workers and improve the quality of information collected by EDD. The next time unemployment spikes, California should be ready to meet the increased demand for services.”

Massachutsetts IT disaster?

On the opposite coast the Boston Globe reported on an entirely separate debacle (which also involved Deloitte):

          None admit fault on troubled jobless benefits system

“… even with the possibility that unemployed workers could face months more of difficulties and delays in getting benefits, officials from the Labor Department and contractor, Deloitte Consulting of New York, testified before the Senate Committee on Post Audit that the rollout of the computer system was largely a success.

“‘I am happy with the launch,’ said Joanne F. Goldstein, secretary of Labour and Workforce Development, noting that she would have liked some aspects to have gone better.

“Mark Price, a Deloitte principal in charge of the firm’s Massachusetts business, acknowledged that software has faced challenges during the rollout, but insisted, ‘We have a successful working system today. ‘’’

NPfIT shield

A shield of denial was up for years at the Department of Health whose CIOs and other spokespeople repeatedly claimed that the NPfIT was a success.

Comment

If you didn’t know that Universal Credit IT wasn’t working, or that thousands of people on the east and west coasts of the US hadn’t been paid unemployment benefits because of IT-related problems, and you had to rely on only the public comments of the IT suppliers and government spokespeople, you would have every reason to believe that Universal Credit and the jobless systems in Massachusetts and California were working well.

Why is it that after every failed IT-based megaproject those in charge can simply blow the truth gently away like soap bubbles?

When confronted by bad news, suppliers and their customers tend to join hands behind their defensive shields. On the other side are politicians, members of the public affected by the megaprojects and the press who have all, according to suppliers and officials, got it wrong.

Is this why lessons from public sector IT-based project disasters are not always learned? Because, in the eyes of suppliers and their clients, the disasters don’t really exist?

None admit fault on troubled jobless benefit system

State fired Deloitte

Complaints continue despite claims system is under control

As many as 300,000 affected by California’s software problems

California’s predictable fiasco?

2 Comments

Posted in Campaign4Change, change management, CIO, consultants, contractors, culture, Deloitte, ERP, Government IT, IT projects, IT-related failures, managing outsourced services, npfit, openness and transparency, operational issues, outsourcing, oversight, private sector lessons for the public sector, public sector, public-private partnerships, supplier relationships, Universal Credit

NPfIT central costs rise by tens of millions – even after “dismantling”

Posted on 18/09/2013 | 1 comment

By Tony Collins

On 22 September 2011 the Department of Health announced the dismantling of the NPfIT. As the press release was being issued some officials at the department were aware that they were continuing to spend tens of millions on central administrative costs of the programme.

Today’s report of the Public Accounts Committee has a figure for the central costs of the NPfIT until the end of March 2012 of about £890m. Before the DH announced the dismantling of the programme, in March 2011, the DH put the central costs at £817m.

So there has been a rise in central admin costs of about £70m since the NPfIT was supposedly dismantled.

The administrative costs are separate from spending on the contracts with BT or CSC. The admin costs don’t include the delivery of a single laptop to the NHS under the NPfIT. They are simply the central costs of administering the programme – including day rates for consultants – such as day rates of £1,700 to help senior officials prepare for appearances before MPs on the Public Accounts Committee.

The central costs have never been explained, not even by the National Audit Office which has published several reports on the NPfIT.  It is known that some central costs are explained by items of questionable benefit such as the commissioning of DVD films that marketed the NPfIT.

Some of the cost categories have emerged as a result of an FOI request (below).  Officials made regular visits to various parts of the globe to promote the success of the NPfIT. It’s thought that the DH has spent more than £100m on consultants for the programme.

Millions of pounds have been spent with public relations companies. The DH spent about £30,000 on press cuttings in two years alone.  Released central costs for just two years of the NPfIT between 2005 and 2007 include:

  • £1.23m with Expotel Hotel Reservations
  • £1.87 Harry Weeks Business Travel
  • BT conferencing – £1.15m
  • Intercall video conferencing – £274,973
  • MWB (Serviced Offices) – £15.8m
  • Regus – offices and meeting rooms – £3.17m
  • Spring International Express (courier and other services) – £192, 662
  • Cision UK (press cuttings) – £30,000
  • Fishburn Hedges (includes public relations) – £559,310
  • Good Relations (public relations] – £1.55m
  • Porter Novelli (public relations and information) – £943,000
  • ASE Consulting – £31.7m
  • Capgemini – £15m
  • Deloitte MCS – £42.8m
  • Atos Consulting – £32.3m
  • Gartner – £3.8m
  • QI Consulting – £14.5m
  • Tribal Consulting – £6.9m

Comment

Central administrative costs of nearly £900m on a single IT programme are breathtaking. That makes the National Programme for IT in the NHS one of the world’s largest public sector IT projects – before a penny has been spent on deliveries of hardware or software to the NHS.

It’s almost as surprising that not even the National Audit Office has been able to obtain a breakdown. Has central spending been properly controlled? Perhaps not, given that the DH, even this year, spent up to £1,700 a day on consultants to brief a senior official for a hearing of the Public Accounts Committee in June 2013.

Maybe the taxpayer should be grateful that the consultants were hired for only 52 days between February and June 2013 to prepare for the Committee’s hearing, and that the DH managed to renegotiate the day rate down from £1,714 to £1,000 a day between April and June.

Maybe the taxpayer should be grateful that the total cost of the consultancy for preparing for the PAC hearing was only £73,563.

But the £73,563 was spent after the DH estimated its central administrative costs on the NPfIT at nearly £900m – which are costs up to 31 March 2012.

It’s also remarkable that some at the DH still consider the NPfIT a success. This was the NAO’s conclusions on the NPfIT in its May 2011 report on the NPfIT Care Records Service:

“Central to achieving the Programme’s aim of improving services and the quality of patient care, was the successful delivery of an electronic patient record for each NHS patient. Although some care records systems are in place, progress against plans has fallen far below expectations and the Department has not delivered care records systems across the NHS, or with anywhere near the completeness of functionality that will enable it to achieve the original aspirations of the Programme.

“The Department has also significantly reduced the scope of the Programme without a proportionate reduction in costs, and is in negotiations to reduce it further still. So we are seeing a steady reduction in value delivered not matched by a reduction in costs.

“On this basis we conclude that the £2.7 billion spent on care records systems so far does not represent value for money, and we do not find grounds for confidence that the remaining planned spend of £4.3 billion will be different.”

But this was the Department of Health’s view on NPfIT Care Records Service value for money:

“The Department considers, however, that the money spent to date has not been  wasted and will potentially deliver value for money… The Department believes that the flexibility provided by the future delivery model for the programme will deliver functionality that best fits the needs of the clinical and managerial community. The future architecture of the programme allows many sources of information to be connected together as opposed to assuming that all relevant information will be stored in a single system. This approach has been proven in other sectors and is fully consistent with the Government’s recently published ICT strategy.”

This contradiction between the DH’s view of the NPfIT, and the NAO’s, indicates, perhaps, that the DH continues to live in a world not entirely attached to reality.

From April 2013, the DH’s central team and some local programme teams responsible for the NPfIT moved to the Health and Social Care Information Centre which has taken over the local service provider contracts with BT and CSC. Will it be able to control central spending on the very-much-alive NPfIT?

Update:

The central costs could rise much further – possibly by more than £100m – if the eventual settlement of the legal case between the DH and Fujitsu works out badly for the taxpayer. Legal costs on the case so far are about £31m.

1 Comment

Posted in BT, Campaign4Change, CRC, excessive secrecy, Freedom of Information, Fujitsu, health records, India, IT projects, IT-related failures, managing outsourced services, NHS, npfit

Why does truth on Universal Credit emerge only now?

Posted on 04/09/2013 | 2 comments

By Tony Collins

For nearly a year the Department for Work and Pensions, its ministers and senior officials, have told Parliament that Universal Credit IT is on track and on budget.

Together with DWP press officers, they have criticised parts of the media and some MPs for suggesting otherwise.

Now the truth can be held back no longer: the National Audit Office is expected tomorrow to report on UC’s problems. Ahead of that report’s publication, and perhaps to take the sting out of it, work and pensions secretary Iain Duncan Smith has allowed Howard Shiplee, the latest DWP lead on delivery of UC, to own up to the project’s difficulties.

IDS has given permission for Shiplee to write an article for the Telegraph on the UC project. Every word is  likely to have been checked by senior DWP communications officers.

It’s the first time anyone on the UC project has publicly acknowledged the project’s difficulties though, as with nearly every government response to critical NAO reports, the administration depicts the problems as in the past. Shiplee’s article says

“… it’s also clear to me there were examples of poor project management in the past, a lack of transparency where the focus was too much on what was going well and not enough on what wasn’t and with suppliers not managed as they should have been.

“There is no doubt there have been missteps along the way. But we’ve put that right…

“I’m not in the business of making excuses, and I think it’s always important to acknowledge in any project where things may have gone wrong in order to ensure we learn as we go forward.

“To that end, the key decision taken by the Secretary of State to reset the programme to ensure its delivery on time and within budget has been critical.

“When David Pitchford arrived from the Major Projects Authority earlier this year, at the Secretary of State’s request, he began this process in line with those twin objectives…

“I’ve also ensured that as a programme we have a tight grip on our spending, and I have put in place a post for a new Director who will be dedicated to ensuring that suppliers deliver value for money. I am confident we are now back on course and the challenges are being handled.”

Parliament has a right to ask why nearly every central government IT project that goes wrong – whatever the government in power – is preceded for months and sometimes years in the case of the NPfIT by public denials.

From the over-budget and fragmented Operational Strategy project for welfare benefits in the 1980s, to the repeatedly delayed and over budget air traffic control IT at the New En Route Centre at Swanwick, Hampshire, and the abandoned Post Office “Pathway” project in the 1990s, to the failed National Programme for IT – NPfIT –  in the NHS in the last decade, ministers and senior officials were telling Parliament that all was well and that the project’s critics were misinformed. Until the facts became only too obvious to be denied any longer.

These are some of the reassurances ministers and DWP officials have been giving Parliament and the media about the UC project. None of their statements has given a hint of the  “missteps along the way” that Shiplee’s article refers to now.

House of Commons, 20 May 2013

Universal Credit (IT System)

Clive Betts (Lab): What assessment he [the secretary of state for work and pensions] has made of the preparedness of the universal credit IT delivery system.

Iain Duncan Smith: The IT system to support the pathfinder roll-out from April 2013 is up and running…

Betts: I thank the Secretary of State for that answer, but will he confirm that three of the pathfinders are not going ahead precisely because the computer system is not ready? …

Duncan Smith: The hon. Gentleman is fundamentally wrong. All the pathfinders are going ahead. The IT system is but a part of that, and goes ahead in one of the pathfinders. The other three are already testing all the other aspects of universal credit and in July will, essentially, themselves roll out the remainder of the pathfinder, and more than 7,000 people will be engaged in it. All that nonsense the hon. Gentleman has just said is completely untrue.”

**

BBC – 9 Sept 2012

 “A Department for Work and Pensions spokeswoman said: “Liam Byrne [Labour] is quite simply wrong. Universal Credit is on track and on budget. To suggest anything else is incorrect.”

**

Iain Duncan Smith, House of Commons, 20 May 2013

“This [Universal Credit] system is a success. We have four years to roll it out, we are rolling it out now, we will continue the roll-out nationwide and we will have a system that works—and one that works because we have tested it properly.”

Howard Shiplee – FT July 2013

“… Howard Shiplee, who has led UC since May, denied claims from MPs that the original IT had been ‘dumped’ because it had not delivered. ‘The existing systems that we have are working, and working effectively,’ he said. He added, however, that he had set aside 100 days ‘not to stop the programme, but to reflect on where we’ve got to and start to look at the entire total plan’.”

**

DWP spokesperson 16 August 2013

“… a DWP spokesperson said: “The IT supporting Universal Credit is working well and the vast majority of people are claiming online.”

**

Howard Shiplee Work and Pensions Committee, House of Commons, 10 July 2013.

“…The pathfinder, first of all, has demonstrated that the IT systems work…”

Mark Hoban, DWP minister, House of Commons, 6 March 2013.

“The shadow Secretary of State has been touting this story for months. No it has been longer than that. The last outing was in today’s Guardian. I want to make it clear that nobody has walked off the project; all the contractors are in place and the project is on schedule to be delivered at the end of April. Now, if he thinks the idea is good in theory, it is about time he supported it. It is working and the contractors are in place, doing the job and ensuring that the pilots will be up and running at the end of April.”

[Hoban’s response was to a question on whether personnel or contractors at Accenture, Atos Origin, Oracle, Red Hat, CACI or IBM UK had been stepped down, or in any way notified by the Department, that they were to suspend work on Universal Credit. The main IT contractors for UC are Accenture, Hewlett Packard and BT plus input from Agile specialists Emergn. The DWP awarded UC IT contracts without any specific open competitive tender.]

Comment

On this site various posts have questioned whether Iain Duncan Smith has been getting the whole truth on the state of the UC IT project. He repeatedly went before MPs of the Work and Pensions Committee and gave such confident reassurances on the state of the UC project that it was difficult to believe that he knew what was really going on.

What we now know about the UC project’s “missteps along the way” shows, if nothing else, how gullible ministers are in believing their officials.

It is hard or impossible to believe that officials would lie but it is probable they would tell their ministers what they want to hear – and IDS has been in no mood to hear about problems.

Every big IT-based project in government that is failing ends up in a pantomime. From the back of the auditorium the media and MPs shout out when they receive leaks about problems. “Look behind you – there’s chaos,” they call out to departmental ministers and officials who don’t look behind them and reply “Oh no there isn’t!”

One reason this pantomime is repeated over decades is that independent reports on the progress or otherwise on big IT-based projects and programmes in central government are kept under departmental lock and key.  Even FOI requests for the keys consistently fail. 

So it’s usual for ministers and officials to answer media and Parliamentary questions about departmental projects without fear of authoritative contradiction.

Until the NAO is in imminent danger of publishing  a revealing report.

Perhaps it’s a lack of openness and accountability that contributes to IT-enabled change projects in central government going seriously awry in the first place.

With openness would come early and public recognition of a scheme that’s too ambitious to be implementable. With secrecy and the gung-ho optimism that seems to pervade projects like Universal Credit many on the project pretend to each other and perhaps even themselves that it’s all doable, while money continues to be thrown away.

When will the pantomime of misinformation and long-delayed revelation stop? Perhaps when Whitehall becomes genuinely open and accountable on the progress or otherwise of its IT-enabled projects. In other words: never.

Thank you to David Moss for drawing my attention to Howard Shiplee’s article in the Telegraph.

Time for truth on Universal Credit

Millions of pounds worth of secret DWP reports

2 Comments

Posted in Accenture, Agile, BT, change management, DWP, Gateway reviews, HP, IT projects, IT-related failures, Major Projects Authority, managing outsourced services, npfit, openness and transparency, operational issues, oversight, procurement, project management, public sector, public services, Real-Time Information, Universal Credit

Tagged , ,