Category Archives: Campaign4Change

Somerset Council publishes “lessons learnt” from IBM contract

Posted on 18/02/2014 | 7 comments

By Tony Collins

It’s rare for any council to publish the lessons learned from its outsourcing/joint venture contract, but Somerset County Council has set an example.

The council has produced its report to “inform future commissioning”. The council held a workshop to help identify the right lessons.

Written by Kevin Nacey, the council’s Director of Finance and Performance, and published by the Audit Committee, the document is diplomatically worded because the South West One joint venture contract with IBM continues; it was renegotiated in 2013 when the council took back some services and about 100 staff that had been seconded to South West One.

IBM is the majority shareholder in the joint venture company, and is its main funder. The minority shareholders comprise the county council, Taunton Deane Borough Council and Avon and Somerset Police. The joint venture company still provides ICT, finance and human resources/payroll.

Dave Orr, a former council IT employee and campaigner for openness, who spotted Nacey’s report, says it misses some important lessons, which are in Orr’s From Hubris to High Court (almost) – the story of Southwest One.

Lessons

From Somerset County Council’s Audit Committee report:

Contract too long and complicated

“One of the most significant lessons learnt is not to make contracts overly complicated. Both the provider and the Council would agree that the contract is incredibly complicated. A contract with over 3,000 pages was drawn up back in 2007 which was considered necessary at the time given the range of services and the partnership and contractual arrangements created.”

Expectations not met

“The partnership between the provider and the three clients has at times been adversarial and at times worked well. What has become clear over time is that any such partnership depends upon having similar incentives and an understanding of each partner’s requirements.

“Requirements change and the nature of local government changed considerably as a result of the national austerity programme. The well-documented financial difficulties faced by the provider early into the contract life also affected its ability to meet client expectations. The net effect is that at times the provider and partner aims in service delivery do not always match and discord and dissatisfaction can occur.”

Client team to monitor supplier “too small”

“The Client function monitoring a major contract needs to be adequately resourced. At the outset the size of the client unit was deemed commensurate with the tasks ahead, such as monitoring a range of performance measures and reporting on such to various management and Member forums.

“Liaison between partners, approving service development plans and approval of payments under the contract were other significant roles performed by the client unit. However, as performance issues became evident and legal and other contractual disputes escalated, the team had to cope with increasing workloads and increasing pressure from service managers and Council Members to address these issues.

“This is a difficult balancing act. You do not want to assemble a large client function that in part duplicates the management of the services being provided nor overstaff to the extent that there is insufficient work if contract performance is such that no issues are created.

“With hindsight, the initial team was too small to manage the contract when SAP and other performance issues were not resolved quickly enough. Sizing the function is tricky but we do now have an extremely knowledgeable and experienced client team.”

Some contract clauses “too onerous”

“Performance indicators need to be meaningful rather than simply what can be measured. Agreement between the provider and the SCC client of all the appropriate performance measures was a long and difficult exercise at the beginning of the contract. Early on in the first year of the contract, there were a large number of meetings held to agree how to record performance and what steps would be necessary should performance slip below targets.

“Internal audit advice was taken (and has been at least twice since under further reviews) on the quality and value of the performance indicator regime. It is regrettable and again with hindsight a learning point that too much attention was paid to these contractual mechanisms rather than ensuring the relationship between provider and SCC was positive. Perhaps the regime was too onerous for both sides to administer.”

Too ambitious

“Contract periods need to be different for different services as the pace of change is different. The range of services provided under the initial few years of the contract were quite extensive. On another related point the provider also had to manage different services for different clients.

“This level of complexity was perhaps too ambitious for all parties. Although there are many successful parts to the contract, it is inevitable that most will remember those that did not work so well.

“The contract period of 10 years is a long time for 9 different services to change at the same pace. Of course, service development plans were agreed for each service to attempt to keep pace with service needs as they changed. The secondment model introduced as part of the contract arrangements had been used elsewhere in the country before this contract used it.

“Nevertheless, it was the first time that 3 separate organisations had seconded staff into one provider. In many ways the model worked as staff felt both loyalty to their “home” employer, keeping the public service ethos we all felt to be important, and to Southwest One as they merged staff into a centre of excellence model.”

Hampered by terms of staff contracts

“The disadvantage was that Southwest One was hampered by the terms and conditions staff kept as they tried to find savings for their business model and to provide savings to the Council in recent years given the changing financial conditions we now operate under.

“Another aspect of this contract in terms of complexity is the nature of the partnering arrangement. It is not easy for all partners to have exactly the same view or stance on an issue. Southwest One had to manage competing priorities from its clients and the partners also had varying opinions on the level of performance provided.”

In summary

“This was a very ambitious venture. The service provided in some cases got off to an unfortunate start with the issues generated by SAP problems and relationships were strained and attracted much inside and outside attention.

“All parties have been working very hard to keep good relationships and to fix service issues as they arise. The sheer size and complexity of this contract has proven difficult to manage and future commissioning decisions will bear this in mind.

“Over the years officers running services that receive support from Southwest One have been surveyed regularly on how they feel the contract has been progressing. Despite all of the issues and lessons learnt outlined in this report, it is worth pointing out that many of the customer satisfaction and performance levels under the contract have been met by Southwest One.”

Comment

Well done to Somerset County Council’s Audit Committee and the Council’s auditors Grant Thornton for asking for the report. The South West One contract has been a costly embarrassment for IBM and the council. It has also had mixed results for Taunton Deane and the local police though officials in these two organisations seem locked into a “good news” culture and cannot admit it.

Perhaps the best thing to emerge from the IBM-led joint venture is this “lessons learnt” report. Without it, what would be the point of the millions lost and the damage to council services?

From contracts that don’t work out as expected, councils and central government departments rarely produce a “lessons” report , because nobody requires them to. Why should they bother, especially when it may be hard to get an internal consensus on what the lessons are, and especially when a report may mean admitting that mistakes were made when the contract was drawn up and signed.

Public sector organisations will sometimes do anything to avoid admitting they’ve made mistakes. Somerset County Council and its Audit Committee have shown they are different. Maybe the rest of the public sector will start to follow their example.

Dave Orr’s well-informed analysis of the lessons from Southwest One.

7 Comments

Posted in Campaign4Change, change management, IBM, Institute for Government, IT-related failures, legal disputes, local government, SAP, shared services, supplier relationships

Is IDS losing his cool over Universal Credit IT?

Posted on 06/02/2014 | 3 comments

By Tony Collins

IDS was polite and calm, almost deferential, when he went before MPs of the Work and Pensions Committee in September 2012.  “Can I say it is always a privilege to be here?” he said.

At at Monday’s hearing of the same committee, though, he was at times tetchy, patronising and mildly bullying. “I don’t think this committee can run the department,” he replied when asked why he hadn’t told the committee in 2012 of problems with the Universal Credit IT project.

Several times he talked over the MP who was asking him questions, with the result neither could be clearly heard.

[If he’s like that at meetings with DWP officials would anyone want to tell him something he doesn’t want to hear? Perhaps his loss of cool on Monday reflected the baffling complexity, and rising costs, of the waterfall part of  Universal Credit’s  IT programme.]

IDS might also have been shaken by the absence of his most authoritative ally, Howard Shiplee, who has been off sick since shortly before Christmas.

Hidden 

Over a period of more than a year, the DWP and IDS fed the work and pensions committee good news about progress on the Universal Credit IT project. The truth didn’t surface until the National Audit Office published its report on UC in September 2013.

Unknown to the committee in 2012, the DWP was struggling at that time to set out how the detailed design of systems and processes would fit together and relate to the objectives of Universal Credit. This was raised repeatedly in 2012 by internal audit, the Major Projects Authority and a supplier-led review. The committee wasn’t told.

Hence Dame Anne Begg, the softly-spoken chairman, came to Monday’s meeting with a direct question. Why, when IDS came before the Work and Pensions committee in September 2012, did he make no mention of having commissioned a red team review into the Universal Credit project several months earlier.

“Because it was an internal review,” replied IDS. “We were looking the results of that and trying to take whatever decisions were necessary. It was about some of the issues that were going on in the UC team…”

Begg: “But why didn’t you tell us a review was going on?”

IDS: “I don’t tell the committee everything that is happening within the Department until we have reached a conclusion about what is actually happening.”

Begg: “It was an ideal opportunity when you appeared before us in September [2012] that you could have said there were concerns about what was happening with Universal Credit but at that session you were very bullish about how successful everything was.”

IDS: “I still remain very confident about how successful it will be. [Note a difference in tenses between the question and answer]. At the time we were working out how we would make the reset.”

Sir Humphrey

At IDS’s sided was Robert Devereux, Permanent Secretary at the DWP, who seemed at times a parody of Sir Humphrey. [Animated in the delivery of some of his answers Devereux looked as if he was saying something interesting until you listened to the words.]

One MP asked Devereux why the DWP had given written evidence to the committee in 2012 that Universal Credit was on track when it wasn’t. Devereux said that UC was a large and complex programme. “You are constantly evaluating and re-evaluating your forward plans … as you go along things change.” MPs were none the wiser.

Misled?

Begg [to IDS]: Did you not think it appropriate that this scrutiny committee of the House of Commons, which oversees the work the department does, [should have been kept] informed about changes?

IDS: “With respect we did keep the Committee informed as and when we had clarified what we were actually doing and what we thought the problem was and where it existed and how you isolate it and what changes you made. I don’t for one moment agree in any way at all that we hid stuff. We knew we would be accountable to the committee and all would become public… I don’t think this committee can run the department.”

Begg pointed out that IDS had failed to mention a report of the Cabinet Office’s Major Projects Authority in February 2013. That report had notified the DWP of flaws in UC governance, management and programme design – despite the same matters having been raised in previous MPA reports.

Begg:  “You gave oral evidence to us on 10 July [2012] … but you did not refer at that session to the critical Major Projects Authority report or the reset which had already taken place earlier that year.”

IDS: “I cannot remember what I said to the committee. I have no desire to look back.”

Begg said the DWP told the committee that the pathfinder projects demonstrated that the IT systems worked. “You cannot get any more definitive than that,” said Begg. IDS gave no clear answer.

Obfuscation

Mid-way through the hearing, the mood of some of the exchanges was summed up by Labour MP Debbie Abrahams who told IDS:

“I cannot say with the strongest feeling my concern about the hubris that you have demonstrated in the tone to this committee. You haven’t explained, certainly to my own satisfaction – anybody who is watching will draw their own conclusions – you have not given any satisfactory explanation about how you have informed, or kept this committee informed, about the difficulties the department was experiencing.

“There have been obfuscation and smoke and mirrors even up to a few weeks before the report from the National Audit Office [in September 2013]. The memorandum that was released in August – this was clearly saying everything was fine and dandy. It is clearly not. I give you one more opportunity to answer, so you can explain to this committee, why such poor information is provided by your department.”

IDS replied: “I just don’t agree with you, and I don’t agree that we have done anything else but be open and honest about what the issues are, as and when they have been identified and what we would do about them as and when we have made our decisions about them…

Open?

“When we found something wrong we went and sorted it out. As we sorted it out we made clear direction about that, and eventually through the NAO, the PAC [Public Accounts Committee and the [Work and Pensions] committee.

“I think we have been pretty open about it. I don’t think there’s anything more. In fact in a sense we are going round and round in circles here at this committee hearing at the moment.”

Begg:  “We are not convinced you have got it sorted out.”

Comment

Monday’s hearing shows how ministers and officials justify the hiding of reports on costly IT-enabled projects that are going wrong. IDS didn’t even tell MPs in July 2013 that the Major Projects Authority had four months earlier recommended an immediate pause in the programme.

Most worrying of all, officials and IDS seem content that the DWP gave the work and pensions committee – in September 2012 and July 2013 – a good news story on the state of the Universal Credit IT project while truth about the project’s problems stayed hidden.

IDS suggested it was not necessary to tell MPs about reports until ministers have “reached a conclusion about what is actually happening” That may be never.

It’s time for public accounts and work and pensions MPs to insist on seeing Major Projects Authority reviews, and other reports, on the progress or otherwise of big government IT-enabled programme such as UC. MPs should not have to wait for an NAO report to get the truth.

Governments, whatever their hue, will always refuse to publish these reports contemporaneously, such is the will of departmental heads. They have been refusing to publish the reports for more than 20 years.

But if MPs keep insisting with an unbreakable tenacity on their publication  – and for publication before they are out of date – it may eventually happen, and gone will be the power of ministers and officials to mislead MPs on the state of big IT-enabled programmes.

Until publication happens, is there much point in MPs questioning IDS or his officials on the UC IT programme? They will get only the public relations version of the truth.

 

3 Comments

Posted in Campaign4Change, DWP, excessive secrecy, IT-related failures, Major Projects Authority, openness and transparency, procurement, project management, public services, Real-Time Information, Universal Credit

Tagged , ,

Officials black out IT security report after it’s published in full

Posted on 05/02/2014 | 1 comment

By Tony Collins

In one of the most bizarre regressions since the FOI Act came into force in 2005, officials at Somerset County Council have redacted an audit report on SAP security weaknesses after the report was published in full.

The result is that anyone can see links to both reports. This is the report with parts of it redacted – blacked out. These are links to the full versions, which were published before the redactions – here and here.

The report was written by auditors Grant Thornton for Somerset County Council and highlights weaknesses in a database that is shared by the council, Taunton Deane Borough Council and Avon and Somerset Police.  The database is part of a SAP system run by Southwest One on behalf of the three authorities.

Southwest One is an IBM-led enterprise that provides IT and other services to the three authorities under a controversial outsourcing contract. Dave Orr has written comprehensively about the deal.

Somerset published the Grant Thornton report in full. The media including Campaign4Change published some details of the IT security weaknesses mentioned in the Grant Thornton report. It appears that Avon and Somerset Police asked officials at Somerset to black out details of some of the weaknesses.

Somerset-based FOI campaigner Dave Orr says the blacking out is to save the blushes of the police.

Says Orr: “Much of the redaction in the Somerset County Council IT Controls report by Grant Thornton, especially generic and available password advice in Section 3, is not based in a genuine security threat, but looks to be rooted in a Police culture that seeks to avoid criticism and/or embarrassment.”

Somerset MP Ian Liddell-Grainger says:

“SAP was built on the cheap by IBM to serve three different customers – the County Council, Taunton Deane district council and the Police. It would have made sense to bung in a few partitions to stop council eyes taking a peek at police matters, or vice versa. But that would have cost money – perish the thought.”   

 Police SAP systems’s “significant” security weaknesses. 

1 Comment

Posted in Campaign4Change, Freedom of Information, IT security, local democracy, local government, outsourcing, public sector, public services, public-private partnerships, SAP, security, supplier relationships

Of mice and IT elephants – guest blog

Posted on 04/02/2014 | 4 comments

By John Pearce

I heard your interview on BBC’s World at One today Tony. You were saying there may be potential for fleet-of-foot small IT firms to access government contracts. It was music to my ears.

You referred to the NHS and Universal Credit IT disasters and the way contracting has been dominated by a few big beasts and multi-nationals.  You killed the myth that “big is beautiful”  and praised “new rules”  to break up projects into smaller units.

Small can be perfectly formed and powerful. Businesses like ours are quick-reflex mice stuck behind the elephants blocking the doors. Why don’t they just sit out of the way, in the room, like other elephants?

We are an SME in IT.  We have great pedigree, an innovative product, a presence in education and are ready to break into the business world and government work more generally. But without the bulk and buying power to advertise, lobby and bid for the current huge projects we have not been able to do much, if anything. We are encountering elephant in the door syndrome.

So we continue doing what we do, scurrying like mad, working unreasonable but happily given hours. It is not in the country’s interest for us to be tired, blocked and trapped. We fear being swallowed up, of losing our identity. I suppose it might be quicker than being slowly squashed under an elephant’s backside.

iAbacus

Dan O’Brien, my business partner, is young, creative, dynamic and rushed off his feet.  I am three of those and old.  He has run a small successful software company for 15 years. I had a successful senior career in education and in business as consultant, evaluator, writer and publisher.  I created a deceptively simple, improvement model for individual, team and organisation. So, Dan and I created an on-line version.

We launched “The iAbacus” in 2012 and were finalists in the BETT2014Awards [hosted by Jo Brand] on 22 January 2014.  There were lots of mice competing with us and the usual elephants. But before we announce the winner let’s have a look at the iAbacus focusing on school governance. 

We dream of developing this and moving into business generally. We see a huge potential for this “empowering personnel” approach applied to NHS and civil service personnel. Up to now the elephants have blocked the way, or grinned through the windows while they ate ice buns. Can elephants grin?

We didn’t win the Bett2014 Big Cheese but it was a great show – it makes people like us feel good.  Yes, coming back to the office was disappointing but we are nibbling away, on-line, working in education but, even in this field I know so well, customers can be sniffy too – “small is ugly and simple is simplistic; let’s go for the big suppliers”. 

New rules

Will the Cabinet Office’s new rules work?

 Not unless there is support for small outfits like ours who, intent on the day to day business, will struggle writing the bids and attending the selections. We’ll keep running and swerving around elephant bottoms but we need muscle power and finance for the advertising and lobbying. We need to elbow past the elephants, get an audience with buyers.  Is there anyone out there?  Echo…echo….echo…

Or, are there friendly elephants out there who could help us, encourage and include us?  Could the regulations persuade them?  How about a clause like the one when planning new houses?  Every housing project has to include a percentage of affordable homes. How about every IT contract having to include a percentage of SMEs?  

I want to one of them. I want to be a Trojan Mouse!

John Pearce is a freelance consultant, working across education, business and community. After a successful career in teaching and headship, he became Deputy Chief Inspector for Nottinghamshire County Council. He was a BETT2014 finalist for The iAbacus which he created with Dan O’Brien.

john@iabacus.co.uk   dan@iabacus.co.uk

BBC’s World at One focus on government IT.

4 Comments

Posted in Campaign4Change, Government IT, SME

Tagged , , , ,

Universal Credit head Howard Shiplee off sick

Posted on 03/02/2014 | 5 comments

By Tony Collins

Howard Shiplee, who joined the Department for Work and Pensions in May 2013 as Director General of the Universal Credit Programme, has been off sick since shortly before Christmas.

A DWP spokeswoman told Campaign4Change that Shiplee is in regular touch with the programme through conference calls and would be back working full-time on UC as soon as he is fit.

Today  [3 February 2013] Shiplee’s boss Iain Duncan Smith is due to appear again before the Work and Pensions Committee  to answer questions about UC. The DWP spokeswoman said Shiplee is looking forward to returning to the UC project.

Comment:

The job of Director General of UC is well paid –  £258,750 according to the Daily Mail. But it may also be the most stressful IT-related job in central government. UC is probably the most high profile of all IT-enabled projects in central government and it faces many challenges, according to the National Audit Office.  IDS is determined that the IT will succeed – though whether he will still be the minister in charge of UC after the general election next year is uncertain.  

Shiplee is well regarded – he was previously the construction director for the successful 2012 Summer Olympics in London. Keeping his reputation for successful delivery on the UC project will not be easy.  Campaign4Change wishes him a speedy recovery.   

5 Comments

Posted in Campaign4Change

Ex BP executive to run Major Projects Authority

Posted on 03/02/2014 | Leave a comment

By Tony Collins

John Manzoni is to join the civil service as Chief Executive of the Cabinet Office’s Major Projects Authority.

The former BP executive brings 30 years’ commercial experience to the job. He was  recently President and CEO of Canadian oil and gas company Talisman Energy Inc. He spent 24 years at BP, holding senior strategic and operational leadership roles at global, regional and local level. 

He was a member of the BP plc Main Board from 2003- 2007.  The Major Projects Authority was set up in 2011 with a mandate from the Prime Minister to turn around poor project management in Government. It reviews major projects and has the power to stop them in extreme cases.

Manzoni will need a stomach for polite confrontation if his Major Projects Authority is to continue to be effective. Some long-standing  executives in departments prefer to carry on without the influence – they call it interference – of the Cabinet Office. 

It may be that the only effective way the CEO of the Major Projects Authority can influence in the civil service is by heaping praise on departmental officials while recommending changes he hopes the department will take seriously.

The formidable David Pitchford was interim head of the Authority until he left last September.

Manzoni says: 

“Some of the largest and most complex projects can be found in Government and it’s the scale of the challenge that makes this role interesting. Working with Departments, and at the service of talented civil servants, the MPA has already delivered remarkable improvements.

“To me, the key to its continued success is the quality of project professionals at the centre and in departments. It always comes down to the people.

“If we get this right, it will be a sustainable agenda that is good for all seasons and all political persuasions. It’s hard to fight against the logic of spending large amounts of money more effectively and improving public services at the same time.”
Cabinet Office minister Francis Maude says: “John has an impressive record of leading global operations and delivering complex, challenging briefs…

“As part of this Government’s long-term economic plan we must to continue to improve the Civil Service’s management of major projects. Last year the Major Projects Authority saved hard-working taxpayers £1.7bn. I’m confident that with John’s leadership we can go even further and make a real difference to the delivery of these projects which matter to all of us.”
Danny Alexander, Chief Secretary to the Treasury adds: “The Major Projects Authority has real power to intervene in failing projects and stop taxpayers’ money being wasted, and it is already turning around our record of delivering Britain’s most important projects.

“John’s experience speaks for itself …”

Sir Bob Kerslake, Head of the Civil Service says: “Improved project delivery is a crucial part of our Civil Service Reform Plan- indeed, in the future, no one will lead a major project without completing extensive training at the Major Projects Leadership Academy.

“It is fantastic that John Manzoni is bringing his extensive experience to lead the next generation of Civil Service project managers.”

Stephen Kelly, Chief Operating Officer says “The MPA’s success lies in delivering quality outcomes on time within budget, and we are passionate to see MPA become the leading authority in world class project implementation.

“The fusion of private sector best practice with strong public service ethos is key to this future. The MPA has already had a strong start working with departments delivering major public projects and I have no doubt that John’s leadership and proven success will accelerate our delivery.”

Leave a comment

Posted in Campaign4Change, Major Projects Authority

Tagged , , , ,

Dare anyone criticise this IT project – with the CEO as leader?

Posted on 03/02/2014 | Leave a comment

By Tony Collins

Croydon Health Services NHS Trust has had mixed success with its go-live of the Cerner Millennium system.

It is said to be a technical success but last week board members of the Croydon Clinical Commissioning Group expressed concerns about ongoing problems with the system.

Fouzia Harrington, director of quality and governance told the Croydon Advertiser: “The implementation [of Cerner] itself went well in technical terms, but there have been some implications about how it has been used by staff.

“It’s had far more impact in terms of the time it takes to book people in, for example. There have also been implications in terms of lost information about patients.

“There has been a lack of information about hospital activity, which has an impact on finances and, potentially,the quality of services patients are receiving…”

David Hughes, a lay member of the board, was not satisfied with that reassurance.

“You say that no harm has occurred,” he said,  “but while we’ve had no direct incident so far, patient care has definitely suffered.

“You talk about increased waiting times and there’s a risk that harm may occur because of the difficulty in getting in touch with clinicians who actually know what is going on with the patient.

“I’m very concerned from a quality point of view that our main provider has a serious problem with its information systems.”

Hughes called for action. Although the trust may not be aware of an incident yet it may “come out through further investigation that there has been”.

Some waiting times have increased,  the CCG cannot be certain of exact levels of activity at the hospital, and missing information has made it difficult to commission some services.

The concerns were raised at a board meeting on Tuesday.

Dr Tony Brzezicki, chairman of the CCG, said new system would eventually lead to improvements.  “Hospital patients had five sets of notes before. That in itself posed a risk that Cerner will mitigate,” he said.

“However, there have been administrative delays which mean longer waiting times for patients.There are also issues for the service to primary care which is a significant risk. Some of the problems have been resolved though I am concerned at the time scale because they are certainly impacting on my practice.”

Success

John Goulston is the Croydon Health Services NHS Trust CEO. One of his previous jobs was as Programme Director of the London Programme for IT at NHS London. The LPfIT was formerly part of the National Programme for IT. 

As well as CEO, he chairs the trust’s Informatics Programme Board which has taken charge of bringing Cerner Millennium to Croydon’s community health services and the local University Hospital, formerly the Mayday.

Goulston reported to his board that the Cerner go-live – on 30 September and 1 October last year – was a success.

“Our partners Cerner, BT and Ideal have commented that the Trust has undertaken one of the most efficient roll-outs of the system they have worked on, with more users adopting the system more quickly and efficiently than other trusts … the success we have achieved to date is the result of the efforts of every single system user and all staff members,” said Goulston.

Goulston has said the trust deployed the “largest number of clinical applications in a single implementation in the NHS”. 

The Department of Health provided central funding, and the trust paid for implementation “overheads”.  The Health and Social Care Information Centre was the trust’s partner for the go-live.

The Croydon Advertiser asked Croydon Health Servicesa series of questions about Cerner, including its cost to the NHS, but was sent a short statement.

A spokesman told the Advertiser the system would improve patient administration and means that nurses have access to “quality, detailed information” when delivering care.

He added: “During the initial switch over of systems in September while staff were getting used to the system, some patients did need to wait slightly longer to check in for their clinic appointments.

“The trust has maintained and surpassed our 18 week referral to treatment targets from the initial roll out.”

Croydon’s response

Campaign4Change put some questions to the Croydon trust. These are the questions and its responses: .

Is the trust being completely open – taking seriously the duty of candour –  about problems arising from the Cerner Millennium go-live?

“The Trust takes its duty of candour on all issues very seriously; we believe that transparency is essential in running a modern NHS organisation. We are held to account by our board at public meetings, where the public are able to attend and question our senior management team, by our local health overview and scrutiny committee and our commissioners.

“Recent press coverage on CRS Millennium appeared in the local press when the system was discussed in a public meeting of our commissioners.”

As the CEO is leading the Cerner Millennium project, does this make it difficult for trust staff and trust directors to say anything even mildly critical about the implementation?

“Staff opinions on the implementation of CRS Millennium, both positive and negative, are welcomed by the Trust. Staff have given their frank opinions of the system directly to the Chief Executive both in our monthly all staff meetings and at the open staff engagement surgeries held by our Chief Executive and Chairman. All staff opinions are taken seriously and are acted upon appropriately.”

Given the CEO’s enthusiasm for the implementation is there a special onus on the press office to defend the implementation and play down problems? [I note that the Croydon Advertiser implied its questions had not been answered, and that the Trust gave a short statement instead.]

“The communications team respond to and facilitate a large number of external requests, including from the media, in a transparent, timely and appropriate manner. This same approach is followed on questions about CRS Millennium.

“CRS Millennium will bring about many improvements to patient care and Trust efficiency and we are enthusiastic about communicating these; it is unfortunate that recent press coverage did not consider these positive benefits in any depth.”

A comment on the Croydon Advertiser’s website says:

“When I checked in to out-patients I supplied all my personal details; however the post code I gave was declared invalid by the new system. That filled me with confidence. I also gave my contact as a mobile; however they tried to ring me using an old landline number.”

Comment

It’s generally accepted that having a high-level sponsor for an IT project is essential but when the lead is the CEO, does that make it difficult for people to challenge and constructively criticise?

A “good news” culture tends to prevail – as happened on Universal Credit, on the BBC’s Digital Media Initiative, and within the Department of Health on the NPfIT. Nobody dared to speak the whole truth to power. The truth tends to surface only when a new administration takes over or, in the case of Universal Credit, the minister obtained his own independent reports on project progress.

Campaign4Change put it to the Croydon trust that board directors see reports on the Cerner implementation only every two months and much can happen in the intervening period. This it did not deny.

Even if the trust’s directors met daily would they dare to challenge the CEO? And will the full facts  ever emerge? Things could be much better than CCG directors believe  – or much worse.

After nearly every major NPfIT implementation of the Cerner Millennium system in London and beyond (such as North Bristol) the facts were scarce, and reassurances that no patients had come to harm were plentiful. 

Here we go again?

**

Should lessons have been learned from these Cerner go-lives?

Barts and The London

Royal Free Hampstead

Weston Area Health Trust

Milton Keynes Hospital NHS Trust

Worthing and Southlands

Barnet and Chase Farm Hospitals NHS Trust

Nuffield Orthopaedic

North Bristol.

St George’s Healthcare NHS Trust

University Hospitals of Morecambe Bay NHS Foundation Trust

Birmingham Women’s Foundation Trust

NHS Bury

GPs asked to contact hundreds of patients who may have missed treatment after hospital’s cancer referrals blunder  – Pulse

London LMCs alert over Imperial cancer waits mix-up – Pulse.

GPs kept in the dark over hospital cancer blunder – Pulse

 IT system has increased waiting times and led to lost patient data.

Patient records go-live success – or NPfIT failure

Leave a comment

Posted in BT, BT Health, Campaign4Change, Department of Health, e-health, excessive secrecy, health, health records, LPfIT, NHS, npfit, openness and transparency, operational issues

When “life and death” NHS IT goes down

Posted on 29/01/2014 | 1 comment

By Tony Collins

Almost unnoticed outside the NHS an email was circulated by health officials last weekend about a national “severity 1” incident involving the Electronic Prescription Service, running on BT’s data Spine .

“The EPS [electronic prescriptions service] database is currently experiencing severe degradation of performance. … BT engineers [are] currently investigating with the database application support team,” said the email.

A severity 1 or 2 incident, which involves a temporary loss of, or disruption to, the Spine or other national NHS system,  is not unusual, according to a succession of emails forwarded to Campaign4Change.

The Department of Health defines a severity 1 incident as a  failure that has the potential to:

— have a significant adverse impact on the provision of the service to a large number of users; or

— have a significant adverse impact on the delivery of patient care to a large number of patients; or

— cause significant financial loss and/or disruption to NHS Connecting for Health [now the Health and Social Care Information Centre], or the NHS; or

— result in any material loss or corruption of health data, or in the provision of incorrect data to an end user.

The Health and Social Care Information Centre, which manages BT’s Spine and other former NPfIT contracts, reports that the spine availability is 99.9% or 100%. But the HSCIC’s emails tell a story of service outage or disruption that is almost routine.

If the spine and other national services  are really available 99.9% of the time, is that good enough for the NHS, especially when ministers and officials are increasingly expecting clinicians and nurses to depend on electronic patient records and electronic prescriptions?  In short, are national NHS IT systems up to the job?

NHS staff access the spine tens of millions of times every month, often to trace patients before accessing their electronic records.  The spine is pivotal to the use of patient records held on Rio and Cerner Millennium systems in London. It is critical to the operation of Choose and Book, the Summary Care Record, Electronic Prescription Service pharmacy systems, GP2GP, iPM/Lorenzo, and the Personal Demographics Service.

According to a Department of Health letter sent to the Public Accounts Committee, payments to BT for the Spine totalled £1.08bn by March 2013.

BT says on its website that its 10-year NHS Spine contract involves developing systems and software to support more than 899,000 registered NHS users. The HSCIC says the Spine is used and supported 24 hours a day, 365 days a day.

“There is a huge amount of industrial-strength robustness, availability, disaster recovery, that you cannot get someplace else,” said a BT executive when he appeared before MPs in May 2011.

Life and death  

Sir David Nicholson spoke of the importance of the spine and other national NHS systems at a hearing of the Public Accounts Committee in 2011. He said they were

“providing services that literally mean life and death to patients today … So the Spine, and all those things, provides really, really important services for our patients…”

When Croydon Health Services NHS Trust went live with a Cerner Millennium patient records system at the end of September a “significant network downtime” – of BT’s N3 network – had an effect on patients.

A trust board paper, dated 25 November 2013 says:

“CRS Millennium (Cerner) Deployment -Network downtime – Week 1.  In particular, the significant network downtime in week 1 (BT N3 problem) led to no electronic access to Pathology and Radiology which resulted in longer waits for patients in the Emergency Department (ED) leading to a large number of breaches. This was a BT N3 problem which has been rectified with BT …”

Below are some of the emails passed to Campaign4Change in the past four months. Written by the Health and Social Care Information, the emails alert NHS users to outages or disruption to GP or national NHS IT systems.

Some HSCIC messages of disruption to service

October 2013

Severity 2
HSCIC
National
CQRS has not received a number of participation status messages.
Also affecting: GPES
USER IMPACT:
CQRS Users are not able to manually submit specific information, this will impact the users’ business process for entry of achievement data.
ACTION BEING TAKEN:
Following a configuration change by the GPES Business Unit a specific code has now been added to the GPET-Q Database. We are currently awaiting confirmation that the addition of the relevant code has been successful. Discussions are taking place regarding the re-submission of status messages. HSCIC conference calls are on-going.

[A severity 2 service failure is a failure [that] has the potential to:

– have a significant adverse impact on the provision of the service to a small or moderate number of service users; or

– have a moderate adverse impact on the delivery of patient care to a significant number of service users; or

– have a significant adverse impact on the delivery of patient care to a small or moderate number of patients; or

– have a moderate adverse impact on the delivery of patient care to a high number of patients; or

– cause a financial loss and/or disruption … which is more than trivial but less severe than the significant financial loss described in the definition of a Severity 1 service failure.]

**

Severity 2
HSCIC
BT Spine
National
Intermittent performance issues on TSPINE.
T-Spine
RESOLUTION:
BT Spine have confirmed that the incident has been resolved and users are able to perform routine business processes without delays.

November 2013

Severity 1
BT Spine
HSCIC
National
Users are unable to log into PDS.
USER IMPACT:
All sites are currently unable to access PDS, this is causing a delay to normal services.
ACTION BEING TAKEN:
BT Spine are working to restore service.

**

Severity 2
BT Spine
HSCIC
National EPS users.
Slow performance on reliable and unreliable messages for EPS.
USER IMPACT:
This is causing delays to routine business processes as some users may be experiencing slow performance with the EPS service.
BT investigating.

**

Severity 2
BT Spine
HSCIC
National
Slow performance on EPS Messaging.
USER IMPACT:
This is causing delays to routine business processes as some users may be experiencing slow performance with the EPS service.
ACTION BEING TAKEN:
BT moved the database to an alternate node following application server restarts. This temporarily restored normal message response times however performance has started to degrade again. BT Investigation continues.

**
Severity 1
Atos
HSCIC
National
Multiple users were unable to log in to the Choose & Book application.
ATOS made some network configuration changes overnight 19th/20th November which restored service. After a period of monitoring throughout the day yesterday the service has remained stable and at expected levels. Further activities and investigation will be carried out by several resolver teams which will be scheduled through change management.

**
Severity 2
BT Spine
HSCIC
National
Slow performance on EPS Messaging.
No further issues of slow response times with EPS messaging have occurred today. BT Spine to continue root cause investigation.

**

Severity 2
Cegedim RX
HSCIC
National
Cegedim RX – Users are experiencing slow performance in EPS 1 and EPS 2.
USER IMPACT:
Users are experiencing slow performance and delays to routine business processes when using EPS 1 and EPS 2.
ACTION BEING TAKEN:
Following a restart of application services, traffic has improved for all new EPS messages. However there is a backlog of EPS messages which may cause delays to routine business processes. Cegedim RX to continue to investigate.

**

December 2012

Severity 1
BT Spine
HSCIC
National
Performance issues have been detected with the transaction messaging system (TMS).
Also affecting: Choose and Book, GP2GP
USER IMPACT:
This may cause delays to routine business processes. This may have an effect on all Spine related systems. This includes PDS, Choose and Book, PSIS, SCR, ACF Services.
ACTION BEING TAKEN:
This has been resolved but BT are currently monitoring performance. Further investigation is required by BT into the root cause.

**

Severity 2
GDIT – CQRS
HSCIC
National
DTS has not processed a CQRS payment file.
CQRS
Also affecting: GPES
USER IMPACT:
This is causing delays to routine business processes.
ACTION BEING TAKEN:
GDIT are currently developing a fix which will be rolled out tomorrow evening, pending successful testing.

January 2014

Severity 1
BT Spine
HSCIC
National
TMS reliable messaging unavailable.
USER IMPACT:
TMS reliable messaging unavailable and users having to implement manual workarounds.
ACTION BEING TAKEN:
Issues experienced due to a planned change overrunning, BT Spine continue to implement the transition activity in order to restore service.

**

Severity 2
BT Spine
HSCIC
National
Users have experienced intermittent issues with the creation and cancellation of smartcards in CMS [Card Management Service for managing smartcards].
CMS
USER IMPACT:
This is intermittently causing delays to routine business processes as some users have been unable to create, cancel, cut or print cards in CMS.
ACTION BEING TAKEN
Users may experience issues with the creation and cancellation of cards in CMS. BT have identified a fix for the issue which is currently undergoing testing prior to deployment into the live environment.

**

Severity 2
BT Spine
HSCIC
National
The maternity browser was unavailable within NN4B.
RESOLUTION:
BT identified a problematic server which was recycled to restore system functionality.

**

Spine scheduled outage for essential maintenance activity.

During critical work to migrate to a new storage solution on Spine an issue was experienced on the Transaction Messaging Service (TMS) in September of this year. The issue resulted in BT failing over the TMS database from its usual site on Live B to Live A to restore service. The failover was completed well within the Service Level Agreement and no detrimental long term impacts to the service were incurred.

On the 15th January 2014 between approximately 22:00-23:30, HSCIC, in conjunction with BT, are planning to relocate the TMS database back to Live B, this is for several critical reasons:

  1. The issues experienced, which prompted the failover, are fully resolved and will not be experienced again as the storage migration work is now complete.
  2. The Spine service is designed to operate with all databases running on Live B so this work supports the optimum configuration for the service.
  3. Most critically the transition for all data on Spine to Spine2 has been designed to operate from a standby site with no live databases on it. Therefore to support the Spine2 transition this work is absolutely essential.

In order to facilitate a safe relocation of the database a 1.5 hour outage is required to TMS. The impact of this to Spine is significant and results in effectively an outage for Spine and its interfaces to connecting systems for that period. The time and date is aimed at the lowest times of utilisation for Spine, to minimise impact to end users, as well as not impacting critical batch processing and Choose & Book slot polls.

 

Date & Time

Change Start Change Finish Services Affected Outage Duration
15/01/2013 22:00 15/01/2013 23:30 Transaction Messaging Service (TMS) 1.5 hours
Service  Impact Description
Choose and Book The Choose and Book service will be available but functionality will be limited until the TMS database has switched over.Users of the web application will experience limited retrievals during the outage window.The system will not be able to create shared-secret for patients who have not been referred via Choose and Book before.Service Providers will be unable to:

  • Perform clinic re-structures and re-arrange appointments for patients for directly bookable services
  • Send DNA messages to Choose and Book.

For directly bookable services the following functionality will be unavailable:

  • Booking appointments
  • Rearranging appointments
  • Creating new patient accounts

Choose & Book systems will need to queue the messages and resend to Spine once the TMS service is enabled.

Due to the timing of the outage slot polls will not be affected.
Summary Care Record application (SCRa) The SCRa application will be available but functionality will be limited until the TMS database has switched over. Simple traces can be completed on PDS data but users will be unable to perform any PSIS updates (e.g. GP summary updates)
DSA The DSA application will be available but functionality will be limited until the TMS database has switched over.Simple traces can be completed on PDS data but users will be unable to perform any PSIS updates (e.g. GP summary updates).
Electronic Prescription Service (EPS)Pharmacy Systems Reliable messaging will be unavailable for the duration of the switchover work as the TMS service will be suspended dual site. All messages received from EPS systems will be rejected and not go into retry.EPS systems will need to queue the messages and resend to Spine once the TMS service is enabled.
EPS Batch The PPA response for any “claim” messages will not be sent to PPA/PPD. However, EPS will send those response(s) again when the retry jobs are re-activated after the switchover exercise is over. Response for any “claim” messages will not be received until after the switchover. Retry jobs will resend the responses once the TMS service is enabled.
Existing Service Providers (ESPs) There will be varying impacts depending on the product, release version and Spine compliant modules of the solution.ESP systems will need to queue the messages and resend to Spine once the TMS service is enabled.
GP2GP GP2GP will be unavailable until the TMS database has switched over.GP2GP systems will need to queue the messages and resend to Spine once the TMS service is enabled.
GP Extraction Service (GPES) GPES functionality will be unavailable until the TMS database has switched over.Messages will be queued on Spine and processed once the TMS service is restored.
GP Systems Functionality for Choose & Book, EPS and GP2GP, SCR will be limited until the TMS database has switched over.For Choose & Book directly bookable services the following functionality will be unavailable:

  • Booking appointments
  • Rearranging appointments
  • Creating new patient accounts

Systems will need to queue the messages and resend to Spine once the TMS service is enabled.
iPM/Lorenzo The real-time connection to Spine will be unavailable during the TMS outage. However both systems can be disconnected from Spine and operate without synchronised PDS data.iPM/Lorenzo will need to queue the messages and resend to Spine once the TMS service is enabled.
Millennium An outage of PDS reliable messaging will impact Millennium users.Users will be unable to:

  • trace patients
  • register new patients on PDS
  • book or reschedule appointments

Millennium will need to queue the messages and resend to Spine once the TMS service is enabled.
NN4B Trusts will need to be aware that during the outage NHS numbers cannot be generated, new-births cannot be registered and blood-spot labels cannot be generated and should plan accordingly.All birth notifications will be queued and processed once the TMS service is enabled.
Personal Demographics Service (PDS) Simple traces can be completed on PDS data.PDS reliable messaging will be unavailable until the TMS database has switched over.
RiO Users will be unable to:

  • trace patients
  • register new patients
  • book or reschedule appointments

The RiO system will need to queue the messages and resend to Spine once the TMS service is enabled.
TMS Event Service (TES) The majority of TES functionality will be unavailable during the outage.Trusts will need to be aware EPS, Death notifications, and Patient Care Provision Notifications (change of pharmacy) will be queued and sent to the receiving systems once the TMS service is restored.Any impacted notifications will be queued and sent to the receiving systems once TMS is restored.
TMS Batch (DBS, CHRIS, ONS) DBS will be unavailable until the TMS database has switched over (DBS processing will be suspended for the duration of the exercise).As the TMS switchover will be scheduled to start at 22:00, CHRIS batch should complete before the outage starts (CHRIS batch runs at 20:00 nightly).ONS processing will start at 18:00 nightly. If it doesn’t complete before 22:00, the messages will be queued and processed once the TMS service is restored.

**

Severity 2
BT Spine
HSCIC
National
Users are unable to grant worklist items within UIM.
USER IMPACT:
This is causing delays to routine business processes as users are unable to complete their worklist items within the UIM application.
ACTION BEING TAKEN:
BT investigating.

**

Severity 1
BT Spine
HSCIC
National
The EPS database is currently experiencing severe degradation of performance.
USER IMPACT:
Delays to routine business processes.
ACTION BEING TAKEN:
BT engineers currently investigating with the database application support team.

Comment

David Nicholson is right. The NHS has become dependent on systems such as the Spine. But can doctors ever trust any aspect of the safety of patients to systems that are not available 24×7 as they need to be in a national health service?

It appears that BT and other suppliers have not been in breach of service level agreements, and the HSCIC has a good relationship with the companies.  But does the HSCIC have too great an interest in not finding fault with its suppliers or the contracts, for finding fault  could draw attention to any defects in a service for which the HSCIC is responsible?

Have national NHS IT suppliers a strong enough commercial or reputational interest  in avoiding  a disruption or loss of service, so long as they keep within their service level agreements? 

If nobody sees anything wrong with the reliability of existing national NHS IT services improvements are unlikely. Diane Vaughan’s book on the culture and organisation of NASA shows that experts in a big organisation can do everything right according to the rules  and procedures – and still have a disastrous outcome.

1 Comment

Posted in Atos, BT, Campaign4Change, culture, Department of Health, e-health, Government IT, IT projects, LPfIT, NHS, npfit, Summary Care Record, supplier relationships

Tagged

MPs to question ex-BBC DG Mark Thompson on DMI failure

Posted on 28/01/2014 | Leave a comment

By Tony Collins

The Public Accounts Committee plans next month to question the BBC’s former Director General Mark Thompson on the failure of the Digital Media Initiative. He left the BBC in 2012 and became CEO of the New York Times Company.

Margaret Hodge, chairman of the committee, said:

“I look forward to discussing this [DMI] with senior, and former senior, members of the BBC, such as former Director General Mark Thompson, when they come before us on Monday 3rd February at 3.15pm.”

Hodge said today’s National Audit Office memorandum on DMI “reads like a catalogue of how not to run a major programme”. She said she was shocked to learn how poor were the BBC’s governance arrangements for DMI.  “There was no Senior Responsible Owner with complete oversight of all aspects of programme’s delivery.”

The report clearly demonstrates why regular reviews are necessary – and why external reviewers should be listened to, said Hodge.

“If the BBC had established clearer accountability and stronger reporting it could have recognised the issues much earlier and set about minimising the astronomic losses for the licence fee payer.

“These failures go right to the top. The executive board applied insufficient scrutiny during 2011 and the first half of 2012. The programme was not subject to any audit or assurance reporting between early 2011 until July 2012.

“The BBC Trust had questioned the executive about slippages in September 2011 but then applied limited challenge until July 2012. After the BBC executive board became aware of the problems it initiated a review in May 2012 of the DMI timetable, costs and benefits.

“The BBC Trust finance committee did not know until July 2012 that the DMI’s risk rating had increased to red for the period October to December 2011.

“The BBC needs to learn from the mistakes it made and ensure that it never again spends such a huge amount of licence fee payers’ money with almost nothing to show for it.”

BBC’s Digital Media Initiative – another fine mess and predictable as well.

NAO memorandum on the DMI

Leave a comment

Posted in Campaign4Change

Did NPfIT flag-bearing trust lose track of 2,000 patients?

Posted on 23/01/2014 | 1 comment

By Tony Collins

Barnet and Chase Farm Hospitals NHS Trust is one of the most experienced Cerner Millennium electronic patient record user sites in England. It started using Cerner in 1999 for clinical documents, electronic orders and results, pathology, radiology and theatres.

In  2007 the trust became one of the first reference sites for the use of Cerner Millennium  within the National Programme for IT.

With BT’s help, and what the trust calls “full” funding, Barnet and Chase Farm implemented Millennium to support A&E, track patient notes and provide a master patient index. 

Millennium has also been used to support the production of waiting lists.  So why is the trust, with its exceptional IT experience, given a special mention today by the National Audit Office for having technology problems with its 18-week waiting lists?

Trust loses track of 108 patients waiting more than a year

In a report published today on 18-week waiting times – the right of patients to receive consultant-led care within 18 weeks of being referred for treatment, usually by their GP –  the NAO says:

“In May 2013, Barnet and Chase Farm Hospitals NHS Trust identified a failure to monitor more than 2,000 patients on the waiting list, of which 651 had waited between 18 and 51 weeks.

“The backlog developed as a result of a failure in an IT reporting system. At the time of the report to the board in May, there were no patients identified as waiting more than 52 weeks for treatment.

“However, further validation of the waiting list in July by trust staff identified 108 patients that had not been tracked and who had waited more than 52 weeks for treatment; these patients were subsequently reported to NHS England.

“In December 2013, Barnet and Chase Farm Hospitals NHS Trust had 181 patients who had waited more than 52 weeks for treatment…”

Responding to the NAO, officials at Barnet and Chase Farm said they had validated waiting times data, including for example: checks for missing services; changes in waiting list volumes; checks for missing clock starts; and reviewing a sample of patient files against recorded data.

Clinical harm?

But the full facts have not emerged. Two months ago the NHS Barnet CCG Board Governing Board Meeting decided to discuss in private Barnet and Chase Farm’s problems, which are not confined to meeting the 18-week list targets. The discussion went into the meeting’s  “part two” section.

In the open part of the meeting, it was reported to Board members that the Trust had reported a serious incident relating to a breach of access to 18 week pathways.

“The backlog position was 2613, patients waiting between 18 to 51 weeks with a
further 214 patients waiting in excess of 52 weeks. A Clinical Harm Steering Group has been set up by the Trust to review all patients affected by the RTT [referral to treatment] backlog.

“This group is being chaired by Henrietta Hughes, NHSE London region Medical Director who will be advising on the assessment of clinical harm…”

Board directors of Barnet and Chase Farm have discussed the 18-week waiting list problems – but in private.

Comment

Keeping track of patients once the 18-week countdown starts is difficult for any trust and Barnet and Chase Farm is not the only trust mentioned by the NAO as having problems.

It’s not always clear in what circumstances – and when – the clock starts on the 18-week countdown.  The clock may be paused, for example if the patient cannot make an appointment because of a work commitment; and patients may not attend appointments for other reasons.

What makes Barnet and Chase Farm different is its long IT-related experience, and the financial and technical support it has had from BT and the Department of Health under the National Programme for IT and later the London Programme for IT. Keeping track of patients requires the trust to be in full control of its IT systems.

Excessive secrecy

Several trusts implementing patient record systems within the NPfIT have had protracted problems keeping track of patients on their waiting lists – Barts and North Bristol for instance. The details are rarely made public – and the problems recur.

Barnet and Chase Farm has held its discussions on the 18-week problems in private, as has the Barnet CCG. This flies in the face of the coalition’s promise of a duty of candour when things go wrong; and if officials know they can keep their IT-related problems secret they can adopt risky implementations without accountability.

The high-risk implementations continue – at Croydon Health Services NHS Trust, for example, where patients and the public have not been given the full story on what has gone wrong after the implementation  of an NPfIT patient record system.

When safety is at risk – which it can be when a trust loses track of patients on their systems – the NHS has a duty to explain. At least so there is a chance lessons will be learned.

NAO report – NHS waiting times for elective care in England.

1 Comment

Posted in Campaign4Change