Category Archives: project management

Lessons from an IT disaster

Posted on 23/07/2012 | Leave a comment

By Tony Collins

Only rarely is an independent report on an IT-related disaster published.  So North Bristol NHS Trust deserves credit for publishing a report  by Pricewaterhousecoopers into the problematic go-live of Cerner Millennium in December 2011.  PwC calls the Cerner system a “business-critical patient record system”.

The implementation, says PwC,  resulted in significant continuing  operational difficulty. PwC was asked to review the implementation, identify what went wrong and make recommendations.

What is clear from PWC’s report is that North Bristol NHS Trust repeated the known mistakes of other trusts that had gone live with Cerner Millennium:

–          A lack of independent challenge

–          Not enough testing of the system and new business processes

–          Inadequate contingency arrangements

–          Not enough time for data migration

–          Training systems not the same as those to be used

–          Preparations treated as an IT project, not a change programme.

–          Differences between legacy and Cerner systems not fully understood before go live

–          Staff did not always understand new or changed business processes

In 2007 the National Audit Office reported in detail on the lessons from the go-live of Cerner Millennium at Nuffield Orthopaedic Centre, Oxford in December 2005.

One of those lessons was that the Trust did not learn lessons from earlier NPfIT Cerner Millennium go-lives. This happened again at North Bristol, suggests the PwC report:

“There were not dissimilar Cerner implementations within the Greenfield [other ex-Fujitsu and now BT-managed Cerner Millennium implementations under the NPfIT] systems running a few months before NBT’s [North Bristol Trust] implementation. Similar difficulties were experienced there, but they were more successfully addressed.”

Below are extracts from PwC’s report “Independent review of Cerner Millennium implementation North Bristol NHS Trust”.

“The success of an implementation of this scale, complexity and timing depends on substantial, robust and enduring programme management focusing on:

–          The IT implementation. Incorporating configuration of Cerner Millennium, infrastructure, security, interfaces and testing;

–          The migration of data from the two legacy PAS systems into Cerner Millennium;

–          Change management to engage and train stakeholders, embed change in the organisation and ensure that processes and procedures are aligned to the new system;

–          Continuous communication with users about changes to business processes as a result of the implementation; and

–          Quality control criteria and the association governance to ensure that go-live went ahead in a safe and sustainable manner.

–          The Trust needed stringent programme management with programme and project managers of the highest quality, to ensure that effective governance and project planning procedures were followed.

–          The go-live decision and assurances needed to pass strict criteria with sufficient evidence to provide assurance to the board that all necessary activities were completed prior to go-live.

The implementation in both the wards and the Emergency Department (ED) went well. Staff in ED were well engaged in the project and as a result were fully aware of the changes to their business processes at go live. There were some minor system issues initially but these were resolved quickly and ED was fully operational with Cerner Millennium soon after go live. One of the underlying factors in the success of the deployment to ED was that there was no data migration required as the historical data remains in the old system.

The launch in the wards went as expected; the functionality was tested well and the data was loaded manually, although there now appear to be issues with staff engaging and using the system as intended.

The majority of problems encountered at go live related to the theatre and outpatient clinic builds.

Outpatients had the most disruption immediately after go live. The Trust’s back office team had not finished building the outpatient clinics in Cerner Millennium, so the new and old systems did not mirror each other and data could not successfully migrate. Changes continued to be made to clinics in the old PAS systems, and these were not all reflected in Cerner Millennium.

Ad hoc clinics were used in the old PAS system to allow overbooking to maximise activity. These were not separated from real clinics at go live and migrated to Cerner Millennium as real clinics. The ad hoc clinics in PAS had deliberately abnormal timings so they could be excluded from time-based reports, for example 12:30am and 5:30am. The system generated letters for these ad hoc out- of-hours clinics, and many were sent to patients.

In the old system, clinics for a number of consultants could be pooled to facilitate patients seeing the next available consultant.  All clinics in Cerner Millennium are specific to a consultant and this caused significant confusion to administration staff using the new system.

PAS [the legacy patient administration system] treats “weeks” differently to Cerner Millennium. On migration, weeks were misaligned and the dates for clinics and theatres was incorrect. This created huge confusion as patient notes did not agree with Cerner Millennium , despite exhaustive work before go live to ensure that all patient notes were ready for the clinics that should have been on the system.  This also affected information in letters, with patients advised to attend their appointment on the wrong date.

There was a further issue in theatres relating to theatre procedure codes. The Trust did not map the old procedure codes to the new to ensure that all the required procedures would be available in Cerner Millennium for the data to migrate successfully. The Trust identified this issue soon after go live and has run a parallel manual process to ensure patients received the correct procedures.

The training provided to staff by the Trust did not equip them to be able to use Cerner Millennium at go live. The training environment did not mirror the system the Trust implemented as certain elements of the system were not complete when the training domain was created. Theatre staff and outpatient appointments could not train on a system with theatre schedules and outpatient clinics built in.

The Trust is now beginning to move out of the crisis and return to normal operations.

Lack of effective quality controls

There was insufficient rigour over the controls criteria and sign off of the gateway reviews.

There was inadequate operational control over the go live process, such as clinic freeze and updates pre-, during, and post go-live. Evidence from the interviews suggests that:

  • There was little challenge to confirm that the gateway criteria had in fact been met.
  • There was no evidence presented to the Cerner Programme Board or the Trust Board to demonstrate that the gateway criteria had been met.
  • There was not enough focus on or monitoring of risks and issues and their impact on go live.
  • The cleansing of old and out-of-date data from the legacy PAS systems was inadequate; as a result, erroneous data became live data in the Cerner system.
  • Data Migration issues were not all resolved and their impact on go live was not considered.
  • The outpatient and theatre builds were neither complete nor accurate, and there were no controls which could have detected this before go live.
  • There were inadequate controls over clinic freeze and clinic changes prior to go live.

Lack of effective programme planning

Programme plans were not rigorously updated as the programme progressed and planning around training, testing and data migration and build was not robust. The Trust failed to recognise this programme as a change programme and did not effectively manage the engagement and feedback from their stakeholders. Evidence from the interviews suggests that:

  • The Trust did not factor contingency into its programme plan to account for changes to the go live date.
  • The Cerner Programme Management Office was not effective because of inadequate resource and programme tools.
  • The Trust had a lack of sufficiently skilled resources for a project on this scale.
  • The Trust’s operational staff were not fully engaged in the Cerner project.
  • The Cerner project was treated as an IT project and not a business change programme.
  • The training was inadequate and did not provide users with the skills they needed to be able to use the system at go live.
  • The testing focused on the functionality of the system and not end-user testing of the outpatient and theatre builds.
  • There was no end-user testing of the final outpatient clinic and theatre builds prior to go live.
  • There was lack of understanding of roles within the wider programme team.
  • External parties offered NBT help and advice. They felt that the advice was not taken and the help was refused.

Lack of effective programme governance

Programme governance processes were not reviewed and updated regularly to ensure that they were adequate and there was inappropriate accountability for key decision making. During the implementation, the Trust established new overarching change management arrangements for the Building our Future programme. Evidence from the interviews suggests that:

  • The Cerner Project team failed to comply with the Trust’s Building our Future governance processes
  • The information presented to the Cerner Programme Board and the Trust board by the Cerner Project team was inadequate for them to make informed decisions;
  • The Cerner Programme Board was not effective; and
  • Significant issues relating to the theatre and outpatient clinic build were not escalated to the Cerner Programme Board or the Trust board.

PwC’s Conclusions

For a programme of this scale and complexity, the management arrangements were not sufficiently extensive or robust. There were many issues with the software and data migration, the training of users and operational go live planning. The Trust Board and the Cerner Programme Board did not plan to have, and did not receive, independent assurance that the state of the programme supported a decision to go-live.

Complex IT implementations are never without risks and issues that need to be managed, even at the point of go live. The scale of the issues in this implementation was not properly understood by those with responsibility, and as a result they were not in a position to make sound decisions.

Many of the problems are associated with poor data and process migration. Staff found that a significant proportion of migrated data was incorrect in the new system, and this had rapid and substantial operational impact which has taken a considerable time to rectify with manual processes. Staff needed to be more directly involved in migration and process testing.

The implementation was manifestly a complex change programme. But IT took the lead, and there was no intelligent customer with sufficient distance from IT to ensure products and progress were properly challenged.

There were not dissimilar Cerner implementations within the Greenfield running a few months before NBT implementation. Similar difficulties were experienced there, but they were more successfully addressed.”

PwC recommends that:

–  the Trust “stop and take stock”. It says  “The Trust needs to take stock of its position and develop a coherent and detailed plan for the remainder of the recovery stage. The Trust then needs to ensure that effective cross programme planning and governance arrangements are enforced for all current projects, especially those under the Building Our Future programme.”

PwC also recommends that the Trust carry out a:

–  Governance review

– Capability/capacity review

– Cross programme plan review

– Operational assessment

– Review of process and controls

– Review of information requirement

– Technical resilience/infrastructure review

– Review of access controls

Comment:

To me the PwC report throws up at least six points:

1) Are NPfIT go-lives more political than pragmatic?

In the 1990s Barclays Bank went live with new systems for all its branches. During the night (I was invited to watch the go-live at head office) the most striking element was a check list that asked questions on progress so far. The answers determined whether the go-live would happen. The check-list was completed repeatedly – seemingly endlessly – during the night.

Many  different types of mishaps could have stopped the go-live.  None did.  Go-lives of Cerner Millennium are different. They seem unstoppable, whatever the circumstances, whatever the problems.  There was nothing political about the Barclays go-live. But NPfIT go-lives are intensely political.

Would North Bristol’s board have accepted with equanimity a last-minute cancellation, especially after go-lives had been postponed at least twice before?

2)  Are NHS boards too focused on “good news” to oversee an NPfIT go live?

North Bristol NHS Trust deserves praise for publishing the PwC report.  But it’s not the whole story.  The report says little about any potentially serious impact on patients. Also it mentions (almost in passing) that the Trust board discussed in November 2011 the readiness of Cerner Millennium to go live. That discussion was probably positive because Millennium went live a month later. But there is no mention of that discussion in the Trust’s board papers for November 2011.

Why did the Trust discuss its readiness to go live in secret? And why did it keep secret its November 2011 report on its readiness to go live?

If North Bristol, like so many NHS trusts, is congenitally beset with a good news culture at board level, can the full truth ever be properly discussed?

3) Isn’t it time Cerner lessons were learnt?

After seven years of Cerner implementations in the NHS, several of them notorious failures, isn’t it time Trusts learnt the lessons?

4)  What’s the current position?

PwC’s report is succinct and professional. It’s also diplomatically-worded. There is little in the report that points to how the Trust is coping with the operational difficulties. Indeed it suggests the Trust is returning to normal. “The Trust is now beginning to move out of the crisis and return to normal operations,” says the PwC report. But that is, in essence, what the Trust has been saying publicly since January 2012.  PwC says nothing about whether the safety of patients has been jeopardized by the go-live.

5) Where were the Trust’s Audit Committee – and internal auditors?

Every NHS Trust has an audit committee and internal auditors to warn about things that are going wrong, or may go wrong. It appears that they were out to lunch when it came to North Bristol’s Cerner Millennium project and its consequences.  The Audit Committee seems hardly to have mentioned the project. Should North Bristol’s board hold the Audit Committee and internal auditors to account?

6) Is the Trust board to blame?

Perhaps rightly PwC does not seek to apportion blame. But did the Trust board ask the right questions often enough?  The tacit criticism in the PwC report is of the IT department and layers of management below board level. But is that criticism misdirected? If the board’s culture of encouraging good news – of “bring me solutions not problems” –  has not changed, perhaps little or nothing will have been learned from North Bristol’s IT-related disaster.

PWC report Independent review of Cerner Millennium implementation North Bristol NHS Trust.

Lessons from Nuffield Orthopaedic’s Cerner Millennium implementation in 2005.

North Bristol apologises over Cerner go-live.

New hospital system caused chaos.

MP asks why two Cerner systems cost vastly different prices.

Leave a comment

Posted in Campaign4Change, culture, e-health, Gateway reviews, governance, Government IT, health, health records, IT projects, IT-related failures, NHS, npfit, openness and transparency, operational issues, project management, public sector, public services

Tagged ,

All change for police IT – again?

Posted on 18/07/2012 | 4 comments

By Tony Collins

Police IT is supposed to have undergone a transformation over the past 13 years, thanks in part to a Home Office national police IT programme called NSPIS – for which Securicor Information Systems was awarded contracts worth more than £140m.

NSPIS contracts awarded in 1999 included:

– Case preparation: acquisition and delivery of forms, photographs, police reports, statements and other materials required in court for trying cases.

– Custody: booking in, tracking and monitoring of individuals held in police cells.

– Command and control: coordination and management of police operations.

– Crime: analysis of case histories and crime statistics.

With some reluctance, dozens of police forces took NSPIS systems with mixed success. The national transformation did not happen, though large sums were spent. NSPIS [National Strategy for Police Information Systems] was followed by another national IT-led transformation programme ISIS [Information Systems Improvement Strategy].

Now the government plans another police IT-led transformation. It is setting up a new company to improve police IT [as if the last so-called transformation programmes had not existed].

In a joint statement, the Home Office and the Association of Police Authorities say the new company will give strategic ICT advice to forces and procure, implement and manage ICT solutions for forces.

The company will “help police forces to improve their information technology and get better value for money from contracts”.

The police ICT company Ltd is now owned by the Association of Police Authorities and the Home Office but will be handed over to police and crime commissioners following elections in November.

In setting up the company Nick Herbert, the policing minister, says

“While some police IT is good, such as the new Police National Database, much of it is not.  There are 2,000 systems between the 43 forces of England and Wales, and individual forces have not always driven the most effective deals.

“We need a new, more collaborative approach and greater accountability, utilising expertise in IT procurement and freeing police officers to focus on fighting crime.

“By harnessing the purchasing power of police forces, the new company will be able to drive down costs, save taxpayers’ money, and help to improve police and potentially wider criminal justice IT systems in future.”

Chairman of the Association of Police Authorities Councillor Mark Burns-Williamson says that when the new company is handed over to police and crime commissioners “we want it to be fit for purpose and efficient in delivering IT tasks”.

The aim of the new company, says the Home Office and the Association of Police Authorities, is to “free chief officers from in-depth involvement in ICT management and enable greater innovation so officers have access to new technology to save time and ensure better value for the taxpayer”.

Police IT in a poor state?

UKAuthority.com reports that Tom Winsor, the new chief inspector of police, is “staggered” at the ineffectiveness of police IT.

Giving evidence to MPs he said

“I was staggered when I did my field work, in the police pay review, at just how low-tech the technology of the police is in volume crime and so on. It is extraordinary. They have computer screens that resemble those that we saw in the early 1980s. I mentioned the police officers doing their own two-finger typing and so on.

“It is the most extraordinarily archaic system. I think it is part of HMIC‘s role to expose inefficiency – and that surely is massively inefficient.”

Winsor said he had watched police officers standing in a queue for up to four hours at a time to book in a suspect. The private sector would not tolerate such delays, and would quickly change the system, he said.

Comment

With 43 forces buying their own IT it seemed sensible for the Home Office to try and introduce national systems.  As Neil Howell, the then IT Director at Hampshire Police Authority, said in March 2006, there was “political pressure to take up some systems – e.g. NSPIS Case and Custody ” but some national systems did not “match current level of functionality or requirements …”

In the NHS, several national IT-led transformation programmes preceded the NPfIT, but nobody in power wanted to know about the past when NPfIT was launched in 2003.

An extraordinary effort – and money – went into NSPIS  but police forces resented being told what to buy and in general were happy with own IT choices. Many were particularly happy with NSPIS rival systems from Canadian company Niche.

Perhaps the Home Office should accept that, apart from natural national systems such as the  Police National Database, Automated Numberplate Recognition, and the “Impact” intelligence sharing system, police IT is too complicated to be done nationally.

Mandating rarely works

Mandation rarely if ever works in the public sector. The Home Office and its agents cannot tell 43 autonomous police forces what technology to buy and implement.  Public bodies can, and do, circumvent mandation, sometimes by simply ignoring it, as National Audit Office reports point out.

The Department of Health  tried to tell trusts what to buy under the NPfIT and that didn’t work. Like police forces NHS trusts are largely autonomous.

Governments don’t have memories when it comes to failed IT-led transformation programmes. It may be good for civil servants and suppliers to learn new skills and experiment with IT on recycled transformation programmes.

But should suppliers learn at the expense of taxpayers? And should new ministers and civil servants keep launching new and exciting IT-led transformation programmes that fail as miserably as the last – giving excuses for a replacement set of ministers and civil servants to renew the cycle?

The Department of Health has finally learnt that what’s needed before the launch of any major  IT-led initiative is a frank appraisal of what has gone wrong in the past, and what can be learnt.  The DH achieves this in the “Impact Assessment” section of its latest IT strategy.  It’s not beyond the wit of police forces, the Home Office and the Association of Police Authorities to follow the DH’s example.

Unless they do, perhaps David Pitchford’s Major Projects Authority at the Cabinet Office should think twice before allowing large sums to be spent on new police IT.

Joint statement of Home Office and Association of Police Authorities

4 Comments

Posted in outsourcing, Politics, private sector lessons for the public sector, procurement, project management, public sector, public services, public-private partnerships, reducing cost strategically

Tagged , ,

Poor IT suppliers to face ban from contracts?

Posted on 28/06/2012 | Leave a comment

By Tony Collins

The Cabinet Office minister Francis Maude is due to meet representatives of suppliers today, including  Accenture BT,Capgemini, Capita, HP, IBM, Interserve, Logica, Serco, and Steria.

They will be warned that suppliers with poor performance may find it more difficult to secure new work with the Government. The Cabinet Office says that formal information on a supplier’s performance will be available and will be taken into consideration at the start of and during the procurement process (pre-contract).

Maude will tell them that the Government is strengthening its supplier management by monitoring suppliers’ performance for the Crown as a whole.

“I want Whitehall procurement to become as sharp as the best businesses”, says Maude. “Today I will tell companies that we won’t tolerate poor performance and that to work with us you will have to offer the best value for money.”

The suppliers at today’s meeting represent around £15bn worth of central government contract spend.

The representatives will also be:

– asked their reactions on the government’s approach to business over the past two years

– briefed on the expanded Cabinet Office team of negotiators (Crown Representatives) from the private and public sectors. Maude says these negotiators aim to maximise the Government’s bulk buying power to obtain strategic discounts for taxpayers and end the days of lengthy and inflexible contracts.

Spending controls made permanent

Maude is announcing today that cross-Whitehall spending controls will be a permanent way of life. The Government introduced in 2010 temporary controls on spending in areas such as ICT  and consultancy. It claims £3.75bn of cash savings in 2010/11, and efficiency savings for 2011/12, which it says are being audited.

The Cabinet Office says: “By creating an overall picture of where the money is going, the controls allow government to act strategically in a way it never could before. For example, strict controls on ICT expenditure do not just reduce costs but also reveal the software, hardware and services that departments are buying and whether there is a competitive mix of suppliers and software standards across government.”

Maude said: “Our cross-Whitehall controls on spending have made billions of cash savings for the taxpayer – something that has never been done before. That’s why I’m pleased to confirm that our controls will be a permanent feature, helping to change fundamentally the way government operates.”

Leave a comment

Posted in cost reduction, efficiency savings, Government IT, IT-related failures, operational issues, outsourcing, oversight, Politics, private sector lessons for the public sector, procurement, project management, public sector, software quality, supplier relationships

Tagged , ,

Why is MoD spending more on IT when its data is poor?

Posted on 28/06/2012 | Leave a comment

By Tony Collins

The Ministry of Defence and the three services have spent many hundreds of millions of pounds on logistics IT systems over the past 20 years, and new IT projects are planned.

But the National Audit Office, in a report published today – Managing the defence investory –  found that logistics data is so unreliable and limited that it has hampered its investigations into stock levels.

“During the course of our study,” says the NAO, “the Department provided data for our analyses from a number of its inventory systems. However, problems in obtaining reliable information have limited the scope of our analysis…”

The NAO does not ask the question of why the MoD is spending money on more IT while data is unreliable and there are gaps in the information collected.

But the NAO does question whether new IT will solve the MoD’s information problems.

“The Department has acknowledged the information and information systems gaps and committed significant funds to system improvements. However these will not address the risk of failure across all of the inventory systems nor resolve the information shortfall.”

MPs on the Public Accounts Committee, who will question defence staff on the NAO report, may wish to ask why the MoD’s is so apparently anxious to hand money to IT suppliers when data is poor and new technology will not plug information gaps.

Comment:

MPs on the Public Accounts Committee found in 2003 (Progress in reducing stocks) that the MoD was buying and storing stock it did not need. Indeed after two major fires at the MoD’s warehouses at Donnington in 1983 and 1988 more than half of the destroyed stock did not need replacing. Not much has changed judging by the NAO’s latest report.

It’s clear that the MoD lacks good management information. Says the NAO in today’s report:

“The summary management and financial information on inventory that is provided to senior staff within Defence Equipment and Support is not sufficient for them to challenge and hold to account the project teams…”

But will throwing money at IT suppliers make much difference? The MoD plans the:

–  Future Logistics Information Services project, which is intended to bring together and replace a number of legacy inventory management systems; and

–  Management of the Joint Deployed Inventory system which will provide the armed services with a common system for the inventory they hold and manage.

But is the  MoD using IT spending as proof of its conviction to improve the quality of data and the management of its inventory?

Managing the defence inventory

Leave a comment

Posted in cost reduction, Defence, Government IT, IT projects, IT-related failures, oversight, private sector lessons for the public sector, project management, public sector, public services, software quality, supplier relationships

Tagged ,

Whitehall to relent on secrecy over mega projects – after 10-year campaign?

Posted on 29/05/2012 | 1 comment

By Tony Collins

The Cabinet Office may be about to change its decade-old policy of not publishing reports on  the progress or otherwise of its large, costly and risky IT-based projects.

A change of policy from secrecy to openness would give MPs and the public warning of when a major project is in trouble and needs rescuing or cancelling.

Parliament last to know

For more than a decade campaigners have sought to persuade successive governments to publish “Gateway” reviews, which are short independent audits on the state of big projects.  The secrecy has meant that Parliament is usually the last to know when new national schemes go wrong. IT-related failures have hit many public services including those related to tax, benefits, immigration, passports, the fire service, prisons, schools examinations, student loans, the police and health services.

Now Sir Bob Kerslake, head of the civil service, has hinted to campaigning Conservative MP Richard Bacon that the Cabinet Office may change its policy and publish the “red, amber, green” status of large projects as they are routinely assessed.

Kerslake was replying to Bacon at a hearing of the Public Accounts Committee meeting on transparency. Bacon pointed out at the hearing that the Public Accounts Committee had, years ago, called for Gateway reviews to be published.

Not learning from mistakes

“Something I have always been puzzled by is why government does not learn from its mistakes particularly but not only in the area of IT where things go wrong again and again, again and again,” said Bacon. “I have come to the conclusion government does not learn from its mistakes because it does not have a learning curve. If you don’t have a learning curve you are not going to learn.”

He cited the example of how Ian Watmore, Permanent Secretary at the Cabinet Office, had, at Bacon’s request, arranged for an “Opening Gate” report on Universal Credit to be published in the House of Commons library.

But, said Bacon, when an IT journalist applied to the Department for Work and Pensions, under the FOI Act, for the release of all Gateway reports on Universal Credit, the DWP would not publish any of them  – and even refused an FOI request to release the report Watmore had arranged to be placed in the House of Commons library, which Bacon obtained.  “So there is still a culture of intuitive, instinctive secrecy,”  Bacon said. Kerslake replied:

“Yes, actually we are looking at this specific issue as part of the Civil Service Reform Plan….I cannot say exactly what will be in the plan because we have not finalised it yet, but it is due in June and my expectation is that I am very sympathetic to publication of the RAG [red, amber, green] ratings.”

Bacon pointed out that the Cabinet Office Structural Reform Plan Monthly Implementation Updates had said Gateway reviews would be published.  But the commitment was removed for no apparent reason. When the Cabinet Office was asked why,  it said the Structural Reform Plans were only ever “drafts”.

Bacon asked Kerslake if the Government now plans to publish the Gateway reports.  “The Cabinet Office Structural Reform Plan Monthly Implementation Updates originally said Gateway reviews would be published  and then it somehow got downgraded into a draft; and from what’s publicly available at the moment the position of the government is not to publish Gateway reviews.  You sound as if you’re saying that’s going to change. Is that right?” asked Bacon.

“Watch this space,” replied Kerslake. “I am sympathetic. I generally broadly welcome, in principle, the idea of publishing information but there are lots of risks …”

Peter Gershon introduced Gateway reviews when he was Chief Executive of the Office of Government Commerce, which is now part of the Cabinet Office’s Efficiency and Reform Group. The reviews are carried out at key decision times in a project and are sometimes repeated:

  • Gateway Review 0 – Strategic assessment
  • Gateway Review 1 – Business justification
  • Gateway Review 2 – Procurement strategy
  • Gateway Review 3 – Investment decision
  • Gateway Review 4 – Readiness for service
  • Gateway Review 5 – Benefits realisation

Are Gateway reviews a success?

Gateway reviews are now supplemented by regular assurance audits carried out for the Cabinet Office’s Major Projects Authority. None of the reports is published.

Gateway reviews have not stopped costly failures such as Firecontrol or the NPfIT.  One permanent secretary told an MP that the reviews in his department were considered unimportant by senior responsible owners, for whom the reports are written. This may be because SROs often have charge of many projects; and even their SRO responsibilities are often in addition to their main jobs.

But Gershon had high hopes of Gateway reviews when they were introduced in February 2001. This is evident from the number of times he referred to Gateway reviews at one hearing of the Public Accounts Committee in December 2001.

 “… as the Gateway review process cuts in, which I have referred to on a number of occasions when I have appeared at this Committee …”

“… Through things like the Gateway process we are helping to sharpen the focus on the whole life aspects of these and other forms of complex projects in public sector procurement…”

“ …First, we have the introduction of the Gateway review process…”

“ … The Gateway process is a demonstrable example of how we have introduced a technique to support that whole life approach…”

“… If you look at the guidelines around the Gateway review process that is one of the things that is tested by these independent reviews …”

“… we recognised that that was a problem some time ago, which is why in the Gateway review one of the things that is explicitly tested is things like the skills and capabilities of the team at the design and build stage and that the skills and capabilities of the team at the procurement stage …”

“… in this area with the Gateway review process, from when we first launched it last February, we have been helping the department take a whole life approach to these forms of complex projects …”

“… Part of the Gateway Review process is to get a much sharper insight on to where we see good things happening where we can encourage other clients to replicate them…”

“… Now, with the Gateway Review process, my experience has been because of where we have deliberately focused the attention on the early life of projects where there is the greater scope for management to take corrective action, the accounting officers are paying a lot of attention to the recommendations that are emerging because, much to my surprise, most of them do not seem to like coming here defending what has gone wrong in the past. They seem to welcome the recommendations that we are providing to them to help try to get projects on to much stronger foundations in the future…”

“… With the Gateway Review, my experience has been that the Accounting Officers respond to the recommendations very positively…”

“…Gateway Reviews explicitly test how the department is planning in the pre-contract phase to secure ongoing value for money in the post-contract phase…”

“… Take, firstly, the Gateway Review process. That is testing various points in the life cycle of the project, from the very earliest stage…”

“… I would certainly expect in Gateway Reviews that the review team would be testing what methods were in place to facilitate the ongoing management of the contract…”

“… I think it is encouraging that Sir Ian Byatt thought the Gateway Review process had sufficient value to recommend it in his own review…”

And so forth.

Comment:

We applaud Richard Bacon MP for his persistent call for Gateway reports to be published.

Gateway reviews have defeated expectations that they would stop failures; and the National Audit Office tells us that central departments don’t even request Gateway reviews on some big and risky projects although they are supposed to be mandatory.

But Gateway and other project assurance reports could prove invaluable if they are published. In the public domain the reports would enable Parliament and Francis Maude’s “armchair auditors” to hold officials and SROs to account for projects that are in danger of failing. That would be an incentive for officialdom to fail early and fail cheaply; and Gateway reviewers may take greater care to be neutral in their findings – not too lenient, or too harsh – on the basis that the reports would be open to public scrutiny. SROs would also have to take the review reports seriously – not just put them in a draw because nobody knows about them anyway.

We welcome Kerslake’s comments but hope that he and his colleagues plan to publish more than the RAG (Red/Amber/Green) status of projects. Otherwise they will be missing an opportunity.  Gateway reports and other assurance reviews are expensive. Reviewers can earn up to £1,000 a day. This money  could be well spent if the reviews are to be published; but it will add to public waste if the reports are kept secret and continue to be deemed pointless or unimportant by departments.

It is ironic, incidentally, that the Ministry of Justice, which introduced the FOI Act, gives advice to departments to keep the RAG status of Gateway reviews confidential. In its advice on Gateway reviews and the FOI, the MoJ tells departments that the “working assumption” is that the substance of the Gateway reports should be kept confidential until at least two years have elapsed.

It’s time for a culture change. Maybe the Civil Service Reform Plan next month will be worth reading.

1 Comment

Posted in change management, cost reduction, culture, excessive secrecy, Freedom of Information, Gateway reviews, Government IT, IT-related failures, Major Projects Authority, managing change, NHS, npfit, openness and transparency, oversight, Politics, project management, public sector, public services, Universal Credit

Tagged , , ,

Queensland audit brings in new broom to sweep away problem IT projects

Posted on 28/05/2012 | Leave a comment
There is nothing like an IT project disaster to spur the arrival of a new broom designed to ensure that it will never happen again.
 
Until next time.
 
According to the Queensland Courier-Mail in Australia, every computer system used across Queensland’s public service will be probed for flaws and inefficiencies under a A$5.2m audit set up to head off another costly IT project.
 
It follows the emergence of problems with a Queensland Health health payroll project which is now being audited after it was revealed that the costs required to put the project right would increase to over $400 million.
 
Newly appointed Queensland IT Minister Ros Bates ordered the audit to uncover how IT is being used across the government’s 19 departments and find where savings can be made.
 
The audit team, which comprises a seconded army of 32 public servants, will present its findings by the end of October.
 
In addition to the audit, as part of a push to achieve government IT efficiencies, departmental chief information officers have been stripped of their autonomy and will now report to the Queensland Government CIO.
 
Meanwhile, despite its payroll project problems, Queensland Health has won an excellence in eGovernment award for a project dubbed ‘The Viewer’ that has streamlined how clinicians access patient information about their patients.
 
The Viewer is a read-only web-based application that sources key patient information from existing Queensland Health systems, providing consolidated information in one place.
 
Before the project’s  implementation, patient data was stored in a range of paper and electronic record systems in over 260 different facilities.
 

Leave a comment

Posted in Australia, Campaign4Change, e-health, Government IT, IT projects, IT-related failures, project management

Tagged , , , , , , ,

Fire ‘superstations’ without software cost £1m a month – The Times

Posted on 24/05/2012 | Leave a comment

By Tony Collins

The Times reports today that taxpayers are paying more than £1m a month on the rent and upkeep of fire control rooms across England that have never been used. The purpose-built control centres look ready for immediate use, with open-plan desks fitted with desktop monitors and keyboards, and huge screens on a wall at the front of the control rooms which are supposed to help fire and rescue crews mobilise appliances and manage incidents.

Only there’s no working software.  The Department for Communities and Local Government negotiated the end of a contract with the main contractor EADS for software to run the regional control centres in December 2010. Officials concluded that the software could not be delivered within an acceptable timeframe. The regional control centres were completed before the IT project was cancelled.

The cost of the centres has been uncovered after a request under the FOI Act. The Times devotes much of its page three to a story under the headline:

Revealed: scandal of the £1m-a-month fire service ‘superstations’ lying empty.

Only one of nine regional centres is in use. The other eight incur rent, electricity, water and repair costs at £1,134,566 a month. Costs will be incurred for years because there are no break clauses in the agreements to lease the buildings. Two leases come to an end in 2027, one in 2028, two in 2032, three in 2033 and one in 2035.

A spokesman for the Department said that agreement has been reached for a further two of the buildings to be used by local fire authorities. Officials are searching for public or private sector tenants to occupy the other regional centres.

Lord Prescott, the former Deputy Prime Minister, who authorised the start of the technology project in 2004,  said he had been kept in the dark by civil servants on the rising costs of the scheme. He said it had been on budget when he left the department in 2007.

Eric Pickles, the Communities Secretary, said the failure of Firecontrol was an “expensive reminder of why you can’t trust Labour to run anything”. But the Coalition’s coming to power has not stopped central government IT-related failures.

Why Firecontrol failed

Firecontrol  followed the same tracks to a cliff edge that have caught out civil servants, ministers and suppliers on other government  computer-related projects.

The National Audit Office and the Public Accounts Committee found that  the Firecontrol project was rushed, had little support from those who would use it, costs and complexity were underestimated, there was an over-reliance on consultants and a lack of accountability for decisions made  – or not made.

The idea was to replace 46 local control rooms with nine, linked regional centres, which would be equipped with new standardised computer systems to handle calls, mobilise equipment and manage incidents.

But the project was cancelled in December 2010 with ministers unsure the technology would ever work. The NAO estimates that £469m will be wasted on the project.

The NAO found that the scheme was “flawed from the outset”, largely because local fire and rescue officers did not want regional centres or major changes in the way they worked.  Introducing any large new system is difficult but with enthusiastic support serious problems can sometimes be overcome; but introducing a complex new system without support from those who would use it means staff will have little incentive to find ways around problems.

The NPfIT [National Programme for IT in the NHS] failed in part because it lacked support among GPs and NHS staff; and the complexity of introducing standardised technology in semi-autonomous hospitals – each one with different ways of working – was underestimated. It was the same with Firecontrol.

The complexity of introducing standardised systems in regional centres with no goodwill among staff – was underestimated.  From the start many local fire and rescue officers criticised the lack of clarity on how a regional approach would increase efficiency. “Early on, the Department’s inconsistent messages about the regionalisation of the Fire and Rescue Service led to mistrust and some antagonism,” said the NAO.

The technology project was rushed while local fire crews were excluded from project discussions. “The project progressed too fast without essential checks being completed. For example Departmental and Treasury approval was given without proper scrutiny of the project’s feasibility or validation of the estimated costs and savings,” said the Public Accounts Committee. The project went ahead before the full business case was written.

A review of the project as early as April 2004 found that the scheme was already in poor condition overall and at significant risk of failing to deliver. But the “Gateway” review report was kept secret for seven years.

Is the stage set for IT disasters in government to continue? So far the Coalition has decided, like Labour, to keep secret all internal reports on the progress or otherwise of its mega projects, including Universal Credit, though the policy on secrecy may be about to change, which Campaign4Change will report on separately.

Firecontrol – same mistakes repeated on other projects.

Leave a comment

Posted in Campaign4Change, change management, consultants, culture, excessive secrecy, Freedom of Information, Gateway reviews, Government IT, IT projects, IT-related failures, local government, managing change, npfit, openness and transparency, Politics, project management, public sector, public services

Tagged , ,

The real reason NHS Risk Register is a State secret?

Posted on 16/05/2012 | 2 comments

By Tony Collins

Yesterday  (15 May 2012) the Information Commissioner Christopher Graham issued a finely-crafted report to Parliament on his concerns about the Government’s use of a ministerial veto to stop publication of the Transition Risk Register relating to health service reforms.

Graham’s concern is that the veto represents a new and worrying approach to Freedom of Information.

Graham cannot do anything about the veto but he can warn MPs when he feels the Government has gone too far. This he has done in his report which says that the previous three occasions on which the ministerial veto has been exercised related to the disclosure of Cabinet material under FOIA. Now the Government has applied the veto to information held by the Department of Health.

Says yesterday’s report: “ The Commissioner would wish to record his concern that the exercise of the veto in this case extends its use into other areas of the policy process. It represents a departure from the position adopted in the Statement of Policy and therefore marks a significant step in the Government’s approach to freedom of information.”

The Government’s decision to ban publication of the health service risk register is particularly relevant to IT-related projects. This is because the government uses exactly the same arguments to ban contemporaneous publication of Gateway reviews and other independent assessments of IT-related projects and programmes.

Risk registers and Gateway reviews of IT-enabled change projects are similar. They are designed to identify all the main risks associated with the project or programme and have a red/amber/green system of rating the risks.

The Government’s argues that risk registers (and Gateway reviews) are researched and written in a “safe space” that allows civil servants to give advice and recommendations in a frank way. This candour would be compromised if the civil servants thought their advice would be published, says the Government.

In issuing a veto on the health risk register Andrew Lansley, the Health Secretary said, in essence, that he could not trust civil servants to be entirely honest if they knew their reports would be made public.

Said Lansley:  “If risk registers are routinely or regularly disclosed at highly sensitive times in relation to highly sensitive issues, or there is legitimate concern that they could be, it is highly likely that the form and content will change: to make the content more anodyne; to strip out controversial issues or downplay them; to include argument as to why risks might be worth taking; to water down the RAG [red,amber, green] system.

“They would be drafted as public facing documents designed to manage the public perception of risk; not as frank internal working tools. These consequences (many of them insidious) would be to the detriment of good government.”

Lansley also wanted to ban publication to pre-empt sensational media coverage.  In this he was repeating the arguments made by civil servants under Labour who refused, under the FOI Act, to publish risk registers and Gateway reviews.  Said Lansley “I consider that the form and the frankness of the content of TRR [health service Transition Risk Register] would have been liable to create sensationalised reporting and debate.

“The content would also have been inherently highly open to misinterpretation by both the press seeking a headline and/or for political reasons. The likelihood of this occurring is particularly acute where the subject matter is, as with the Transition programme, controversial and the proposals at a highly sensitive stage.”

But the Commissioner did not accept that disclosure of the Transition Risk Register would affect the frankness and candour of future risk registers. The Commissioner also said that a ministerial veto should, by law, be made only in exceptional circumstances.  But the Government has failed to explain why there are exceptional circumstances in this case.  Said the Commissioner:

“The Commissioner does not consider that sufficient reasons have been given as to why this case is considered to be exceptional, particularly in light of the [Information] Tribunal’s decision dismissing the Department’s [Department of Health’s] appeal.

“The Commissioner notes that much of the argument advanced as to why the case is considered to be exceptional merely repeats the arguments previously made to Commissioner and the Tribunal and which were in part dismissed by the Tribunal.”

Graham concludes:

“In light of previous commitments he has made, and the interest shown by past Select Committees in the use of the ministerial veto, the Commissioner intends to lay a report before Parliament under section 49(2) FOIA on each occasion that the veto is exercised. This document fulfils that commitment.

“ Laying this report is an indication of the Commissioner’s concern to ensure that the exercise of the veto does not go unnoticed by Parliament and, it is hoped, will serve to underline the Commissioner’s view that the exercise of the ministerial veto in any future case should be genuinely exceptional…

“The arguments employed by the Department at the Tribunal and by the Secretary of State in explanation of the subsequent veto, both in the Statement of Reasons and in exchanges in the House of Commons around the Ministerial Statement, certainly use the language of ‘exceptional circumstances’ and ‘matter of principle’. But the arguments are deployed in support of what is in fact the direct opposite of the exceptional – a generally less qualified, and therefore more predictable, ‘safe space’.

“As such, the Government’s approach in this matter appears to have most to do with how the law might be changed to apply differently in future. This question falls naturally to consideration by the Justice Committee who have been undertaking post-legislative scrutiny of the Act.”

Comment:

The reason for the veto in the case of the health service risk register has little to do with protecting a safe space for frank discussion.

Civil servants already compile risk registers, Gateway reviews and similar reports on the basis that they may, at some point, be published. Officials are no more likely to be frank if they know their reports will be confidential than more guarded if they know the documents will be published. They will do what their job entails. Their job requires honesty. They will do that job whether or not reports are published.

The real reason for the veto – and the refusal of departments to publish all contemporaneous internal reports on large and complex programmes, particularly those with a large IT element – is that some new schemes within Government operate at a shambolic level.

Any new government, whatever its hue, soon learns to keep secret the fact that such programmes are sometimes characterised by near anarchy.

One outsider to the UK government, Australian David Pitchford, discovered the truth when he became Executive Director of Major Projects within the Efficiency and Reform Group which is part of the Cabinet Office. Pitchford may not have realised his comments would be reported when he told a project management conference in 2010 that “nobody in the UK Government seems to know how many projects they have on the books, nor how much these are likely to cost”.

He found that projects were launched, and continued, without agreed budgets or business cases.  Today, there is better scrutiny of major projects, by the Cabinet Office’s Major Projects Authority. But the MPA is limited in what it can do or scrutinise. Which leaves government in a general mess when it comes to implementing anything new.

Evidence for this mess comes from the National Audit Office. Its auditors tend to investigate departments as a whole more than they do specific projects but when they do the careful reader can see that projects such as the Rural Payments Agency’s Single Payment Scheme (a scant regard for public funds, said the NAO) and the C-NOMIS project for the prison service (kindergarten mistakes, said chair of Public Accounts Committee) were without a structure. Chaos prevailed – and ministers were among the last to know.

Publication of project reports encourages professionalism. Departmental heads can be held to account if Parliament knows what has gone wrong. That’s precisely the reason departmental heads don’t want risk registers and other project reports published. It’s why all internal reports on Universal Credit, the government’s biggest IT-related project, are kept secret in spite of FOI requests.

The ministerial veto in the case of the NHS risk register is the government and civil service colluding in keeping the public and Parliament in ignorance of internal management’s inability to run complex new projects and programmes in a professional way.

Ministers and permanent secretaries don’t especially mind media criticisms that are based on speculation. They don’t want their critics having authoritative internal reports. That’s why the Cabinet agreed the health service veto – and it’s one reason the government has a not-very-hidden aversion to the FOI Act.

The coalition cannot justly claim to cherish open government while it is refusing so many requests under FOI to publish contemporaneous taxpayer-funded reports on its major schemes.

We agree with the Information Commissioner that use of the ministerial veto is a step too far. No number of announcements by the Cabinet Office on open government will gloss over the fact that the coalition is even more secretive about mega-projects than Labour. That’s saying something.

2 Comments

Posted in Campaign4Change, Freedom of Information, Gateway reviews, Government IT, IT projects, IT-related failures, Major Projects Authority, NHS, operational issues, Politics, private sector lessons for the public sector, project management, public sector, public services, Universal Credit

Tagged , ,

The story behind India’s struggling Aakash IT project

Posted on 07/05/2012 | Leave a comment

By David Bicknell

The New York Times has carried a couple of excellent blog posts reporting on India’s struggling “Aakash” IT project.

The India Ink posts detail the story behind a plan to introduce a cheap computer built for Indian students. As the blog explains, last October, the Indian Ministry of Human Resources Development unveiled the new, $35 computer.

Now, more than six months later, with thousands of university students still waiting for the laptop, “the tale of the Aakash looks a bit like an Indian soap opera, complete with a convoluted storyline, multiple characters, and massive personality clashes.”

As India Ink says, the Aakash project, if successfully completed, could enable millions of students to connect with the larger digital world, and is being closely watched outside India as the national government tries to attract foreign investment in public-private partnerships for everything from infrastructure to vocational training.

“The original idea behind the Aakash seemed pleasantly simple. A cheap computer would benefit Indian university students by enabling them to watch lectures or get lecture notes and other class information online. In 2009, a team of government researchers developed the basic design for the low cost device.

“The job of putting the project out to bid fell to I.I.T. Rajasthan, which by spring of 2011 had received 477 million rupees — about $9.2 million — in government funds to pay for procuring and testing 100,000 low-cost tablets. In writing the tender, I.I.T. Rajasthan detailed the technical specifications for the tablet but did not specify the criteria for testing and approving the devices, according to a government source involved in the project. That omission was to prove disastrous.

Here is Part One of the tangled tale of the project, which involves issues with procurement, outsourcing, testing and governance.

And here is Part Two.

India’s $35 Aakash tablet comes apart
Aakash Tablet Problems: India’s $35 slate slammed by testers

Leave a comment

Posted in Campaign4Change, change management, governance, Government IT, India, outsourcing, procurement, project management, public-private partnerships

Tagged , , , , , , , , , ,

Whitehall defies NAO and Cameron on publishing status of big projects

Posted on 02/05/2012 | 1 comment

By Tony Collins

Government action to cut the number of failures of big projects including those with a major ICT component has made a difference, the National Audit Office reports today.

In its report “Assurance for major projects”  the NAO is largely supportive of actions by the Government, , the Cabinet Office’s Major Projects Authority and the Treasury in setting up reviews of major high-risk projects, including ICT-based programmes, to ensure that if they are failing they are put back on track or cancelled.

The NAO says the Government’s decision to “dismantle” the NPfIT was taken after the project was assessed by the Major Projects Authority.

But the report also shows how civil servants have managed to defy a mandate from the Prime Minister, and a separate NAO recommendation in 2010, for information on the status of big ICT and other high-risk projects to be published.

Says the NAO report

“The ambition to publish project information, as part of the government’s transparency agenda, has not been met.

“Our 2010 report recommended that the government should publicly report project status. We consider that public reporting of project information is key to providing greater accountability for projects and improving project outcomes… Regular transparent reporting of performance which highlights successes and non-compliance would also help to build an enduring assurance system.”

Separately in the report the NAO says

“There has been a lack of progress on transparency.  The [Cabinet Office’s Major Projects] Authority has not yet met its commitment to publish project information in line with government’s transparency agenda. The Authority cannot deliver this objective on its own. Senior level discussions are ongoing, between Cabinet Office, HM Treasury and departments, on the arrangements for public reporting.”

Should ministers intevene to force publication?

But the NAO report does not raise the question of why ministers have not intervened to force civil servants to publish the status information on high-risk projects.

Campaign4Changehas argued that publishing status reports on big ICT projects and programmes would be the most effective single action any government could take to reduce the number of failures. (see “Comment” below)

Prime Minister’s 2011 mandate

The NAO’s 2010 recommendation for status information on major projects to be published was backed by a mandate from the Prime Minister in January 2011 which included the undertaking to “require publication of project information consistent with the Coalition’s transparency agenda”.

The House of Commons’ Public Accounts Committee has recommended that departments publish information on the state of their major IT-based projects and programmes; and the Information Commissioner has rejected civil service arguments for not publishing such information.

In addition Francis Maude, the Cabinet Office minister, said, when in opposition, that the Conservatives, if they gained power,  would publish “Gateway” review reports soon after they are completed.  Gateway reports are similar to the assurance reviews carried out for the Major Projects Authority.

Yet none of this has happened.

The “rebel” civil servants

How is it that a group of civil servants who are opposed to publishing information on the status of large risky projects can defy the Prime Minister, Francis Maude, the National Audit Office, and the all-party Public Accounts Committee? Those recalcitrant civil servants argue that assurance reviewers would not tell the whole truth if they knew their assessments would be made public.

But how do we know they tell the whole truth when the reports are kept confidential? The Information Commissioner has pointed out in the past that civil servants have a public duty to be candid and honest. If they are not because their reports are to be published, they are failing in their public duty.

Today’s NAO report says there are differences of opinion among civil servants over whether to publish status information on projects.

Says the NAO

“There has been some support for greater transparency from departments who believe that tracking and publishing major milestones could create helpful tension in the system.

“However, concerns have been raised that increased transparency could limit the value of assurance, as it could inhibit assurance reviewers and project staff holding full and frank discussions.

“Some senior project staff also have concerns that public reporting could have a negative commercial impact, and would prefer delayed rather than real-time public reporting.”

The Cabinet Office told Campaign4Change in 2010 and 2011 that instead of publishing status reports on each major project, it will publish an annual report on the state of its programmes.

But that hasn’t happened either.

Says the NAO:

“As well as the objective to publish project information, the [Major Projects] Authority has not yet met its objective to publish an annual report on government’s major projects.

“The Authority initially expected to publish an annual report in December 2011 but is now expecting the report to be published in May or June 2012. The format of the annual report, and the information it will contain, has yet to be decided.”

Comment:

Many times over the last 20 years I have said that publishing status reports on major IT-based projects and programmes would be the most effective single action any government could take to deter departments from going ahead with overly ambitious schemes that are doomed to fail. If, against good sense, impractical schemes are approved, publishing status information will make all the difference.

Permanent secretaries will not lose sleep over a failing project, but they will not want information on it published – which is why that information should be published.

Publishing status information would give civil servants a good reason to tackle weaknesses as they developed.  Permanent secretaries may not mind losing public money on a failing project or programme. They will always fear embarrassment, however.

Who is really in control of Whitehall – civil servants or No 10? David Cameron’s office has issued a mandate that requires status information on projects to be published. The NAO has issued a similar recommendation. How long can the civil service hold out against the political will?

Links:

NAO report – Assurance for major projects

Firecontrol – same mistakes repeated on other projects

1 Comment

Posted in cost reduction, culture, excessive secrecy, Freedom of Information, Gateway reviews, Government IT, IT projects, IT-related failures, Major Projects Authority, openness and transparency, operational issues, private sector lessons for the public sector, project management, public sector

Tagged , , , ,