Category Archives: change

How to cost-justify the NPfIT disaster – forecast benefits a decade away

Posted on June 7, 2013 | 2 Comments

By Tony Collins

To Jeremy Hunt, the Health Secretary, the NPfIT was a failure. In an interview with the FT, reported on 2 June 2013, Hunt said of the NPfIT

“It was a huge disaster . . . It was a project that was so huge in its conception but it got more and more specified and over-specified and in the end became impossible to deliver … But we musn’t let that blind us to the opportunities of technology and I think one of my jobs as health secretary is to say, look, we must learn from that and move on but we must not be scared of technology as a result.”

Now Hunt has a different approach.  “I’m not signing any big contracts from behind [my] desk; I am encouraging hospitals and clinical commissioning groups and GP practices to make their own investments in technology at the grassroots level.”

Hunt’s indictment of the NPfIT has never been accepted by some senior officials at the DH, particularly the outgoing chief executive of the NHS Sir David Nicholson. Indeed the DH is now making strenuous attempts to cost justify the NPfIT, in part by forecasting benefits for aspects of the programme to 2024.

The DH has not published its statement which attempts to cost justify the NPfIT. But the National Audit Office yesterday published its analysis of the unpublished DH statement. The NAO’s analysis “Review of the final benefits statement for programmes previously managed under the National Programme for IT in the NHS” is written for the Public Accounts Committee which meets next week to question officials on the NPfIT. 

A 22 year programme?

When Tony Blair gave the NPfIT a provisional go-ahead at a meeting in Downing Street in 2002, the programme was due to last less than three years. It was due to finish by the time of the general election of 2005. Now the NPfIT  turns out to be a programme lasting up to 22 years.

Yesterday’s NAO report says the end-of-life of the North, Midlands and East of England part of the NPfIT is 2024. Says the NAO

“There is, however, very considerable uncertainty around whether the forecast benefits will be realised, not least because the end-of-life dates for the various systems extend many years into the future, to 2024 in the case of the North, Midlands and East Programme for IT.”

The DH puts the benefits of the NPfIT at £3.7bn to March 2012 – against costs of £7.3bn to March 2012.

Never mind: the DH has estimated the forecast benefits to the end-of-life of the systems at £10.7bn. This is against forecast costs of £9.8bn to the end-of-life of the systems.

The forecast end-of-life dates are between 2016 and 2024. The estimated costs of the NPfIT do not include any settlement with Fujitsu over its £700m claim against NHS Connecting for Health. The forecast costs (and potential benefits) also exclude the patient administration system Lorenzo because of uncertainties over the CSC contract.

The NAO’s auditors raise their eyebrows at forecasting of benefits so far into the future. Says the NAO report

“It is clear there is very considerable uncertainty around the benefits figures reported in the benefits statement. This arises largely because most of the benefits relate to future periods and have not yet been realised. Overall £7bn (65 per cent) of the total estimated benefits are forecast to arise after March 2012, and the proportion varies considerably across the individual programmes depending on their maturity.

“For three programmes, nearly all (98 per cent) of the total estimated benefits were still to be realised at March 2012, and for a fourth programme 86 per cent of benefits remained to be realised.

There are considerable potential risks to the realisation of future benefits, for example systems may not be deployed as planned, meaning that benefits may be realised later than expected or may not be realised at all…”

NPfIT is not dead

The report also reveals that the DH considers the NPfIT to be far from dead. Says the NAO

“From April 2013, the Department [of Health] appointed a full-time senior responsible owner accountable for the delivery of the [the NPfIT] local service provider contracts for care records systems in London, the South and the North, Midlands and East, and for planning and managing the major change programme that will result from these contracts ending.

“The senior responsible owner is supported by a local service provider programme director in the Health and Social Care Information Centre.

“In addition, from April 2013, chief executives of NHS trusts and NHS foundation trusts became responsible for the realisation and reporting of benefits on the ground. They will also be responsible for developing local business cases for the procurement of replacement systems ready for when the local service provider contracts end.”

The NAO has allowed the DH to include as a benefit of the NPfIT parts of the programme that were not included in the original programme such as PACS x-ray systems.

Officials have also assumed as a benefit quicker diagnosis from the Summary Care Record and text reminders using NHSmail which the DH says reduces the number of people who did not attend their appointment by between 30 and 50 per cent.

Comment

One of the most remarkable things about the NPfIT is the way benefits have always been – and still are – referred to in the future tense. Since the NPfIT was announced in 2002, numerous ministerial statements, DH press releases and conference announcements have all referred to what will happen with the NPfIT.

Back in June 2002, the document that launched the NPfIT, Delivering 21st Century IT for the NHS, said:

“We will quickly develop the infrastructure …”

“In 2002/03 we will seek to accelerate the pace of development …

“Phase 1 – April 2003 to December 2005 …Full National Health Record Service implemented, and accessible nationally for out of hours reference.”

In terms of the language used little has changed. Yesterday’s NAO report is evidence that the DH is still saying that the bulk of the benefits will come in future.

Next week (12 June) NHS chief Sir David Nicholson is due to appear before the Public Accounts Committee to answer questions on the NPfIT. One thing is not in doubt: he will not concede that the programme has been a failure.

Neither will he concede that a fraction of the £7.3bn spent on the programme up to March 2012 would have been needed to join up existing health records for the untold benefit of patients, especially those with complex and long-term conditions.

Isn’t it time MPs called the DH to account for living in cloud cuckoo land? Perhaps those at the DH who are still predicting the benefits of the NPfIT into the distant future should be named.

They might just as well have predicted, with no less credibility, that in 2022 the bulk of the NPfIT’s benefits would be delivered by the Flower Fairies.

It is a nonsense that the DH is permitted to waste time on this latest cost justification of the NPfIT. Indeed it is a continued waste of money for chief executives of NHS trusts and NHS foundation trusts to have been made responsible, as of April 2013, for reporting the benefits of the NPfIT.

Jeremy Hunt sums up the NPfIT when he says it has been a huge disaster. It is the UK’s biggest-ever IT disaster. Why does officialdom not accept this?

Instead of wasting more money on delving into the haystack for benefits of the NPfIT, it would be more sensible to allocate money and people to spreading the word within Whitehall and to the wider public sector on the losses of the NPfIT and the lessons that must be learnt to discourage any future administrations from embarking on a multi-billion pound folly.

→ 2 Comments

Posted in Atos, BT, BT Health, Campaign4Change, change, change management, CSC, e-health, excessive secrecy, Fujitsu, health records, IT projects, IT-related failures, legal disputes, Major Projects Authority, managing change, managing outsourced services, NHS, npfit, public sector, public services, Summary Care Record

Tagged

Francis Maude boasts of £10bn savings but …

Posted on June 3, 2013 | Leave a comment

By Tony Collins 

This morning Cabinet Office minister Francis Maude held a press conference with his senior officials to announce that civil servants have radically changed the way they work to save £10bn in 2012/13.

The savings are nearly £2bn higher than originally planned and, according to the Cabinet Office, have been “reviewed and verified” by independent auditors.

With a little journalistic licence Maude says: “…we are on the way to managing our finances like the best-run FTSE100 businesses.”

The breakdown of the £10bn savings:

Procurement   £3.8bn
Centralisation of procurement for common goods and services  £1.0bn
Centrally renegotiating large government contracts  £0.8bn
Limiting expenditure on marketing and advertising, consultants and temporary agency staff   £1.9bn
Transformation savings   £1.1bn
IT spend controls and moving government services and transactions onto digital platforms  £0.5bn
Optimising the government’s property portfolio  £0.6bn
Project savings   £1.7bn
Reviewing performance of major government projects  £1.2bn
Taking waste out of the construction process  £0.4bn
Workforce savings   £3.4bn
Reducing the size of the Civil Service   £2.2bn
Increasing contributions to public sector pensions   £1.1bn

Comment

It’s good news and the figures don’t seem plucked out of thin air which sometimes happens when central government announces savings.

The big question is whether the savings are sustainable. Maude has inspired the Cabinet Office’s Efficiency and Reform Group to be motivated and hard-working. But bringing about long-term change in Whitehall – as opposed to restricting consultancy contracts and cutting annual costs of supplier contracts by reducing what’s delivered – is like peddling uphill. How long can you do it without losing motivation and energy? It’s not just parts of the civil service that are resistant to the savings agenda – it is also some IT suppliers, according to Government Computing.

It’s likely that only profound changes in central government operations and working practices will outlast the next general election. At the moment the civil service is like a rubber band that has been stretched a little. It wants to return to its standard shape, which the next government may allow it to do.

The National Audit Office said in its report in April 2012 on the Efficiency and Reform Group in 2011/12:

“Savings to date have differing degrees of sustainability.”

The NAO also said this:

“It is not fully clear how ERG intends to make the reforms necessary to secure enough savings over the rest of the spending review. ERG has yet to translate its ambition for saving £20 billion by 2014-15 into more detailed plans.

“ERG has made progress in developing strategies across its wide range of responsibilities, and is focusing on core activities likely to produce savings. However, until recently ERG’s focus has mainly been on the savings themselves, with less emphasis on delivery of the longer-term changes and improvement in efficiency necessary to make them sustainable.”

And this:

“Departments have still tended to lack a clear strategic vision of what they are to do, what they are not, and the most cost-effective way of delivering it. Much of departments’ 2014-15 savings are likely to come from further reductions in staff. Sustainability of these savings will depend on developing skills and working in new ways while maintaining staff motivation and engagement.”

But the NAO was generally positive about the ERG’s contribution to savings.

“ERG’s actions to date, particularly its spending controls, have helped departments deliver substantial spending reductions.”

We hope the Cabinet Office’s diligent efforts continue  – sustainably.

Efficiency and Reform 2012/13 savings. Summary report.

Some suppliers still resistant to change? – Government Computing.

→ Leave a comment

Posted in Campaign4Change, capacity building, change, change management, Cloud, Cloud Computing, consultants, cost reduction, culture, G-Cloud, Government IT, Green IT, IT projects, Major Projects Authority, managing change, procurement, project management, public sector, public services, reducing cost strategically

Tagged , , , , ,

Is Major Projects annual report truly ground-breaking?

Posted on May 29, 2013 | 2 Comments

By Tony Collins

Francis Maude, the Cabinet Office minister, describes as “nothing short of groundbreaking” a report of the Major Projects Authority which gives the RAG (Rred/Amber/Green) status of more than 100 major projects.

That the report came out late on Friday afternoon as most journalists were preparing to go home, some of them for the whole bank holiday weekend, suggests that the document was a negotiated compromise: it would be published but in such a way as to get minimal publicity.

Indeed the report is a series of compromises. It has the RAG status of projects but not the original text that puts the status into context.

Another compromise: senior civil servants in departments have persuaded Maude to publish the RAG decisions when they are at least six months old.

This enables departmental officials to argue their case in the “narrative” section of the MPA annual report that a red or amber/red decision is out-of-date and that there has been significant improvement since. This is exactly the DWP’s justification for the amber/red status on Universal Credit.

The DWP says in the MPA report: “This rating [amber/red] dates back to September 2012, more than seven months ago. Since then, significant progress has been made in the delivery of Universal Credit. The Pathfinder was successfully launched and we are on course both to expand the Pathfinder in July 2013 and start the progressive national roll-out of Universal Credit in October.”

That the Pathfinder was launched successfully might have nothing to do with Universal Credit’s amber/red status which could be because of uncertainties over how the IT will perform at scale, given the complexities and interdependencies.  The MPA report says nothing about the uncertainties and risks of Universal Credit.

More compromises in the MPA annual report: the Cabinet Office appears to have allowed departments to hide their cost increases on projects such as HMRC’s Real-time Information [RTI] in the vague phrase “Total budgeted whole life costs (including non-government costs).”

The Cabinet Office has also allowed departments to write their own story to accompany the RAG status. So when HMRC writes its story on RTI it says that “costs have increased” but not by how much or why. We know from evidence that HMRC gave to the Public Accounts Committee that RTI costs have risen by “tens of millions of pounds”. There is nothing to indicate this in the MPA annual report.

Another compromise in the MPA annual report: there are no figures to compare the original forecast costs of a project with the projected costs now. There are only the 2012/13 figures compared with whole-life projected costs (including non-government projected spend).

And the MPA report is not comprehensive. It came out on the same day the BBC announced that it was scrapping its Digital Media Initiative which cost the public £98m. The MPA report does not mention the BBC.

The report is more helpful on the G-Cloud initiative, showing how cheap it is – about £500,000. But there is little information on the NHS National Programme for IT [NPfIT] or the Summary Care Record scheme. 

Yet the MPA annual report is ground-breaking. Since Peter Gershon, the then head of the Office of Government Commerce, introduced Gateway reviews of risky IT projects about 12 years ago with RAG decisions, they have remained unpublished, with few exceptions. The Cabinet Office is now publishing the RAG status of major departmental projects for the first time. Maude says

“A tradition of Whitehall secrecy is being overturned. And while previous Governments buried problems under the carpet, we are striving to be more open. By their very nature these works are high risk and innovative.

“They often break new ground and dwarf anything the private sector does in both scale and complexity. They will not always run to plan. Public scrutiny, however uncomfortable, will bring about improvement. Ending the lamentable record of failure to deliver these projects is our priority.”

Comment

The MPA annual report is a breath of fresh air.

Nearly every sentence, nearly every figure, represents compromise. The report reveals that the Universal Credit project was last year given an amber/red status – but it doesn’t say why. Yet the report has the DWP’s defence of the amber/red decision. So the MPA report has the departmental defences of the RAG decisions, without the prosecution evidence. That’s a civil service parody of openness and accountability: Sir Humphrey is allowed to defend himself in public without the case against him being heard.

But it’s still useful to know that Universal Credit is at amber/red.  It implies well into the project’s life that the uncertainties and risks are great. A major project at amber/red at this stage, a few months before go-live, is unlikely to turn green in the short term, if ever.

Congratulations

The Cabinet Office deserves congratulations for winning the fight for publication of the RAG status of each major project. Lord Browne, the government’s lead non-executive director and a member of the Cabinet Office’s Efficiency and Reform group, has said  that billions of pounds of taxpayers’ money is being frittered away because of “worryingly poor” management of government projects.

“Nobody ever stops or intervenes in a poor project soon enough. The temptation is always to ignore or underreport warning signs,” he says.

The management of some large projects – usually not the smaller ones – is so questionable that departments ignore advice to have one senior responsible owner per major project, says the MPA.

The MPA annual report will not stop the disasters. Its information is so limited that it will not even enable the public – armchair auditors – to hold departments to account. Senior civil servants have seen to that.

But the report’s publication is an important development: and it provides evidence of the struggle within Whitehall against openness. Francis Maude and Sir Bob Kerslake, head of the civil service, have had to fight to persuade departmental officials to allow the RAG status of projects to be published. The Guardian’s political editor Patrick Wintour says of the MPA annual report

“Publication led to fierce infighting in Whitehall as government departments disputed the listings and fought to prevent publication.”

Large-scale change

If Maude and Kerslake struggled to get this limited distance, and there is still so much left to reform, will large-scale change ever happen?

Maude and his officials have as comprehensive mandate for change from David Cameron as they could hope for. Yet still the Cabinet Office still seems to have little influence on departments. When it comes to the big decisions, Sir Humphrey and his senior officials hold onto real power. That’s largely because the departments are responsible to Parliament for their financial decisions – not the Cabinet Office.

Maude and his team have won an important battle in publishing the MPA annual report. But the war to bring about major change is still in its very early stages; and there’s a general election in 2015 that could halt Maude’s reform plans altogether.

The Major Projects Authority Annual Report.

→ 2 Comments

Posted in capacity building, change, change management, DWP, excessive secrecy, Gateway reviews, Government IT, HMRC, IT-related failures, Major Projects Authority, private sector lessons for the public sector, procurement, project management, public sector, public services, Universal Credit

Tagged , , , , , , ,

Does a Mid Staffs culture still pervade the NHS?

Posted on March 6, 2013 | 1 Comment

By Tony Collins

The Francis report on Mid Staffordshire NHS Foundation Trust highlighted appalling record-keeping among other problems.

One of the case studies in the report was that of an insulin-dependent diabetic, Gillian Astbury,  who entered Cannock Hospital for a urinary tract infection, had a fall in the hospital, was discharged, and later admitted to Stafford Hospital on 1 April 2007 because of bones she damaged in the fall. She died ten days later, probably after not being given insulin.

Francis highlights the lack of records on her need for insulin. There was a “failure to keep nursing records adequately or at all … there was a failure to comply with professional guidelines on note taking …”

Astbury’s partner Ron Street told hospital staff that she was diabetic, a point which went into her medical notes – initially.  But, said Francis,  nursing records for Astbury were almost non-existent.

“There is no evidence of what care took place … during interview nursing staff admitted that they did not check or read the notes regularly (if at all) and there was no linkage with notes from other wards …” 

Francis’s recommendations included a call for trust staff and managers to be open and accountable when things that go wrong.

This isn’t happening.

Campaign4Change picked an NHS trust to test whether the pre-Francis culture still prevails: whether there is the same old secrecy and defensiveness over standards of record-keeping, and whether positive news suffocates real and potential problems in trust board reports.

North Bristol NHS Trust

North Bristol NHS Trust has a chronic problem with record-keeping. It installed the Cerner Millennium electronic patient record system in December 2011, prompting a “crisis”.

Later the trust’s PR officer said in response to an FOI request that there had been 16 clinical incidents in two months relating to the new electronic patient record system. “These were all clinical incidents where the new system was cited as a causal factor, such as wrong patient wrong notes, lack of notes, incorrect clinic list,” she said.

She added:  “However our robust safeguarding processes, as well as additional checks and balances in all departments, ensured that clinical safety was not compromised and no patients were put at risk. Our priority is always patient safety and there is no indication that this has been affected.”

Last year North Bristol asked PWC to review the Cerner implementation. In its report PWC claimed that the “Trust is now beginning to move out of the crisis and return to normal operations”. That was in July 2012.

The Trust has still not returned to normal operations. Last month the Department of Health singled out North Bristol as one of only two trusts in England that failed to submit to the DH “incomplete RTT” pathway data. Incomplete pathway data refers to patients still waiting for consultant-led treatment. RTT means referral to treatment.

In August and September 2012 North Bristol was the only trust in England that failed to submit to the DH “incomplete RTT” pathway data.

Trust’s “numerous difficulties”

With little explanation, a North Bristol trust board paper in January this year referred to numerous difficulties relating to IT systems. This was in the context of an increasing number of overdue responses to complaints from patients. Said the board paper:

“Difficulties with appointment bookings and notification letters are still numerous. These are all reported to IM&T.” Again with little explanation another North Bristol board report, in November 2012, referred to “ongoing pressure in Cerner recovery …”.

So what are the Cerner problems, why have they continued for more than a year and has the North Bristol Trust’s board of directors been properly informed about them?

To test North Bristol’s openness on its Cerner problems I asked the Trust’s press officer and its media relations manager whether they could send me any trust report on the problems with the Cerner implementation.

Two days later they said that “some patience would be appreciated” but declined to say when they would respond to my question, so I asked it under FOI. The Trust gave no acknowledgement.

Perhaps North Bristol is too busy to deal with external questions and challenges on its record keeping. But that was one of the big problems highlighted by Francis in his report on Mid Staffs: that the Trust did not respond to external questions and challenges.

Worryingly, North Bristol’s reporting culture seems to prefer the positive over the negative.  This was one of its replies to an FOI request in 2012:

“With respect to inpatients, during November (before the implementation of Cerner) 40 patients were cancelled on the same day as admission for non-clinical reasons. During December (after the implementation of Cerner) 33 patients were cancelled on the same day as admission for non-clinical reasons – 7 fewer than in November.”

This reply – and others  - gave the impression, without giving contextual evidence,  that things were better since the Cerner implementation than before.

Francis in his report on Mid Staffs said,

“… for all the fine words printed and spoken about candour, and willingness to remedy wrongs, there lurks within the system an institutional instinct which, under pressure, will prefer concealment, formulaic responses and avoidance of public criticism.”

This would, it seems, apply to North Bristol – and every one of the other NHS trusts that have had electronic patient record implementations go wrong.

Indeed it is unfair to pick on North Bristol. The positive tone of its board reports is standard practice for trust board reporting across the NHS in England.

Francis said the NHS needs to change. In his letter to Jeremy Hunt on his report, Francis referred to an “institutional culture which ascribed more weight to positive information about the service than to information capable of implying cause for concern”.

But can NHS boards change in the absence of compulsion?

Audits of trust board reports?

One thing Francis did not suggest was that trust boards should have their board reports audited independently for honesty and openness.  An audit would detect an overly buoyant tone that downplayed concerns.  “There were 5 serious falls in December an increase of 3 from November. There were 185 falls in December compared to 139 falls in November, which had the lowest number of falls in one month this year.”

This was from a North Bristol board report that gave no explanation of the five serious falls. But the report made the point that November (2012) had the lowest number of falls in one month this year. If you were among the five who’d had a serious fall in hospital – and in Gillian Astbury’s case a fall in Stafford Hospital led to her death – you would probably want the trust’s board to focus on an analysis of the five serious falls, rather than be told how good a month November was for falls.

Board reports are a window on the culture of a public sector organisation. In the NHS nobody in authority seems not to have noticed that an American corporate positivism pervades many NHS board reports.  It’s within this culture that needless deaths such as those at Mid Staffs went unnoticed.

Until NHS trust board reports become more business-like and deal with concerns and potentially serious problems as would a private sector board – instead of giving the impression that they are trying to celebrate so-called achievements – the Francis report may make little difference.

North Bristol’s apparent unwillingness to disclose any detail of its Cerner problems – perhaps to its own board – is to be expected; but that natural reluctance to disclose may be symptomatic of one of the NHS’s biggest problems. The unnecessary deaths at Mid Staffs will be for nothing if the NHS does not change in the light of the Francis report. Complacency, arrogance, a preoccupation with good news and a culture of downplaying or even trying to ignore bad news are the enemy. Unless a board approach of honesty and openness is independently audited and enforced, Francis’s recommendations may bring little lasting change.

→ 1 Comment

Posted in Campaign4Change, change, change management, culture, e-health, health records, NHS, openness and transparency, oversight, Politics, private sector lessons for the public sector

Frustrated with the system – Govt CIOs, executive directors, change agents

Posted on January 14, 2013 | 2 Comments

By Tony Collins

Today The Times reports, in a series of articles, of tensions in Whitehall between ministers and an “unwilling civil service” over the pace of change.

It says a “permanent cold war” is being conducted with the utmost courtesy. It refers to Downing Street’s lack of control.

In one of the Times articles, Sir Antony Jay, co-creator of the “Yes Minister” TV series, writes that the civil service is more prepared to cut corners than in the 1970s  but hasn’t really changed. “If a civil servant from the 1970s came back today they would probably slot in pretty easily,” says Jay.

Politicians want “eye-catching” change while civil servants “don’t want to be blamed for cock-ups”, he says.

Separately, Mike Bracken, Executive Director of Digital in the Cabinet Office, has suggested that a frustration with the system extends to CIOs, executive directors to corporate change agents.

Bracken created the Government Digital Service which is an exemplar of digital services.  His philosophy is it’s cheaper and better to build, rent or pull together a new product, or at least a minimum feasible product, than go through the “twin horrors of an elongated policy process followed by a long procurement”.

Bracken has the eye of an outsider looking in. Before joining the Cabinet Office in July 2011 he was Director of Digital Development at the Guardian.

Bracken’s blog gives an account of his 18 months in office and why it is so hard to effect change within departments. I’ve summarised his blog in the following bullet points, at the risk of oversimplifying his messages:

Collective frustration

-  After joining the Cabinet Office in 2011 Bracken made a point of meeting senior officials who’d had exalted job titles, from CIOs and executive directors to corporate change agents. “While many of them banked some high-profile achievements, the collective reflection was frustration with and at the system,” says Bracken.

Civil service versus citizen’s needs

-  “I’ve lost count of the times when, in attempting to explain a poorly performing transaction or service, an explanation comes back along the lines of ‘Well, the department needs are different…’ How the needs of a department or an agency can so often trump the needs of the users of public services is beyond me,” says Bracken.

- Policy-making takes priority over delivery, which makes the civil service proficient at making policy and poor at delivery. “Delivery is too often the poor relation to policy,” says Bracken. Nearly 20,000 civil servants were employed in ‘policy delivery’ in 2009. Each government department produces around 171 policy or strategy documents on average each year. Bracken quotes one civil servant as saying: “The strategy was flawless but I couldn’t get anything done.”

Are citizen needs poisonous to existing suppliers?

- Departmental needs take priority over what the public wants. Bracken suggests that user needs – the needs of the citizen – are poison to the interests of policymakers and existing suppliers. “Delivery based on user need is like kryptonite to policy makers and existing suppliers, as it creates rapid feedback loops and mitigates against vendor lock-in,” says Bracken.

- “When it comes to digital, the voices of security and the voices of procurement dominate policy recommendations. The voice of the user [citizen] barely gets a look-in. ( Which also explains much of the poor internal IT, but that really is another story.)”

A vicious circle

Bracken says that new IT often mirrors clunky paper-based processes. [It should usually reflect new, simplified and standardised processes.] “For digital services, we usually start with a detailed policy. Often far too detailed, based not just on Ministerial input, but on substantial input from our existing suppliers of non-digital services. We then look to embed that in current process, or put simply, look for a digital version of how services are delivered in different channels. This is why so many of our digital services look like clunky, hard-to-use versions of our paper forms: because the process behind the paper version dictates the digital thinking.”

Then things take a turn for the worse, says Bracken. “The policy and process are put out to tender, and the search for the elusive ‘system’ starts. Due to a combination of European procurement law and a reliance on existing large IT contracts, a ‘system’ is usually procured, at great time and expense.

“After a long number of months, sometimes years, the service is unveiled. Years after ‘requirements’ were gathered, and paying little attention to the lightning-quick changes in user expectation and the digital marketplace, the service is unveiled to all users as the finished product.

“We then get the user feedback we should have had at the start. Sadly it’s too late to react. Because these services have been hard-wired, like the IT contract which supplied them, our services simply can’t react to the most valuable input: what users think and how they behave.

“As we have found in extreme examples, to change six words the web site of one of these services can take months and cost a huge amount, as, like IT contracts, they are seen as examples of ‘change control’ rather than a response to user need.

“If this 5-step process looks all too familiar that’s because you will have seen it with much of how Government approaches IT. It’s a process which is defined by having most delivery outsourced, and re-inforced by having a small number of large suppliers adept at long-term procurement cycles.

“It is, in short, the opposite of how leading digital services are created, from Amazon to British Airways, from Apple to Zipcar, there is a relentless focus on, and reaction to, user need…”

GOV.UK the civil service exemplar?

Bracken says: “In the first 10 days after we released the full version of GOV.UK in October 2012, we made over 100 changes to the service based on user feedback, at negligible cost. And the final result of this of this approach is a living system, which is reactive to all user needs, including that of policy colleagues with whom we work closely to design each release.”

Bracken says long procurements can be avoided.  “When we created GOV.UK, we created an alpha of the service in 12 weeks … We made it quickly, based on the user needs we knew about… As we move towards a Beta version, where the service is becoming more comprehensive, we capture thousands of pieces of feedback, from user surveys, A/B testing and summative tests and social media input.

“This goes a long way to inform our systems thinking, allowing us to use the appropriate tools for the job, and then replace them as the market provides better products or as our needs change. This of course precludes lengthy procurements and accelerates the time taken for feedback to result in changes to live services.”

Comment:

More big government projects could follow GOV.UK’s example, though some officials in their change-resistant departments would say their systems are too complex for easy-to-reach solutions. But a love of complexity is the hiding place of the dull-minded.

The Times describes the conflicts between the civil servants and ministers as a “crisis”. But conflicts between civil servants and ministers are a good thing. The best outcomes flow from a state of noble tension.

It’s natural for some senior civil servants to oppose change because it can disrupt the smooth running of government, leading potentially to the wrong, or no payments, to the most vulnerable.  It’s up to ministers like Francis Maude to oppose this argument on the basis that the existing systems of administration are inefficient, partly broken and much too costly.

A lazy dependence on the way things are will continue to enfeeble the civil service. Ministers who push for simplicity will always come into conflict with civil servants who quietly believe that simplicity demeans the important work they do. To effect change some sensible risks are worth taking.

The reports of a covert and courteous war between parts of the civil service and ministers are good news. They are signs that change is afoot. Consensus is far too expensive.

→ 2 Comments

Posted in Campaign4Change, change, change management, cost reduction, efficiency savings, EU procurement, G-Cloud, Government digital Service, Government IT, Politics, public sector, public services

Tagged , ,

We spend more on IT per capita than any other government – Maude

Posted on November 8, 2012 | 1 Comment

By Tony Collins

Cabinet Office minister Francis Maude, in a speech at the FT Innovate Conference on 6 November 2012, said:

“In the last decade our IT costs have gone up – while our services remained patchy. According to some estimates, we spend more on IT per capita than any other government.” Estimated annual IT spend in the public sector is between £14bn and £20bn.

And is the spend worthwhile?

“The same people who do their shopping, banking and social networking online are still interacting with Government on the phone, in person or on paper at less convenience to them and more cost to us…

“Government provides more than 650 transactional services, used about 1 billion times every year – but presently there are only a handful where a large majority of people who could use the online option do so.

“Half don’t offer a digital option at all – and apart from a handful of services, if there is a digital option few people use it because it’s not a sufficiently fast or convenient option.

Car tax online - under-used

“In some cases users try online and then have to revert back to other channels – in 2011 around 150 million calls coming into government were self-reported as avoidable.

This leaves us with a situation where, for example, three-quarters of people use the internet for car insurance, but only half buy car tax online.

“This is simply not good enough …”

GOV.UK

He praised the agile-based GOV.UK government website as easier to use and faster than Directgov and Businesslink which it replaces.

Mosquitoes

The Cabinet Office is also reducing the “incomprehensibly large number of Government websites”  – down from 424 to 350 in the last year.

“We closed a site dedicated to British mosquitoes – no doubt mosquitoes is a serious issue. We just didn’t feel it warranted a whole website.”

£15,000 to change a line of web code

“Departments can be asked to pay £15,000 to change a single word on a website because they are locked into legacy contracts negotiated at a time when the digital capacity lay almost entirely outside government.

“This is changing. We are moving away from legacy IT and our reliance on a few large System integrators. And introducing smaller contracts; shorter terms; a more diverse supplier community that is welcoming to SMEs; open standards; open source; more use of commodity. These are the new parameters.”

Francis Maude’s speech in full.

 

→ 1 Comment

Posted in change, change management, cost reduction, Government IT, managing change, private sector lessons for the public sector

Tagged , ,

DWP starts media campaign on Universal Credit IT tomorrow

Posted on September 19, 2012 | Leave a comment

By Tony Collins

The Work and Pensions Secretary Iain Duncan Smith has told MPs his department is launching a “major exercise” tomorrow to inform the media about Universal Credit, including progress with the IT project.

The public relations push will include a demonstration to journalists of the Univeral Credit front-end, and an explanation of the ability of “agile” to rectify problems as you go along. Duncan Smith said there is a lot of ignorance in the media, and suppositions, that need tackling head on.

His full statement on the PR campaign is at the foot of this article.

Comment

Iain Duncan Smith’s remarks to MPs sound remarkably like the statements that were made in the early part of the National Programme for IT in the NHS, when DH ministers and senior officials were anxious to correct ignorance and suppositions in the media – and to show journalists the front end of new electronic patient record systems.

Several times journalists were invited to Richmond house in Whitehall, the HQ of the DH, to hear how well the NPfIT was going. So anxious were the minister and leading officials to give a good impression of the programme that, on one occasion, trade journalists who had an insight into the NPfIT’s progress and could ask some awkward questions in front of the general media were barred from attending.

I would like Universal Credit to succeed. In concept it simplifies the excessively complex and costly benefit system. The worrying thing about the scheme, apart from the DWP’s overly sensitive reactions to scepticism in the media, is the way UC seems to be following the path which led to NPfIT’s downfall.

The Secretary of State attacks the media while trying to show UC in a glowing light and at the same time keeps secret all the DWP’s interview reviews and reports on actual progress. Duncan Smith says that the DWP wants to be open on UC but his department is turning down FOI requests.

There is no doubt that Duncan Smith has a conviction that the programme is on course, on budget, and will deliver successfully. But there still a morass of uncertainty for the DWP to contend with, and lessons to be learnt from pilots, some of which could be important enough to require a fundamental re-think. That’s to say nothing of HMRC’s Real-Time Information project which is part of UC.

Duncan Smith says the UC project is not due to be complete until 2017 which gives the DWP ample time to get it right. But ministers and officials in the last administration gave the NPfIT 10 years to complete; and today, nine years later, the scheme is being officially dismantled.

Did NPfIT ministers really know or understand the extent of the project’s true complexities and uncertainties?  Did they fully grasp the limited ability of suppliers to deliver, or the willingness of the NHS to change?  But they were impressed with the patient record front-end system and they organised several Parliamentary events to demonstrate it to MPs.

The NPfIT public relations exercises – which included DH-sponsored DVDs and a board game to market the NPfIT – were all in the end pointless.

Should Duncan Smith be running Universal Credit?

This is another concern. Duncan Smith is much respected and admired in Parliament but he appears too close to UC to be an objective leader. At a hearing of the Work and Pensions this week Duncan Smith took mild criticism of UC as if it were a verbal attack on his child.

It is doubtful anyone working for Duncan Smith would dare give him bad news on UC , though he attends lots of departmental meetings. Doubtless he listens to all those who agree with him, those who are walking press releases on the progress of the UC programme. He’d be a good marketing/PR man on UC. But surely not its leader. Not the one making the most important decisions. For that you would need somebody who’s free from the politics, who is independently minded, and who welcomes informed criticism.

Is there any point in a demo of front-end systems?

Seeing a front-end system means little or nothing. The question is will it work in practice when it is scaled up, when exceptions come to light, and when large numbers of people try to contact the helpdesks because they cannot get to grips with the technology and the interfaces,  or have particular difficulties with their claim.

What will a media campaign achieve?

If the NPfIT experiences are anything to go by, journalists who criticise the UC project will be made to feel stupid or uninformed.

In a totalitarian regime the media could be forced to publish what the government wants people to believe. Will the DWP’s PR campaign be designed to achieve the same end without the slightest attempt at coercion?

Duncan Smith’s comments to MPs

Below is some of what Iain Duncan Smith told Work and Pensions Committee MPs this week. He had been asked by a Committee MP to have a dialogue with the media to ensure that people believe that Universal Credit is a good thing.

Duncan Smith:

“On Thursday we are carrying out a major exercise in informing the median about what we are doing, looking at the system front-end, about budgets and all the elements the committee has been inquiring into.

“We will take them through that, show them that. We are going to open up much more. It is such an important system that I want people to learn what it is all about.  There is a lot of ignorance in the media and suppositions made; things that are important to tackle head on. Everyone says you mustn’t have a big bang; you are not going to be ready in time. The time we deliver this is 2017 when it is complete.  That is over four years…”

→ Leave a comment

Posted in Agile, Campaign4Change, change, change management, culture, DWP, e-health, excessive secrecy, FOI, Government IT, IT projects, managing change, managing outsourced services, NHS, npfit, Politics, procurement, project management, public sector, public services, Real-Time Information, Universal Credit

Tagged ,

Is Universal Credit really on track? The DWP hides the facts.

Posted on September 11, 2012 | 6 Comments

By Tony Collins

The Department for Work and Pensions has told Campaign4Change that consultancy reports it commissioned on Universal Credit would, if disclosed under FOI, cause “inappropriate concern”.

Who’s to say the concern would be inappropriate?

At the weekend a spokesman for the Department for Work and Pensions told the BBC: “Liam Byrne (Shadow Work and Pensions Secretary) is quite simply wrong. Universal Credit is on track and on budget. To suggest anything else is incorrect.”

But the DWP has decided not to disclose reports by consultants IBM and McKinsey that could throw light on whether the department is telling the truth. Though the reports cost taxpayers nearly £400,000, the public has no right to see them.

The DWP told us: “Disclosure [under FOI] would … give the general public an unbalanced understanding of the [Universal Credit] Programme and potentially undermine policy outcomes, cause inappropriate concern (which in turn would need to be managed) and damage progress to the detriment of the Government’s key welfare reform and the wider UK economy.”

Comment

In refusing to publish the costly reports from IBM and McKinsey the Department for Work and Pensions makes the  assumption that the Universal Credit IT programme will be better off without disclosure. But does the  DWP know what is best for the Universal Credit project?  Is the DWP’s own record on project delivery exemplary? Some possible answers:

-  The DWP has a history of big IT project failures, some of which pre-date the “Operational Strategy” project in the 1980s to computerise benefit systems. MPs were told the Operational Strategy, as it was called, would cost about £70om; it cost at least £2.6bn.  Today, decades later, the DWP still has separate benefit systems and relies on “VME” mainframe software that dates back decades.

- NAO reports regularly criticise the DWP’s management of projects, programmes or  suppliers. One of the latest NAO reports on the DWP was about its poor management of a contract with Atos , which does fit-to-work medical assessments.

- The DWP hasn’t broken with tradition on the awarding of megadeals to the same familiar names. Though Universal Credit is said to be based, in part, on agile principles, Accenture and IBM are largely in control of the scheme and the department continues to award big contracts to a small number of large companies. HP, Accenture, IBM and Capgemini are safe in the DWP’s hands.

-  The NAO has qualified the accounts of the DWP for 23 years in a row because of “material” levels of fraud and error.

So is the DWP in an authoritative position to say that the taxpayer and the Universal Credit IT project are better off without disclosure of consultancy reports when the DWP has never done it differently; in other words it has never disclosed its consultancy reports?

Can we trust what DWP says?

Without those reports being put in the public domain can we trust what the DWP says on the success so far of the Universal Credit programme?

Unfortunately departments cannot always be trusted to tell the truth to the media, or Parliament, on the state of major projects.

In 2006 the then health minister Liam Byrne praised the progress of the NHS National Programme for IT, NPfIT. He told the House of Commons that the NPfIT had delivered new systems to thousands of locations in the NHS. “Progress is within budget, ahead of schedule in some areas and, in the context of a 10-year programme, broadly on track in others.”

That was incorrect. But it was what the Department of Health wanted to tell Parliament.

Now it is the DWP that is praising Universal Credit and it is Liam Byrne criticising the programme. This time Byrne may have a point. The problem is we don’t know; the DWP may or may not be telling the truth – even to its Work and Pensions Secretary Iain Duncan Smith.

It would not be the first time ministers were kept in the dark about the real state of big IT projects: ministers were among the last to know when the Rural Payment Agency’s Single Payment Scheme went awry.

And while the NPfIT was going disastrously wrong, progress on the programme was being praised by ministers who included Caroline Flint, Lord Hunt, Lord Warner, John Reid, Andy Burnham, Ivan Lewis and several others. Even a current minister, Simon Burns, gave Parliament a positive story on the NPfIT while the programme was dying.

So while DWP spokespeople and Iain Duncan Smith praise the Universal Credit IT programme can anyone trust what they say? Though Duncan Smith sits on an important DWP steering group on Universal Credit, does he know enough to know whether he is telling the truth when he says the programme is on track and on budget?

At arm’s length to ministers, officialdom owns and controls the facts on the state of all of the government’s biggest projects – and the facts on Universal Credit’s IT programme will continue to stay in locked cupboards unless the Information Commissioner rules otherwise, and even then the DWP will doubtless put up a fight against disclosure.

The IBM and McKinsey reports were so well hidden by the DWP that, for a time, it didn’t know it had them.

The DWP gave the reasons below for rejecting our appeal against the decision not to publish. The DWP’s arguments against publishing the reports on Universal Credit are the same ones that, hundreds of years ago, were used to ban the publication of Parliamentary proceedings: that reporting would affect the candour of what needed to be said. That proved to be nonsense.

By hiding the reports the DWP gives the impression it doesn’t want the truth about Universal Credit to come out – leaving the department and Iain Duncan Smith free to continue saying that the scheme is on track. Indeed Duncan Smith said yesterday that he “has nothing to hide here”. That is evidently not true.

The reports we’d requested were:

- Universal Credit end-to-end technical review” (IBM – cost £49240).
- Universal Credit delivery model assessment phases one and two. ( McKinsey and Partners – cost £350,000).

DWP’s letter to us:

7 September 2012
Dear Mr Collins,

…You asked for a copy of the Universal Credit Delivery Model Assessment Phase 1 and 2, and the Universal Credit End to End Technical Review.

I am writing to advise you that the Department has decided not to disclose the information you requested.

The department has conducted an internal review and the information you requested is being withheld as it falls under the exemptions at section 35(1)(a) and (formulation of Government policy) and Section 36 (2) (b) and (c) (prejudice to the effective conduct of public affairs) of the Freedom of Information Act. These exemptions require the public interest for and against disclosure to be balanced.

These reports from external consultants discuss the merits or drawbacks of the UC delivery model and an assessment of whether the IT architecture is fit for purpose. This must be candid otherwise; the Department and the taxpayer will not secure value for money. Such reports can therefore be negative by nature in their outlook.

The Department considers that premature disclosure of these reports could lead to future consultants’ reports being less frank. In addition, there is a risk that this may lead to an absence of a recorded audit trail of the more candid elements. This is not in the public interest. Similarly, key staff selected to be interviewed by consultants are likely to be inhibited if they think their candour is likely to be recorded and released.

It is vital that the Department’s ability effectively to identify, assess and manage its key risks to delivery is not compromised. The willingness of senior managers to fully engage in a timely manner and support consultants assessment and assurance of key IT projects in an unrestrained, frank and candid way is vital to the effectiveness of the process.

Disclosure would also give the general public an unbalanced understanding of the Programme and potentially undermine policy outcomes, cause inappropriate concern (which in turn would need to be managed) and damage progress to the detriment of the Government’s key welfare reform and the wider UK economy.

While we recognise that the publication of the information requested could provide an independent assessment of the key issues and risks, we have to balance this against the fact that these reports includes details of ongoing policy formulation and sensitive information the publication of which would be likely to prejudice the effective conduct of public affairs.

The Department periodically publishes information about the introduction of Universal Credit, and this can be found on the Departments website here http://www.dwp.gov.uk/policy/welfare-reform/universal-credit/

Yours sincerely
Ethna Harnett

We have appealed the DWP’s refusal so the matter is now before the Information Commissioner’s Office.

Universal  Credit programme on course for disaster – Frank Field

Has the DWP lost £400,000 of reports it commissioned on Universal Credit?

Millions of pounds of secret DWP reports

NAO criticises Atos benefits contract

DWP scraps £141m IT project three months after assurance to MPs

→ 6 Comments

Posted in Agile, Campaign4Change, change, change management, DWP, excessive secrecy, Freedom of Information, Government IT, HMRC, IBM, IT projects, managing outsourced services, npfit, outsourcing, Universal Credit

Tagged ,

IBM in dispute with its joint venture partners on £585m contract

Posted on August 31, 2012 | 1 Comment

By Tony Collins

IBM says it is currently in dispute with the joint venture partners on a number of contractual matters relating to South West One, a joint venture between IBM and three public authorities. IBM owns the joint venture company.

South West One’s annual report says that a mediation was held on 4 and 5 July 2012 between IBM and Somerset County Council, which is the main public authority partner, on a confidential basis.

“No settlement has been reached and accordingly the board [of South West One] will be reviewing which of the remaining options in the contractual procedure should now be pursued,” says SW1′s annual report.

South West One’s report doesn’t give any detail on the “contractual matters” in dispute.

Possible matters under discussion might have included a withholding of money (the councils are expected to pay IBM about £585m over 10 years, from 2007),  contention over KPIs (IBM did not meet all of its key performance indicators and indeed met fewer of Somerset’s KPIs in 2011 than in 2010), changes to the contract which is being re-negotiated, a lack of remedial action over accounting problems in Somerset’s finance department following a major SAP implementation , a shortfall in expected savings, and the council’s extra costs of working around SAP-related problems .

It is known that a contract renegotiation has been underway for some time.

The contract was subjected to review after the Conservatives took control of Somerset County Council from the Liberal Democrats in May 2009.

The review in June 2010 found that some aspects of the contract had been successful but “figures provided do, however, tend to indicate that the anticipated procurement savings are currently falling short of projections”.

On service delivery the review said there had been “major and minor system problems and difficulties in implementation have been experienced which have often involved Somerset County Council staff in additional time and effort in working around these issues”.

It said that a “significant area of difficulty has been in relation to financial and processing components of SAP which have also had a serious effect on others outside Somerset County Council.

“As a result, there appears to have been substantial but unquantified additional direct and indirect costs incurred by the County Council and others in resolving the various difficulties encountered.

“Southwest One has also provided intensive additional resources at its own expense, notably in addressing the issues that arose in relation to the SAP phase one roll out where lessons have clearly been learned and applied to the more successful phase two implementation. More work is, however, still required as a priority in some key areas where concerns remain around the efficiency and effectiveness of service delivery and financial systems.”

South West One is dependent on the financial support of IBM to continue trading, says  company’s annual report. It adds that the “difficult political and economic environments in which the company has been operating have not shown any signs of easing”. Somerset has taken back from South West One finance, an HR advisory service, design and print.

“The difficult environment for business, both public and private, will continue to place strains upon opportunities for South West One,” said the annual report.

“There will be specific challenges in the forthcoming year due to the implementation of Universal Credit, the requirements of the Winsor report and changes in regard to the move from Police Authorities to Police Crime Commissioners.”

South West One made a loss in 2011 of £6.8m (a loss of £22.7m in 2010) and has accumulated net liabilities of £43.2m. The company can continue trading, in part because it has the support of IBM UK’s parent:  International Business Machines Corporation based at Armonk New York.

IBM owns 75% of the shares in South West One. Somerset owns 11.75%, Avon and Somerset Police Authority 8.25%, and Taunton Deane Borough Council 5%.

This article owes much to Dave Orr who has campaigned tenaciously for the facts of the South West One deal to be made known.  

Comment

The unsettled dispute suggests that the “partnership” aspect of the contract between IBM and the three public authorities – Somerset County Council, Taunton Deane Borough Council and Avon and Somerset Police Authority –  is at an end. A partnership normally implies a harmonious relationship between the parties.

Is it any surprise that things have come to this?

The South West One contract was signed in 2007, in the early hours, at a weekend, amid great haste and secrecy.  The deal was driven by a senior official at Somerset who wanted to take the council “beyond excellence”. But the joint venture had little support from many of the council staff who were seconded to South West One. Most councillors took little interest in the setting up of South West One.

IBM has found to its cost that signing a major contract with just an inner circle of enthusiasts is not enough to make such a deal work. Though some have changed many of Somerset’s councillors remain. It could be said that they deserve the deal they have got, given that so few of them took any interest in the negotiations in 2007.

Besides, it is unlikely that any joint venture which doesn’t have the support of most staff will work, which makes mutuals a potentially better shared-services option.

IBM struggles with SAP two years on – a shared services warning?

IBM-led model partnership based on SAP makes loss

→ 1 Comment

Posted in Campaign4Change, change, change management, cost reduction, culture, IBM, IT-related failures, legal disputes, local government, managing outsourced services, mutuals, openness and transparency, operational issues, outsourcing, Politics, procurement, public-private partnerships, SAP, shared services, Universal Credit

Tagged

IBM won bid without lowest-price – council gives detail under FOI

Posted on July 10, 2012 | Leave a comment

By Tony Collins

Excessive secrecy has characterised a deal between IBM and Somerset County Council which was signed in 2007.

Indeed I once went to the council’s offices in Taunton, on behalf of Computer Weekly, for a pre-arranged meeting to ask questions about the IBM contract. A council lawyer refused to answer most of my questions because I did not live locally.

Now (five years later) Somerset’s Corporate Information Governance Officer Peter Grogan at County Hall, Taunton, has shown that the council can be surprisingly open.

He has overturned a refusal of the council to give the bid prices. Suppliers sometimes complain that the public sector awards contracts to the lowest-price bidder. But …

Supplier / Bid Total cost over 10 years
BT Standard bid £220.552M
BT Variant Bid £248.055M
Capita Standard Bid £256.671M
Capita Variant Bid £267.687M
IBM Standard Bid £253.820M
IBM Variant Bid £253.820M

The FOI request was made by former council employee Dave Orr who has, more than anyone, sought to hold Somerset and IBM to account for what has turned out to be a questionable deal.

Under the FOI Act, Orr asked Somerset County Council for the bid totals. It refused saying the suppliers had given the information  in confidence. Orr appealed. In granting the appeal Grogan said:

“I would also consider that the passage of time has a significant impact here as the figures included under the exemption are now some 5 years old and their commercial sensitivity is somewhat eroded.

“Whilst, at the time those companies tendering for the contract would justifiably expect the information to be confidential and that they could rely upon confidentiality clauses, I am not able to support the non-disclosure due the fact that the FOI Act creates a significant argument for disclosure that outweighs the confidentiality agreement once the tender exercise is complete and a reasonable amount of time has passed.

“I therefore do not consider this exemption [section 41] to be engaged. Please find the information you requested below…”

[In my FOI experience - making requests to central government departments - the internal review process has always proved pointless. So all credit to Peter Grogan for not taking the easy route, in this case at least.]

MP Ian Liddell-Grainger ‘s website on the “Southwest One” IBM deal.

IBM struggles with SAP two years on – a shared services warning.

Council accepts IBM deal as failing.

Was Audit Commission Somerset and IBM’s unofficial PR agents?

→ Leave a comment

Posted in Campaign4Change, change, change management, excessive secrecy, Freedom of Information, governance, IBM, IT-related failures, managing change, managing outsourced services, SAP, shared services, software quality, supplier relationships

Tagged ,