Lessons from an IT crisis – Tesco Bank

By Tony Collins

In a crisis customers want truth and openness. So was Tesco Bank truthful and open when a data migration left its customers locked out of their accounts? As its chief executive Benny Higgins says:“The test of an organisation is what they do when things go wrong.”

This is the story of how Tesco Bank handled a crisis – what it did well and not so well – and what the public sector can learn from the difficulties. 

Many large companies – and government departments – go through IT-related crises. How they deal with them could determine whether their reputation and credibility suffer lasting damage.

Below is one of is one of most useful case studies in recent years. It’s a study of what happened on a day to day basis when IT changes at Tesco Bank locked out thousands of their customers from their online accounts – and the Bank’s helpline couldn’t cope with the volumes of calls. The Bank was accused of not telling the truth to the media and its customers, and of being in IT chaos.

In the way it handled the matter Tesco Bank unwittingly copied the way some central government departments act when faced with an IT-related crisis by, for example, comparing the small number of people affected with the high number who are unaffected, an approach that can belittle the experiences of the thousands who suffer the consequences of poor customer service.

By looking back over the period of the crisis at Tesco Bank it’s possible to draw out the mistakes that would otherwise be lost in the melee of the media reports on how the debacle affected individuals. Just as quickly as the media picks up and reports on a crisis, it can quickly forget all about it when new stories take priority.

For the thousands of Tesco Bank customers unable to log into their accounts, the crisis will not be quickly forgotten. Some have said they will remove their money from the Bank, if they have not done so already, and some will be seeking compensation. The crisis may remain a stain on the reputation and credibility of Tesco Bank for years.

In a personal tweet, the presenter of BBC R4’s Money Box Paul Lewis, who has done much to bring the Tesco Bank story to public attention, summed up his reading of the Bank’s handling of its crisis: “An apology but no real understanding of how to deal with a big mistake”.

So what did Tesco Bank get wrong and how should it have reacted?

Tesco Bank did some things well.  It:

 –          engaged with the media. George Gordon, the Edinburgh-based head of communications at Tesco agreed to be interviewed on BBC R4’s Money Box Live on Wednesday 22 June, day two of the crisis. A few days later, on Saturday 25 June, Tesco Bank’s CEO Benny Higgins went on Money Box to answer questions from the programme’s presenter Paul Lewis. [Tesco Bank also responded quickly to my questions.]

–          apologised to customers, gave out information about the numbers of people affected and conceded that its service to customers had been unacceptable.

–          said it would deal with requests for compensation on a case by case basis, and made this clear on its website.

–         set up a Q&A on its website to help customers with log-in problems

–          eventually unlocked customer accounts and elicited praise from some customers for its helpfulness in doing so

What it didn’t do well. It: 

 –          continued to tell customers the technical problems were sorted when many people still could not access their bank or savings accounts. Customers accused the bank of not telling the truth. One, on BBC R4, asked Tesco Bank’s head of communications why the Bank was “lying” to customers.

–          told customers that calls to its helplines had been answered in an average of 15 minutes when, the next day, it took Moneybox’s Paul Lewis 54 minutes to get through.

–          used, initially, an 0845 number on its helpline which for some customers was a premium-rate number.

–          apologised for its poor customer service but at no point answered directly any questions about whether it had given out inaccurately positive information.

–          allowed an in-store customer whose “Clubcard Plus” credit card transaction was declined to face embarrassment and blame rather than say the problem was Tesco Bank’s fault.

–           compared numbers of customers locked out of their accounts with the apparently much larger number who had successfully logged in

–          suggested some customers were at fault, saying that in a typical week up to 1,000  people will have trouble logging on because they “have inaccurate security credentials in their possession … do not have the right details”.

–          might have underestimated the volumes of attempted log-ins on the first operational day after a major migration of customer accounts from one system to another. It’s unclear whether testing before go-live took a pessimistic view on possible volumes of log-ins.

Anatomy of an IT crisis – how it unfolded and how Tesco Bank responded

Wednesday 15 June 2011

ComputerworldUK.com reveals that Tesco Bank is in the final stages of moving customer accounts from one system to another. The Bank plans “imminently” to switch customers from the old Royal Bank of Scotland systems to its own computers. Tesco Bank had bought RBS three years before to provide an alternative to the familiar customer service failings of High Street rivals.

Tesco Bank says in a financial statement that it “continues to make good progress and is now in the final stages of the transition to its own systems and infrastructure”. Tesco Bank launched in 1997 and has 6.5 million customers. It offers savings accounts, loans and credit cards.

Weekend 18/19 June

Tesco switches 605,000 savings accounts and 320,000 personal loan accounts from Royal Bank of Scotland to Tesco Bank’s systems.

Monday 20 June

Tesco Bank’s online services are unavailable for 12 hours to all customers – nobody with Tesco Bank can access their savings or loan accounts

Tuesday 21 June

Tesco bank systems are down again, this time for six hours. Complaints start to mount, some from people who say that the Bank is not telling the media the truth.  ComputerworldUK.com has 52 comments after it reports that some Tesco Bank customers cannot access their accounts while the bank undergoes system migration to its Fiserv Signature bank platform. The BBC quotes Tesco Bank as saying the problems have been fixed and that “all customers can access their accounts online as normal”.

Says the BBC, quoting a Tesco Bank spokesman:

“For a brief period some customers were unfortunately unable to access their accounts. We apologise for this, but can reassure them that the process is now complete and all customers can access their accounts online as normal.”

Jon is among those who have spent more an hour getting through to Tesco Bank’s helpline.

“I can’t understand comments that Tesco appear to have made to the media suggesting that the problem has been ‘fixed’.  It’s pretty clear that Tesco is having a huge IT collapse and, as soon as I’m able, I’ll be moving my funds to another bank. If they can mess up this so badly, then people need to question whether their funds are even safe with Tesco.”

Another customer comments: “I still cannot access my savings account…Not happy as needed to transfer some money to a relative for deposit on a flat . May now lose it. Hope they are proud of this monumental cock-up. They really need to compensate people for this.”

People are angry that Tesco Bank’s website says that people can now access their accounts and that extra staff have been brought in to answer phones.

“The front page of the Tesco bank website is now saying that access has now been restored and that extra staff are in place to answer phone enquiries. I still can’t access my account and they are still not answering their phones. Tescos are treating their customers disgracefully.”

Wednesday 22 June

BBC R4’s Money Box Live says it has been “inundated” with calls from Tesco Bank customers who cannot get through to their accounts. It takes a call from a customer who accuses Tesco Bank of “lying”, which the Bank denies. The caller speaks to George Gordon, head of communications at Tesco Bank, who has agreed to take part in the programme.

The caller, Alistair, says: “I have been trying to access my Tesco account since Monday to do a small transfer. We are not talking millions. You have six and half million bank customers. Why are you lying to them through your website and through other media by telling them that your system is now working. I can assure you that as of 30 seconds ago it is not working.”

Gordon says: “I just want to say how sorry I am that you have had these difficulties getting onto our website. For the vast majority of our customers, the site and our phone lines are working but we are aware there area few technical issues there. We have an IT helpdesk and some online information on our website which we are actually improving this afternoon to try and help people in your position. I am very happy after the call to take your details and make sure we pick up on that.”

Vincent Duggleby (presenter) “I have to say I looked at your website and I don’t think it was at all helpful. It simply said everything was alright and you were sorry.”

In his reply Gordon says the technical issues have been sorted out. “Obviously we are sorry for our savings customers who have had technical issues over the last couple of days. As our website says we have suffered some intermittent technical problems and these problems follow a significant piece of work that we did over the weekend – a planned piece of work that we told customers about –moving from Royal Bank of Scotland  platforms to our own Tesco Bank Platforms.

“Now we put messages out there today to let the vast majority of our customers who want to access their accounts know that we have sorted out technical issues and that we have actually increased the number of people in our call centre so that if people have issues they can get in contact with us and we can help them –

Alistair: “It says on your website all customers can access their accounts online as normal but that obviously is not the case. I spent 46 minutes today in a queue to get through only to be told by the person at the other end: ‘I am sorry we cannot access your account.’”

Another caller “Mike” says that Tesco Bank’s 0845 number costs a premium rate to ring, for some people. “Is Tesco Bank making money out of our discomfort?” asks Mike.

In his reply Gordon says – again – that technical problems have been sorted out. His reply: “Absolutely not. We are trying to sort things out. We have had some technical issues over the last couple of days which have been sorted out. We are looking at the calls we are getting at the moment. In terms of the time Mike has spent on the phone, that’s clearly not acceptable and we are sorry about that. In our call centre today we have significantly increased the number of people who are answering the phones so we can deal with the very high call volumes.”

James Hillon, head of Retail Banking at Co-operative Bank, emphasises the importance of trust.

“One of the things we have learned over the last two to three years is that trust is of paramount importance in financial services and it is things like this that can knock peoples’ trust. I am not going to pretend that the co-operative bank has never had systems issues but what customers want at this time is to be dealt with honestly and to be kept informed about what’s going on. That’s the sort of thing we try to do.”

Tesco Bank customers have particular problems logging in with IE9. Commenting on ComputerworldUK’s website they say they have had more success when using Google Chrome or Firefox, though these browsers don’t work for everyone .  Tesco Bank issues guidance to customers with IE9 saying they need to enable “compatibility mode”. Says one customer:

“I did download Firefox and managed to get in to my account (I also had to download Flash V9 – God help anybody who doesn’t know what they are doing). It sort of worked ok but the new level of security wouldn’t let me move any money to my high street bank account without sending them a bank statement first.”

Thursday 23 June. The website of Moneysavingexpert [Martin Lewis] reports that the “Tesco Bank online log-in pain” is “now over”. The site quotes Tesco Bank as saying:

“All of our savings customers can access their accounts online. However some customers need to make a small change to their browser settings to do so.

“We’ve improved the step by step guidance on our website and this should help those having problems. We have also contacted a limited number of customers who had a specific issue on Monday.

“Waiting times on the phone are falling, but customers are still waiting longer than any of us would like and we’re doing absolutely everything we can to sort this out as quickly as possible.”

Saturday 25 June

It’s clear some customers are still locked out.

Paul Lewis on BBC Money Box says some people with online savings accounts at Tesco Bank have been unable to access their money “all week”.

One customer says on Money Box: “The problem is essentially step six in the brochure about changing to the new security system where it says they need a password of at least seven characters long. I entered nine and it said it was not acceptable. It then took me over an hour to be able to ring back and get some help and it’s still not satisfactory. Quite frankly my attitude is I want to withdraw all my money from Tesco’s and bank somewhere else.”

Another Tesco Bank customer says: “I’ve been trying to log into my Tesco savings account since Monday. I think that Tesco should have come clean about this in the beginning and admitted they’d made rather a pig’s ear of it and given customers a regular update, and then I think everybody would have been a lot happier.”

One customer says that a store manager allowed his wife to take the blame for a Tesco Bank problem.

“My wife tried to buy some goods from Tesco’s on Wednesday using Clubcard Plus. Payment was refused. A manager was called and after another attempt he said that the problem was with my wife’s account and that she should call the Clubcard Plus helpline. He did not admit that the fault could be Tesco’s. She was stressed out by this. They could have admitted at the shop that it was Tesco’s fault and not my wife’s.”

Paul Lewis makes the point that the main source of customer frustration is the “lack of clarity from Tesco Bank about what the problems are, how they could be fixed, and indeed when”.

Says Lewis: “On Monday a Tesco Bank spokesperson told the BBC that for a brief period some customers were unable to access their accounts, but all customers could go back online as normal. Similar claims were made on Tuesday and on Wednesday, made indeed on Money Box Live in the afternoon, then on Thursday; and again on Friday we were told most of the problems were resolved.

“But throughout the week listeners kept emailing to say they were still having problems logging on and particularly contacting the helpdesk.”

To his credit, Benny Higgins, Chief Executive of Tesco Bank, goes on Money Box to answer Lewis’s questions, only he doesn’t quite answer questions about whether the Bank has been open about the seriousness of the problems.

Lewis asks Higgins: “Will you now admit that these problems are far more serious than your spokespeople have been telling us all week?”

Higgins: At Tesco we strive to deliver service of the very highest standard. And this week …

Lewis: Well you’ve not been doing that, have you?

Higgins: … and this week Tesco Bank has for a significant minority of customers failed to do so, and it’s not good enough and we apologise unreservedly for that. What I would like to make clear is what has happened and what we’re doing about it.

Lewis: Tell us what’s happened because I’ve heard a number of things have gone wrong. People have said it’s the passwords; it’s the wrong Internet browser. Some people, I was told, had their identities effectively trashed because they were online when the glitches occurred on Monday and Tuesday.

Higgins: I’d be delighted to tell you what happened …we transferred 605,000 customer accounts in respect of savings and another 320,000 in respect of personal loans. The system, the web availability, went down on Monday for 12 hours and on Tuesday for six hours, so regrettably during those periods customers clearly had no access to online. Thereafter we have had continuous availability of our online service…

Lewis: … it’s all very well saying you’ve resolved that [problems with IE9]. You haven’t resolved it for a huge number of customers.

Higgins: … As I say, there has been the issue with Internet 9 and that has affected a number of customers. Of the 100,000 active customers, 25,000 have already re-registered successfully. We’ve had 9,000 customers over each of the last few days active online. One of the two things I think you need to be aware of too is that there were a number of customers online when the system failed. Those customers were locked out. There was also …

Lewis: And they’re still locked out, aren’t they, until they write to you and you write back to them?

Higgins: No, that’s not the case if I can make it clear.

Lewis: Well that’s what some of them have told us.

Higgins: Well let me make it clear. There’s also a number of customers who routinely – and this happens every week … Prior to this week, in a typical week we would have between 750 and 1,000 customers who can’t sign on because they have got inaccurate security credentials in their possession, so they themselves don’t have the right details.

Lewis: Well so you’re blaming the customers now?

Higgins: No, I’m absolutely not. I’m just saying typically there will be 1,000 customers who have that difficulty every week. We had 2,500 customers, including any that had the details wrong themselves, plus those that were online. So we had 2,500 customers who were locked out as a result of Monday and Tuesday and we have been contacting those customers by telephone and by email to ensure that they have every chance to be back online.

**

Lewis makes the point that customers cannot get through on the phone, or they wait on the line seemingly indefinitely.

Higgins says that calls were being answered within 15 minutes on average the previous day; he doesn’t explain why Lewis was hanging on for 54 minutes that morning.

Lewis:  I hung on to your helpline with that lovely music for 54 minutes this morning before we got an answer. That’s just not good enough when you know there are thousands of people needing to contact you.

Higgins:  It’s absolutely the case that we have failed in that regard, but …

Lewis:  What are you doing about it? That’s what people want to know.

Higgins: …It is our highest priority to focus on doing the right thing to put customers in the right place. At the start of the week when we had the failure online, we had a waiting time, an average waiting time, of over 40 minutes. We have between doubled and trebled the number of people on the telephone since Wednesday. Our average waiting time yesterday was 15 minutes. Still not good enough, but we are …

Lewis: It was 54 when I rang, unless you’re saying I was kept on hold exceptionally. It was 54 minutes.

Higgins: Yesterday …

Lewis:  And also we were told today faster payments aren’t working, are they, so it’s still not back to normal even for the customers who can log on?

Higgins:  The average yesterday was 15 minutes. I don’t know what the average this morning has been, but I would expect it to be around the same time.

Lewis:  Well I can only tell you I rang at half-past ten and it was 54 minutes. Let me ask you this. You’ve got this transfer of, as you say, nearly 900,000 people. Didn’t you test the new system to destruction before you actually went live with it? This is a banking system with people’s money at stake.

Higgins:  Of course we did, and I have …

Lewis: Well not very well then, did you?

Higgins: …  I have been involved in a number of large migrations in my career in financial services. We tested very, very thoroughly. It is no consolation to customers and no consolation to us that a relatively small number of issues have gone wrong and created disproportionate damage to the customers. We do apologise unreservedly. It is absolutely our focus to put this right. We’re doing absolutely everything we can and we’ve made huge progress since earlier in the week. A number of …

Lewis: Well I have to say we’re getting more emails than we were earlier in the week. It doesn’t seem to us that things are getting better… A lot of people have said to us we’re going to take our money out of Tesco. What do you say to them?

Higgins:  Customers are entitled to make their own choice. We will not be deflected by the incidents this week from seeking to provide customers with the very highest level of service. The test of an individual and the test of an organisation is what they do when things go wrong as much as any other time. That’s what we’re focused on. Customers will make their own choice. We will not be deflected in our pursuit of serving customers well.

Lewis:  And you think you’ve passed that test because, of course, you want to become a force in banking, don’t you?

Higgins: We have not passed that test yet.

Lewis:  You haven’t passed it. But you want to become a force in banking to compete with other banks when one of the big problems is service?

Higgins: Absolutely, that is indeed what we are setting out to do. This is one of a series of migrations. Many are behind us very successfully. It is no consolation that things have gone wrong this week. We are very focused on doing the right thing and focusing on what customers need sorted. That’s what we’re doing.”

 **

Paul Lewis interviews Mike O’Connor, Chief Executive of Consumer Focus, which describes itself as the “statutory consumer champion” for the UK. O’Connor says:

“We’ll never have accident free. But when things go wrong banks should admit it; they should communicate clearly; they should fix it; they should apologise, and compensate not just for loss of money but also if you’ve spent an hour on the phone…”

Monday 27 June (week 2 of the crisis) 

Tesco Bank gives an address on its website for customers to make claims for compensation.

Thursday 30 June

 Tesco Bank’s website announces:  “Our online service is now fully operational.” The Bank apologises to “any of our savings customers who continue to have difficulties registering online”. It adds:

“We continue to receive a high volume of calls, but are beginning to see waiting times return to more acceptable levels. Your call will be answered as quickly as possible. In some instances to ensure that your request can be dealt with fully we will arrange a call back…”

Saturday 2 July

Some customers remain locked out, despite the assurances that customers could access their accounts. BBC Money Box says emails are “still arriving from listeners desperate to access their own money”.

One is from David of West Sussex who is unable to close his Tesco Bank account. Initially he’d wanted to pay a bill from his Tesco account. After a 40 minute wait on the phone David was told he couldn’t because the system was down.

Tesco promised to phone him back but didn’t. So David phoned twice on Wednesday 22 June and still didn’t get through. By Thursday 23 June, more than a week ago, he decided to close the account and transfer the money to another bank.

Says David on Money Box:

“On 23 June I waited for 45 minutes and eventually the phone was answered by a guy called Scott. He said that he would make the arrangement to close my account and transfer the funds to my nominated bank. He told me this would take three to five working days to complete.

“The next Wednesday, 29 June, that gave them 6 days, I went to my nominated account and there was no money in there. I am talking £7,000. I phoned Tesco again and then found my £7,000 was reinvested due to their – Tesco’s – human error into the Tesco account that was already closed. I couldn’t believe it. I spent a long time on the phone. I was frustrated and very angry. It is just not a way to do business for a company that size.”

A former member of Tesco Bank’s IT team emails to say he is not in the slightest bit surprised at the current fiasco at Tesco Bank. It would take three weeks just to get a computer on your desk, says the man. Another IT project that Tesco forecast would take six months to complete took in the end three years, he claims. He describes a scene to Money Box of total management chaos. “A great organisation bogged down by a management who can’t get things done as he put it.”

Tesco Bank denies it is a slow-moving bureaucratic organisation and says it has a reputation for getting things done, and done well.

Alex Fiatcosky, who was at Tesco Bank for a meeting at the height of the problems, implies that Tesco might have gone live too soon after migrating RBS accounts to Tesco Bank’s systems. “I think that would be something for Tesco to look at themselves to make they have the timings right on this and haven’t inadvertently inconvenienced their customer as part of timings to evacuate from the RBS systems.”

He adds: “Where they were perhaps undone on some of the system challenges is the rising volumes on the first day of full operation on the Monday. One could say: ‘should you not have anticipated the likely rise in volume and tested the system accordingly based on an uplift in attempted online log-ins’. There we have perhaps some areas of learning.”

Paul Lewis concludes by saying that most customers can now access their accounts, some have emailed to praise the Bank’s helpfulness and he got through to the helpline almost immediately.

3 and 4 July

I ask Tesco Bank:

– what lessons it has learned

– whether it had been open and accurate in its communications with customers and the media.

Its spokesman is helpful but doesn’t answer those questions directly . He says Tesco Bank has engaged with the media, made information available on its website and put more people on the phones.

This is Tesco Bank’s statement which, while helpful, still doesn’t answer questions on whether the Bank had learnt anything from telling customers they could access their accounts when some couldn’t.

“Our online service is fully operational and the overwhelming majority of our customers are able to access their accounts online. We are working to tackle remaining registration questions and continue to update the step-by-step guidance on our website to help customers quickly resolve any issues.

“We continue to receive a high volume of calls, but customer waiting times have returned to more acceptable levels. We’ve introduced a dedicated team to call back customers to ensure their requests are dealt with fully.

“We apologise to any of our Savings customers inconvenienced by the issues over the past two weeks.”

Comment:

Most people will welcome Tesco Bank’s entrance into financial services. The more competition the better.

But in saying problems are fixed when customers still cannot access their money, the Bank has inadvertently behaved like government departments that have played down the effect of IT-related problems on taxpayers, NHS patients, junior doctors, farmers, child support claimants, pupils and teachers involved in SAT tests, magistrates’ courts officials, prison officers and MoD workers.

Nothing would surprise people more in an IT-related crisis than plain speaking, truthfulness and openness. It’s time for people to be surprised.