Category Archives: IT projects

The State Auditor in California recently criticised a planned high speed rail system between San Francisco and Los Angeles because the project suffered from critical, on-going oversight problems. In this guest blog, Bob Evans, founder and Managing Director of TestIT Software Assurance, explains why it is critical to have a ‘single source of truth’ for any IT development, to provide the required level of cohesion, discipline, control and transparency that should be expected from any engineering project.

The risks associated with developing or changing IT systems are well documented. IT failures always bring despair and reputational damage to all involved.

Do you remember the scene in Blackadder where General Melchett suggested that walking slowly (again) towards the German machine guns would be “the last thing they’d expect” – and so “that’s what we’re going to do”? This seems to be the attitude found in many IT procurements. What’s needed is a fresh, radical and innovative approach, especially in these changed times where senior management focus is on delivery and cost.

Keeping Control

Establishing and maintaining control of an outsourced project is the single, most important part of IT development; it is imperative for the purchaser to remain in full control at all times. In all of the failure scenarios that we have researched, it is obvious in every case that control has been lost, with always calamitous consequences.

The nature of the procurement process and inevitable time pressure means that the requirements of a new software system aren’t always well-defined at project inception. Typically, specifications are incomplete or ambiguous and put together from a range of disparate sources, with non-existent internal process and control. The “silo” attitude that is often encountered across departmental organisations is usually a key factor here.  Requirements that are not made clear at the start of the project will inevitably lead to confusion, delays, additional cost and even to dispute.

And if a contract is awarded on the basis of a flawed specification, the chosen vendor is in a strong position to justify delays and ramp up the costs. When planned schedules are missed and costs spiral out of control, well intended remedial actions can then create new, unforeseen problems, leading to even more rework.

The culture of care and diligence found in many traditional engineering environments is sometimes lacking in the IT industry and in our experience, engineering discipline is usually the first casualty when pressure is applied. Very quickly, the focus shifts to fire-fighting and a project loses sight of its objectives. It’s like building a bridge – but when halfway across, only fitting every other bolt.

 Contrived Tests

It’s no surprise that applications that have seemingly passed an Acceptance Test, when deployed on a live system, prove to be problematic. Validating the effectiveness of an IT system is typically a contrived process – put simply, tests are designed to pass! They would be – they are usually designed, controlled and often even run by the software vendor.

If you were buying a used car from a local garage, you’d get the RAC to come along and check out the mechanics. You wouldn’t even think of asking the mechanic who worked at that garage to do the inspection would you? But invariably this is the case in IT.

As discussed above, it is imperative that full control of the project remains with the purchaser. But without, continual up to date feedback and independent metrics, it’s hard to know what’s really going on. And the critical decision – when to go live – cannot be made with confidence.

Agile Development

The concept of Agile Development has been round for many years. However, it is still to be seen whether this approach can really work on a large, complex project. A properly managed set of sprints – with precise objectives, proper controls and diligent, independent validation at appropriate times may indeed lead to a more rapid and successful conclusion. However, it should be noted that as development continues, the needs of the business will necessarily change. If controls are not adequately maintained, no process – not even “Agile” ones – can keep up. What is delivered is what the vendor believes is required, rather than what is actually required by the business. And no matter what methodology is used for developing software, at the end of the development cycle it is essential to verify, independently and thoroughly, that the product meets the needs of the business.

The Agile Manifesto focuses on “working software over comprehensive documentation”. Documentation is of course often cumbersome. However, minimising the amount of information while maximising the value of the information is what’s really needed.

When it’s too late – and organisations end up mired in a legal dispute, we’ve been called in to play the part of the pathologist. It is soon apparent that the project was doomed from the start – no spec, no metrics, no control, no hope. Suppliers who didn’t listen, purchasers who didn’t actually know what they needed in the first place.

But there is another way. The Maritime and Coastguard Agency recently used independent software assurance to ensure success of its high-profile CERS/SVD project. There was an acknowledgement right from the start that requirements were bound to change and controls put in place to manage this. There were efficient, timely tests – not at the project end, but continuous tests, right from project inception. There was an approach that asked:  “What can we do given the available time and budget?”, rather than the popular “one-shot, whistle and bells” approach. And there was pro-active searching for and identifying issues early on – when the supplier was still on site – which meant that remedial actions were fixed by the supplier, at no additional cost to the project. Independent, constantly updated metrics also meant that the Maritime and Coastguard Agency stayed in complete control.

It isn’t rocket science. But for IT projects, the discipline driven by an engineering-based approach is generally more likely to lead to success. That’s why bridges get built – and IT projects often don’t. 

TestIT Software Assurance