Data migrations are usually greeted with dread by business managers. IT-driven projects to consolidate servers, move to a new application suite or platform, or just to upgrade systems are often the trigger for a data integration exercise as well. Months down the line when the business gets handed the new live solution, they often discover problems.

As a result, migrations and integrations can fail because they do not get adoption by the business. Either data has been modelled in a way that does not readily support their process, or business- critical data has been lost in the process of matching and deduplication.

But if these flaws are the fault of IT, perhaps business should bear some of the blame. After all, by sitting out of the project, it has not brought its needs and knowledge to bear on the specification. That changes the critical success factors and also reduces the opportunity to make process improvements.

Where these have been a driver, rather than a hidden factor, the benefits can be significant. Bupa has just completed the largest data migration project in the UK insurance industry which now enables it to implement its One Plan initiative. Lasting three years, the project involved migrating data on 12.7 million members, nearly 100,000 groups and 21 million claims.

With a unified view of this data, Bupa is now able to bring new products to market more quickly, allowing it to expand its product range and reduce premiums. Staff productivity has been increased, customer experience enhanced and obligations under Financial Services Authority requirements for "treating customers fairly" have been met.

"We were very aware that consolidation of so many systems was a risky strategy and that the data migration element presented significant challenges and pitfalls", says Steve Felton, director of business change at Bupa.

"For the new systems to be implemented successfully, it was imperative that all the relevant customer and financial information was transferred to the new applications accurately. With the migration of data from flat file, mainframe technology to the multi- level hierarchies of relational database systems together with the multiple mappings required to transform data from source to target systems across different lines of business, this was an incredibly complex project," he notes.

Indeed, the insurer had to gain both FSA and HMRC approval. That meant the company had to account for data being transferred accurately and in its entirety, as well as providing visibility for auditing early on. Although old customer data could have been archived, the business wanted to clean and extract all the data in its legacy systems in order to analyse its business KPIs.

Kognitio worked with the client on delivering the data migration into the new Swift policy administration system and Peoplesoft financial accounting system out of the nine previous solutions. In the process, the number of individual records was reduced by two- thirds through accurate matching and deduplication. That involved 700 million rows of data which were ultimately translated into 2.5 billion rows being loaded in less than two days.

"Involving the lines of business is definitely important to the success of data migration. It is their data - it belongs to them," says Phillip Magnall, head of data migration at Kognitio. "They understand the data and need to input into what the final solution is."

"In the Bupa project, there was a great deal of discussion about how to match and dedupe customer records and the setting up of groups for healthcare policies in the new system," he says. Involving business teams was something Kognitio asked for as soon as it was appointed to the delivery contract.

Says Magnall: "As the project progressed, there was more business involvement than at the start. I have seen data migration projects where the whole thing is driven from an IT perspective and business is not involved at all. What Bupa did is the only sensible way to approach this type of project."

Practitioners with similar experience of major data migration projects agree. At KPMG, advisory partner Steve Gallagher says: "I have been involved in both types, unfortunately.