The DWP Turns to the G-cloud to Solve Big Data Issues

The Data and Analytics Directorate sits within the Department for Work and Pensions (DWP), providing a range of services to customers across government: from analytical data sets to inform Spending Reviews and policy research, to fraud/error reporting within the benefits arena, to data matching for the verification and validation of claims eligibility, National Insurance numbers, electoral registration etc.

The Directorate is a highly complex Big Data environment, housing over 200TB of data, handling over 200 data feeds and supporting 600 users across the Directorate, the DWP and various central government organisations.

When the DWP determined that the on-premises hosting of its core data warehouse was no longer viable, it turned to G-Cloud to procure a fit-for-purpose alternative that would fully leverage the benefits of an assured private Cloud provision.

Challenge

The DWP had two main issues with its existing data warehouse, both essentially corollaries of an in-house hosting solution that just grew organically over time without the requisite control or investment. First, it reached that classic tipping point where the legacy hardware was end-of-life and much of the software out of support; and secondly, the historical physical hosting space was subject to regular power outages and lacked any of the redundancy and resilience consistent with its status as a key piece of infrastructure. At its lowest point, system availability was measured at just 92%, and given the value of the services the Directorate supplied and the high costs of performance-related penalty clauses it was subject to, this was deemed unsustainable.

Requirement

The DWP decided to make its number one priority to rehost the data warehouse on a modern, reliable, secure and fully supported platform. Procuring via G-Cloud, the solution requirement was for:

• An Oracle Database-as-a-Service, delivered as a Platform-as-a-Service (PaaS) private Cloud, incorporating migration of large-scale legacy datasets through to live system and database administration
• The provision of OFFICIAL-SENSITIVE security level services from multiple sovereign (UK) data centres and the ability to demonstrably meet DWP departmental accreditation security requirements
• An architecture based on commodity (x86) infrastructure and standards-based technology
• An advanced service management and expert Oracle support wrap

How Redcentric helped

Colin Mee, Head of Technical Support and Solutions Group, Department for Work and Pensions, takes up the story: “While G-Cloud enabled us to select a supplier fairly painlessly, in as much as Redcentric’s catalogue service description and commercial offering outscored its competitors, in one sense it also gave us concerns, at least at the outset. G-Cloud is a bit like online dating – the profiles look great on paper but how will you get on in real life? This project was never going to be a hands-off, remote, automatic and anonymous lift and shift, where it didn’t matter if the parties didn’t get on as long as the new technical environment was delivered to spec. The complexity of the migration, the criticality of system availability, the magnitude of the project and the inherent risk, these all demanded the closest, most trusting, open, committed relationship from day one. Redcentric’s raw technical capabilities could easily be gauged from the outset – but its ability to work effectively with us to deliver our number one priority? That was an unknown initially.

As fate would have it, we found out earlier than we might have about the character and calibre of our new supplier. We had, in essence, a change of scope, one that would significantly add to the work products, introduce yet more complexity, add to the risk factors, but that was highly desirable both operationally in the short and medium term, and strategically over the long-term.

Redcentric could have quite legitimately used the contract to make this an issue but the business was refreshingly pragmatic and phlegmatic, and far more interested in pushing on and seeing how together we could get this modified project over the line. So the change in operating systems we’d unexpectedly added in to the mix simply got addressed without fanfare; this was an all the more commendable response given the existing pressures around on-going file conversion work, and the challenges of working with a live running scenario where we had to keep the data synchronised.

And to be honest, that really set the tone for the rest of the engagement, not least when Redcentric’s preferred measured go-live plan needed radical amendment to accommodate our insistence on a single weekend cutover! Of course there were bumps in the road but we had the partnership and dialogue going that ensured we always had common purpose and could work through our issues collectively and with shared responsibility. The potential on paper has been fully realised in a very good working relationship.”

Realising the Benefits

“Because of the complexity of the environment, and because we had previously already had a couple of unsuccessful attempts at achieving this, we really didn’t think it could be one. But Redcentric have proved us wrong. They really pulled out all the stops and we now have a highly secure, extremely reliable, Cloud-based platform which has helped us save costs and improve performance.” – Colin Mee, Head of Technical Support and Solutions Group, DWP

Business benefits

Colin Mee: “Back in 2015 this was perceived as ‘a project that couldn’t be done’. The fact that as we sit here today we have enjoyed more than 150 days of continuous uptime demonstrates in the clearest possible terms that not only could it be done, but done really, really well. Redcentric has shown consistent proficiency and professionalism, with a technical quality allied to a customer centricity that gave us confidence from the off that anything was indeed possible.”

Fit for purpose and future-proof The DWP’s core atomic data store is now housed in an environment tailored to its exacting requirements, supported, managed and refreshed

Enhanced resilience

The data warehouse is currently running at 100% availability. This compares with a low of 92% with the previous legacy infrastructure.

Commercial advantage – The project not only came in within budget but proved that the DWP could successfully move away from long-term supplier contracts and benefit from the inherent competitiveness of G-Cloud

Improved performance – One example of the many significant improvements is that the monthly processing time for a critical citizen service function has been reduced by 66% from the previous 150 hours (6 days) to 50 hours.

Reduced risk – Given its key infrastructure status, any issues with the warehouse affect service to other departments, incurring financial penalties for the DWP. The new hosting arrangements are now safeguarding requisite performance levels.

Improved flexibility – The project has secured the transition from an end-of-life proprietary platform to a commodity (x86)-based secure, scalable and highly available private Cloud, securing a more adaptive, agile environment going forward

Solution Info

• Platform as a Service (PaaS)
• Database as a Service (DBaaS)
• Infrastructure as a Service (IaaS)
• Professional Services
• Project Management
  – Infrastructure, Technical and Solution Architecture
  – Network Architecture and Network Engineering
  – Migration Planning
  – Hardware Engineers
  – VMware Engineers
  – Operating System Engineers, Including Windows, Oracle Solaris and Linux
  – Oracle Database Migration Consultancy
  – Oracle Database Administration
  – Backup, Recovery and Disaster Recovery Consultancy
  – Storage Engineers
  – IT Security Expertise
• Management and Monitoring Administrators