Our Client is one of the largest banks in the APAC region. They wanted to migrate from an on-prem data model to an Azure cloud data warehouse that is robust, secure, and scalable. The goal was to improve performance, reduce cost, and enable deep learning modelling. To achieve this, the company wanted to create a data strategy plan and a migration roadmap.
• Created and executed a migration plan
• Moved all data from current on-premise HDFS systems to the cloud
• Migrated 80+ on-premises databases plugged into SAP Sybase IQ using BladeBridge Converter to Databricks PySpark (Procedures in Scope)
• Migrated BODS Code manually on Azure Data Factory
• Assessment and migration of 200+ Cloudera jobs having 130TB data to Azure Databricks/HD Insights.
• Integrated all data assets in one central data repository allowing all data requests to flow to one place.
The migration allowed for enterprise level support for all mission-critical workloads from a central place. The data warehouse provided a single version of the truth for all data and enabled the use of machine learning models and AI algorithms.
The migration also resulted in a data storage cost reduction of 27% and the Client achieved an over 30% query performance improvement. A risk mitigation plan was also created as part of the implementation to ensure security, accuracy, and reliability of the data.
