Challenge
An international life sciences organisation was locked into an proprietary ERP data platform delivered by a third party managed service. The costs had increased significantly over the past few years and levels of service were not sufficient. In addition they had no ownership of the data platform and reporting.
The organisation wanted to build a data platform under their own control and in their existing Azure infrastructure, therefore enabling their existing data team to run the platform. However, they did not have experience with the underlying ERP system (Business Central) and its integration requirements.
Solution
Subscriptions were set up within the organisations existing Azure Tenant and dev, test and prod environments were created.
Business Central to Azure Data Lake Storage (BC2ADLS) was used to incrementally ingest the data from the ERP solution into the data lake. Ingested data was transformed into Parquet data format for efficient storage and processing using Data Factory pipelines. It was then validated and passed through environments using DevOps pipelines.
Data was then available via the data lakes to ingest into their existing data platform. The clients team were trained in use of the solution and supported through a hypercare period.
Project Details
Project Duration - 1 week design, 2 week build, 2 week validation
Project Team - 1 delivery lead, 1 senior data engineer, 1 junior data engineer
Benefits
The organisation gained 100% ownership of their ERP data platform.
Storage and processing costs were significantly reduced.
Data was available on a much more frequent timeline (daily->hourly)