Data Virtualization Using Denodo for Multi-National Logistics and Real Estate Client
Trianz teamed up with a multi-national logistics and real estate investment company to modernize its data warehousing platform. The aim was to help the client scale and cater to growing business needs, enabled by on-demand virtual data models in Denodo.
Challenges for the Logistics and Real Estate Company in Data Operations
The client had numerous business departments which needed reliable, real-time access to datasets. Data sources were diverse and varied, complicating any attempts to integrate data and form a single-source-of-truth.
This logistics and real estate company wanted a quick turnaround on development, and for the new solution to eliminate the need for excess data replication and extract-transform-load processes. They also wanted a self-service option to further reduce workloads for data and IT personnel.
Technology Components Identified by Our Experts
After assessing the current state of its infrastructure and project requirements, Trianz and the client agreed on the following technology components:
Denodo is a data virtualization platform that provides agile, high-performance data integration and abstraction services with zero data replication as-standard.
Snowflake is a cloud data warehousing and data lake platform that supports pay-as-you-go (PAYG) billing and high-throughput raw data processing.
AWS Lambda is a serverless platform for compute operations that runs software processes with zero provisioning for a minimal IT operations management (ITOM) burden.
Informatica Intelligent Cloud Services (IICS) is a cloud data integration service that supports synchronization, replication, and the mass ingestion of datasets.
Tableau was chosen as a data visualization and business intelligence platform, creating more valuable insights by converting raw data into visual dashboards and scorecards.
How Trianz Helped the Logistics and Real Estate Client Meet its Data Objectives
There were three primary pillars to the approach taken by our data experts.
First is the Canonical Building Blocks Model. This constituted the building of a reusable, consistent, and governed accessibility layer using the Denodo Logical Data Warehouse, Data Catalog, Embedded Metadata and Row-level Security.
Second is a Data Science Toolkit. Denodo uses a microservices architecture—this allowed our experts to develop HTTP data inputs and data API outputs, alongside data caching parameters. As a toolkit, the client could use this in the future to develop on-demand data analytics models.
The third and final pillar is the Pipeline-based Deployment Model. We used Terraform to automate data ingestion and loading, building CI/CD pipelines to connect back to Snowflake and AWS Lambda. Denodo Config Backup was adopted to reduce routine downtime, enable effective configuration management, and streamline disaster recovery operations.
Transformational Effects for the Logistics and Real Estate Client with Denodo
Virtualized building blocks in Denodo were targeted to well-documented business areas for faster analysis, insight generation and return on investment. Other business areas included as definitions were established.
On-demand data models reduced initiation times for analytics projects, reducing the time needed to generate insight. This also helped with the development of API-based models and faster go-to-market.
CI/CD pipelines streamlined data orchestration for the client. The end-state was one-click data deployment in Denodo with streamlined upgrade processes, a centralized configuration management database (CMDB), and auto-scaling of data and compute resources in Snowflake and AWS Lambda.
Contact Us Today
Contact Us Today
