Empowering Users, Reducing Costs: How This Manufacturer Revolutionized Business Intelligence
Stuck in Excel hell? This manufacturer ditched slow cubes for real-time dashboards and self-service BI. Discover how they reduced costs, empowered users, and gained faster business insights.Introduction
We were engaged by a manufacturing company which was being effectively run on disparate Excel sheets despite having an ERP. In addition, Finance Team had an OLAP Cube base Reporting System which was being used to generate Sales Reporting. They were trying to make sense of all the data and data consolidation and relationship between various data points was missing.
Challenge
We did a deep dive and find out why the Analytical systems were running slow and not delivering value.
The existing OLAP tool aggregated and then cached subsets of data within its nested array. Thus, Sales Analysis was done within this cube and often new cubes had to be created for new business scenarios. The data transformation into cubes was taking an extraordinary time.
In theory, all the ETL pipelines would run overnight to do the heavy lifting before the business users started work in the morning. This also meant that the data within the databases needed to be modelled for analytical processing using a Kimball Data model. There was also another Data Pipeline team being used too maintain the second set of pipelines from the modelled data to the cube. This approach came with a number of problems of its own. IT Team felt powerless as it was not able to meet the demands of business using this solution architecture.
Approach
We soon realised that that business needs to move towards a modern cloud-based platform which would give them unlimited storage and compute power. With Columnar database structures in the cloud data warehouse, BI teams could begin performing OLAP-type workloads at high performance without needing to extract data and build cubes. By eliminating the need for separate cubes or pre-aggregated tables, this organization could perform complex analytical tasks directly within the data warehouse environment.
We suggested Qlik Sense, self-service BI platform for data discovery and visualization. Qlik Sense Associative Engine indexes and connects relationships between data points for creating actionable insights.
The self-service capabilities, collaboration features, business automation, and service-oriented paradigm aligned well with the goals of improving data discovery, visualization, and reducing TCO for this organisation. We were able to achieve Business buy-in by demonstrating the benefits of applying lineage to such an important business function.
Benefits
- Real-time Analytics: Existing cubes and pre-aggregated tables required periodic updates, leading to latency in reporting. By performing analytics directly in the data warehouse, business could access real-time or near-real-time insights.
- Simplified Architecture: Maintaining separate cubes or pre-aggregated tables was adding complexity to the architecture. By consolidating both transactional and analytical workloads in the data warehouse, business could streamline their infrastructure and reduce maintenance overhead.
Results
- Towards the end, we enabled Self Service and empowered users to explore and create visualizations without extensive IT involvement, enabling faster insights and decision-making. We reduced the total cost of ownership (TCO) for reporting and provided cloud-based deployment options and subscription-based pricing models.
Key Learnings
Process of proper planning of a cloud deployment greatly increases the chances of success.
Modern cloud-based platforms and applications will accelerate your business intelligence.
Access to live row level detailed data instead of building rigid pre-modelled exploration via cubes leads to reduction in Total Cost of Ownership.