Seeing is Believing: How Data Cataloguing Transformed Decision-Making for a Logistics Giant

We were engaged by a logistics company that used a 3rd party product for freight logistics. Each quarter the application vendor upgraded their software. Whenever the upgrade happened, the client’s IT Team needed to upgrade their application to stay compliant and leverage the full product functionality. Since this was a relatively large organisation, the client had a lot of custom code in downstream applications for reporting and sending this data to its sister companies and service providers. The choice for them was to delay the upgrade and expose themselves to security flaws or do throw its IT resources to conduct a manual impact analysis. They were trying to make sense of all the data and relationship between various data points was missing, which required a comprehensive Data Governance Program.

The importance of Data Lineage was not known to the client executives and they feared adding new products to their application stack. We also encountered resistance from a few data stakeholders who had a lack of awareness and training on the importance and use of data lineage and held a belief that Data Lineage across systems was not possible to generate. We also encountered resistance from the 3rd party product vendor who were not keen to allow access to their internal schema.

We suggested a data lineage solution that would automatically find this impact and offer many other capabilities as a bonus. We helped the client deploy a Data Catalouging solution which crawled a variety of data sources like the 3rd party products, Sharepoint, SQL Server and SSRS Reports an and build the data lineage from source systems to reporting software. Now finding the impact took a fraction of the time it took earlier. We had workshops with the 3rd party vendor and convinced them that we need to access only the Metadata and not the real data for us to generate our Data Lineage and that their data would stay protected. We proposed a list of questions that needed to be resolved satisfactorily before going for a full-scale enterprise wide implementation and had workshops for elucidating more specific user requirements. Seeing the value in this exercise, the client also started its Data Governance journey with the appointment of Data Stewards and Data Owners. These key people helped start the development of a business glossary and were willing to spend time and effort to curate and define business metadata. We started with highly used and accessed data assets and worked our way down to the data in a systematic way. We were able to achieve Business buy-in by demonstrating the benefits of applying lineage to such an important business function.