Whether talking traditional big stack business intelligence (BI) solutions or new visual analytics tools, it's an unfortunate fact that enterprises still buy into the candy-coated vision of BI, without fully addressing the underlying factors that make BI successful, cost-effective and sustainable.
Many companies wrongly assume that in data, nothing changes.
Information management is a double-edged sword. Well architected, governed and sustainable BI will deliver the kind of data business needs to make strategic decisions. But BI projects built on ungoverned, unqualified data/information and undermined by 'rebel' or shadow BI will deliver skewed and inaccurate information. Any enterprise basing its decisions on bad information is making a costly mistake. Too many companies have been doing the latter, resulting in failed BI implementations and investment losses.
For more than a decade, I have been urging enterprises to formalise and architect their enterprise information management (EIM) competencies based on best-practice or industry standards, which follow an architected approach and are subjected to governance.
EIM is a complex environment that needs to be governed and which encompasses data warehousing, BI, traditional data management, enterprise information architecture, data integration, data quality management, master data management, data management life cycle, information life cycle management, records and content management, metadata management and security/privacy management.
Keeping up
Effective governance is an ongoing challenge, particularly in an environment in which business must move at an increasingly rapid pace where information changes all the time.
For example, to tackle the governance issue in context of data quality starts with the matching and merging of historical data to ensure design and storage conventions are aligned and all data is accurate, but according to set rules and standards.
It is not just a matter of plugging in a BI solution that would give the results. It may require up to a year of careful design and architecture to integrate data from various departments and sources in order to feed the BI system. The conventions across departments within a single organisation are often dissimilar, and all data has to be integrated and qualified. Even data as apparently straightforward as a customer's ID number may be incorrect - with digits transposed, coded differently between source systems, or missing - so the company must decide which data source or integration rule to trust in order to ensure data warehouses are compliant with quality rules. Data warehouses must also be compliant with legislation standards needed to build the foundation of the 360-degree view of the customer that executive management aspires to. But, integrating the data and addressing data quality is only one area where effective governance must be applied.
Many companies wrongly assume that in data, nothing changes. But, in reality, the company must cater for constant change. For example, when reporting in a bank, customer records can be dramatically incorrect if the data fails to reflect that certain customers have moved to new cities, or that bank branch hierarchies have changed. Therefore, linking and change tracking is crucial in ensuring data integrity and accurate current and historic reporting.
Automation can only take the company so far: it can automate to the nth degree, but it still requires data stewards to carry out certain manual verifications to ensure the data is correct and remains so. Companies need to know who is responsible and accountable for their data and be able to monitor and control the life cycle process from one end to the other. The goals are to eliminate multiple versions of the truth (results), have a trail back to sources and ensure only the trusted version of the truth is integrated into systems.
Rebel without a cause
Another challenge in the way of effective information management is the existence of 'rebel' or shadow data systems. In most companies, departments frustrated by slow delivery from IT or with unique data requirements start working in siloes, creating their own spreadsheets, duplicating data and processes, and not inputting all the data back into the central architecture.
This undermines effective data governance and results in huge overall costs for the company. Instead, all users should follow the correct processes and table their requirements, and the BI system should be architected to cater for these new requirements. It all needs to come through the central architecture. In this way, the entire ecosystem can be governed effectively and data/information can be delivered from one place, also making management thereof easier and more cost-effective.
The right information management processes also have to be put in place, and they must be sustainable. This is where many BI projects fail - a company builds a solution and it lasts only a year, because no supporting frameworks were put in place to make it sustainable. Companies need to take a standards-based, architected approach to ensure EIM and governance is sustained and perpetuated.
New BI solutions and best practice models emerge continually, but will not solve the business and operational problems if they are implemented in an ungoverned environment, much the way a beautiful luxury car may have all the features a driver needs, but unless the driver is disciplined, it will not perform as it should.
Share