In today's data-driven business, maintaining data accuracy and reliability is critical. Data observability offers a solution to monitoring and managing data across systems and pipelines.
This is the insight from a 'fireside chat' at the 2024 ITWeb BI Summitin Johannesburg this week, where industry thought leader Johan Steyn and Sanlam's data experts discussed the significance of data observability.
They explained that data observability involves proactively monitoring data quality, system health, and data lineage to ensure reliable insights and informed decision-making. This is not a one-and-done solution, but a continuous journey.
Elma Salt, technical delivery manager at Sanlam, underscored the importance of establishing robust processes to ensure data quality and reliability, avoiding errors and building trust within the organisation. “The challenges of data observability often lie in prioritising the processes needed, yet the business value it brings forth is immense,” she noted.
Iaan Theron, head of quality and integration at Sanlam, outlined the tangible benefits of data observability, including improved predictability in data outcomes, fewer errors, and reduced operational risks as notable achievements within Sanlam.
"With data observability, we prioritise predictability ensuring the seamless functionality of our dashboards and the pre-emptive identification of potential errors before they compromise decision-making processes," explained Theron.
Sanlam leverages a range of platforms to achieve data observability. Salt elaborated on the company's utilisation of the Splunk platform for data monitoring, the Tableau platform for data analysis, and Microsoft Teams for collaborative needs.
“You have to research and see which platforms work for your business, factoring in variables such as cloud-based environments, technological infrastructure, and organisational climate,” said Salt
Theron said Sanlam’s approach combines advanced tools with a focus on knowledge-sharing. “We can't underestimate the value of our people as we implement data observability. Their experience with our older systems, paired with the right tools, creates a culture of continuous improvement,” he concluded.
Share