Surveyed South African data scientists want a seat at the boardroom table, noting they are dissatisfied with their company’s use of data analytics to inform decision-making.
This is one of the key findings of the Accelerating Digital Transformation report commissioned by SAS in 2021. The research delved into the state of data science across the globe, by assessing the impact of the pandemic, challenges faced and overall satisfaction with the analytics environment.
It is based on a survey of almost 300 global and local data professionals, with the quantitative portion of the research augmented with qualitative interviews with data experts.
It notes the work of data scientists has grown in importance since the onset of COVID-19, with many organisations accelerating digital transformation projects by using technology to improve business operations. More than 90% of respondents indicated greater importance and demand for their work compared to before the pandemic.
According to the research, digital transformation has accelerated significantly due to the pandemic, but the extra demands placed on data scientists by employers revealed significant barriers to effective working and high levels of job dissatisfaction in some areas.
For example, around four in 10 respondents were dissatisfied with their company’s use of analytics and model deployment, while more than 20 barriers to effective working were identified,caused by the radical shift in the way companies deliver products and services.
This was backed by respondents noting that data extracted for data science purposes within the organisation was not used by business decision-makers, making it one of the main barriers they faced.
This, according to the report, suggests a problem with how analytical insights are used by organisations to inform decision-making. Among the recommendations, the respondents noted that, as people responsible for analysing and extracting value from data, data scientists should get a seat at the boardroom table.
“There have clearly been more demands placed on data scientists as the pandemic has accelerated digital transformation projects that many organisations were planning anyway,” says Dr Iain Brown, head of data science at SAS UK and Ireland.
“A major source of frustration is finding a way for organisations to implement the insights from analytics projects and use them in their decision-making, which means giving data scientists a seat at the boardroom table might be a way forward.
“Linked to this, we found concerns around support for data science teams and a lack of talent, which has been an issue for some time, with demand outstripping supply.”
The report also highlighted some specific skills gaps – less than a third of the respondents reported having advanced or expert proficiency in program-heavy skills, such as cloud management and database administration. This is an issue given that use of cloud services is up significantly, with 94% saying they experienced the same or greater use of cloud since COVID-19 struck.
The amount of time spent on data preparation versus model creation is another challenge experienced by data scientists. Respondents spend more of their time (58%) than they would prefer on gathering, exploring, managing and cleaning data.
“Organisations must realise that investing in a team of data scientists with complementary skills could reap huge value for the business, so the cost of hiring needs to consider the return on that investment as we move to significantly more digital and AI-driven business processes,” noted Brown.
The research also identified gaps in consistent organisational emphasis on artificial intelligence ethics, with 43% of respondents indicating their organisation does not conduct specific reviews of its analytical processes with respect to bias and discrimination. Only 26% of respondents reported that unfair bias is used as a measure of model success in their organisation.
“Data science should be represented at any boardroom table, to bridge the gap between the technical and business. However, everyone at the table should also be moving forward towards analytics and recognise its importance,” says Kelly Lu, lead on advanced analytics and artificial intelligence at SAS South Africa.
“Although we have come a long way and many organisations are aiming to become more data- and analytics-driven, data scientists complain about their colleagues sometimes ignoring the analytics or overriding decisions based on gut feel or experience. While this can be very frustrating, it’s something firms can overcome with effective communication and better collaboration.”
Share