Deep dive into impact of IT infrastructure failure

Keenan Naidoo, divisional system manager, Argility Technology Group.

---

While the obvious impact of load-shedding is literally a matter of keeping the lights on, the ripple effect of the power outages is a deep one for retailers that have incurred:

• Significant increased costs due to investments in backup generators, fuel and water storage facilities. Research shared by Trade Intelligence states that organisations allocated millions of rands to provide facilities that ensure operational continuity.
• Increased prices for goods and services being passed back to consumers due to these added costs.
• Stock shortages and decreased ability to fulfil demand forecasting needs. Disruptions in manufacturing processes results in delays in production and reduced output. Stock outages impact on revenue targets and customer experience.
• Different shopping habits in certain verticals, specifically in FMCG. According to a survey conducted by Chirp and Trade Intelligence, 70% of shoppers reported their grocery purchasing and food preparation approach has changed.
• Decreased consumer spending due to increased costs and economic uncertainty.

Retailers are a resilient bunch, and there is a saying from Brene Brown that vulnerability spurs creativity. Apart from investment in backup power solutions, the following are some of the interesting measures that retailers have taken to cope with the impact of load-shedding:

• Adjustment of business hours to facilitate trading to times when power is available, thereby reducing reliance on backup power sources.
• Enhancement of customer in-store experiences due to backup power planning and approach.
• Change of route to market strategies by streamlining supply chain.
• Creative approaches in the prevention of passing costs across to the consumer.
• Adoption of predictive analytics to cater for adjustments of inventory, pricing and promotional strategies to meet the changes in product and shopping patterns.
• Adoption of cloud always-on technologies like Google Cloud and Google Workspace to limit the impact on systems, payment structures and operational teams, while improving security.
• Development and focus on business continuity plans to prepare and manage grid impacts.

Let’s take a deep dive into the impact of IT infrastructure failures. These system failures occur when there are either no backup power solutions available or they fail. This situation is usually amplified by the absence of procedural shutdown processes. It is important to understand that where there is a sudden loss of power, almost all IT hardware is susceptible to failure, especially equipment with moving parts.

Data and its integrity is considered one of the most valuable and important assets within the IT infrastructure; therefore, the failure of storage mediums pose the highest risk to retailers. For example, most furniture retailers run a fully operational back-office which allows them to trade when networking systems fail.

The back-offices provide a full feature set within the store environment if there is no networking, and this is dependent on data being stored on-premises, on servers. The most economical way of storing data is on hard disk drives that utilise spinning magnetic disks.

Peripheral devices can also fail when there is a loss of power, which can sometimes leave the business inoperable.

This is a mature, well-tested approach; however, due to the age of the technology, it is ultimately dependent on the availability of power. When there is a sudden power loss, in-process transactions fail. This can lead to data integrity issues or file system corruption. There are redundancy solutions that can be implemented; however, these are also dependent on power so the solution can update.

In recent years, retailers have been making the switch to solid-state drives (SSD). While still prone to failure due to power outages, the speed at which SSDs process transactions means that in-process transactions are given the best opportunity to finalise.

Component failures include IT components like motherboards, power supplies, chipsets, etc. When these occur, they can leave the system totally incapacitated; however, the chances of these failures causing a loss of data or corruption are miniscule.

Moreover, retailers usually have agreements in place with vendors to replace broken components as soon as possible. This in turn reduces downtime but of course, as with everything – this comes at a cost.

Peripheral devices can also fail when there is a loss of power, which can sometimes leave the business inoperable. Certain peripherals are cheap enough to keep spares; however, larger items such as enterprise printers are extremely expensive and can cause delays in business operations. These failures have no impact on the loss or corruption of data.

There are numerous costs that retailers face in the event of IT infrastructure failures – these can be described as tangible and intangible.

Tangible costs include:

• Replacement or repair cost for the equipment.
• Service provider call out fees, plus professional time and material costs.
• Providers’ software restoration fees – again plus time and material costs.
• Loss of revenue generation activities due to downtime.

Intangible costs include:

Brand damage in the face of retailers with no backup systems in place. This can have serious negative impact on consumer confidence in the brand.

Service providers to retailers must ensure their resources are geared up and adequate enough to cater to the increasing demands placed on them due to failures related to load-shedding.

Retailers seeking technology providers need to look for companies with dedicated teams that aim to meet their needs in project management, development management, business and systems analytics, software development and support specialists.

In our business, we regularly see the rise in disk failures entirely attributable to load-shedding. Retailers need to look to suppliers that have innovative, developed solutions that assist businesses to mitigate the negative effect on their ability to operate. They must seek solutions that aim to drive value by preventing other costs related to failures and which are triggered by load-shedding.