Gartner defines continuous delivery (CD) as a software engineering approach where teams constantly produce valuable software in short cycles, while ensuring the end product can be reliably released at any time.
Continuous testing is also referred to as the final frontier of continuous delivery and crucial to any hope of releasing with confidence.
Of course, the most crucial aspect is to be able to achieve the latter with a clear view of the metrics that provide insight into the quality of deployments and value stream visibility, end-to-end.
To attain true CD, organisations today must rethink the way they carry out quality assurance (QA). This means rewriting the role that QA professionals play within their companies, striving for automation at every turn and utilising test harnesses to accommodate minimum viable product outcomes, and shorter test cycles.
To effectively compete in the app-focused environment, success is often measured in terms of the effective, and timely, release of software to obtain that competitive advantage.
If release processes are not aligned, poor QA on software can result in loss of that advantage, as consumers turn to competitors, simply because their software works better.
In today's application economy every company is a software company, regardless of industry sector.
To compete, every business must be able to quickly build and tweak software to adjust to ever-evolving market demands.
Ultimately, business success depends on faster deployment iterations while maintaining the high quality of service expected by customers, stakeholders and end-users.
What is CD's number one bottleneck?
Even as organisations have moved away from waterfall development practices and worked to bring together developers and operational staff to streamline the delivery pipeline, they still struggle to truly achieve CD.
The interloper that has yet to be fully integrated in the BizOps stream is testing. It is still seen as the necessary gateway and is often viewed as the roadblock to release.
Shifting how testing is done utilising a change in thought processes is the key to speeding up releases. This is, in short, the release with confidence value stream.
In today's application economy every company is a software company, regardless of industry sector.
The objective of continuous software delivery has recently driven numerous paradigm shifts in IT thinking. The Agile and DevOps philosophies and patterns of operation have gone a long way towards helping many leading enterprises to speed up their software delivery processes.
However, QA and testing practices are turning out to be one of the number one bottlenecks impeding the move to CD. Traditional testing procedures and technologies simply aren't equipped to handle the ever- increasing demand for faster testing. As a result, businesses are unable to truly achieve CD.
Studies show 63% of organisations that have adopted DevOps processes report their current QA processes are a bottleneck; 28% confirm their desire to deploy hourly, while 14% are doing just that.
Businesses cannot afford to let quality concerns go by the wayside. According to Forrester, it's not just about speed of delivery but rather speed with quality. If organisations are to speed up their delivery pipeline, they need to remember the success of CD hinges on the effective employment of continuous testing.
If software is to be continuously delivered, companies must ensure tests, procedures and processes around quality are designed to fit in line with the flow of development, integration and deployment. While continuous testing is dependent on strategic automation, it is not synonymous with automated testing.
A continuous testing process encompasses other QA activities that impact overall product quality. The truth of the matter is that the QA mindset is still shackled to old ideas and processes.
So, what are some of the outdated methods that create bottlenecks in the continual delivery and release with confidence process?
Delayed testing is a disaster in the making: 22% of organisations still don't write tests until after development work is completed. Waiting to test applications after the user interface is finalised is a recipe for disaster.
It is known that the later testing starts, the greater the cost of each defect found. The concept of shifting testing left is aimed at addressing these costs.
Delaying testing, on any of the parts of the application, is to invite expensive rework and poorly constructed user interactions. This strikes at the heart of the business object of application development, which is customer experience.
Three challenges facing continuous testing
Firstly, testing remains in a waterfall in its approach − basically testing operates in the same mode and process it has for decades; secondly, highly manual test processes; and thirdly, existing tools are too rigid.
To combat these key challenges, testing needs to shift to open architecture tools, agile modes of functioning, and test cases that are primed for automation, that build both functional and non-functional testing processes into every task that is performed.
It is crucial for organisations to shift their thinking and culture towards a continuous testing model. In this model, everybody in a business is accountable for quality or lack of it, and this includes business analysts, product owners, Devs and QA.
QA professionals need to ensure quality is front and centre of all endeavours. This can be done by seeking to fulfil three important duties:
Test strategist: Developers may be writing test scripts, but QAs should be advising and mentoring on how to effectively design tests. QA professionals bring unique perspectives around product quality issues that many developers may not have.
Toolsmith: This is a new breed of QA professional, not quite tester, not quite developer, but is a master of the right tool for the right test. The toolsmith's purpose is to evaluate new tools, ensure integration functions between tools, and integrate the QA environment into the DevOps pipeline.
Risk manager: The quality assurance risk manager's job is to guarantee that testing aligns with business processes. This ensures the right level of testing is performed at all steps in the pipeline, that scope and feature changes are properly understood, and testing remains agile enough to handle these changes. Ultimately, the risk manager is there to help the company think about how changes need to be deployed and when the level of risk is not acceptable. They are also there to build rollback procedures when problems crop up.
In the continual testing pipeline, QA should be supporting developers from the first line of code, allowing them to focus on what they do, and not on building unit tests.
Ultimately, the goal of these fundamental changes to QA is to enact a continuous feedback loop that shifts testing both to the left and to the right of the development process.
Businesses doing continuous testing understand that what tests well in the lab can still fail in production. Testing components early is crucial, but it is equally important to monitor performance from user transactions within applications once they go live.
Share