Continuous Testing Report

• New report reveals that Quality Assurance and Testing is a bottleneck to implementing DevOps and Continuous Delivery for many organizations
• Fragmented test automation landscape is slowing down the adoption of Continuous Testing
• With 58% of enterprises surveyed deploying a new build daily (and 26% at least hourly), Continuous Testing is becoming a necessary part of the DevOps and agile development ecosystem

Paris, March 11, 2019 – Capgemini and Sogeti, part of the Capgemini Group, today released a new report on Continuous Testing, in collaboration with Broadcom Inc.’s Enterprise Software Division. According to the report, the practice of Continuous Testing – the process of fast and efficient validation of software releases in agile developments through highly automated tests – is gaining ground in large enterprises, with almost a third of IT executives (32%) stating that their IT departments had ‘fully embraced Continuous Testing’. However, with 58% of enterprises surveyed deploying a new build daily (and 26% at least hourly), the report highlights that companies must work to improve their continuous testing effectiveness by streamlining their test environment within an agile development ecosystem.

The report is based on 500 interviews with senior-level IT executives from large and medium-sized companies (over 1,000 employees) in industries including financial services, high tech, healthcare and life sciences, telecommunications, media and entertainment, and manufacturing. It found:

Companies must unlock the potential of automation in testing

The study strongly highlights that there is a significant scope to optimize Continuous Testing processes through technology. It found that automation was only being used to execute 24% of test cases, 24% of end-to-end business scenarios, and to generate 25% of the required test data.

Greater use of automation could significantly improve the velocity of testing activities in the agile teams; for example, over a third (36%) of those surveyed said that more than 50% of testing time is spent searching, managing, maintaining and generating test data. The report recommends that to respond to customer and market needs, all enterprises need to take significant action.

Use cases, detailed in the report, include a leading Australian bank, which had over 5,000 builds for more than 100 applications. By harnessing a single platform for automated build and deployment, integrated with testing and automation tools, it was able to reduce build cycle time by 40%, improve time to market and significantly increase environment uptime.

Enterprises need smarter orchestration and enablement for testing

The self-empowerment of autonomous teams resulted for many enterprises in an uncontrolled landscape with a broad diversity in QA and test automation approaches.  To regain control, cites the report, organizations need to improve the central quality enablement of the agile teams with clearer QA guidelines and smarter QA technology provisioning. A promising development is to make the test orchestration and execution much smarter via artificial intelligence (AI) technologies which provide “smart” test orchestration. With the addition of machine learning capabilities, systems will be able to automatically determine the tests that are required in the release and production cycles.

The report showed a clear need for improved transparency and orchestration in agile testing. Among the executives surveyed, 35% identified a ‘complete audit trail of testing activities’ and a ‘consolidated test and release pipeline’ as the most important test orchestration capabilities, with 32% highlighting a need for a ‘single-place for cross-team collaboration’ and ‘continuous delivery pipeline visibility’.

The shortage of centralized enablement on technology provisioning becomes most apparent when investigating the challenges with test environments. Teams wasted too much time procuring complete test environments. Four in ten (40%) of the respondents said their teams spend more than half of their time building and maintaining their test environments.

“The next two to three years will be a critical time for Continuous Testing as organizations need to solve the dilemma of transitioning to self-empowered autonomous teams where quality is everyone’s responsibility, while improving the central quality support and central quality oversight to create an agile development environment, that is truly able to deliver value in an increasingly complex IT landscape.” said Mark Buenen, Global Leader, Digital Assurance and Testing for the Capgemini Group. “This report clearly demonstrates the need to expand the levels of test automation, make provisioning of test environments and test data smarter, implementing meaningful quality metrics and smart quality dashboards as the core focus areas.”

Inter-disciplinary teams require new skills and support

In the last three to five years, the roles of both developers and testers have evolved significantly. According to the report, developers are now much closer to the customer, with a prominent role in shaping the user experience, while testers have moved out of siloed teams to work in parallel with developers and business teams, meaning that they are involved much earlier in the development life cycle. The roles and responsibilities of developers and testers are blurring, but clearly it remains important to have QA and test focused experts in the agile teams.

These inter-disciplinary teams mark a move forward, but also create challenges, details the report. They require every team member to have a holistic understanding of the entire process, with testers needing to upgrade their technical skills. Companies must address the requirement for upskilling and a new, integrated approach to truly achieve Continuous Testing’s full potential.

“It’s difficult to fully capitalize on the benefits of Agile and DevOps without Continuous Test automation, which enables organizations to leverage data and intelligence to accelerate the creation of high quality software at the pace of digital business,” said Ashok Reddy, Senior Vice President and General Manager, Enterprise Software Division at Broadcom. “As the chasm between IT and business leaders continues to close, new DevOps practices and techniques, such as automating continuous testing processes, will begin to emerge as necessary tools to drive digital transformation in a hyper-competitive marketplace.”

Notes to Editors

Methodology

The Continuous Testing Report (CTR) 2019 brings together survey data and subject matter expert contributions to outline the challenges and potential approaches to transforming test practices in the age of Agile and DevOps. It is based on the considered opinions of several subject matter experts from Capgemini, Sogeti, and Broadcom, buttressed by the results of a global survey of 500 senior decision-makers in corporate IT functions, working for companies and public-sector organizations across eight different regions.

About Capgemini and Sogeti:

A global leader in consulting, technology services and digital transformation, Capgemini is at the forefront of innovation to address the entire breadth of clients’ opportunities in the evolving world of cloud, digital and platforms. Building on its strong 50-year heritage and deep industry-specific expertise, Capgemini enables organizations to realize their business ambitions through an array of services from strategy to operations. Capgemini is driven by the conviction that the business value of technology comes from and through people. It is a multicultural company of over 200,000 team members in more than 40 countries. The Group reported 2018 global revenues of EUR 13.2 billion.

Part of the Capgemini Group, Sogeti operates in more than 100 locations globally. Working closely with clients and partners to take full advantage of the opportunities of technology, Sogeti combines agility and speed of implementation to tailor innovative future-focused solutions in Digital Assurance and Testing, Cloud and Cybersecurity, all fueled by AI and automation. With its hands-on ‘value in the making’ approach and passion for technology, Sogeti helps organizations implement their digital journeys at speed.

Visit us at www.capgemini.com. People matter, results count.

Broadcom, the pulse logo, Connecting everything, CA Technologies and the CA technologies logo are among the trademarks of Broadcom. The term “Broadcom” refers to Broadcom Inc. and/or its subsidiaries.