Achieving continuous automated testing and smart test data provisioning

One of the key success factors with Agile and DevOps is building a continuous integration and deployment pipeline that can help replace manual activities with a fully automated process. And it’s crucial that testing activities be fully integrated in this. In reality, most Agile teams struggle with successfully automating their testing. Capgemini’s market surveys from our World Quality and Continuous Testing reports show that 68% of organizations consider their test automation levels to be too low. And this can have serious consequences with decisions to not execute all tests before deployment or stretched feature delivery timelines. Both are clearly not acceptable.

In our analysis, we’ve found that teams typically experience five challenges in attaining successful test automation:

• The time given per sprint is often too short to successfully build the test automation script
• Previously automated test cases fail to run after the application changes in subsequent sprints, and repair of the test scripts takes too long
• The test environment configuration and test data are not consistent across multiple test runs and, as a consequence, previously automated tests no longer run
• Each team in the organization has their own approach and tool stack for test automation, making it a challenge to re-use automated tests across teams.

To address these challenges, the following measures should be considered:

• Installing a test automation support squad to help teams set up their test automation strategies and script in the right way at an early stage
• Providing an enterprise automation platform and automation guidelines so that individual teams do not have to (re-)invent their own wheel
• Training teams in test automation tools and techniques
• Focusing test automation at API-level testing, as this is more resilient to application changes
• Implementing test environment and test data generation solutions to ensure a reliable, repeatable, and consistent ecosystem for automated testing
• Implementing solutions that automatically generate test scripts from requirements to design.

And while the level of automation activities is never a target in itself, I truly recommend implementing one simple indicator for every team – the percentage of automated test cases. This will give the team and product owners more transparency and enable better targeting by the test automation support squad.

Capgemini’s embedded Quality Engineering in ADMnext has over 10,000 test automation specialists at its disposal, which can help teams create and enhance resilient and maintainable test automation scripts. Naturally, these are fully integrated into the CI/CD pipeline.

Bringing smart test data provisioning for Agile teams

Having the right test data is a straightforward pre-requisite for any test. But it’s easier said than done. You need very different types of data, depending on each type of test. Early-stage functional tests require carefully selected examples of customers, orders, transactions, etc., with specific characteristics.

For performance tests you need real-life volume data, while end-to-end business scenario tests require selected data instances in a completely realistic and consistent test data set. Most teams spend 30% to 40% of their efforts just on getting the test data right. Then there’s the added challenge of complying with regulations like GDPR and other privacy and security rules. It’s clear with all of this that the provisioning of test data is an Achilles heel for the speed of software deliveries.

Copying data sets from production and using data masking (also known as data obfuscation) technologies is a common approach to generate test data. However, this can have a few flaws: extraction of data from production can be a cumbersome and time-consuming process. Masking data can destroy some of the referential integrity of the data set. And the masking of data is not always sufficient to comply with privacy regulations. For example, intelligent tools can often re-engineer masked data, and with that, you are no longer compliant.

As part of Capgemini’s embedded Quality Engineering in ADMnext, we’ve developed ADA (Artificial Data Amplifier) to address this issue. ADA is an AI-based solution that can generate a complete synthetic data set that has all the real-life characteristics required for your test. This solution can generate any type of data record (tabular data, images, text, etc.), based on a small sample set of your actual data. ADA can do this for any type of system, as long as there is a sample set of real-life data to work with. We implement ADA as a service or on-premises offering, depending on your needs and demands.

Essentially, ADA guarantees full compliancy and consistency of test data – enabling a major part of the effort for creating test data to be prevented.

In my next post, we’ll look into how you can bring complete quality transparency to your product owners and business leaders.

In the meantime, to learn more about what Capgemini’s embedded Quality Engineering in ADMnext can do for your business, visit us here .

You can also learn more about Capgemini’s wider ADMnext offering here. And to get started right now, drop me a line here.

Mark Buenen

Expert in Quality Assurance, Test Automation

I pro-actively drive  the change of test services to stay ahead of the evolving client expectations. The major objectives of the past have been higher quality and higher efficiency of testing. The future objective will be truly transformational services that lead to ongoing quality improvement in all areas of IT development and maintenance.