Unlocking the power of AI with data management

This article first appeared on Capgemini’s Data-powered Innovation Review | Wave 3.

Written by:

Jitesh GhaiChief Product OfficerInformatica

In today’s data-driven economy, artificial intelligence (AI) and machine learning (ML) are powering digital transformation in every industry around the world. According to a 20 https://www.weforum.org/agenda/2021/01/ here-s-how-to-flip-the-odds-in-favour-of-your-digital-transformation21 World Economic Forum report, more than 80 percent of CEOs say the pandemic has accelerated digital transformation. AI is top of mind for boardroom executives as a strategy to transform their businesses. AI and ML are critical to discovering new therapies in life sciences, reducing fraud and risk in financial services, and delivering personalized digital healthcare experiences, to name just a few examples that have helped the world as it emerges from the pandemic.

For business leaders, AI and ML may seem a bit like magic: their potential impact is clear but they may not quite understand how best to wield these powerful innovations. AI and ML are the underpinning technology for many new business solutions, be it for next-best actions, improved customer experience, efficient operations, or innovative products.

“AI IS MOST EFFECTIVE WHEN YOU THINK ABOUT HOW IT CAN HELP YOU ACCELERATE END-TO-END PROCESSES ACROSS YOUR ENTIRE DATA ENVIRONMENT.”

Machine learning in general, and especially deep learning, is data-hungry. For effective AI, we need to tap into a wide variety of data from inside and outside the organization. Doing AI and ML right requires answers to the following questions:

• Is the data being used to train the model coming from the right systems?
• Have we removed personally identifiable information and adhered to all regulations?
• Are we transparent, and can we prove the lineage of the data that the model is using?
• Can we document and be ready to show regulators or investigators that there is no bias in the data?

The answers require a foundation of intelligent data management. Without it, AI can be a black box that has unintended consequences.

AI needs data management

The success of AI is dependent on the effectiveness of the models designed by data scientists to train and scale it. And the success of those models is dependent on the availability of trusted and timely data. If data is missing, incomplete, or inaccurate, the model’s behavior will be adversely affected during both training and deployment, which could lead to incorrect or biased predictions and reduce the value of the entire effort. AI also needs intelligent data management to quickly find all the features for the model; transform and prepare data to meet the needs of the AI model (feature scaling, standardization, etc.); deduplicate data and provide trusted master data about customers, patients, partners, and products; and provide end-to-end lineage of the data, including within the model and its operations.

Data management needs AI

AI and ML play a critical role in scaling the practices of data management. Due to the massive volumes of data needed for digital transformation, organizations must discover and catalog their critical data and metadata to certify the relevance, value, and security – and to ensure transparency. They must also cleanse and master this data. If data is not processed and made usable and trustworthy while adhering to governance policies, AI and ML models will deliver untrustworthy insights.

Don’t take a linear approach to an exponential challenge

Traditional approaches to data management are inefficient. Projects are implemented with little end-to-end metadata visibility and limited automation. There is no learning, the processing is expensive, and governance and privacy steps can’t keep pace with business demands. So how can organizations move at the speed of business, increase operational efficiency, and rapidly innovate?

This is where AI shines. AI can automate and simplify tasks related to data management across discovery, integration, cleansing, governance, and mastering. AI improves data understanding and identifies privacy and quality anomalies. AI is most effective when you think about how it can help you accelerate end-to-end processes across your entire data environment. That’s why we consider AI essential to data management and why Informatica has focused its innovation investments so heavily on the CLAIRE engine, its metadata-driven AI capability. CLAIRE leverages all unified metadata to automate and scale routine data management and stewardship tasks.

As a case in point, Banco ABC Brasil struggled to provide timely data for analysis due to slow manual processes. The bank turned to an AI-powered integration Platform-as-a-Service and automated data cataloging and quality to better understand its information using a full business glossary, and to run automated data quality checks to validate the inputs to the data lake. In addition, AI-powered cloud application integration automated Banco ABC Brasil’s credit-analysis process. Together, the automated processes reduced predictive model design and maintenance time by up to 70 percent and sharpened the accuracy of predictive models and insights with trusted, validated data. They also enabled analysts to build predictive models 50 percent faster, accelerating credit application decisions by 30 percent.

With comprehensive data management, AI and ML models can lead to effective decision-making that drives positive business outcomes. To counter the exponential challenge of ever-growing volumes of data, organizations need automated, metadata-driven data management.

INNOVATION TAKEAWAYS

Accelerate engineering
Data engineers can rapidly deliver trusted data using a recommender system for data integration, which learns from existing mappings.

Boost efficiency
AI can proactively flag outlier values and predict issues that may occur if not handled ahead of time.

Detect relationships among data
AI can detect relationships among data and reconstitute the original entity quickly, as well as identify similar datasets and make recommendations.

Automate data governance
In many cases, AI can automatically link business terms to physical data, minimizing errors and enabling automated data-quality remediation.

Interesting read?

Data-powered Innovation Review | Wave 3 features 15 such articles crafted by leading Capgemini experts in data, sharing their life-long experience and vision in innovation. In addition, several articles are in collaboration with key technology partners such as Google, Snowflake, Informatica, Altair, A21 Labs, and Zelros to reimagine what’s possible.