Building Gen AI applications for business growth – actions behind the scenes

Over the last few years, we have been witnessing a strong adoption of artificial intelligence and machine learning (AI/ML) across industries with a wide variety of applications. Use cases range from cost reduction via automation to the generation of additional business via the introduction of AI-infused products and services. The launch of the generative AI (GenAI) application ChatGPT by OpenAI in November 2022 only accelerated AI adoption. At present, many of the tech giants including the leading cloud platform vendors like Google, Microsoft, and Amazon have strong GenAI offerings along with those from many smaller vendors and open-source platforms.

In short, GenAI is an AI discipline where the AI foundation models (FMs) are trained on vast amounts of multimodal data (i.e., text, image, audio, video, terabytes of data, trillions of parameters). With proper user requests on the input, FMs can generate a large variety of multimodal synthesized outputs. Large language models (LLMs) are a subclass of FMs specializing in text. An added benefit of GenAI is its highly superior natural language processing (NLP) capabilities, in many cases using multimodal input/output, making it a great and not-realized-before technology for human-computer interfaces. This is one of the key reasons for the heightened interest in GenAI.

GenAI, with applicability in virtually all industries, can significantly improve many of the day-to-day operations of a business as well as help launch new business capabilities. While some GenAI-based autonomous products like certain types of text, image, and audio/video processing are emerging, many of the enterprise-grade usage scenarios that are currently in focus involve GenAI-based digital assistants to humans.

These assistants can help chatbots (and copilots):

• Respond to open-ended questions in a more human-like manner
• Improve overall customer experience
• Detect features and anomalies in images and transactions
• Help with code writing and testing
• Expand work automation
• Improve a wide range of document processing
• Make cognitive and semantic content searches more efficient and effective
• Provide advanced analytics to assess what-if scenarios
• Assist in creative content generation.

Typical metrics for business growth are revenue increase and healthy profitability. Productivity, innovation, and time-to-market are the key enablers of business growth. Depending on the situation, the discipline of GenAI can positively impact some or all of these enablers. A recent McKinsey study [1] estimates that GenAI-enhanced productivity and innovation could add between $2.6 and $4.4 trillion to the global economy annually and identified that around 75% of the value delivered would fall under four use case categories:

• Customer operations
• Marketing and sales
• Software engineering
• R&D

An early 2023 Capgemini Research Institute report [2] that explored a wide variety of industry use cases and surveyed nearly a thousand executives shows the broad applicability of GenAI and high ROI expectations from GenAI adoption. Of course, to realize significant business growth benefits, GenAI-based applications need to be functionally completed using additional application components besides the GenAI piece and need to be scalable, reliable, and integrated with other enterprise systems as necessary.

Example: A GenAI-enhanced multimodal and omnichannel B2C commerce application

Figure 1. Modular and component-based architecture for “Casey” – A GenAI-powered virtual retail assistant

We, at Capgemini, recently developed a virtual retail assistant, named “Casey” to accept orders and drive the order-to-cash process for partner stores (see figure 1). Casey is voice-activated and GenAI-enabled. Capgemini solution accelerator components power,[3] Google/GCP, and Soul Machines. For the end-to-end application, we layered a ‘digital human’ with conversational AI and cloud-native headless commerce APIs,[3] all pre-integrated for conversational commercial kiosks. It serves as a store-in-store order kiosk allowing the partner stores to maximize their channel reach with minimal investment. Casey is a business growth enabler – it opens a new revenue channel where it is easy to market innovative offers and whose cost does not grow rapidly with business growth, i.e., highly productive, and the solution construction allows for fast time-to-market implementation. Casey’s solution architecture is modular which has enabled us to use this as a basis for many other digital channel use cases in a variety of industries, for example, grocery, general retail, call center, telco, and automotive.

As this example illustrates, to build GenAI-powered applications that cover full customer journeys thus yielding tangible business value, we need to either combine several other application components and technologies or integrate the GenAI parts into otherwise functionally complete existing applications in case suitable ones are available

Creating enterprise-grade GenAI-based apps: Key considerations

To build a GenAI-based enterprise-grade application delivering substantial business growth, we need to consider:

• Opportunity formulation. Identification of the right business-relevant opportunities with realistic ROI projections is a critical success factor (CSF) for eventual success with GenAI-based applications. Especially as companies embark on GenAI adoption, it can reduce the risk of failure if GenAI is used to augment existing activities and processes. For example, the addition of GenAI into an existing customer churn prediction algorithm could process unstructured data like call recordings from customer interactions and customer reviews, capture additional insights like ‘sentiment’, specific store or product issues, competition strengths and weaknesses, possible new product bundles, and suggest appropriate ‘white glove’ treatments to reduce churn. As another example, GenAI could assist in a customer’s product exploration by improving existing user interfaces with visuals and helpful hints and by simplifying the actual purchase action by making the supporting processes more transparent.
• Solution design. One of the first considerations in drafting a GenAI-based solution strategy is to recognize that GenAI-powered interactions with customers or end users can produce actions that may not follow strict workflows, i.e., the complete application needs to have the flexibility to appropriately react to more free-flowing human-GenAI conversations. If the solution is built from scratch such flexibilities can be developed from the ground up which, of course, means a larger development burden. Cloud-first development and use of pre-built components (such as Capgemini’s Digital Cloud Platform [3]) can significantly reduce this burden. If the GenAI components are incorporated in an existing solution then the existing solution most likely will have to be refactored for proper integration of the new and the old including changing/upgrading some of the functionalities of the older components, for example, from batch processing to real-time response, etc. The choice of the appropriate GenAI tool/platform and the availability of data required for the proper functioning of the solution are also key considerations.
• Customer/employee experience and data orchestration. The value of GenAI in chatbots (and copilots) is the level of personalization and context an unscripted conversation can provide to a customer and employee. To retain this value, an enterprise must think through how to orchestrate various interaction points (or digital teammates) for consistency, as well as share interaction and customer data so the next conversation at a different interaction point is able to pick up the conversation where that customer left off last time. These chatbots are also a tool to empower employees to assist customers more broadly, where previously, an employee used to rely on what she knew at that moment now has access to comprehensive and granular data on-demand. Enterprises must also consider an orchestration layer to connect the various GenAI initiatives and data.
• Scale-out. GenAI is still an emerging technology; hence, it is advisable to start small, prove concrete business value, and then scale out to realize the target business benefits. However, in GenAI use cases where technical feasibility has already been proven elsewhere and a realistic business case for the solution is deemed positive, it can be worth the time and effort to create solution architectures with possible scale-out in mind. Such architectures would consider solution performance under production workload, availability and disaster recovery, security and data privacy, identity and access management, error handling, development and run cost optimization, and sustainable development practices. In the scale-out phase, a cloud-based solution approach is often superior and should be duly considered. Some of the GenAI-specific considerations are enterprise data foundations and trust (solid source of truth for customers, vendors, products, promotions, knowledge base, etc.), LLM selection, LLM lifecycle management, prompt version control across environment tiers, UX design for free-flowing conversations, balancing intent-based and generative-based interactions, incorporation of human-in-the-loop, response feedback loop, cost monitoring and optimization, technical debt management, and responsible AI governance.
• Measure and improve. Adequate measurement of solution performance is essential to understanding the current maturity of the solution and possible future enhancements; thus, measurement mechanisms should be built into the solution as first-class citizens. As such, high-level KPIs from traditional solutions can be reused in GenAI-powered solutions, for example, reduction in churn rate, increase in revenue per customer, efficiency in anomaly detection, and the like. However, it would be insightful to also add some metrics related to the model and system quality, and the performance of the GenAI components (see, for example, a summary of relevant metrics in [4]) which could include response error rate, range of input over which response accuracy stays acceptable, system latency, throughput, and run cost.
• Learn and grow. Capturing and sharing experiences as the solutions are developed and rolled out – and learning from them – is extremely valuable for fast-developing technologies like GenAI. Some design documentation, decisions taken along with the rationales, and stakeholder and end-user feedback are good ways to capture experiences from which lessons learned can be derived. This process would help in improving the solution over time as well as increase the organizational maturity to take on higher value (and potentially higher complexity) GenAI-based projects down the line. Over time, defining a robust set of build patterns across use cases would be helpful for asset reuse, solution management, and acceleration of new use case implementations.

Concluding remarks:

Done right, GenAI has tremendous power to push most enterprises forward with healthier business growth and higher market competitiveness. As a productivity enabler, GenAI is expected to accelerate automation by ten years with nearly half of the current tasks having been automated by the end of this decade.[1] Not to be left behind, enterprises should focus both on identifying what GenAI-powered applications are the most valuable for them as well as acquiring, either in-house or via partners, adequate skills to understand the ‘what’ and the ‘how’ of GenAI. In the early stages of GenAI maturity, spot solutions can bring quick wins while as the maturity grows, incorporation of GenAI in broader and across enterprise value chains should be considered for reaching higher benefit goals – and this will take some foundational investment in data, UX strategy, integration strategy, and building a GenAI platform.

References:

[1] https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/the-economic-potential-of-generative-ai-the-next-productivity-frontier

[2] https://www.capgemini.com/in-en/insights/research-library/generative-ai-in-organizations/

[3] https://www.capgemini.com/us-en/solutions/digital-cloud-platform-for-retail/

[4] https://cloud.google.com/transform/kpis-for-gen-ai-why-measuring-your-new-ai-is-essential-to-its-success