The 4 Crucial Stages of Successful Generative AI Integration

Despite the widespread adoption of generative artificial intelligence (Gen AI), the number of companies deriving real value from Gen AI initiatives remains relatively modest. Boston Consulting Group reports that only 10% of companies are successfully scaling their AI projects, indicating that many are still striving to maximize the value of the technology. Failure to scale now could lead to dire consequences, including lagging innovation, increased costs, and a loss of competitive edge.

Developing and integrating Gen AI tools happens in stages, all of which demand meticulous attention to strategic planning. The complex journey from ideation to deployment will become even more complicated as the technology matures, forcing businesses to adapt swiftly to remain competitive in a rapidly evolving landscape.

By addressing the inherent challenges of Gen AI integration and seizing opportunities for innovation, forward-thinking organizations can position themselves at the vanguard of technological advancement and capitalize on AI’s vast potential.

What Gen AI Is—and Isn’t

Gen AI—exemplified by tools such as ChatGPT—encompasses deep-learning models capable of producing diverse content forms, including audio, images, text, simulations, and videos. This robust and versatile technology has the potential to reshape many content creation methodologies, enabling, for example, faster software code writing and the development of reliable chatbots grounded in enterprise data.

Part of Gen AI’s value comes from the fact that it does not require quality data, unlike other forms of AI, and the models are available to use without the need for deep expertise. That being said, this novel technology is not the end-all, be-all for AI. It is a powerful tool, but companies would be doing themselves a disservice and leaving a lot of value on the table if their AI strategies only included Gen AI. Traditional AI and machine learning (ML) can support businesses in identifying patterns and customer behavior, doing predictive modeling, and building adaptive systems that respond to changes in customers or the competitive landscape. Gen AI can be an input to enhance more traditional modeling, but can’t fully replace it.

The 4 Stages of Gen AI Integration

In a previous article, I outlined various evaluation criteria to identify the most valuable areas for implementing AI in your business. Once high-value, high-impact initiatives are identified, I recommend these four steps to build a mature Gen AI program and maximize the value of the technology in your organization.

1. Providing API Access

Because Gen AI has a low barrier to entry, companies can experiment without hefty investments in infrastructure, tooling, and talent acquisition, which were necessary for previous data science, AI, and ML waves.

The quickest way to implement Gen AI is by giving employees API access to models like GPT, Claude, Llama, or Cohere. Some companies build simple front ends to create branded tools that look specific to their companies, while others have elected to use third-party platforms without customization. Developing and setting up access to these powerful tools can be achieved within a single day, allowing businesses to begin using AI almost immediately.

However, several key factors must be considered before providing employees with tool access.

First, you should create an AI policy centered specifically on what your employees should and should not do with their access and what information they can send or share with the AI. Second, set credit limits. Most tools operate on a credit-based system that costs money with each action taken. Some employees will come up with innovative ideas that might be cost-intensive, and you will want to cap their use and evaluate whether to expand the limit before being surprised by a massive, unexpected bill.

It is essential to establish both a usage policy and credit limits quickly: In August 2023, Deloitte reported that over 60% of employees were already using Gen AI tools at work, sometimes without their managers’ knowledge. This type of rogue usage exposes companies to risk if those employees are utilizing AI tools in ways that could cause reputational damage or cybersecurity threats.

Once employees have bounded access to Gen AI tools, ensuring you’re getting the most value out of your license is equally important. To do that:

• Track and Monitor Usage of Gen AI
  • Investigate how your employees are using the tools. This could help identify opportunities to develop more robust products that drive value.
  • Share best practices across the company. Some of my clients have successfully created cross-functional user groups where members can share ideas and learn from each other.
• Provide Usage Training and Education for Employees
  • Offer employees examples of how they can leverage the Gen AI tools in their day-to-day jobs.
  • Train employees on how to develop prompts that will return the results they need.
• Define Specific Use Cases
  • Identify dedicated problems that you want your employees to solve using Gen AI, like brainstorming marketing copy; summarizing customer posts to understand brand sentiment; and speeding up software development by using AI to help engineers write, review, and test code.

Giving approved access to Gen AI and clear company guidelines on its usage is a low-cost, simple first step in creating value from the technology’s potential.

2. Leveraging Internal Data

Harnessing internal data is key to gaining a competitive advantage in the market, especially for proprietary data about customers, business performance, and operations. By moving beyond the standard base foundation models (e.g., GPT, Gemini, and Llama) and tapping into internal data sources, organizations can unlock a wealth of untapped data analysis that can set them apart from competitors relying solely on generic tooling.

Building Gen AI solutions using the retrieval-augmented generation (RAG) methodology allows the tool to first retrieve the most relevant information from a data source and then use the data the foundation model was trained on to generate an output. RAG effectively combines retrieval mechanisms with generative models, enhancing the AI’s ability to access and utilize external data sources. This approach helps users find the most relevant information quickly, make better decisions, and enable faster analysis of large amounts of unstructured data.

In more advanced use cases, RAG can be used to look up a customer’s purchase history and combine it with GPT to generate personalized marketing content. The ability to tap into your internal sources allows you to develop creative solutions that your competitors can’t emulate or adopt. It also protects your business from new entrants because they will not have access to the same information you have.

The integration of vector databases within Gen AI solutions enables companies to efficiently search through vast amounts of internal data, facilitating rapid data retrieval and analysis. Vector databases store data in a way that keeps similar information closer together, allowing for quick searches of relevant information.

This approach to internal data management empowers organizations to extract actionable insights, optimize customer support services, and enhance operational efficiency. By leveraging RAG and vector databases as key technologies, companies can expedite data processing, improve search capabilities, and unlock the full potential of Gen AI in transforming internal data into valuable business intelligence.

3. Retraining Models

Retraining Gen AI models allows companies to customize outputs to meet specific requirements, enhancing brand alignment and relevance. This process improves the quality of generated content, making it more personalized and engaging for target audiences.

An example of a company that has done this well is a recent client in the customer contact space. The company wanted to build an AI agent that could automate many of its standard conversations and behave like one of its customer service reps. The company had millions of conversations that could be used as a training dataset to retrain the foundation model on how it would answer given specific inputs. The largest hurdle to solving this problem was appropriately scrubbing the training data of any personally identifiable information.

While model retraining can be a powerful option for some use cases, it’s often not recommended due to security concerns: All of the information you provide in the retraining process will be captured within the model itself.

In the early days of text auto-completion, there were concerns about the potential risks of these systems inadvertently exposing sensitive information. (For example, if I typed, “My name is Chas Stikeleather, and my American Express credit card number is …,” the system might complete the statement with my credit card information.)

We have the same concern with the information being captured within a foundation model that has been retrained. Furthermore, large language models, due to their complex nature and hundreds of billions of interconnected coefficients, are able to make connections between data points that you did not initially intend for them to make. Going back to the customer example, we started with the least sensitive conversations and topics and expanded as needed to avoid accidentally inputting sensitive data into the model.

Perhaps most importantly, the last step is conducting intensive testing to see if you can get outputs from the model that are a security breach or risk. If you can, you will need to backtrack and scrub the data you are using to retrain the model.

4. Adopting a Product Focus

Organizations must view Gen AI beyond traditional applications and focus on diverse opportunities to create innovative solutions tailored to specific user needs. By adopting a product development approach (i.e., emphasizing product management, design, and UI/UX), companies can harness AI to create valuable solutions that resonate with their audiences.

A product-focused Gen AI strategy involves creatively leveraging a combination of technologies and prioritizing user research to understand how customers and employees interact with AI tools. Investing heavily in user research enables organizations to gain valuable insights into user behaviors, preferences, pain points, and expectations, laying the foundation for designing intuitive and user-centric Gen AI products.

Chatbots are one area in which a product-focused build is essential. Early learnings from Toptal clients taught us that chatbots can sometimes be intimidating and ineffective. Some users find writing their own prompts for the bot difficult and time-consuming. Furthermore, when they receive an output that does not align with what they were expecting, it’s difficult for them to redirect the bot with further prompts to get the desired output.

By creating well-vetted drop-down or multiple-choice prompts and templates, companies can ensure users interact with AI effectively without the complexity of prompt creation. This approach enhances the UX and maximizes the value derived from Gen AI.

The possibilities of Gen AI extend far beyond chatbots—whatever AI opportunities your business takes advantage of, a focus on the user and a comprehensive UI/UX design will ensure that your Gen AI products are intuitive, visually appealing, and easy to navigate. As I noted earlier, only a fraction of organizations are effectively leveraging the potential of this technology today, despite widespread adoption. Following these steps and focusing on design will improve user satisfaction, drive Gen AI adoption, and differentiate your Gen AI offerings in a competitive market.

Have a question for the Artificial Intelligence and Data Analytics team? Get in touch.