Hybrid Platforms Trends Series - AI - View from the Bridge 

This next series of Hybrid Platforms Trends will dive into the world of AI and some of the hot topics of conversations, our partnerships and the state of the hardware platforms being used to run the workloads. 

The Trending Conversation: 

We see three core trends emerging outside of the GPUaaS providers and large-scale LLM training requirements driving the headlines for Generative AI. Those super-large deployments have easy-to-define outcomes and value to justify the investments. However, once we drop down a layer, some numbers are still significant but the value can be harder to prove.  

Over the past 6-9 months, we have seen conversations shift as organisations digest the news and hype and ask the pertinent question: How do we take advantage of this new technology? How do we ensure alignment with our ethics and responsibilities? And how do we do all this in a commercially attractive manner? We have defined three core conversations for our customers to consider and to help them innovate at the required speed and scale.  

AI Strategy 

Aligning an AI strategy is critical to success and avoiding those 'CV' projects that will consume valuable funds, time, and resources without any reasonable chance of return. The list of AI use cases that different departments compile will contain some real gems and some complete duds. Ensuring all ideation is centralised and controlled as part of an organisation-wide strategy will be essential. A critical first step is developing a process to validate each idea and uncover those golden use cases that will drive valuable outcomes. Aligning everything to the organisation’s goals and bringing the relevant stakeholders on the journey will help maximise any investment requests. 

Once you have defined the first one or two use cases, building out the entire business case and blueprint for deployment will ensure a solid foundation to proceed. 

AI Governance 

With the increasing legislation around AI coming into force around the globe, legal and GRC teams have started to pay attention and are putting the brakes on projects. Building a solid governance framework that stretches from People and Process to the technology deployed to enforce compliance will be critical. Ethics will ensure that adopting AI technologies (in all forms) aligns with your organisational standards and that effective management of reputational and legal risks is ensured.  

Another emerging conversation in this space concerns AI usage detection, Deepfake challenges and broader LLM security concerns. These are topics for a future series. 

AI Platforms    

The platform conversation in the AI space has exploded into a potentially complex web of technology requirements. Representing this as a hierarchy of needs can help add some context to this web and ensure decisions are made from a strategic and outcome-aligned position. Without early alignment in the platform space, there is a risk that project timelines will be impacted as your teams look to untangle all these interactions.  

*note: Vendor logos are not exhaustive 

Luckily, we can help simplify this landscape and guide you through the choices and decisions from our position of experience and independence. Not being locked to a single vendor solution allows us to provide an opinion based on facts and outcomes rather than hype or misinformation. We align these requirements into five essential layers; allowing specialist teams to focus on delivering value.  

Predictive v Generative - The LLM reversal potential    

When the industry talks about responsible AI adoption, the normal views centre around use case ethics, transparency, bias, and accountability. While these things are front and centre, another core principle needs to be considered: Commercial Responsibility.  

Some would argue that predictive AI has been around for a long time since the '50s, but certainly over the last 10-15 years. We define this as those AI models that are good at looking at data and identifying patterns that can be used to inform decisions. This could be in Computer Vision or areas like Document Intelligence. We define Generative AI and the underlying Large Language and Foundation Models as the ability to create new content or data instances.  

While the opportunity is massive for Generative AI, it also carries increased costs, governance requirements, risk, and complexity. This is both at inception and over the run-time lifecycle of the model. Our commercially responsible approach is to ensure you don’t adopt technology you don’t need or a simple analogy to avoid using a sledgehammer to crack a nut!  

This is why we focus on four key areas of AI use cases to ensure we can deliver responsibly by combining Predictive and Generative technologies to solve that critical challenge. Sometimes, solving a problem with a predictive use case will yield better returns. 

• Computer Vision 

• Natural Language Processing 

• Document Intelligence 

• Generative AI 

There’s no one way to adopt AI 

One thing that has become clear over the last 12 months is how quickly the market is evolving in capability and optionality. We have also noticed that knowledge levels across industries are not achieving the same pace. This lack of understanding will lead to misconceptions and missed opportunities for many organisations.  

I expect everyone to interact with everyday AI; those AI features are built into the tools and products we use in our home or work lives. These will be tools like Microsoft CoPilot, Google Gemini, ServiceNow and GitHub copilot. They are provided as a service or 'off the shelf' without the need to deploy infrastructure or complicated data science work. They can bring massive value when the right adoption programs are associated with enabling coworkers.  

The challenge is that once people leave this comfortable bubble, the next step is to assume they need to train or build a proprietary foundation model and all the costs and complexity that entails. The reality is that over the past 12 months or so, we have seen a middle ground evolving by leveraging approaches like fine-tuning and retrieval augmented generation (RAG). While not new (RAG, for example, has been around since 2020), its use has exploded in the last 12 months.  

So, what are these different terms, and how do they relate to the cost and complexity requirements we keep referring to? 

Large Language Model/Foundation Model 

Firstly, we need to be clear on a Large Language Model (LLM) versus a Foundation Model; then, we can understand how this middle ground of AI adoption has unfolded. This will be a brief overview, not an entire history of AI over the last seven years. 

Before we had Foundation Models, it would have been the responsibility of every organisation to gather enough data and compute power, time, and money to train their own model. We must not underestimate the scale of completing such a task. Meta’s Llama 3, for example, had 15 trillion data tokens and took 7.7 million GPU hours to train! 

Now, we have access to paid and open-source foundation models where someone else has taken the time to gather the data and pay for the compute to train the model. This means only some people need to pay that initial cost or have access to the required amount of data. At a fundamental level, this is what has made AI open to a wider audience. 

LLMs are a subtype of Foundation Models specifically designed to understand and generate human-like text. They excel in tasks like text completion, translation, summarisation, question answering, and conversational chatbots. A foundation model's wider scope can be used in other AI disciplines, like Computer Vision or Robotics.  

Fine Tune a model 

So why do we need anything other than Foundation Models? Well, part of the problem with training on such a vast data set is that AI can need more context, specialist knowledge, or domain-specific understanding. The other concern is that you will only sometimes have full knowledge of the data used to train the model, which might raise concerns about bias or hallucinations (when AI makes a mistake).   

Think of the Foundation Model training as going through an education system, such as school. You get a broad and general 'training' on a wide variety of subjects, but generally, you are not an expert in any. Plus, the geography you are educated in will dictate, to some degree, your view on various topics (potential bias in our foundation model). 

Following our (loose) analogy, fine-tuning is when you go to university and spend a few years being educated on a particular topic, ironing out some misconceptions (bias), and making yourself an expert in a specific area. When we fine-tune, we take that Foundation Model and give it context with industry or organisation-specific data so it can deliver more accurate and contextual results.  

Fine-tuning builds on that base training (those first 18 years), reducing the compute and data requirements by an order of magnitude. Those 7.7 million GPU hours for Llama 3 could have taken months to run, while tuning a model might only need hundreds of GPU hours and a matter of days. 

Retrieval-Augmented Generation (RAG) 

RAG involves augmenting a Foundation Model with access to an external curated dataset to facilitate improved contextual output. RAG leverages the foundation model's generative capabilities to extract data from a domain-specific source; allowing for accurate responses without the complexity of training fine-tuning or the risk of using a base foundation model. 

For example, a generative AI model supplemented with a medical index could be an effective assistant for a medical professional. Another example would be a financial analyst benefiting from a chat assistant linked to market-specific data. 

Historically, RAG required you to train the base model first, so it had limited overall benefit, but now we have access to a wealth of pre-trained Foundation Models. This means RAG can be deployed on another order of magnitude lower level of compute. 

The outcome 

AI and Generative are becoming more accessible to every organisation, and keeping up with technological advancements is critical. Some potential implementations could have been too expensive last year, but with advancements in AI technology, prices could become a differentiator this year as they become ever-more affordable. 

Summary: 

Navigating this fast-paced and evolving market will be a challenge for many organisations. What was compliant and affordable last month could be a compliance nightmare next, while that use case that was commercially out of bounds could now be achievable to unlock a new revenue stream. With our Global AI network, we have significant resources continually monitoring and reviewing all aspects of that hierarchy of needs, allowing us to help your teams accelerate to meaningful outcomes.