Practical business applications of AI; does my business need RAGs anywhere except for the kitchen sink?

From a business leadership perspective it can seem as if AI is everywhere, yet besides the occasional chatgpt or copilot discussion, the tangible benefits from the available technologies seem elusive. The aim of this blog post is to give a practical example of the use of AI technologies in improving the efficiency of your business, along with some thoughts on ROI of a solution like Retrieval-Augmented Generation (RAG).

There is a whole variety of acronyms surrounding the AI space, and most of them say very little to the average business leader. Most of us are mainly familiar with Multi-Modal models like ChatGPT, which might represent the entirety of AI as we understand it. We might have had some frustrating experiences with early stages of chatbots in our role as a consumer, whether the chatbot experience was “AI powered” or not, and these experiences quite understandably make us question whether an AI -based solution can be robust and fit for purpose in the core of our business. However, there are AI technologies that can be implemented in a robust and effective manner, and those can drastically improve the usability of one of the most critical assets of a modern company; data and the information the data can give us.

“Check the network drive, it should be in one of the folders…” is a sentence I would bet most of us have heard when looking for certain piece of information, for example regarding certain detail in one of the company’s dozens of products. At best this data actually is well defined and maintained in a structured and logical manner on the company network drive. But even with best practices implemented for the data management, it can take a while to find, especially if there are multiple types of fact / data sheets and you are not sure which type holds the info you are looking for. If your company is growing, data management can be a challenge, and onboarding new team members can subsequently become more challenging than it needs to be. New people often need to check things about the company’s products and services, and whilst interaction with more experienced team members should be encouraged to build the team spirit and internal networking, it might not be the most efficient use of their or the new hire’s time to spend it walking through where to find information. This is where a Retrieval-Augmented Generation -based solution can immensely help your business. RAG works like a smart assistant that instantly finds the most relevant information from your documents to answer questions. It doesn’t just search for keywords; it understands the context and provides comprehensive answers. Plus it can pinpoint you exactly to the document where the information can be found.

In this first blog post the focus is on the basic concept, the greyed out part shown in this picture is to be discussed in another blog post.

A custom-built RAG solution indexes the data that you give it, creating a searchable database of information. When the user asks a question, RAG retrieves the most relevant documents from the index. This information is then passed on to a Large Language Model (LLM) to synthesize this information into a clear and concise answer, and it can also provide the list of references in your data that it bases this answer on. This is the power of RAG; it can be customized with your data to fit your specific needs. With the right expertise, a RAG solution can also be built on infrastructure directly under your management; meaning that if your solution handles sensitive data, you do not need to store it on cloud servers that reside in a geographical location that would be difficult from your contractual obligations perspective. Depending on the scale of the solution you need, the infrastructure needed to run the system can even reside in your company’s premises, only accessible from your intranet. Or if feasible from your data management perspective, it can be made with dynamic cloud infrastructure.

ROI of a RAG-based solution naturally depends on a lot of things, and there are no two identical cases, but the good questions to ask yourself are: do you have a lot of complicated products or services? Does your business grow rapidly, do you often need to do onboarding of new team members? How much could it cost your business if an employee was forced by circumstances to give a customer a misinformed guesstimate instead of accurate information?

What’s the catch then, why doesn’t everybody already have a RAG-based solution if the idea of it was presented already in 2020?(*) Well, despite the idea being presented already years ago, practical implementations and frameworks take time to develop, and only roughly over the last year we have seen wider emergence of various RAG-frameworks. One thing that is also good to keep in mind is, that a RAG-based solution can only be as good as the data you feed it, so it isn’t an off-the-shelf answer to a situation where the data is not maintained and the documents are scattered in random places. Like for all data-intensive endeavors, the first step is data management; you have to have a well-defined and implemented process of handling your master data, and you need to have it stored in a well structured manner. Only then you can feed RAG what it needs in order to become a tool that will improve the efficiency in the information usage in your business. Developing a RAG-based solution for your business can (and most probably should) be an iterative process, so a good way to start on this path is to identify and define the information availability pain points in your organization, and solve these one at a time. HTGP has the resources to help you in every step of the way; from identifying the most urgent need, planning and implementing data management, to building the RAG-based solution and rolling it out, and defining the change management process to make sure the solution is put to efficient use.

In this blog post I focused on a simple approach to RAG to introduce the core concept from business management perspective, but besides unstructured text documents, a RAG-based solution can also incorporate data from production databases like a CRM system, as well as programmatic data over real-time API queries, and even multimodal data sources utilising a Vision Language Model (VLM) -based document retrieval architecture. To discuss these aspects in the length and depth they deserve from a practical business perspective warrants another blog post, so stay tuned for the next one!

(*) “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks” by Patrick Lewis et al. (Facebook AI Research, 2020).

Written 1/2025 by Juho Alatalo, Advisor @ HTGP. Juho.alatalo@htgp.fi, 040 250 0925