Kahuna Labs Blog

By Sanjeev Gupta, CEO, Kahuna Labs

Retrieval-augmented generation, or RAG search, is often the starting point for introducing GenAI into the support experience. Search vendors have promoted RAG as a breakthrough that leverages large language models to improve how we find information and answer questions.

But while RAG looks good in demos, it doesn’t solve the real problems faced by enterprise support organizations. In most complex technical environments, RAG alone is not enough.

Where RAG Works — and Where It Does Not

RAG works well when questions are generic, use cases are limited, and information already exists neatly in documents. For example, in B2C scenarios or simple Q&A environments, RAG can retrieve relevant content, collate and rephrase it, and provide a good answer.

However, when the required information does not exist in documents — which is often the case in technical troubleshooting — RAG relies on incomplete or outdated information to produce incorrect answers with confidence. This sends both the customer and the support engineer down the wrong diagnostic path, increasing time to resolution and frustrating everyone involved.

Vendors often respond by blaming the lack of documentation and urging companies to hire more people to document more thoroughly — usually resulting in negative ROI for the support leader.

The key question is simple:
 Can RAG help if you don’t have complete troubleshooting documentation for every scenario, product version, and customer configuration?

Unfortunately, the answer is no.

Why Case Summaries Don’t Fix the Problem

Many support organizations try to overcome RAG’s limitations by summarizing old tickets and feeding those summaries back into RAG. That still doesn’t work well.

Case summaries highlight the root cause and resolution for a specific customer scenario. A different case is likely to involve a different context. Searching through summaries might give you a distribution of possible root causes or resolutions — but not the immediate next step: “What diagnostic should I run next to find the actual root cause in this new case?”

Key Limitations of RAG:

• No awareness of versions or context. RAG search is not aware of product versions or customer context – the ‘retrieval’ treats every piece of information as equally pertinent to the current situation, which it often is not.
• Incomplete information. RAG is unaware of the incomplete information. In the support landscape, the troubleshooting information for a case may not be documented completely within a ticket or a document – partly due to lack of diligence in documenting steps, and partly because many steps happen outside the case management system. Slack conversations in support channels, zoom calls, references to documents or other cases, escalations to Jira tickets are some examples. Unless you diligently bring them together, and string them in a chronological order, you won’t have all the steps. Even after that, some steps may still be missing – so each piece of information needs to be assessed for completeness and the gaps filled-in from complimentary sources.
• Unequal credibility of sources. Not all information sources have equal credibility. A past case that was confirmed resolved by the customer due to a specific action has high credibility. A past case that was transient in nature, and went away on its own, or one where the customer never responded has low credibility. A document written last week has a high credibility. A document written 2 years ago, that has not been referred to by anyone in the last 10 months, has low credibility.
• Evolving problem statements. During the journey of a case, the problem statement evolves – as you collect more diagnostics, the problem statement becomes more clear. Sometimes the problem completely changes direction. In order to find the correct next step, you need to look at the specific next steps taken in the past cases that were at the same stage of problem clarity.
• Action ownership. The “next step” could be a next step that only a support engineer can execute, or it could be a next step that only a customer can execute. And some customers may have specific situations due to their unique security considerations. Unless you diligently separate out which steps can be performed by the support engineer, and which ones by the customer, or which ones are the questions and which ones are the actions, the recommendations will not be actionable.
• High noise levels. Past cases are highly noisy. We have seen examples where RAG search recommended getting a coffee or escalating to Nicole as a next step, just because one of the past issues had a mention of that. This noise must be removed, if you’d like the next step recommendations to be accurate.
• PII and confidentiality. Past cases contain PII and confidential data from your customers. Usually, you’d anonymize that before feeding it for RAG search to avoid cross-customer contamination. However, the common ways of anonymizing tend to lose the chain of thought within and across cases from the same customer. So this anonymization needs special treatment before you can consider learning from past cases.
• No validation of correctness. Since RAG is a linear pipeline, there is no validation or evaluation of the completeness or correctness of the result, before it is presented to your customers. The generation and augmentation steps assume that the retrieval is accurate, even if it is grossly distanced from the original request.
• No learning loop. RAG search is a read-only operation – it does not learn from the user feedback, or use that to improve over time.

Troubleshooting complex support issues is not a language problem that can be solved by a large language model. It is not an information retrieval problem. It is a reasoning problem — one that requires constructing a logical troubleshooting map for your specific product.

How Kahuna Labs Approaches the Problem

What support engineers really need is a Troubleshooting Map™ — a guided sequence of diagnostic steps showing how to move from one stage of a problem to the next.

Kahuna AI uses multiple deep models to break each case into problem snapshots — the smallest meaningful representations of an issue and its context. These snapshots are then clustered to identify similar situations, even when steps are missing or documentation is incomplete.

From there, Kahuna AI reconstructs the most probable step-by-step path to resolution, ranking possible next actions by their credibility and completeness. Each part — identifying missing steps, clustering similar problems, assessing completeness, and ranking credibility — is powered by specialized models.

With this approach, Kahuna AI can recommend the correct next step for over 80% of all case progressions.

A Message to Support Leaders

If your product is simple, stable, and well-documented, RAG may be sufficient — and it shouldn’t cost much. It’s available out-of-the-box from many vendors today.

However, if your support organization handles complex issues across multiple versions, configurations, and customer environments, RAG won’t be enough. You need AI that reasons, not just retrieves — one that learns from every past case and guides your engineers to the next best step.

That’s exactly what we’re building at Kahuna Labs.

Watch the full webinar here to learn more about how Kahuna AI enables support teams to automate complex problem diagnostics and resolution.