Kahuna Labs Blog

By Hitesh Sharma, VP of Engineering, Kahuna Labs, and Chaitanya Potluri, Co-Founder, Kahuna Labs

Kahuna Labs just completed a Proof of Value (PoV) to assess the Kahuna Platform’s potential impact on technical support cases for a recent customer. The goal was straightforward: provide AI recommendations that support engineers (and customers) actually find useful.

Methodology & Feedback

We trained Kahuna AI on two years of historical data to build a Troubleshooting Map™. The system was refined through “friendly feedback” sessions with the customer’s support SMEs. These sessions enabled customer-specific tuning, ensuring the AI recommendations aligned with and benefited from the customer’s internal best practices and domain knowledge.

Customer Profile

• Multi-product global leader in a specific class of infrastructure products
• ~3,500 customers globally
• Support team: 250 engineers across three tiers
• Cost per case: $120–$500
• Documentation: ~30–40K documents, ~80% believed to be stale (specific documents are difficult to identify)
• Existing state:
  • Case deflection maximized using chatbots and self-service
  • Homegrown advanced AI-based knowledge search tool in place

Limited Scope

The PoV was limited to a single product line and trained only on past cases and centrally maintained troubleshooting documents.

Excluded from training:

• Case attachments
• Zoom call transcripts (32% of cases had Zoom calls)
• Microsoft Teams conversations
• Jira tickets
• Fragmented troubleshooting documents maintained locally by Support Engineers

These data gaps will be addressed prior to a production rollout.

Product in scope for PoV: Infrastructure product with both on-prem and cloud deployments. Case volume for the product in scope: 1,900 cases per month.

Data Quality

As part of the PoV, we first evaluated the quality of historical case data. Key findings by Kahuna AI:

• 52% of cases had a Completeness Score™ of 3, 4, or 5; the rest lacked meaningful documented steps
• 26% of cases had more than one grammar or spelling error in outbound customer communications
• 21% of outbound messages showed minimal or no empathy
• 15% of customer messages had moderate-to-high negative sentiment
• 32% of cases required Zoom calls
• 20% of outbound messages contained customer-sensitive information
• In 25% of cases, every troubleshooting step was performed by the customer; these cases were fully self-serviceable and could have been deflected using multi-turn troubleshooting driven by the Troubleshooting Map

Evaluation

After training, the customer selected a representative set of cases that Kahuna had not seen before, at various stages of troubleshooting and asked Kahuna AI to generate next-step recommendations.

Findings:

• ~70% of Kahuna recommendations matched the Support Engineer’s eventual actions (fully or partially)
  • Low confidence recommendations would be suppressed in production
  • Expected accuracy is significantly higher after ingesting Zoom transcripts (~32% of cases), case attachments, and enabling reinforcement learning in production
• 25% self-service potential: One-quarter of cases were identified as candidates for Kahuna Auto-Resolve, requiring no support engineer involvement
  • For remaining cases, the customer observed potential for left-shift from L3 to L2 and L2 to L1
• Faster diagnostics: AI reduced manual back-and-forth for diagnostic data collection
  • Zoom call predictions were highly accurate and could eliminate 3–4 message exchanges for ~32% of cases
  • Meaningful first responses enabled contextual probing questions before case assignment
• Quality communication: Suggested responses improved professionalism and empathy while reducing engineer time spent drafting emails

Conclusion

The readout demonstrated that Kahuna can materially improve support operations for this customer by providing troubleshooting recommendations based on a comprehensive Troubleshooting Map. The PoV showed potential to reduce resolution time, increase self-service, enable support tier left-shift, and improve consistency and quality of customer communications.