Moveo’s custom Large Language Model (LLM) tuned for Customer Experience (CX) outperforms GPT-4 in all grading dimensions.

Moveo.AI announced that after rigorous comparison, its custom LLM tuned for CX outperforms GPT-4-0613 in all grading dimensions, except Markdown, where GPT-4 performs better. The evaluation was based on a random sample of hundreds of entries from Moveo’s production data, which neither our LLM nor GPT-4 had encountered before. Each entry was converted into a prompt consisting of the user question, conversation history, grounding knowledge from the collection documents, live instructions, and custom instructions.

This press release features multimedia. View the full release here: https://www.businesswire.com/news/home/20240723855013/en/

As can be clearly seen in this table, Moveo’s custom LLM outperformed GPT-4 in four critical dimensions that are the cornerstone of a great Customer Experience: Hallucination, Repetitions, Disambiguation, and Readability. The two models are equal in Language while GPT-4 performs better only in Markdown use. (Graphic: Business Wire)

Methodology

The grading process assessed Moveo’s LLM and GPT-4 responses across 8 dimensions that capture critical traits within the CX setting:

  • Hallucination
  • Repetition
  • Disambiguation
  • Live agent handover
  • Readability
  • Language
  • Markdown, and
  • Latency

Each dimension received a score, determining which LLM provided a better response. To evaluate the performance of the different models, Moveo used a separate GPT-4 instance as a “grader,” performing a single API call for each of the samples.

Results

Moveo’s custom LLM outperforms GPT-4-0613 in all grading dimensions, except in Markdown, where GPT-4 performs better in stylistic formatting. Most importantly, it is worth mentioning that in terms of hallucination, GPT-4 performs worse, which could hurt Customer Experience. For example, if GPT-4 provides incorrect information about a product, it could lead to potential liabilities, customer dissatisfaction, and increased support requests.

Moveo’s LLM responds in only 5 seconds, while GPT-4 takes at least 18 seconds. In that time, Moveo.AI could have handled more than 4 inquiries, significantly enhancing support efficiency and customer satisfaction.

According to Panos Karagiannnis, CEO of Moveo.AI, “Enterprises need vertical-specific LLMs as every customer interaction is an opportunity to build trust and loyalty. By minimizing hallucinations and connecting to real-time information systems, our LLM significantly beats GPT-4, reduces the risk of customer dissatisfaction and potential liabilities, and sets a new standard in CX”.

To learn more about Moveo’s proprietary LLMs, please visit: https://moveo.ai/

About Moveo.AI

Moveo.AI is a Conversational AI platform transforming how enterprises interact with customers. Moveo’s LLM, trained on historical and real-time CX data, powers GenAI agents to seamlessly connect to real-time data and unstructured knowledge bases to provide accurate and contextually relevant answers to inquiries.

Moveo.AI Panagiota Gkotsi +306981928344 panagiotagkotsi@moveo.ai