Moveo’s LLM vs GPT-4 for Customer Experience
24 Julio 2024 - 8:10AM
Business Wire
Moveo’s custom Large Language Model (LLM) tuned for Customer
Experience (CX) outperforms GPT-4 in all grading dimensions.
Moveo.AI announced that after rigorous comparison, its custom
LLM tuned for CX outperforms GPT-4-0613 in all grading dimensions,
except Markdown, where GPT-4 performs better. The evaluation was
based on a random sample of hundreds of entries from Moveo’s
production data, which neither our LLM nor GPT-4 had encountered
before. Each entry was converted into a prompt consisting of the
user question, conversation history, grounding knowledge from the
collection documents, live instructions, and custom
instructions.
This press release features multimedia. View
the full release here:
https://www.businesswire.com/news/home/20240723855013/en/
As can be clearly seen in this table,
Moveo’s custom LLM outperformed GPT-4 in four critical dimensions
that are the cornerstone of a great Customer Experience:
Hallucination, Repetitions, Disambiguation, and Readability. The
two models are equal in Language while GPT-4 performs better only
in Markdown use. (Graphic: Business Wire)
Methodology
The grading process assessed Moveo’s LLM and GPT-4 responses
across 8 dimensions that capture critical traits within the CX
setting:
- Hallucination
- Repetition
- Disambiguation
- Live agent handover
- Readability
- Language
- Markdown, and
- Latency
Each dimension received a score, determining which LLM provided
a better response. To evaluate the performance of the different
models, Moveo used a separate GPT-4 instance as a “grader,”
performing a single API call for each of the samples.
Results
Moveo’s custom LLM outperforms GPT-4-0613 in all grading
dimensions, except in Markdown, where GPT-4 performs better in
stylistic formatting. Most importantly, it is worth mentioning that
in terms of hallucination, GPT-4 performs worse, which could hurt
Customer Experience. For example, if GPT-4 provides incorrect
information about a product, it could lead to potential
liabilities, customer dissatisfaction, and increased support
requests.
Moveo’s LLM responds in only 5 seconds, while GPT-4 takes at
least 18 seconds. In that time, Moveo.AI could have handled more
than 4 inquiries, significantly enhancing support efficiency and
customer satisfaction.
According to Panos Karagiannnis, CEO of Moveo.AI, “Enterprises
need vertical-specific LLMs as every customer interaction is an
opportunity to build trust and loyalty. By minimizing
hallucinations and connecting to real-time information systems, our
LLM significantly beats GPT-4, reduces the risk of customer
dissatisfaction and potential liabilities, and sets a new standard
in CX”.
To learn more about Moveo’s proprietary LLMs, please visit:
https://moveo.ai/
About Moveo.AI
Moveo.AI is a Conversational AI platform transforming how
enterprises interact with customers. Moveo’s LLM, trained on
historical and real-time CX data, powers GenAI agents to seamlessly
connect to real-time data and unstructured knowledge bases to
provide accurate and contextually relevant answers to
inquiries.
View source
version on businesswire.com: https://www.businesswire.com/news/home/20240723855013/en/
Moveo.AI Panagiota Gkotsi +306981928344
panagiotagkotsi@moveo.ai