In the dynamic world of banking and finance, the ultimate goal we’re striving towards with large language generative models (LLMs) is to recreate the age-old personal banker in digital form. Imagine a future where interacting with your bank is as simple and intuitive as saying: "Hey ABC Bank, please help me pay a few invoices and update my home address." This reality is no longer a distant dream, but is now within reach.
“Generative AI is currently the hot topic, reaching the peak of the Gartner Hype Cycle. However, as we witness this rise, we're also starting to see signs of the inevitable trough of disillusionment among clients. The excitement is palpable, but so are the challenges and misconceptions surrounding the implementation of these advanced models,” explains Kevin Staples, specialist software engineer and CEO of BBD Group, a custom software solutions company with 40 years of experience building and integrating complex solutions for corporates.
Over the past two decades, Staples has dedicated his career to enhancing customer service solutions for large enterprises. In the last four years, his focus has shifted to transforming client interactions using LLM generative models.
Understanding the maturity roadmap
Staples says one major reason behind the growing disillusionment with LLMs is the lack of awareness about the necessary maturity roadmap. Too often, organisations attempt to leap directly to sophisticated chatbot implementations without laying the proper groundwork. This approach almost invariably leads to failure.
“In my experience, you cannot shortcut this journey.” According to Staples, success begins with establishing a mature customer service infrastructure, which he refers to as your ‘ticket to the game’. This involves creating a deeply integrated and comprehensive service landscape. He explains that only then can you begin to adopt the basics of LLM technology, gradually building towards what he calls the ‘all-knowing and capable bot’.
The road to an enhanced customer experience
Starting at the foundational level provides the groundwork to ensure the advanced capabilities of generative models can be achieved, culminating in the use of features like voice-to-text and text-to-voice capabilities. Staples believes this methodical approach ensures the journey towards leveraging LLMs is not only sustainable and effective, but provides the required guardrails to ensure customers are protected from the well-known hallucinations that LLMs are prone to.
LLMs in action
BBD spent the early years upskilling on generative AI, LLMs in particular; learning, playing and building POCs at this bleeding edge of innovative software solutions. The company is now actively working on a solution with a leading financial client.
Some of the learning in this space so far:
- To ensure customer service quality and minimise hallucinations in the solutions, it’s important to design the solution using an optimal choice of strategies and design patterns to ensure accuracy, but to have very effective guardrails in place to intervene when required.
- Model evaluation is critical, and steps need to be implemented within pipelines to test models in terms of truths. Such steps include collecting truths dataset, evaluating outputs from a testing pipeline and including human testers.
- It’s often beneficial to package models and pipelines into more deployable components and services.
- Retrieval-augmented generation (RAG) as a model is an effective way of improving accuracy and reducing hallucinations by providing more context to the trained LLM model.
- Fine-tuning models usually results in better responses, according to BBD’s results, but can be computationally heavy, time-consuming and does not fully alleviate the risk of hallucination.
Final thoughts
The future of customer service is bright and vastly different from today’s landscape. It’s an exhilarating journey, and while it promises substantial rewards, it requires a strategic and phased approach. Staples encourages organisations to embark on this journey with a clear understanding that it is a progressive path to the future, not a quick-fix project.
By embracing this mindset, we can all move closer to achieving a seamless, AI-driven customer service experience that mirrors the personalised touch of a human banker, making banking interactions more intuitive, efficient and satisfying.
Share