May 11, 2023 – Duration 23:40

How OVO Energy builds better bots — faster

The best customer service bots deliver on consumers’ preference for fast, first-interaction resolution. And when an interaction is too complex, seamless integration enables the bot to transfer a consumer and the context of their interaction to an agent. These boss bots are essential as consumer use of chatbots continues to rise, but their satisfaction with the experience declines. According to the benchmarking report “The State of Customer Experience,” 38% of consumers interacted with chatbots for a service interaction last year, up from one-third in 2021. But the percentage of consumers who were highly satisfied with their bot experience declined from 25% to 21% in that same timeframe. In this episode of Tech Talks in 20, Lucas Woodward, a software engineer at OVO Energy, joins us to explain how to use automation to build better bots, faster. He’ll also reveal how he automates and tests bot integration with other CX technologies.


Listen and Share

Listen on Apple Podcasts
Listen on Google Play Music
Listen on Spotify
Listen on Stitcher
Lucas Woodward

Lucas Woodward

Software Engineer, OVO Energy

Lucas Woodward is a full stack developer with over 15 years’ experience who enjoys writing software for a living. Outside of work, he enjoys developing personal projects and attempting to draw.

TTi20 S3E15 Conversation Highlights

Here are conversation highlights from this episode, edited and condensed. Go to the timestamps in the recording for the full comments.

Welcome, Lucas. Tell us about yourself and your role, and a bit about OVO Energy.

Lucas Woodward (01:15): 

I work at OVO Energy, one of the big energy companies in the UK. My job is a software engineer on a team that writes bots and the backend services that integrates into Genesys to pull out data for reporting purposes. We’ve been working a lot on bots to achieve fairly complex user journeys.

Tell us how OVO Energy is using bots in the customer experience.

Lucas Woodward (02:05): 

At the moment, the bot is primarily the first thing a customer will hit when they’re going through a WhatsApp or web messenger journey, or in some cases the IVR. Depending on the menu option they choose, they might go straight through to a bot, which are being used for FAQs and trying to answer simple questions using intent recognition. They’re also used for understanding routing as well: how to route a customer based on bits of metadata we can glean from the information they provide. And some bots have complete journeys, like submitting a meter reading.

Where an agent hasn’t been involved, we also have bots to wrap up with a survey.

You’ve created an automation strategy to help build bots. Tell us about your approach and why you built that automation.

Lucas Woodward (03:38): 

When we were first given the task of looking into bots, I’m a software engineer, so I started coming at it from a software engineer’s perspective. You start by thinking about how much automation you can have in the pipeline for creating a bot. You can automate steps, with less reliance on a human who could potentially make mistakes. The process becomes quicker, and you can do things in a more reproducible way.

Testing is one example. When we make a change to a bot, we have over 30 regression tests that take place to ensure that the functionality we’ve built hasn’t caused anything else to regress. We’ll then have exploratory testing.

But just being able to define a bot in code, reproducibly deploy that to different environments, and put the creation of the bot in the code review process has meant that the rest of the team are able to offer comments on any improvements they can see with the bots.

We have a single source of truth where the bot’s code is kept. We’ve reduced that feedback cycle, as well. We’re able to quickly get changes in front of the customer with a certain amount assurance that nothing is broken.

Similarly, you built automation for Genesys Web Messenger and designed it to ensure the bot integration. Can you walk us through that?

Lucas Woodward (05:45): 

With that integration, the pipeline will deploy bots using the Terraform provider that Genesys offers, as well as the dependencies that it has — because a bot rarely does anything without communicating with an external system.

So, all these data actions and data tables, they’re defined as code and deployed along with the bot. And because Terraform allows you to define your infrastructure, you can point it to a particular environment. You could have an organization for development, an organization for production, and you can point it at either of these, and it will deploy the same way every single time. We then know that what we test in development is going to be in production. That’s what the customers are going to interact with.

Then I created another tool which allows us to define test interactions. It’s essentially like a script: You say this to the bot, you’ll expect the bot to say this back, and then you will provide this account number and so forth. And that’s using the client SDK for web messenger. You can create a simple web messenger deployment, give it a deployment ID, and then you can point any scripts that you want and it’ll run through the tests.

As part of our route to production, one stage that a bot has to pass is against [Microsoft] Azure. And we’ve cut lot of time from that with regression testing. The exploratory testing is still human-based because humans know how to break these things better than any script.

That’s been really helpful because previously we might’ve had to have reduce the amount of regression testing that we were doing in order to get something through quickly, whereas now we don’t have to. All these tests are done in parallel, so we’ll have 10 tests at a time kicked off in parallel, which will simulate 10 customers. That type of automation has really sped things up.

It sounds like your development process is iterative and you’re tracking what customers are doing when they’re interacting with the bots. Tell us a little bit more about that.

Lucas Woodward (08:42): 

The [voice of customer team] allows developers to come to meetings where they’ll ask a customer some questions about their interaction and will look at how they interact with these services. You get real-time feedback of why they’ve done what they’ve done or said what they’ve said. As a developer, sometimes you can be quite removed from that.

You can have a scenario of how you think a journey will play out, but then a customer will provide a completely different answer to what you expected. And then reviewing the transcripts enables you to see how people interact with your bots or where the bots have not understood a question in a particular way because it was phrased in a way you hadn’t catered for. So, there’s a constant iterative improvement.

There’s a tremendous amount of data that’s coming back to you. Tell us about how you’re consuming that data.

Lucas Woodward (10:08): 

Along with the milestones and outcomes that are available for flows, we have another tool in the background that utilizes participant data. Within a flow, we’re able to use participant data to essentially mark a metric we’d like to capture. If there’s a yes/no question, we could have participant data formatted in a way that is consumable every time that data has been updated, and then store it in an observability platform we use regularly for software. So, we’re able to have the same metrics that we’d have for software for the flows, as well.

What’s useful is it comes in real time. There’ve been occasions where there’s been something in the news that has changed the customer’s behavior, and we can instantly pick up on that. Or there’s been some problem within the bots and then that’s flagged. You can do things like anomaly detection or monitoring on that.

We also take a lot of data from Genesys to do reporting that brings together bits of different information across the business. We’ll have data coming into a data warehouse where we’re then able to structure queries on that to understand more about customer behavior across different products.

You mentioned that you can update bots in real time. How are you using data to evolve your flows and your bots?

Lucas Woodward (12:10): 

A human will review it to understand what’s happening. Anomaly detection could tell us that something’s changed in the way people are answering questions, and then it’d be a case of reviewing that to understand from a human perspective what’s happening and make adjustments.

Some services that are dependent on bots are rarely up 100 percent of the time. We’re able to detect that, mitigate for it and a gracefully handle a service going down temporarily. We can redirect people to different part of a bot or gather a bit of information and then pass them to an agent. Then the agent has that information, which is really useful.

What’s next on the roadmap for how you’re using bots? More journeys, more personalization?

Lucas Woodward (13:02): 

Yes, more journeys.

We’re currently work on another one. We tend to look at what has a lot of complexity and could value from having automation and data pulled in from the business in a quicker fashion than is possible for an agent. So, we’re looking for journeys like that and automating.

And then in the background, we’re looking into new things around AI, including generative AI.

We’re also testing voice recognition for different dialects.

What’s behind how you choose bots?

Lucas Woodward (14:59): 

It’s a multitude of things, like speech recognition capabilities; integrations and restrictions around those integrations; how it plugs into development tools; how easy it is to create a flow with it. Some may have a UI that’s very intuitive but lack the ability to export a file that we could keep in source control. There are lots of different factors. It’s an interesting space to be in because it’s moving so quickly and there are so many different options …

We have bots that try to glean information, that ask for details about a customer so we’re able to gather information on how to route them. The type of technology we choose would have an impact on how we ask the customer for that information and impact their experience. If you’re just restricted to touch tones on your phone, there’s a limited amount of information you can ask the customer for with that before they either get frustrated or they make a mistake. You could employ different technology that’s very good at speech recognition instead.

But then, some technologies are great at names, but they might not have the classifiers for postcode. So, it’s trying to find the right technology or the right bot for that particular journey or what you’re asking customers to do.

What’s one thing that surprised you with bots?

Lucas Woodward (17:44): 

We have a bot that allows people to put in their meter reads from the electricity meter. You’ll have a reading and it’s a long set of numbers, often with a decimal point in there, and you have to ignore anything on the other side of the decimal point. So, we put together a nice script that asks the person for this number, explains what they need to do, and people submit photos of their meters, which is perfectly normal. You’d expect that in hindsight, why wouldn’t you? That’s something I would do. It’s a complex ask for a long number. And they might not know they’re going to be talking to a bot.

That comes into the question about what technologies there are. Because it could be that you go down the journey of looking at image recognition, some way of pulling that information from the photo and then asking the customer, “Is this the correct number in your meter?” And then use that to further train image recognition.

If you only had time to give one piece of advice on bots, what would it be?

Lucas Woodward (19:42): 

Pay attention to how customers are interacting with your bots. They’re the only reason you’re creating your bots, and it’s very easy when you’re creating a bot to have a preconceived notion of how people will interact with it — and to stick to that as well.

Sometimes you just look at the success metrics. You might not look at the failure metrics as closely as you should. But when you start looking at those transcripts of how people interact compared to what you expect, it might be completely different. So, keep paying attention to those transcripts and recordings and iterate based on those.

Tell us how to find your blogs about software engineering.

Lucas Woodward (20:29): 

I post to the OVO Energy tech blog and I have a personal one, which is — because I enjoy sketching and I’m a developer.