Thread by @mredstone

Welcome to #chatbotvoice2023 from Berlin.

I'm throwing up some tidbits around #conversationalai live from the 5th World Chatbots & Voice Summit from Berlin.

First up, Christoph from @GetCyara

We're talking testing and assuring quality in #conversationalai

Testing is an impossible manual task and it needs to be automated. Done wrong, it can cost heavily. So it is a huge, but important challenge.

Think about lab testing Vs real world testing. An in car voice assistant needs to cope with background noise, connectivity etc.

At home, expect kids, dogs, TVs in the background.

For chat, expect random input.

Banks see questions about weather. Or asking for fashion advice. Plan and rest for that, not just the happy path

Testing adds value in every stage of the bot development lifecycle.

Testing is all about shifting left and finding issues earlier in the process.

And the first slide on #ChatGPT - using reverse psychology in #prompting

So how can we test to ensure guardrails are in place when using #LLMs?

Now onto Sascha from @dbsystel

He'll be demystifying #ChatGPT for an effective #customerexperience and #EmployeeExperience

The German Ministry of Education has released guidelines for #GenerativeAI in schools.

"A ban cannot be a viable reaction"

So how do we reduce the gulf between the individual and the enterprise using #conversationalai and #GenerativeAI

And here's how to do it.

Replace the #NLG with an #LLM as it's just way too expensive to maintain

So how can we use #GenerativeAI in our workflow?

Most #enterprise #conversationalai vendors have implemented some form of #LLM into their platform for developers to take advantage of.

But you can lose control of the experience so be aware.

Sascha is now demonstrating how @cognigy has implemented #LLMs into their platform for AI enhanced outputs within their #NLG engine

#GenerativeAI and #conversationalAI working alongside each other in the tooling

Useful to remember that #ChatGPT had a knowledge cutoff, with no dynamic data (currently), so it will generate based on probability not data.

Which is what causes the dreaded #hallucinations associated with #GenerativeAI

Lifting the lid on what's to come with using #ChatGPT within @dbsystel for #cx and #ex using a mix of classic intents and #GenerativeAI for the #NLG orchestration layer.

Here's what the architecture looks like and a quick demonstration of it running in #microsoftteams

Then a quick demo of using a #ChatGPT orchestration to switch languages.

Next up. Ingo from @koredotai

Ingo is talking us through how @koredotai has implemented #LLMs into their technology for dialogue generation, use case generation and other areas to speed up time to market for their client assistants.

Ingo is now demonstrating knowledge extraction from documents in @koredotai with Zero coding

#nocode #LLMs

Now onto how you can use @koredotai to enhance both the #cx and #ex of contact centres with their agent assist feature.

With a little demo and example of this slide.

Using @koredotai agent assist, has seen the contact time reduced by 30% for the 2000 agents in @Citibank

Now we move onto the #chatbot track of the summit.

First up, Caspar Doing from @Politie and Casper from @studiowinegum talking about how chatbot Wout connects citizens and police.

There's a declining trend in citizen sentiment in government and police. Changing the way that you interact can reverse that sentiment.

A shifting society needs a new approach to being able to contact and communicate with the public services.

And thank you Studio Winegum for the winegums

It's taken lots of pilots and proof of concept testing to get to this point.

Persona and personality was exceptionally important in the development of Wout.

Even down to the name. Wout is a popular name but also a slang word for police in the Netherlands (like UK Bobby). Also a swear word in the past relating to the police (like pig in the US)

Tone of voice has removed the formality of the police to bring it closer to the citizens engaging with Wout.

Avatar identity reflects the multi cultural society served.

Even the website is becoming more informal based on positive sentiment from Wout.

Now onto the #conversationdesign of Wout.

Part of the persona planning determined that Wout is open-minded and looks beyond the police organisation to signpost users.

Wout should always have a solution.

Wout can now fully process a disturbance claim without human assistance.

Massive automation opportunity of one of the biggest types of calls received

When chat might not be the right fit (sexual abuse, for example), Wout refers to colleagues with specialist experience.

The start would be #livechat, but can handoff to a call as well.

If the user cannot be assisted by police there is a referral to an appropriate partner.

Obviously, the police may not be somebody's best friend

It will handle abuse, set boundaries and stay professional.

UK police need to think about the #scunthorpeproblem

Stats behind Wout are impressive.

700 agents across 11 contact centres connected and they are the livechat fallback as well.

Tracking the intents and their popularity helps inform the roadmap for further development of automation via Wout.

Top intent is asking to talk to a human. Wout responds with asking what do you want to talk to them about and bring this back around to the automation, if possible

So what does the data say for Wout from @Politie about society usage of this technology?

Fascinating case study of a school child being bullied with true risk to life.

They would not have connected with the police service if it wasn't for Wout.

The future of Wout is bright! And for the human capital element of the Dutch Police service @Politie

Overall massively positive sentiment for this initiative

Up next, is Radomir from @olxgrouptech

What happens once a business decides that they want to implement #automation

Start with the goal.

What do you want to do?

Is it a budget driven decision?

What technology do you use?

Do you set up a #CoE Automation Team?

Here's what that team needs to look like

Most use cases are siloed to the technology.

Work to unblock those silos and get as much tech used for each use case.

One automation under a use case umbrella.

So where do you start?

Your people are important as they know the business and its operations better than an external consultant.

Don't hire a developer as the starting point.

2nd thing to do

Create playbooks, scripting standards, best practices, training processes

Document everything

If you lose a key member of the team, it makes for a quick handover.

What do you document?

Simple answer. Everything

How do you determine the financial success of the project?

Is it ROI or CCCC?

To bring this to life, you need a strong team.

What does an Automation Team look like?

Creating automation is easy.

Exception and error handling is the challenging part.

How to approach the business.

First error, don't tell people you are replacing them.

Put a positive spin - saving time, increasing productivity, helping employees.

Remove the robot from the human.

Increase mental well-being by removing repetitive tasks

Great quote from Henry Ford about time waste.

Train people to create their own automations and they will continue to find use cases, which makes automation feel safer to other colleagues.

And we are back from lunch. Keeping the twitter thread going of tidbits from this year's #chatbotvoice2023 in Berlin.

As I'm chairing the #chatbot track, most of this is biased away from voice, but we've got some great afternoon sessions coming up.

First up this afternoon, is Kamel Nebhi from @EF

This is a case study on leveraging #LLMs for language testing

It's important to remember the context here.

@EF is all about culturally immersive education. It started as an initiative to bring students from Sweden to the United Kingdom to learn English.

EFSET is a free online standardised English language assessment.

Used by over 2500 companies and over 700 schools.

It was built in the traditional, expensive, digital way over the past 5 years.

How can AI enable effective assessments?

Leverage #LLMs to create the ability to:

Automate scoring
Automate item generation
Automate feedback

How was this new type of digital development process work?

Leverage #nlp to score the items, #LLMs to generate new questions and assessments.

Pilot to a set of students.

Human in the loop iterations and learning's

Large scale piloting

So what does multitask learning look like from an architecture perspective?

It learns multiple tasks at the same time by information sharing, which improves the performance of each task compared to separate learning.

How well does this assessment predict future performance?

There is a high correlation between EFSET Vs #IELTS and #tofl

It costs a lot of money to generate new assessment items.

So we can prompt #GPT3 to generate new items, based on the topic and assessment level.

So prompt templating and few-shot learning to generate and iterate.

Use of human validators to confirm similarity.

Up next, Carmen from @FlixBus talking to us about best practices for #conversationdesign from her key takeaways after transforming a legacy #IVR into a #conversationalIVR

Here's their journey to automated #customerservice conversations

We all know the pain of calling into a traditional IVR contact centre.

It's painful and it needs to be better.

Automating a contact centre through conversations creates a massive transformation of #customerexperience

So what are the 7 best practices learned?

1) avoid apologetic language and vocabulary with negative sentiment

2) use caller-centric vocabulary

Don't use business jargon etc.

3) control the density of politeness markers

Systems shouldn't be flattering and the use of 'we' is inconsistent with the persona of the system.

Two thankyous is too many!

Make your system sound like a system. Convey efficiency and use "I"

4) avoid lengthy instructions

Human short term memory collapses after 10 seconds of auditory input.

5) provide instructions according to the mental model of the user

Humans have difficulty replicating tasks based on instructions that don't follow an optimised or logical order.

6) keep menus below a maximum of 5 options

Humans will forget the options quickly, our working memory is limited.

7) offer the most relevant options at the beginning of the menu

It will be difficult to optimise this for every user.

Use implicit menu navigation. Use as much context as possible, use intent recognition through natural language.

Next up, the #TOBi team from @VodafoneGroup presenting the potential of #omnichannel engagement and collaborative workflows across #customerexperience

TOBi has been on a bit of a journey since 2017.

2018 first launch in Italy
2019 first voice product
2021 first common use cases
2022 Vodafone composer

To create a global, standardised experience, there needs to be coordination across the markets. Although each market has different priorities etc.

Here's how TOBi is governed across the group.

Where's the TOBi standardisation happening across the group?

There are three key building blocks

Platform
Language understanding
Journeys

By standardising journeys across group-lead and market-lead design creates efficiencies and maintains a central library of journey management.

It allows for the sharing of best working practices across different markets.

Now we see what's happening in @vodafone_de since TOBi launched in April 2018

Very cool capabilities here.

I especially like the proactive services, allowing Vodafone to create a proactive #customerexperience

Now we get a demo of animated TOBi, which is a beta of their Employee Version. It's in German and the voice is synthesised from an employee's voice.

Lots of applause from the audience.

Next up, I'll be hosting a panel discussion titled:

"Beyond the Chatbot: The Future of Conversational AI and Generative AI models."

As I'm hosting the panel, I won't be live tweeting.

#chatbots #conversationalai #generativeai