S2E deployed and now manages conversational robots for 4 major banking and insurance companies, in order to answer their recurring customer questions about employee savings. Stéphane Palvadeau, the Chatbot Administrator and Call Centre Supervisor at S2E, shares his experience managing these 4 bots using dydu’s software.
Who is S2E (Service Epargne Entreprises)?
S2E was born in 2006, grouping together 4 major players in employee savings plans, to support their development. S2E manages the administrative processing of employee savings schemes for 70,000 companies and close to 3 million employees.
S2E works with dydu’s solution to manage and monitor 4 chatbots aimed at employees with an employee savings scheme.
Stéphane Palvadeau, who are you and what is your role?
I joined S2E in October 2018 to finalise the employee savings business aspects of the chatbot, drawing on the expertise I previously acquired working in account and business administration. With regards to the shared management of the 4 bots, I am currently responsible for publishing content, analysing the dialogues, and reporting to the custodian account keepers.
More generally, my role is to continually improve the customer experience by identifying how the online customer journey can be improved, as well as any navigation issues. I also liaise with the custodians’ call centres to ensure that the chatbots and operators are providing consistent information. I begin most days by analysing the previous day’s dialogues and I optimising the knowledge base accordingly. The rest of my day is then spent improving and enriching content, liaising with call centres, producing reports, requesting updates, etc.
What issues do company pension plan bots respond to?
The bots specialise in employee savings in the banking and insurance industry. They answer generic user questions and help navigate the custodian’s website. The aim is to relieve call centres of requests with low added-value. One of the custodian account keepers already had their own chatbot. So, when S2E was contacted to deploy the same service on other custodian websites, we merge the content and management for several clients with the same issues. A large part of the knowledge base is therefore based on a regulatory framework, which can be shared among our clients. Similarly, the insights gained from analysing the dialogues can often be shared too.
In order to share this knowledge, we therefore first had to define nearly 200 common knowledge articles in collaboration with the custodian account keepers. This phase took about a year. During this time, we also identified a dozen differentiating topics to be managed outside of the collective framework, by the custodian’s individual bot. Some content is customised for each of our 4 clients, based on their schemes’ specificities or their communication strategy.
This enables us to offer each of our custodian clients a personalised and scalable service, while ensuring a common knowledge base, particularly on the regulatory side. In order to target the best period for each client, the final deployment phase for the 4 chatbots took place between October 2019 and February 2020.
How does this project work?
The 4 bots benefit from the same overall services, including the knowledge base administration, bot management or the production of reports. Content is updated and added by S2E, who then informs the custodians. This mode of operating ensures maximum agility when using the product.
The number of potential users for each custodian varies greatly, so the volume of dialogues observed on each bot is naturally heterogenous. The most used chatbot carries out more than 5,000 conversations per month. But whatever the numbers, the volumes observed justify this chatbot self-care approach.
Drawing on user questions to the 4 bots to update and add to the shared knowledge articles, also ensures that all our clients’ users share the same experience. This is a particularly virtuous model that has a positive impact on the bots’ performance.
How do you measure the bots’ performance?
We send a monthly custom report to each of our clients, based on indicators made available by dydu and the 30 most used shared knowledge articles. The bots have now reached a level of maturity that will enable us to make these reports quarterly.
Once a year, we send the custodians a global report, to provide them with an overview of the activity, and identify areas for improvement and development. The initial objective was to maintain a high qualification rate for the interactions. We therefore monitor this indicator carefully. We continually monitor the knowledge articles, in a centralised manner, and improve them accordingly, which explains why the 4 bots have interaction qualification rates that are over 98%, compared to a 93%* industry average.
We also, of course, monitor and regularly seek ways to improve the level of other customer satisfaction indicators. User feedback and reasons for dissatisfaction are therefore always analysed to ensure that the content published is continually improved.
How have the bots been used during the health crisis?
The first lockdown didn’t change user behaviour, but we have observed a change in questions, particularly with regard to stock market trends and the impact of the fall in prices on employee savings plans. We were able to quickly create new knowledge articles and update others, in order to answer these questions. Dydu’s product makes it possible for us to react quickly, because we can adapt our answer model to the situation in a very short amount of time.
What are your chatbot administrator best practices?
When it comes to employee savings plans, certain key terms that appear in user questions, such as “payment” or “transfer”, can have different meanings depending on the context or even the time of year.
To overcome this issue, I avoid updating knowledge articles with ambiguous phrases and allow the chatbot to suggest alternative phrasings instead (suggestion of several knowledge articles), so as to not provide a potentially irrelevant direct answer. In my experience, users are able to select the knowledge article that reflects their situation. In some cases, I think that indirect matching should be favoured over direct matching. If I notice that a knowledge article’s phrase is misleading users, I delete it.
It is also very important, in my opinion, to take great care when adding phrases to knowledge articles. Indeed, during my daily dialogue analysis, I add to the knowledge articles if I think a user question is likely to be asked again. This means that the bot can now understand a question that it misunderstood the day before. Before adding these new phrases, I correct spelling mistakes, delete any irrelevant contextual elements, and chose the most suitable matching groups. This process has enabled to gradually improve the matching rate over time.
What developments are you planning for the future?
Some of our clients want to expand their service with LiveChat, so that users can continue a conversation they started with the chatbot, with an agent. Dydu’s solution allows to define the triggering of knowledge articles based on behavioural targeting rules (time of inactivity, determined number of unsatisfactory answers, etc.).
We would also like to add an onboarding step before users can enter a question, in order to contextualise the relationship and explain the chatbot good practices. This will avoid any confusion between the chatbot and LiveChat.
The more personalised the information, the more relevant and appreciated by users it is. With this in mind, we plan on providing the chatbot with custom information, so that it can publish individualised content. This will meet the specificities of some of our clients’ operations that are not currently covered by the shared knowledge.
And finally, why not deploy the chatbot on a mobile app, which has proved hugely successful over the past few years?
* dydu data for the entire year 2020, based on more than 15 customer relationship chatbots in the banking/insurance industry.
