Optimize Chatbots Using the Crowd

October 10, 2018


The term chatbot is probably familiar only to a few people, but it is likely that everyone has come into contact with one at some point. For example when you ask your smartphone a question via voice input, or when you contact a service provider’s customer support. Virtual assistants like Alexa, Google Assistant or Cortana are based on chatbots – programs that enable simple and intelligent communication between people and computer-based systems.

Communication with Artificial Intelligence

Chatbots are algorithms that continue to develop with the help of artificial intelligence. They give a user answers to questions, in written form or spoken word. The secret to a good chatbot is that the program becomes more familiar with the characteristics of human users and makes them its own when processing data. The artificial assistant gets better the more he operates in a human manner. That means: the ideal chatbot is inconspicuous. The user does not even realize he is talking to a machine that is translating his questions into zeros and ones and literally calculating the correct answer.

The use of chatbots is continually increasing. Optimistic estimations predict that by the year 2020, 85 percent of all communication services will rely on machines instead of people. Automation pays off for all companies that want to offer their clients effective customer support. Acceptance of the use of artificial intelligence for communication with people is continually growing. The main focus for customers always lies on how useful the response to a subject is. Whether that answer is received from a person or a program is of secondary importance.

The Communication Standard of the Future?

Chatbots will play an increasingly larger role in the future. However, intelligent communication technologies replace a human workforce only partially. A new scope of actions opens itself through chatbots as well, and the extent of their possibilities cannot even be imagined today. Chatbots make completely new, highly efficient and at the same time affordable communication channels possible for many businesses.

Bots also help relieve employees so they can be deployed in other important areas. This enables a preselection with all customer inquiries: Which questions can be answered by machine, and which questions, because they are more complex or unusual for example, should be routed to support personnel? The effectiveness of communication with customers can be significantly improved with chatbots.

How do you Optimize Chatbots?

The user asks questions, artificial intelligence answers. The fact that the answers sometimes miss the target shows that the implemented technology is not yet fully developed. And this is true although initial trials with virtual conversation partners took place in the year 1964.

What does the optimization of bots depend on? Clearly, intelligent algorithms are the first requirement to recognize what the user even wants. A broad database is essential in order to gain correct results from these algorithms.

Three elements are part of communicating with a machine:

  • speech recognition (transformation of the audio signal to text)
  • understanding the asker’s intention
  • the effective search for answers

The development of chatbots requires solid training data, the larger and more nuanced the quantity, the better. This is because the quality of a chatbot increases with the amount and quality of the input. The program recognizes coherencies and optimizes its detection patterns. The more training data is processed, the better the output becomes.

The Crowd Trains Chatbots

The development of algorithms for chatbots is based on its adaptation to training data. But that means that the quality of the program depends on how well developed the individual training scenario is. The crowd provides a way to implement these scenarios.

certain inquiries come up again and again for a telephone support service. Crowd workers formulate different individual sentences, for example in the first person (“I can’t access my account”) or passive (“My account can’t be accessed.”). And thanks to the international network of crowd workers it is simple (and cost effective) to gather the questions in different languages.

When training chatbots there are fundamentally two different approaches:

  • Training data is provided by the customer and the crowd undertakes the interpretation. In this way already existing dialogs are organized and evaluated by the crowd.
  • The crowd itself generates the training data. An intention is defined and the individual crowd worker formulates the inquiry. In this way companies receive a broad spectrum of possible question formulations for the same intention.
Read this case study: How the crowd generates training data for chatbots.

Millions of Trainers Worldwide

With a workforce of over 1.3 million people around the world, clickworker supports businesses with the training of chatbots. Whether it is a question of delivering training data or validating results, the use of various different people provides fertile ground for artificial intelligence. Examples of projects that can be broken down into micro jobs are:

  • Crowd workers receive individual audio files of spoken language (even dialects) and transcribe what is heard.
  • Crowd workers formulate similar commands in different variations.
  • In individual audio files audible noise is to be detected and classified (background noise, yawns, throat clearing, etc.).
  • Crowd workers use a camera to record gestures or facial expressions, which are meant to communicate certain intentions, feelings or requests.
Learn more about clickworker’s “AI Training Data Service“.

And this list is not nearly complete. To recognize the commonalities in similar questions or speaker intentions, requires input that is as varied as possible. The development of accurate and targeted answers depends on having the most multifaceted data basis possible for training. The crowd is an ideal tool for this.

Dieser Artikel wurde am 10.October 2018 von Jan Knupper geschrieben.


Jan Knupper