January 2020
Text by Dr. Arle Lommel

Image: © metamorworks/istockphoto.com

Arle Lommel is a senior analyst with independent market research firm Common Sense Advisory (CSA Research). He is a recognized expert in quality processes and interoperability standards. Arle’s research focuses on technology, quality assessment, and interoperability.




Beyond translation: Conversational content in a multilingual world

Conversations flow very differently in different languages and countries. But does your chatbot know that? Here is what to keep in mind when localizing conversational agents.

Chatbots, machine-authored text, and automated information retrieval and summarization: These topics are increasingly important to global businesses seeking to interact more efficiently with customers and meet their needs. Recent developments in this area have been spectacular, and AI-driven intelligent agents (chatbots and virtual digital assistants such as Siri, Cortana, and Alexa) are among the most visible success cases for machine-generated content. Enterprises look to these applications as ways to reduce costs, engage users, and create entirely new classes of goods and services.
Unfortunately, as they seek to build these services, they very quickly run into walls that threaten the viability of their development efforts:

1.     Most intelligent content frameworks support only a handful of languages.

The most common frameworks are essentially monolingual and built around English or Chinese, with some support for French, Spanish, German, or other major languages. Although it might seem trivial to localize strings in these development frameworks, this lack of language support is a problem.


2.     Speech technology still lags for voice-enabled applications.

Although it is getting better, automatic speech recognition (ASR) still struggles with accents and nonstandard dialects, slang, unclear speech, and out-of-vocabulary items. Even a few misunderstandings can change the course of an interaction. Privacy concerns can make it hard to bring humans into the loop to improve outcomes.



3.     Lack of state in applications frustrates users.

Data privacy concerns prevent developers from storing information on the history of user interactions with services. For example, if a phone owner asks Siri or a similar service for directions to Germersheim in Germany, it can launch a mapping application. However, if, five minutes later, the user asks, "What are the top tourist attractions there?" these applications will not know what "there" means because their developers have no way of tracking what happened in the conversation moments before.
However, an even bigger problem from a localization perspective is that conversations do not follow the same rules and expectations across cultures and languages. For example, a chatbot developed in the U.S. might ask, "What’s your name?" early in a discussion, store the answer in a session variable, and use it frequently in dialogue, for example: "Thank you, William. When do you want to book that flight?" Individuals interacting with a chatbot in other markets might find the use of a first name in this fashion to be too informal or "American." Although it may be simple to re-engineer a conversational agent to avoid using first names, other problems are not as easy to resolve. Conversations flow differently in different languages and countries: When to ask for specific information – or even what to ask for – may not be the same.
Some translation vendors promote the idea that enterprises leverage machine translation as an "intercept layer" on top of a chatbot in order to avoid the expense and difficulty of translating it. However, even in cases where differences may not be immediately apparent, factors such as indirect ways of asking or answering questions can throw agents into confusion. For example, if a chatbot is built on U.S. English training data and asks a yes-or-no question to which a British user responds, "That would be lovely," this typical English response may cause the agent to stop, even without the need for translation. Add in the factor of machine translation from German to English or French to Chinese and small differences can quickly add up.

Conversational agents pose a challenge, even for major enterprises

In CSA Research’s examination of chatbots, many developers expressed a pessimistic outlook about their ability to deliver them in multiple languages. One of the largest developers stated that it assessed the likelihood of success for these projects at less than 30 percent. It subsequently abandoned its efforts to translate chatbots due to the high failure rate. It found that each language version, rather than being a translation, was effectively an independent development effort.
Some of the factors that lead to failure for localization of intelligent agents are:

1.     Lack of relevant training data outside of English.

The current excitement around machine learning applications can lead companies to unwarranted optimism around their projects. Data-driven chatbots and other agents require substantial amounts of relevant examples that have been labeled – usually by human curators – but this information may be unavailable in most languages. Data manufacture – the practice of creating relevant training data – is currently an expensive and labor-intensive task, and the return on investment may not justify it for most markets.


2.     Lack of natural language processing (NLP) tools in many languages.

Chatbots and other agents frequently rely on technologies such as sentiment analysis – which determines whether comments are positive or negative – and entity identification, which finds references to specific objects, concepts, dates, locations, or other information and links it back to authoritative sources of information. These tools today exist primarily in English and a handful of European languages, but cannot be assumed to exist everywhere. To work around this limitation, some developers use machine translation, but errors in the results this technology yields can lead to disappointing outcomes.



3.     Lack of local knowledge.

Most development efforts start in English and prepare scripts and templates to highlight expected use cases. Unless teams involve experts in the languages and countries they intend to do business in, they may find that seemingly reasonable design decisions end up creating downstream headaches. This is particularly the case when legal concerns apply to specific markets or when conversation sequence or expectations vary considerably from the source version.


4.     Feature creep.

If development teams do not exercise discipline, the scope of their agents may expand over time. As they become more complex, the points of potential difficulty in localization multiply. Enterprises report the best success across multiple languages when they keep their conversational agents focused on specific tasks and limit their scope.
The situation with intelligent agents today is in many respects similar to software development in the 1980s and 1990s before internationalization best practices emerged. As a result, companies are experimenting with various solutions, but clear guidance for how to localize intelligent agents is lacking.



Steps to improve chances of success

Not every project will succeed. Work with conversational agents is a new and exciting field, but developers still have to figure out how to make them work around the world. Your goal is to improve the likelihood that you will deliver conversational agents successfully in various markets. If you are localizing them, you can boost your chances by doing the following:


1.     Do more than translate strings.

Rather than creating translated versions of resource files and plugging them back into the source agent, treat each localized version as an independent product. Although you may be able to borrow some translated assets, be prepared to engineer language-specific solutions. The results may be more like transcreation than traditional localization, but they are more likely to meet expectations.


2.     Involve local experts early on.

Involve them in more than advisory roles. They need the authority to influence plans and production schedules, or you run the risk that development teams will ignore them and move ahead with plans that will not work internationally. As a result, language experts need to be an integral part of the development effort.


3.     Develop around a common set of capabilities.

Different frameworks for creating conversational agents have different feature sets. Many developers report difficulty when they build chatbots or other agents on a platform that supports certain functionality only to find that it is not available on common platforms in other countries. To address this difficulty, list the platforms and frameworks you will need to support for each market. These will vary from fully speech-enabled ones in some markets to simple SMS-based text agents in others. Based on what you want the agent to accomplish, find a path to success on a common set of features. If this is not possible, do as much as you can and document the areas where you will need separate engineering effort for some markets.


4.     Be willing to scale back plans.

Management may want an intelligent conversational agent that can do everything, but such efforts will almost certainly fail when multiple markets are factored in. You are much more likely to achieve success with smaller, focused agents. This approach will usually deliver better results in your source language as well. If you need an agent that can address many needs, create a selection mechanism up front that routes customers to a purpose-built agent that meets their needs. In addition, even if you use a machine learning-based framework in your home market, you may find that a "rails-driven" agent will provide better outcomes in your target markets because it does not depend on the ability to understand unconstrained speech. This type of chatbot instead relies on simple questions with answers such as "Press or Say 'Yes' or 'No'" to guide users down pre-defined paths, i.e. "rails", to meet their needs.

Although localizing conversational agents is a challenge, increasing numbers of enterprises are successful in this area. Growing awareness of international issues and how they affect these applications is also helping to drive better support in frameworks, thus facilitating these projects over time.