October 2011
By Dr. Ralf Wienken

Dr. Ralf Wienken is responsible for technical documentation in the Xinfo Wieland Sacher Company. The main activity of this company is the development of software for industrial applications.

ralf.wienken[at]xinfo.de
http://app-solut.com

Augmented Reality in technical communication

The term technical communication describes the provision of instructions for the manufacturing and modification of products as well as for the safe usage of products. The items to be described are mostly machines or software, but also things for everyday use that require explanation like pharmaceuticals, cooking pots or toys. Almost always, the information is transmitted visually, either on paper or on displays. In the near future, there is a good chance that these two visual channels will be supplemented by inserting data directly into the perception of reality itself. This is known as Augmented Reality.

Let’s take a look at what Augmented Reality is and what role it can play in technical communication.

Augmented Reality

Augmented Reality (AR) generally refers to the enlargement of the human perception of reality by information generated by a computer. An artificial second level is added to the natural environment that provides additional data in real-time.

This second level is usually generated by a mobile phone like the iPhone or Android or by a tablet computer like the iPad. Captured by the camera, the display shows the reality section to which appropriate information and objects are added.

Some examples: Equipped with a smartphone, a user may start the search for a new apartment. As he walks through the desired area, an app for real estate provides him with the needed information in real time: an image of the road as seen by the user, whether suitable apartments are available, data about the flats and even the direction in which a further search promises the most success. If interested, the apartment seeker may retrieve detailed data about an object.

Such applications are based on a connection of GPS and compass data from the smartphone with a database that provides information about real estate. The app detects exactly where the apartment seeker is located and in what direction he’s looking. In addition to his natural sensory perception of reality appears a second level reality with artificially added data: the user sees the same buildings on the smartphone as through his eyes, but enriched with appropriate information. The reality is thus extended, in other words augmented.

The objects that are added to reality have the following characteristics:

  • They are related to the real world. The reality is augmented, not replaced - otherwise we would speak of virtual reality.
  • They cannot be perceived with the natural sensory organs.
  • They correspond exactly to the place where the user is located.
  • They are context-sensitive. The information is localized and selected according to the purpose - the user sees only what he needs to see to solve his problem.
  • They are interactive - the user can manipulate the data according to his requirements. For example: in addition to the automatically presented information, the user may request more detailed data describing a product he is especially interested in.
  • They are supplied both visually and acoustically.


In addition to apps that work with geographic data, other types of apps recognize objects and react accordingly.

Let's assume our smartphone user from the previous example has now found an appropriate apartment and needs help to set it up. He walks through his new home and, on his smartphone, places small-sized objects where he imagines furniture. These objects are known by the app and connected with data of furniture. If he points the camera to an object, the app reacts and, for example, places a wardrobe in front of the wall. The wardrobe can be modified in its characteristics such as color, height and width. Again, the reality is extended: the camera provides a three-dimensional duplicate of the apartment, that the owner can fill with virtual furniture.

Four weeks later, the apartment owner wants to put a picture on the wall, and has bought an electric drill. He directs his phone onto the machine, and a tutorial starts automatically. This tutorial is not a simple video, but a 3D reproduction of the apartment with the real drill. The software constantly and interactively changes the reproduction according to the current situation, so that the tutorial looks as if a demonstration is taking place in reality.

Interaction with other people

However, Augmented Reality is not restricted to a single user who wants to obtain information about the environment, or who wants to place objects into the environment and manipulate them. It is possible to create virtual objects for other users, and, for example, leave messages on house walls or set huge posters into the sky. Such objects will remain and are visible to all users with the same equipment. It's like an online computer game, with the difference that there is still a connection with reality.

Location-based or object-based applications

As shown in the examples, there are two ways to create an Augmented Reality:

  • by location-based applications that work with geographic data, as shown in the example with the apartment search (location-based services)
  • by object-oriented applications that work with images, as shown in the examples with the furniture and the drill (image-based services)

The basis of such applications are mostly so-called Augmented Reality Browsers (ARB), such as junaio from the company Metaio, or layar from the company with the same name. They are available for iOS, Android and Symbian.

Technical communication and Augmented Reality

Currently, applications based on an Augmented Reality are mostly designed for marketing, sales and advertising. In technical communication, the new technology so far is only used playfully as to demonstrate its possibilities. This will change. Augmented Reality applications will play an increasing role in the industrial sector. Short reaction times, mobility and immediate communication of information promise productivity gains which no company can afford to ignore.

In the industrial sector, mainly image-based services are used in the following areas:

  1. Customer service and maintenance

    Augmented Reality is an ideal technology for customer service and maintenance, because the required information can be accessed immediately and directly on-site. 
Imagine that a service technician is ordered to repair a complex machine. He directs his iPad to the machine, and immediately the current performance data is retrieved and the problem identified. The software creates a visual overlay that indicates the problem area and demonstrates a step-by-step repair process. The technician on his part enters information about the repair either using the display or the voice command.

  2. Manufacturing and assembly
    
Information and instructions are provided to the employees exactly where and when needed, maybe even projected onto the component.

  3. Quality management
    
It is possible to localize problems with a tablet computer and to lead an employee there, for example in an engine factory. The employee then directs the computer to the engine and immediately the deviations of the currently measured data are displayed, along with suggestions on how to solve the problem.
  4. Prototype construction and design

    In this area, Augmented Reality has been used for some time already to visualize changes in design or components. 

  5. Planning and reconstruction of factories
    
Here, too, Augmented Reality has been used for some time to visualize the planning process. It's not only about design, but also about construction: for example, in order to see at an early stage whether a new pipe system fits into the existing system.



Augmented Reality provides technical writers with a new challenge: In the future they will be expected to write for a medium that is much more interactive and situational than anything before. This undoubtedly changes the creation of technical documents.

Technical writers need to get a clear idea of the visual and audible impression that is intended to be achieved. Augmented Reality cannot be compared with the simple reading of documents, whether on paper or on display. Rather, it is a comprehensive sensorial input, which can be very impressive for beginners: semi-submerged in the artificial reality, while still semi-connected with the real environment.

Producing technical documentation for Augmented Reality will also necessitate more permanent coordination with the software developers.

Augmented Reality enables the exact adaptation to the user’s situation. Thus, technical writers need to consider not only the users location, time and his problem with the product, but also ways in which the user can interact with Augmented Reality.

Summary

The obvious reason for technical writers to be interested in Augmented Reality is: it exists. If this business segment is not occupied at an early stage (which is now), marketing people and developers will write the contents, and later on technical communication providers will have to convince customers that they can do better.

Apart from business prospects, there are the extended possibilities of information transmission that make Augmented Reality so interesting for technical communication. It replaces the traditional manual with a mix of visual and acoustic reality and added information.

Whether Augmented Reality can be established fully as an independent third channel in addition to paper and display is to be expected, but not finally settled. In any case, it would have to be separated from the display, to be fed directly into the senses. This has been discussed in the laboratories for some time now – however, until now there are no serious industrial applications due to missing technical possibilities. But that is only a matter of time.

Further reading

See the following links for further information: