March 2013
By Juergen Lumera

Juergen Lumera has more than 15 years experience in XML authoring and delivery solutions. In his current role as director TIS Product Management and Innovation, he is defining the next generation of technical documentation systems. He is especially interested in new trends and technologies for technical documentation.



Is Augmented Reality the future of technical documentation

A few years ago, Alan Brandon asked the following question in an article about an Augmented Reality research project (AMAR) at the Columbia University: Is Augmented Reality the future of technical documentation? He did not really answer the question, but he described the huge potential of this technology. The research project did not change the technical documentation world at that time. This article will describe what has changed since then and why the answer to his question is now a definitive YES – this technology is changing the technical documentation world and we need to be prepared.

Before we start discussing in greater detail the possibilities of Augmented Reality in the context of technical documentation, it is necessary to define what Augmented Reality is exactly, what the requirements are, and how it is already used for technical documentation.

What is Augmented Reality?

Wikipedia defines Augmented Reality in the following way: “Augmented Reality (AR) is a live, direct or indirect, view of a physical, real-world environment whose elements are augmented by computer-generated sensory input such as sound, video, graphics or GPS data. …“

Image: R vs VR vs AR – what are the differences (Source: Re-flekt GmbH)


In other words Augmented Reality

  • combines real and virtual content
  • is interactive in real time
  • registers in 3D

Samples of Augmented Reality can be found in our daily life – but very often we are not even aware that it is Augmented Reality. Everyone has seen those overlaid lines in a televised football game to indicate the position of a player relative to the ball. A more advanced usage of Augmented Reality can be found in Smartphone Applications like Wikitude or Layar, which show search results through the view of the camera. For technical documentation, applications like Junaio augment cartridge exchange information on a printer.

Image: AR Owner guide – overlay video of a printer with information about changing the cartridge (Source: Metaio GmbH)

What is required to use Augmented Reality?

The major components of an augmented reality solution are:

  • a sensor to capture reality (typically a camera)
  • 3D data about the content to be recognized
  • display to show captured and overlaid (augmented) content, such as a smartphone, tablet, goggles, or head-mounted display.
  • component(s) to interact with AR application – touch screen or microphone
  • the augmented content (3D data, 2D data, text, audio)
  • application to feed content to AR client application

How can Augmented Reality be used for technical documentation?

The above mentioned AMAR project at the Columbia University had implemented a prototype to allow a service technician to execute a repair guided by augmented information. The technician sees the real world and additional content through his goggles. This additional content helps him to find the right position, to use the correct tools and to execute the necessary steps in the right order without being distracted by consuming and interpreting “classical” technical documentation. Many variations with other display devices have been built but they all follow more or less the same basic concept.

Image: Repair information – overlay live video of an engine compartment with parts, tools and steps

Advantages of Augmented Reality

Using Augmented Reality eliminates the need to transfer written content to the location where it needs to be applied. In a classical environment a technician or owner typically flips through pages (paper or Web) to identify the relevant information. This information then needs to be transferred to the place where the task (repair, operation, etc.) has to be executed. This requires the technician to interpret the written content and align existing graphics with the physical object.

In an Augmented Reality world this transformation (and search) step is not necessary because the information is already displayed at the place where it is needed. It is information at the point and place of need. It can even adjust to the current context: a technician can start by himself and request support from an Augmented Reality application when he cannot continue based on his own knowledge.

What does this mean for technical authors?

The most important part of an Augmented Reality solution, beside the software component, is the content used for augmentation. This content is what a technical author has to produce in the future. He needs to work in a 3D view of the main object and place the content for augmentation. The textual part will to a fair degree be replaced by 3D objects showing activities, tools, parts, etc. New authoring tools for those tasks are needed and require a different skillset – graphic-oriented thinking, focused on “real” task execution instead of abstract descriptions, aiming to minimize/eliminate the non-graphical information. It is not enough to use a new tool; it requires us to change the mental approach of how to “interact” with the consumers of technical documentation: in an Augmented Reality world you don’t explain, you SHOW.

What is required to produce AR content?

Similar to the traditional authoring approach, an author requires an authoring tool allowing management of the augmented content (e.g. parts, tools, navigation) and, in addition, a model of the actual real world part (e.g. vehicle, printer). Currently the market for these authoring tools is limited (e.g. Metaio Creator from Metaio GmbH or D’Fusion Studio from Total Immersion) and the tools are in their early stages in relation to technical documentation authoring. For this reason, most AR content developers use repurposed tools from other business areas, for example CAD or design tools. Irrespective of the tool an author is using, there is always the need to deal with the same set of input data (mainly engineering models) to produce the augmented content.

Currently there is only limited training offered around AR content creation. At present the recommended approach for AR use cases is to develop expertise ‘on-the-job’ using support from a technology supplier. This enables authoring tools to be developed for the specific user cases while building up skills and expertise.

What has changed since the original question has been asked?

During the last three years the technology to enable Augmented Reality has changed dramatically. All modern smartphones have the required sensors and display capabilitiy already built-in. Tablets offer the same set of functionality and provide a larger area for interaction. Voice recognition has tremendously improved and can be used in industrial solutions. Inexpensive goggles to display augmented content are ready for the mass market.

But not only the technology has changed. The massive use of Augmented Reality in marketing as well as extensions to existing applications and media types (navigation systems, catalogues, search engines, and so on) have paved the street for this technology.

We have reached a state where it is no longer a question if it will happen because it is already there. Now it is rather the question of how quickly we can set up a complete process and structure to support Augmented Reality in the same way as we are doing it for the existing technical documentation.

To get back to the original question: YES Augmented Reality is the future of technical documentation because the market has the tools and will demand the content for it.

Image: Wire harness – overlay live video of a vehicle with wire harness