The aim of IMPROVE is to improve lightweight near-to-the-eye displays and tiled stereoscopic large size displays. The improvements on the hardware level consist in developing a unique stereoscopic head mounted display (HMD) using emerging display technology such as LCOS and OLEDs. For tiled stereoscopic large screen displays improved calibration techniques will be developed to ease and accelerate their use. On the software level improvements comprise the fidelity of the content to be displayed (rendering quality), the interfacing between the user and the displays through innovative 2D/3D interaction techniques for mixed realities and advanced tracking systems. The achievements of IMPROVE are integrated into a collaborative mixed reality product development environment, showcased and evaluated in two application scenarios: collaborative product design in the car industry and architectural design.

Motivation

Advanced applications in automotive and architecture such as collaborative design review and augmented reality sketching require advances in the following fields:

• Lightweight stereoscopic transparent HMDs
• Calibration techniques for high resolution tiled displays
• Augmented reality rendering techniques to seamlessly embed virtual objects into real scenes
• Interaction techniques for collaborative mobile VR and AR
• Marker-less large-area tracking algorithms to support AR interactions
• Real-time video transmission to untethered mobile users

Application scenario 1:

Automotive design review

Today, design review sessions in automotive industry are mainly performed using large screen displays and virtual reality visualisation systems.
The design team is sitting in front of the display wall examining various aspects of the design from arbitrary points of view by using an input device to rotate the virtual model at will.
In contrast to that, when reviewing a physical model, the observer is moving relative to the physical prop - actually in a review more than one person is visually exploring the looks of a new design.
IMPROVE's automotive application scenario is motivated exactly by this real world example.

Combining tracking, interaction, augmented reality rendering and real-time video transmission techniques, several users are supported in doing collaborative mobile mixed reality design review sessions. They can virtually annotate models in 3D to suggest changes. Annotations can later be reviewed to check whether changes have been performed between successive review sessions.

Application scenario 2:

Architectural design and review

The architectural scenario is even somewhat more demanding.

The architects involved in the project want to do first sketches of their model on the construction site, then carry home those first sketches for refinement with their desktop architectural modelling tools such as VectorWorks and finally show and review the results to their customers either indoors or outdoors. When showing an architecture model, correct lighting simulation is essential - without shadows the virtual object looks floating above the ground and disconnected from its physical environment.

Enabling Technologies

To realize the above scenarios several technologies are under advancement in IMPROVE: The OLED-based HMD To facilitate augmented reality design review a new kind of head mounted display (HMD) is developed by two project partners.

The new HMD makes use of organic light emitting diodes (OLEDs). OLEDs are very efficient in term of power consumption and costs. The HMD is very lightweight and provides stereoscopic optical see-through capabilities. The physical layout of the HMD provides an excellent view of the physical environment - no tunnel effect.

Physically based augmented reality rendering

Another key technology is physically based augmented reality. The aim is to seamlessly integrate virtual objects into real environments by using the lighting information present in the physical scene when rendering virtual objects.

Large-area marker-less tracking

Large-area tracking is also crucial for realizing the IMPROVE scenarios. For large areas, marker-based tracking becomes less and less attractive since many markers or many cameras are needed -or both.
A more elegant and less expensive alternative is marker-less tracking. Based on information extracted from the images that are captured by a small video camera attached to the HMD, the pose and position of the head is estimated. For interaction with the virtual objects in the field tablet PCs are used.

For more information see http://www.improve-eu.info