Intelligent Support for Skilled Workers Using Extended Reality

Extended Reality (xR) — including the Mixed Reality (MR) continuum from Virtual Reality (VR) to Augmented Reality (AR) — opens up powerful learning and working environments while providing intelligent, context-aware support.

To fully leverage this potential, we develop intelligent solutions that enable efficient creation and use of Mixed Reality applications — supporting people in training, manufacture, maintenance, and repair.

Intelligent assistance means our systems recognize situations and provide context-specific guidance — whether in VR or AR — across the full Mixed Reality spectrum.

Our solutions adapt flexibly to specific task requirements, data flows, and constraints in data preparation.

Our solutions combine advanced technologies to enable adaptive and context-sensitive support systems that assist users across a wide range of work environments.

Overview of our technological foundation:

• Visual recognition and analysis
  • Computer vision for real-time interpretation of real-world scenes
  • Object recognition using machine learning, e.g. based on CAD data for visual inspection
  • Pose estimation using markerless tracking for precise spatial positioning of objects as the basis for accurate overlay with digital content
• 3D capture and semantic representation
  • Neural methods for fast, approximate 3D capture of physical environments in XR
• Comparison and interaction
  • Target–actual comparison of components with nominal CAD geometry, including components made of deformable materials such as plastic molded parts
  • Practical applications:AR-supported sorting processes, Assembly assistance with step-by-step guidance, 3D instructions for maintenance and repair

The mixed reality continuum ranges from immersive virtual reality to augmented reality—implemented through a wide variety of hardware and software solutions, from head-mounted displays (HMDs) to tablets and smartphones.

Fraunhofer IGD has been researching and developing VR and AR technologies since the early 1990s. Despite major advances in performance, cost, and form factor, the following key challenges remain for practical deployment:

• Automated data integration: 3D models from development processes should be processed without manual adjustments—manual post-processing and training based on outdated data are counterproductive.
• Efficient scenario creation: MR applications must be created and adapted efficiently, for example for training or assembly support.
• Stable object recognition and tracking: Despite progress in machine learning, collecting or generating training data and training with company-specific 3D data remains resource-intensive.
• Fast and robust environment capture and semantic analysis: Intelligent assistance systems require real-time capture of reality with sufficient precision and semantic understanding in order to provide adaptive support.
• Robust target–actual comparisons: Necessary even when objects move or deform, for example in quality assurance.
• Development of industrial metaverse platforms: For sustainable, interconnected applications across the entire product lifecycle and value networks.

We address these and other challenges in our research and development activities—practical and future-oriented.

Are you looking to use Extended Reality (xR), Mixed Reality (MR), Virtual Reality (VR), or Augmented Reality (AR) to support your workforce?

Contact us — we bring deep experience, practical solutions, and technological excellence.

Our technologies are used across multiple industries and are supported in part by our spin-off companies.