Adaptive Camera View Clustering for Fast Incremental Image-based 3D Reconstruction
Darmstadt, TU, Bachelor Thesis, 2020
Photogrammetry, more precisely image-based 3D reconstruction, is an established method for digitizing cultural heritage sites and artifacts. This method utilizes images from different perspectives to reconstruct the geometry and texture of an object. What images are necessary for a successful reconstruction depends on the size, shape, and complexity of the object. Therefore, an autonomous scanning system for 3D reconstruction requires some kind of feedback during acquisition. In this thesis, we present an evaluation of different state-of-the-art photogrammetry solutions to identify which of them is most capable of providing feedback that predicts the quality of the final 3D reconstruction during acquisition. For this, we focused on the open-source incremental reconstruction solutions COLMAP, Alicevision Meshroom and MVE. Additionally, we included the commercial solution Agisoft Metashape to evaluate how it compares against the open-source solutions. While we were able to identify some characteristic behaviors, the accuracy and runtime of all four reconstruction solutions vary based on the input dataset. Because of this, and the fact that all four solutions compute very similar results under the same conditions, our tests were not conclusive. Nevertheless, we chose COLMAP as the back-end for further use as it provided good results on the real dataset as well as an extensive command-line interface (CLI). Based on these results, we introduce an iterative image-based reconstruction pipeline that uses a cluster-based acceleration structure to deliver more robust and efficient 3D reconstructions. The photogrammetry solution used for reconstruction is exchangeable. In this pipeline, images that portray common parts of an object are assigned to clusters based on their camera frustums. Each cluster can be reconstructed separately. The pipeline was implemented as a c++ module and tested on the autonomous robotic scanner CultArm3D®. For this system, we embedded the pipeline in a feedback loop with a density-based Next-Best-View (NBV) algorithm to assist during autonomous acquisition.
Automated 3D Mass Digitization for the GLAM Sector
Archiving 2020 online. Final Program and Proceedings
Archiving <2020, Online>
The European Cultural Heritage Strategy for the 21st century has led to an increased demand for fast, efficient and faithful 3D digitization technologies for cultural heritage artefacts. Yet, unlike the digital acquisition of cultural goods in 2D which is widely used and automated today, 3D digitization often still requires significant manual intervention, time and money. To overcome this, the authors have developed CultLab3D, the world’s first fully automatic 3D mass digitization technology for collections of three-dimensional objects. 3D scanning robots such as the CultArm3D-P are specifically designed to automate the entire 3D digitization process thus allowing to capture and archive objects on a large-scale and produce highly accurate photo-realistic representations.
Erweiterter 2D Farbmanagement Workflow für 3D Digitalisierung
Darmstadt, Hochschule, Bachelor Thesis, 2020
Konsistente und der Realität entsprechende Farbwiedergabe ist bei Digitalisierung oftmals ein wichtiger Faktor. Insbesondere bei der Erfassung von Kulturerbe spielt dies eine große Rolle. Für den 3D Bereich gibt es jedoch bislang keine veröffentlichten Untersuchungen bzgl. der Farbqualität im Vergleich zu 2D Aufnahmen oder spezielle Empfehlungen für einen vollständigen Farbmanagement Workflow. Das Ziel der vorliegenden Arbeit war daher, zu untersuchen, welche Faktoren sich auf Farbe und Farbwahrnehmung auswirken und wie sie dies tun. Insbesondere wurde ermittelt, wie sich Farbmanagement im 3D Bereich im Vergleich zum 2D Bereich auswirkt und wie ein optimales Ergebnis durch Übertragung und Erweiterung eines 2D Workflows erreicht werden kann. Betrachtet wurden die drei Kriterien Farbabstand (Delta E), Rauschen und Farbneutralität. Dazu wurde anhand der Technical Guidelines for Digitizing Cultural Heritage Materials ein 2D Workflow entwickelt. Dieser wurde auf eine digitale 3D Abbildung (Rekonstruktion) eines flachen Objekts übertragen und entsprechend erweitert. Die Ergebnisse bei verschiedenen Einstellungen des Tools Agisoft Metashape für die Rekonstruktion, sowie bei Anwendungsvarianten von ICC-Profilen und Maskierung, wurden mit der 2D Abbildung und untereinander verglichen. Die Untersuchungen zeigen, dass der Farbmanagement Workflow allgemein eine sichtbare und messbare Verbesserung bewirkt. Die 3D Rekonstruktion weist gegenüber dem 2D Vergleichsbild einen deutlich schlechteren Farbabstand, jedoch bessere Farbneutralität und Rauschwerte auf. Optimale Farbwiedergabe wird durch den Blending Mode Mosaic oder Max Intensity bei der Rekonstruktion und die Anwendung eines ICC-Profils auf den Textur- Atlas erreicht. Zudem sollte keine weitere Color Calibration durch Agisoft durchgeführt werden. Der Lichtabfall zeigt einen starken Einfluss, daher sollte dieser durch z.B. Depth of Field Masking ausgeglichen werden. Als problematisch haben sich auch Reflexionen und der Weißabgleich herausgestellt. Letzterer muss für die gesamte Aufnahme fest eingestellt sein. Die Reflexionen sollten mit einem Polfilter entfernt werden.
Extended 2D Image Quality Assessment for Photogrammetric 3D Digitization
Darmstadt, Hochschule, Master Thesis, 2020
With recent technical progress the digitization of cultural heritage has become increasingly practicable and important. Many museums capture their artifacts to document and preserve their current state or to have a digital copy in case the original is damaged, destroyed or being restored. The majority of this digitization today is achieved by photographing or flatbed scanning of artifacts, thus producing a 2D image suited to capture 2D documents but limited to only a specific view angle for a 3D object. Therefore, in recent years, the application of new 3D scanners that can completely capture an object’s surface from all sides have become more popular. For both, 2D and 3D digitization methods, the goal is to digitally represent the original artifact in the most realistic and authentic way. Therefore, in 2D digitization, guidelines have been developed to estimate and assure the quality of the captured data. One widely used standard is the ISO norm 19264 which considers a large number of image quality criteria, such as sharpness and color accuracy. However, currently no comparable guidelines exist for the 3D digitization of artifacts. This thesis bridges part of the gap between 2D and 3D. It focuses on increasing the achievable resolution of the reconstructed texture and geometry. For this purpose, the quality characteristics from ISO 19264, which describe the level of detail of a 2D image, are extended to 2.5D so that they describe the complete depth of field of the image. Afterward, the effect of different camera-, image- and reconstruction-settings on the 3D results are investigated. For this purpose, the obtained point cloud is investigated both visually and based on the VDI/VDE 2634 guideline. A clear correlation between the quality of the camera parameters and the achievable reconstruction result can be shown. Thus, best practices for the settings can be derived, which allow for reconstruction details in the texture and geometry, which are not visible to the naked eye. The best practices describe, among other things, that the used image sections must be limited by the depth of field, how to calculate the texture size of the reconstruction based on the 2D sampling rate and that it is better to use a smaller depth of field with better quality then a larger one with slightly worse quality. At the end of the thesis, the current limitations of the 3D quality evaluation methods are discussed and an outlook is given describing how these can be even further improved in the future to enable a better evaluation of the 3D results.
Phenomenological Roughness Extraction for Physically Based Rendering
Many modern rendering systems use physically based shading techniques, one of which bases its surface property description on the "metalness" and "roughness" of a material. In this work a method for the automatic extraction of the surface roughness parameter from images of an object is developed. The extraction employs polarized light to filter out reflections and uses the difference in reflectivity visible in the images compared to an unpolarized lighting environment to deduce a surface roughness value.
Towards 3D Digitization in the GLAM (Galleries, Libraries, Archives, and Museums) Sector – Lessons Learned and Future Outlook
The IPSI BgD Transactions on Internet Research
The European Cultural Heritage Strategy for the 21st century, within the Digital Agenda, one of the flagship initiatives of the Europe 2020 Strategy, has led to an increased demand for fast, efficient and faithful 3D digitization technologies for cultural heritage artefacts. 3D digitization has proven to be a promising approach to enable precise reconstructions of objects. Yet, unlike the digital acquisition of cultural goods in 2D which is widely used and automated today, 3D digitization often still requires significant manual intervention, time and money. To enable heritage institutions to make use of large scale, economic, and automated 3D digitization technologies, the Competence Center for Cultural Heritage Digitization at the Fraunhofer Institute for Computer Graphics Research IGD has developed CultLab3D, the world’s first fully automatic 3D mass digitization technology for collections of three-dimensional objects. 3D scanning robots such as the CultArm3D-P are specifically designed to automate the entire 3D digitization process thus allowing to capture and archive objects on a large-scale and produce highly accurate photo-realistic representations. The unique setup allows to shorten the time needed for digitization from several hours to several minutes per artefact.
End-to-end Color 3D Reproduction of Cultural Heritage Artifacts: Roseninsel Replicas
Eurographics Workshop on Graphics and Cultural Heritage (GCH) <17, 2019, Sarajevo, Bosnia and Herzegovina>
Planning exhibitions of cultural artifacts is always challenging. Artifacts can be very sensitive to the environment and therefore their display can be risky. One way to circumvent this is to build replicas of these artifacts. Here, 3D digitization and reproduction, either physical via 3D printing or virtual, using computer graphics, can be the method of choice. For this use case we present a workflow, from photogrammetric acquisition in challenging environments to representation of the acquired 3D models in different ways, such as online visualization and color 3D printed replicas. This work can also be seen as a first step towards establishing a workflow for full color end-to-end reproduction of artifacts. Our workflow was applied on cultural artifacts found around the “Roseninsel” (Rose Island), an island in Lake Starnberg (Bavaria), in collaboration with the Bavarian State Archaeological Collection in Munich. We demonstrate the results of the end-to-end reproduction workflow leading to virtual replicas (online 3D visualization, virtual and augmented reality) and physical replicas (3D printed objects). In addition, we discuss potential optimizations and briefly present an improved state-of-the-art 3D digitization system for fully autonomous acquisition of geometry and colors of cultural heritage objects.
Acceleration of 3D Mass Digitization Processes: Recent Advances and Challenges
Mixed Reality and Gamification for Cultural Heritage
In the heritage field, the demand for fast and efficient 3D digitization technologies for historic remains is increasing. Besides, 3D has proven to be a promising approach to enable precise reconstructions of cultural heritage objects. Even though 3D technologies and postprocessing tools are widespread and approaches to semantic enrichment and Storage of 3D models are just emerging, only few approaches enable mass capture and computation of 3D virtual models from zoological and archeological findings. To illustrate how future 3D mass digitization systems may look like, we introduce CultLab3D, a recent approach to 3D mass digitization, annotation, and archival storage by the Competence Center for Cultural Heritage Digitization at the Fraunhofer Institute for Computer Graphics Research IGD. CultLab3D can be regarded as one of the first feasible approaches worldwide to enable fast, efficient, and cost-effective 3D digitization. lt specifically designed to automate the entire process and thus allows to scan and archive large amounts of heritage objects for documentation and preservation in the best possible quality, taking advantage of integrated 30 visualization and annotation within regular Web browsers using technologies such as WebGI and X3D.
Methodology for Evaluation of Precision and Accuracy of Different Geometric 3D Data Acquisition Methods
München, TU, Master Thesis, 2017
3D optical scanning systems have been gaining considerable space in metrology, being largely applied in industry sectors and in the cultural heritage domain. The amount of available sensors on the market has grown considerably. Thereby, deciding for the right technique that fits-to-a-purpose or the most cost efficient technology, is a challenging task. When deciding in which technology to invest, the user often relies on the manufacturer’s instructions. However, manufacturers generally do not state under which conditions such values were acquired and thus, the system’s reproducibility is not assured. If measurements could be traced back to a common standard, this problem could be easily addressed. As such a solution is still not available, specialist often tend to solve this issue by associating terms like precision, accuracy and uncertainty to a measurement. Nowadays, the most applicable solution to define the accuracy of a system relies on the VDI/VDE 2634. This master thesis aims to develop a common solution to assess accuracy for different geometric 3D data acquisition models, considering the specifications of the VDI/VDE 2634 Part 3. The methodology proposed here encompasses the entire process from the acquisition to its processing stage. The study-case comprehend triangulated methods, as photogrammetry and laser line sensor. During the acquisition, a calibrated probing body and adapted test are proposed. The processing stage includes a best-fit algorithm and an evaluation of measurement uncertainty. The result comprehends the quality parameters together with the visualization of measurement uncertainty supporting the entire system. Therefore, providing to the end user enough information about the capability of the evaluated system.
Projecting our Past to the Future - Challenges and Results: The Case of Asinou Church
Eurographics Workshop on Graphics and Cultural Heritage (GCH) <15, 2017, Graz, Austria>
In this paper, we present some of the novel results of the Marie Curie Initial Training Network for Digital Cultural Heritage (ITN-DCH) project, describing briefly the work done focusing on the project's first case study: the Panagia Phorviotisa of Asinou, an UNESCO World Heritage Listed (WHL) monument in Cyprus. The paper introduces some challenges and the importance of multidisciplinary, sustainable research and development in the emerging domain of DCH in Europe. The different methodologies address these challenges through a professional network of partners including Academia, Research and Industry. The paper describes the 3D documentation of the church and how the data acquired can be used and re-used in Mixed Reality (MR) applications using also Deep Learning techniques, as well as in Education.
c-Space: Time-evolving 3D Models (4D) from Heterogeneous Distributed Video Sources
Eurographics Workshop on Graphics and Cultural Heritage (GCH) <14, 2016, Genova, Italy>
We introduce c-Space, an approach to automated 4D reconstruction of dynamic real world scenes, represented as time-evolving 3D geometry streams, available to everyone. Our novel technique solves the problem of fusing all sources, asynchronously captured from multiple heterogeneous mobile devices around a dynamic scene at a real word location. To this end all captured input is broken down into a massive unordered frame set, sorting the frames along a common time axis, and finally discretizing the ordered frame set into a time-sequence of frame subsets, each subject to photogrammetric 3D reconstruction. The result is a time line of 3D models, each representing a snapshot of the scene evolution in 3D at a specific point in time. Just like a movie is a concatenation of time-discrete frames, representing the evolution of a scene in 2D, the 4D frames reconstructed by c-Space line up to form the captured and dynamically changing 3D geometry of an event over time, thus enabling the user to interact with it in the very same way as with a static 3D model. We do image analysis to automatically maximize the quality of results in the presence of challenging, heterogeneous and asynchronous input sources exhibiting a wide quality spectrum. In addition we show how this technique can be integrated as a 4D reconstruction web service module, available to mobile end-users.
Data Provenance in Photogrammetry Through Documentation Protocols
XXIII ISPRS Congress Prague, Commission V
International Society for Photogrammetry and Remote Sensing Congress (ISPRS) <23, 2016, Prague, Czech Republic>
ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, III-5
Documenting the relevant aspects in digitisation processes such as photogrammetry in order to provide a robust provenance for their products continues to present a challenge. The creation of a product that can be re-used scientifically requires a framework for consistent, standardised documentation of the entire digitisation pipeline. This article provides an analysis of the problems inherent to such goals and presents a series of protocols to document the various steps of a photogrammetric workflow. We propose this pipeline, with descriptors to track all phases of digital product creation in order to assure data provenance and enable the validation of the operations from an analytic and production perspective. The approach aims to support adopters of the workflow to define procedures with a long term perspective. The conceptual schema we present is founded on an analysis of information and actor exchanges in the digitisation process. The metadata were defined through the synthesis of previous proposals in this area and were tested on a case study. We performed the digitisation of a set of cultural heritage artefacts from an Iron Age burial in Ilmendorf, Germany. The objects were captured and processed using different techniques, including a comparison of different imaging tools and algorithms. This augmented the complexity of the process allowing us to test the flexibility of the schema for documenting complex scenarios. Although we have only presented a photogrammetry digitisation scenario, we claim that our schema is easily applicable to a multitude of 3D documentation processes.