From the Digitization of Cultural Artifacts to the Web Publishing of Digital 3D Collections: an Automatic Pipeline for Knowledge Sharing
Summary (4 min read)
Introduction
- In the field of cultural heritage (CH), knowledge sharing is one of the most essential aspects for communication activities between museal institutions, that conserve and take care of cultural collections, and the public.
- For this purpose, multimedia technologies are becoming more and more widespread in the CH field, where these surrogates are then represented by digital copies.
- Moreover, inasmuch as high speed scanning systems are often prone to a lack of accuracy with respect to more classic digitization technologies, the authors also estimate a normal texture from these data, once again in an automatic manner.
- Hence, the archival and the sharing of vast item collections becomes possible and easy also for non expert users.
- Section III presents the two first stages of their system, namely the in-hand scanner used for the acquisition, as well as their processing step for generating a digital copy from the acquired data.
A. Real-time 3D scanning
- An overview of the 3D scanning and stereo reconstruction goes well beyond the scope of this paper.
- Their main issues are the availability of technology and the problem of aligning data in a very fast way.
- Among the latter, the most robust approach is based on the use of fast structured-light scanners [1], where a high speed camera and a projector are used to recover the range maps in real-time.
- This is essentially due to the low resolution of the cameras, and to the difficulty of handling the peculiar illumination provided by the projector.
- Other systems have been proposed which take into account also the color, but they are not able to achieve real-time performances [6] or to reconstruct the geometry in an accurate way [7].
B. Color acquisition and visualization on 3D models
- The most flexible approach starts from a set of images acquired either in a second stage with respect to the geometry acquisition, or simultaneously but using different devices.
- Image-to-geometry registration, which can be solved by automatic [8]–[10] or semiautomatic [11] approaches, is then necessary.
- Due to the lack of consistency from one image to another, artifacts are visible at the junctions between surface areas receiving color from different images.
- In particular, Callieri et al. [20] presented a flexible weighting system that can be extended in order to accommodate additional criteria.
- The first tools aimed at visualizing 3D models in Web pages were based on embedded software components, such as Java applets or ActiveX controls [25].
III. DIGITIZATION AND PROCESSING OF CH ARTIFACTS FOR GENERATING DIGITAL COPIES
- Cultural heritage has been a privileged field of application for 3D scanning since the beginning of its evolution.
- This is due to the enormous variety and variability of the types of objects that can be acquired.
- Moreover, archival and preservation are extremely important issues as well.
- The acquisition of a large number of objects can be expensive both in terms of hardware and time needed for data processing.
- They usually need the placement of markers on the object, which is something that is hard to make on CH artifacts.
A. Acquisition by in-hand scanning
- The first stage of their workflow, namely the one producing all data required for the creation of digital copies from cultural artifacts, is based on an in-hand scanner whose hardware configuration is shown in Figure 2.
- This scanner, like most of the high speed digitization systems, is based on structured light.
- The scanning can be performed in two different ways.
- Occlusions are detected by a hue analysis which produces, for each video frame, a map of skin presence probability.
- These data are then used in the next stage of the workflow in order to produce the color texture and the texture of details, as explained hereafter, in section III-B.
B. Recovery of a diffuse color texture
- The texturing method extends the work proposed in [20] so as to adapt it to the data flow produced by the scanning system presented above.
- These criteria have been chosen so as to penalize image regions that are known to lack of accuracy, in order to deal with data redundancy from one image to another in © 2012 ACADEMY PUBLISHER a way that ensures seamless color transitions.
- Since the complete geometric configura- tion of the scene is known, the authors can use a simple shadow mapping algorithm to estimate shadowed areas, to which a null weight is assigned.
- During the subsequent passes, highlights are identified by computing the luminosity difference between the texture obtained at the previous pass and the input picture.
- Each elementary mask contains values in the range ]0, 1], zero being excluded in order to ensure that texturing is guaranteed for every surface point that is visible in at least one picture.
C. Recovery of a texture of details
- It often leads to a loss of accuracy with respect to traditional scanning devices, thus preventing the acquisition of the finest geometric details.
- The authors use here a similar approach for extracting a normal map from the video flow produced by the in-hand scanner.
- This uneven sampling distribution may result in an estimated normal which is reliable along this plane but really uncertain along the orthogonal direction.
- To alleviate this problem, the authors propose to analyze the sampling distribution at each point p by performing a PCA on the set of light directions.
- By definition, ν2 is the direction along which the sampling is the poorest.
IV. WEB PUBLISHING
- After geometry and texture images have been processed, the third and last stage of their pipeline optimizes the generated data for network transmission and realtime rendering on standard Web browsers.
- The optimized version of the 3D model is stored in the server file system and is accessed by a standard HTTP server to serve requests of visualization clients.
- In the following, the authors describe the steps they use for preparing and storing the data.
A. Data optimization
- The optimization phase is composed of two sequential steps: geometry partitioning and rendering optimizations.
- To this end, the authors use a simple greedy method that iteratively adds triangles to a chunk until the maximum number of vertices is reached.
- One advantage of the indexed triangle mesh representation is that vertices referenced by more than one triangle need to be stored only once, also known as Rendering optimizations.
- To convey this advantage from memory occupancy to rendering performances, graphics accelerators have introduced a vertex cache capable of storing data associated to up to 32 vertices, thus allowing to reuse the results of a considerable amount of per-vertex calculations.
- Even though the problem does not have a polynomial-time solution, several works have been developed [42], [43] that produce a very good approximate solution in a relatively small amount of time.
B. Data storage and retrieval
- One of their goal is to exploit standard and easily available technologies for making the produced models accessible on the Web platform.
- To this end, the authors decided to use the well-known Apache HTTP server and use the server file system as the storage database.
- Model data is saved under standard file formats: to store geometry information the authors use the Stanford polygon file format (PLY), which support multiple vertex attributes and binary encoding, while Portable Network Graphics (PNG) images are used for color and normal textures.
- Even though those formats are already compact, the authors take advantage of the automatic compression (gzip) applied by the Apache server on data transmission, as well as automatic decompression executed by browsers on data arrival.
- To access the remote 3D model, visualization clients use JavaScript to issue a HTTP request with a base URL of the form http://example-data-domain.org/modelname/, and appending predefined file names to discriminate among geometry and texture files, such as geometry.ply, color.png and normal.png.
V. RESULTS
- The authors present in this section some results of their pipeline, as well as some implementation details.
- The proposed objects are a sample of a group of artifacts which were used to test the entire system.
- The proposed results and processing times show that an extension to large collections is straightforward.
- For the texture of details, both matrices (LTL) and (LTC) of equation 6 can be constructed incrementally by processing input pictures one by one on the GPU and accumulating intermediate results using buffer textures.
- This permits to obtain the 3D models of several objects within an hour of work.
B. Diffuse color texture reconstruction
- The authors texturing results are shown in the bottom row of Figure 6.
- The most obvious difference that can be noticed is clearly the drastic loss of luminosity that occurs in the case of the naive approach.
- For the Pot model , the big vertical crack in the white rectangle results from the fact that one portion of the surface was depicted by a much greater number of frames with respect to the adjacent one: this produces an imbalance among the number of summed color contributions, and the consequent abrupt change of color.
- During this texturing phase, the only parameters that must be set by the user are the number of applications for each elementary mask.
- The set of parameters is then really small and can be tuned in an easy and intuitive manner.
C. Detail texture reconstruction
- Figure 8 illustrates the efficiency of their normal correction procedure by showing the normal field computed for the same object with and without correction.
- Since the eigenvalue ratio decreases, the estimated normal is forced to get © 2012 ACADEMY PUBLISHER closer to the original mesh normal along the direction of highest uncertainty.
- The frames on the right side of these images highlight once again the improvement resulting from the estimated texture of details.
- Thus, the user do not have to perform on purpose an exhaustive measurement just to satisfy the fitting constraints.
- As shown at the bottom of each Web page snapshot in Figure 13, the rendering performance is in the order of thousands of frames per second (FPS) for models that range from 50K to 100K triangles.
VI. CONCLUSION
- The authors presented a complete pipeline for the creation of Web browsable digital content from real objects, consisting in 3D models enhanced by two textures respectively encoding artifact free color and fine geometric details.
- Even though the proposed approach is generic enough to be used in any application for which producing and sharing digital content about real artifacts present an interest, its three main advantages (namely its ease of use, its high automation and its quickness) make it particularly appropriate to cases where huge collections have to be processed.
- It is not easy to know in advance if enough frames have been acquired so that accurate color or fine geometric detail can be extracted safely.
- Hence, a “Photo Tourism-like” [46] image navigation could be possible.
Did you find this useful? Give us your feedback
Citations
References
24,320 citations
"From the Digitization of Cultural A..." refers background or methods in this paper
...Photometric stereo consists in computing high quality normal/range maps by taking several photographs from the same viewpoint but with different illumination directions [37]–[39], or by moving the object in front of a camera and a light source that are fixed with respect to each other [40], [41]....
[...]
...This is not a one-off fortuitous occurrence, but generally true (see additional examples at [37])....
[...]
...The figures for other values of d are at [37]....
[...]
...EXPERIMENTAL RESULTS We have designed all experiments such that they are reproducible, and as such, all data and code are freely available at [37]....
[...]
13,993 citations
"From the Digitization of Cultural A..." refers background or methods in this paper
...They differ from each other by the programming paradigm they use, ranging from scene-graph-based interfaces, like Scene.js [31] and GLGE [32], to procedural paradigms, like SpiderGL [33] and WebGLU [34]....
[...]
...do consider “Finding Motifs in a Database of Shapes” [34]....
[...]
...In 2007, Xi et al.[34] proposed a method to find image motifs or the most similar pair of images in the image database....
[...]
...js [31] and GLGE [32], to procedural paradigms, like SpiderGL [33] and WebGLU [34]....
[...]
6,938 citations
5,046 citations
"From the Digitization of Cultural A..." refers background or methods in this paper
...West: A Monograph of the British Desmidiaceae [32], which is still referenced in modern scientific texts, and some of them are vanity publications by “gentlemen scholars”....
[...]
...They differ from each other by the programming paradigm they use, ranging from scene-graph-based interfaces, like Scene.js [31] and GLGE [32], to procedural paradigms, like SpiderGL [33] and WebGLU [34]....
[...]
...js [31] and GLGE [32], to procedural paradigms, like SpiderGL [33] and WebGLU [34]....
[...]
...They are typical examples from the perhaps hundreds of books on Diatoms published during the Victorian era [23][28][32]....
[...]
4,107 citations
"From the Digitization of Cultural A..." refers background in this paper
...Photometric stereo consists in computing high quality normal/range maps by taking several photographs from the same viewpoint but with different illumination directions [37]–[39], or by moving the object in front of a camera and a light source that are fixed with respect to each other [40], [41]....
[...]
Related Papers (5)
Frequently Asked Questions (19)
Q2. What have the authors stated for future works in "From the digitization of cultural artifacts to the web publishing of digital 3d collections: an automatic pipeline for knowledge sharing" ?
Other appealing directions of work could include the possibility to enrich the Web publishing phase, by auto- matically formatting a Web page based not only on the 3D model, but on other types of data, like text and images.
Q3. What is the way to store the geometry data?
Model data is saved under standard file formats: to store geometry information the authors use the Stanford polygon file format (PLY), which support multiple vertex attributes and binary encoding, while Portable Network Graphics (PNG) images are used for color and normal textures.
Q4. What is the correct way to describe the material properties of an object?
The most correct way to represent the material properties of an object is to describe them through a reflection function (e.g. BRDF), which attempts to model the observed scattering behavior of a class of real surfaces.
Q5. What is the effect of the projector illumination mask?
As expected, the projector illumination mask tends to increase the influence of image regions that correspond to the most illuminated surface parts, which leads to a conservation of luminosity.
Q6. Why are artifacts visible at the junctions between different images?
due to the lack of consistency from one image to another, artifacts are visible at the junctions between surface areas receiving color from different images.
Q7. How long does it take to recover the texture?
The diffuse color and detail textures recovery can take up to 5-10 minutes, while the data optimization for Web publishing is almost instantaneous.
Q8. What is the use of the interaction metaphor?
The interaction metaphor known as world-in-hand or trackball has been used to facilitate the artifact inspection by using the mouse.
Q9. How can the authors use a simple shadow mapping algorithm to estimate shadowed areas?
Since the complete geometric configura-tion of the scene is known, the authors can use a simple shadow mapping algorithm to estimate shadowed areas, to which a null weight is assigned.
Q10. How can the authors recover the occlusions of a video?
thanks to the fact that the authors know the light position for each video frame, it is possible to partially recover them by using a photometric stereo approach.
Q11. How many high quality 3D models can be made available every day?
Tens of high quality 3D models can be made available every day, for any kind of use (archival, study, presentation to the public).
Q12. What is the only light source in the scene?
During an acquisition, the only light source in the scene is the scanner projector itself, for which the position is always perfectly known.
Q13. What is the main reason for the lack of standardization of 3D content?
Beside the developers fragmentation that arises due to this wide variety of available tools and to their incompatibilities, the burden incumbent upon the user for the installation of additional software components prevented a wide adoption of online 3D content.
Q14. What other ways to make a Web page more appealing?
Other appealing directions of work could include the possibility to enrich the Web publishing phase, by auto-matically formatting a Web page based not only on the 3D model, but on other types of data, like text and images.
Q15. What is the common issue with the cited methods?
Another issue, which is common to all the cited methods, is the projection of lighting artifacts on the model, i.e. shadows, highlights, and peculiar BRDF effects, since the lighting environment is usually not known in advance.
Q16. What is the common method of avoiding lighting artifacts?
In order to correct (or to avoid to project) lighting artifacts, two possible approaches include the estimation of the lighting environment [23] or the use of easily controllable lighting setups [24].C. 3D graphics on the Web platformSince the birth of Internet, content of Web document has been characterized by several types of media, ranging from plain text to images, audio or video streams.
Q17. What is the advantage of the indexed triangle mesh?
It has the property to save a significant amount of space for the vast majority of 3D models, for which, on average, a vertex is referenced by six triangles.
Q18. What is the main advantage of the proposed approach?
Even though the proposed approach is generic enough to be used in any application for which producing and sharing digital content about real artifacts present an interest, its three main advantages (namely its ease of use, its high automation and its quickness) make it particularly appropriate to cases where huge collections have to be processed.
Q19. What is the advantage of the vertex cache?
To convey this advantage from memory occupancy to rendering performances, graphics accelerators have introduced a vertex cache capable of storing data associated to up to 32 vertices, thus allowing to reuse the results of a considerable amount of per-vertex calculations.