Limits...
Three-dimensional reconstruction of highly complex microscopic samples using scanning electron microscopy and optical flow estimation

View Article: PubMed Central - PubMed

ABSTRACT

Scanning Electron Microscope (SEM) as one of the major research and industrial equipment for imaging of micro-scale samples and surfaces has gained extensive attention from its emerge. However, the acquired micrographs still remain two-dimensional (2D). In the current work a novel and highly accurate approach is proposed to recover the hidden third-dimension by use of multi-view image acquisition of the microscopic samples combined with pre/post-processing steps including sparse feature-based stereo rectification, nonlocal-based optical flow estimation for dense matching and finally depth estimation. Employing the proposed approach, three-dimensional (3D) reconstructions of highly complex microscopic samples were achieved to facilitate the interpretation of topology and geometry of surface/shape attributes of the samples. As a byproduct of the proposed approach, high-definition 3D printed models of the samples can be generated as a tangible means of physical understanding. Extensive comparisons with the state-of-the-art reveal the strength and superiority of the proposed method in uncovering the details of the highly complex microscopic samples.

No MeSH data available.


3D red-cyan anaglyphs generated by combining the two rectified stereo views of the microscopic samples as well as the solid 3D models created using Meshmixer [70] for (a) Arabidopsis Anther 1, (b) Arabidopsis Anther 2, (c) Graphene, (d) Pseudoscorpion and (e) Fly Ash sample sets.
© Copyright Policy
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC5383242&req=5

pone.0175078.g008: 3D red-cyan anaglyphs generated by combining the two rectified stereo views of the microscopic samples as well as the solid 3D models created using Meshmixer [70] for (a) Arabidopsis Anther 1, (b) Arabidopsis Anther 2, (c) Graphene, (d) Pseudoscorpion and (e) Fly Ash sample sets.

Mentions: Having the height estimate for each point, a dense 3D point cloud can be generated and further used for 3D surface reconstruction. To eliminate the effects of smoothing introduced by general purpose mesh generating toolsets, similar to that of used in MeshLab [69], Delaunay triangulation is done by utilizing the image grid as the set of mesh nodes. The triangular mesh nodes are then transformed from the two dimensions of the image plane to the three dimensions of the model using the computed height estimates. This practically eliminates the smoothing effects which generally happen near the edges of the objects and in areas that contain sudden jumps due to sharp changes in the depth estimate. Using simple MATLAB scripts, the generated 3D surface can be transformed and saved as standard STL files which can be later used for further mesh modification and processing using more specialized software. Use of edge aware mesh smoothing procedures can be considered in order to have a more pleasing appearance without losing details of the edges and sudden changes of depth. Fig 8 shows 3D red-cyan anaglyphs generated by combining the two rectified stereo views of the microscopic samples as well as the solid 3D models created using Meshmixer [70]. The generated models can also be sent out for 3D printing as the ultimate means for creating a tangible representation of the complex microscopic structure [34, 36, 71, 72]. Fig 9 (a) shows one image from Fly Ash sample, while (b) shows another view of the 3D solid model created using the computed disparity estimates and modified using MeshMixer. Finally, (c) is an image captured from the 3D printed model.


Three-dimensional reconstruction of highly complex microscopic samples using scanning electron microscopy and optical flow estimation
3D red-cyan anaglyphs generated by combining the two rectified stereo views of the microscopic samples as well as the solid 3D models created using Meshmixer [70] for (a) Arabidopsis Anther 1, (b) Arabidopsis Anther 2, (c) Graphene, (d) Pseudoscorpion and (e) Fly Ash sample sets.
© Copyright Policy
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC5383242&req=5

pone.0175078.g008: 3D red-cyan anaglyphs generated by combining the two rectified stereo views of the microscopic samples as well as the solid 3D models created using Meshmixer [70] for (a) Arabidopsis Anther 1, (b) Arabidopsis Anther 2, (c) Graphene, (d) Pseudoscorpion and (e) Fly Ash sample sets.
Mentions: Having the height estimate for each point, a dense 3D point cloud can be generated and further used for 3D surface reconstruction. To eliminate the effects of smoothing introduced by general purpose mesh generating toolsets, similar to that of used in MeshLab [69], Delaunay triangulation is done by utilizing the image grid as the set of mesh nodes. The triangular mesh nodes are then transformed from the two dimensions of the image plane to the three dimensions of the model using the computed height estimates. This practically eliminates the smoothing effects which generally happen near the edges of the objects and in areas that contain sudden jumps due to sharp changes in the depth estimate. Using simple MATLAB scripts, the generated 3D surface can be transformed and saved as standard STL files which can be later used for further mesh modification and processing using more specialized software. Use of edge aware mesh smoothing procedures can be considered in order to have a more pleasing appearance without losing details of the edges and sudden changes of depth. Fig 8 shows 3D red-cyan anaglyphs generated by combining the two rectified stereo views of the microscopic samples as well as the solid 3D models created using Meshmixer [70]. The generated models can also be sent out for 3D printing as the ultimate means for creating a tangible representation of the complex microscopic structure [34, 36, 71, 72]. Fig 9 (a) shows one image from Fly Ash sample, while (b) shows another view of the 3D solid model created using the computed disparity estimates and modified using MeshMixer. Finally, (c) is an image captured from the 3D printed model.

View Article: PubMed Central - PubMed

ABSTRACT

Scanning Electron Microscope (SEM) as one of the major research and industrial equipment for imaging of micro-scale samples and surfaces has gained extensive attention from its emerge. However, the acquired micrographs still remain two-dimensional (2D). In the current work a novel and highly accurate approach is proposed to recover the hidden third-dimension by use of multi-view image acquisition of the microscopic samples combined with pre/post-processing steps including sparse feature-based stereo rectification, nonlocal-based optical flow estimation for dense matching and finally depth estimation. Employing the proposed approach, three-dimensional (3D) reconstructions of highly complex microscopic samples were achieved to facilitate the interpretation of topology and geometry of surface/shape attributes of the samples. As a byproduct of the proposed approach, high-definition 3D printed models of the samples can be generated as a tangible means of physical understanding. Extensive comparisons with the state-of-the-art reveal the strength and superiority of the proposed method in uncovering the details of the highly complex microscopic samples.

No MeSH data available.