Limits...
A Novel 2D-to-3D Video Conversion Method Using Time-Coherent Depth Maps.

Yin S, Dong H, Jiang G, Liu L, Wei S - Sensors (Basel) (2015)

Bottom Line: In this paper, we propose a novel 2D-to-3D video conversion method for 3D entertainment applications. 3D entertainment is getting more and more popular and can be found in many contexts, such as TV and home gaming equipment. 3D image sensors are a new method to produce stereoscopic video content conveniently and at a low cost, and can thus meet the urgent demand for 3D videos in the 3D entertaiment market.Global depth gradient is computed according to image type while local depth refinement is related to color information.The experimental results prove that this novel method can adapt to different image types, reduce computational complexity and improve the temporal smoothness of generated 3D video.

View Article: PubMed Central - PubMed

Affiliation: Institute of Microelectronics, Tsinghua University, Beijing 100084, China. yinsy@tsinghua.edu.cn.

ABSTRACT
In this paper, we propose a novel 2D-to-3D video conversion method for 3D entertainment applications. 3D entertainment is getting more and more popular and can be found in many contexts, such as TV and home gaming equipment. 3D image sensors are a new method to produce stereoscopic video content conveniently and at a low cost, and can thus meet the urgent demand for 3D videos in the 3D entertaiment market. Generally, 2D image sensor and 2D-to-3D conversion chip can compose a 3D image sensor. Our study presents a novel 2D-to-3D video conversion algorithm which can be adopted in a 3D image sensor. In our algorithm, a depth map is generated by combining global depth gradient and local depth refinement for each frame of 2D video input. Global depth gradient is computed according to image type while local depth refinement is related to color information. As input 2D video content consists of a number of video shots, the proposed algorithm reuses the global depth gradient of frames within the same video shot to generate time-coherent depth maps. The experimental results prove that this novel method can adapt to different image types, reduce computational complexity and improve the temporal smoothness of generated 3D video.

No MeSH data available.


Original 2D images (first column), generated depth maps of [16] (second column), [17] (third column), [18] (fourth column) and the proposed algorithm (fifith column).
© Copyright Policy
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC4541829&req=5

sensors-15-15246-f017: Original 2D images (first column), generated depth maps of [16] (second column), [17] (third column), [18] (fourth column) and the proposed algorithm (fifith column).

Mentions: To evaluate the proposed method, we also compare the proposed algorithm with other algorithms. Four video sequences, “Air”, “Arctic”, “Fashion” and “Cod” from [15], are tested. The algorithms [16,17,18] are adopted as references. Figure 17 shows the original 2D images and generated depth maps of these algorithms. We can find that the proposed algorithm has advantages in terms of accuracy of the generated depth maps. The algorithm as in [16] relies on the motion vector. If an object in the image does not have relative motion, the depth cannot be extracted correctly. The edge-based algorithm in [17] will be not accurate when the foreground object is large. The algorithm as [18] cannot adapt to different scene types compared to the proposed algorithm. Besides, the latter two algorithms use only a single image to generate depth maps and ignore the temporal coherence of depth maps between frames in the original 2D video, thus they may produce temporal flickering and redundant computation in the process.


A Novel 2D-to-3D Video Conversion Method Using Time-Coherent Depth Maps.

Yin S, Dong H, Jiang G, Liu L, Wei S - Sensors (Basel) (2015)

Original 2D images (first column), generated depth maps of [16] (second column), [17] (third column), [18] (fourth column) and the proposed algorithm (fifith column).
© Copyright Policy
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC4541829&req=5

sensors-15-15246-f017: Original 2D images (first column), generated depth maps of [16] (second column), [17] (third column), [18] (fourth column) and the proposed algorithm (fifith column).
Mentions: To evaluate the proposed method, we also compare the proposed algorithm with other algorithms. Four video sequences, “Air”, “Arctic”, “Fashion” and “Cod” from [15], are tested. The algorithms [16,17,18] are adopted as references. Figure 17 shows the original 2D images and generated depth maps of these algorithms. We can find that the proposed algorithm has advantages in terms of accuracy of the generated depth maps. The algorithm as in [16] relies on the motion vector. If an object in the image does not have relative motion, the depth cannot be extracted correctly. The edge-based algorithm in [17] will be not accurate when the foreground object is large. The algorithm as [18] cannot adapt to different scene types compared to the proposed algorithm. Besides, the latter two algorithms use only a single image to generate depth maps and ignore the temporal coherence of depth maps between frames in the original 2D video, thus they may produce temporal flickering and redundant computation in the process.

Bottom Line: In this paper, we propose a novel 2D-to-3D video conversion method for 3D entertainment applications. 3D entertainment is getting more and more popular and can be found in many contexts, such as TV and home gaming equipment. 3D image sensors are a new method to produce stereoscopic video content conveniently and at a low cost, and can thus meet the urgent demand for 3D videos in the 3D entertaiment market.Global depth gradient is computed according to image type while local depth refinement is related to color information.The experimental results prove that this novel method can adapt to different image types, reduce computational complexity and improve the temporal smoothness of generated 3D video.

View Article: PubMed Central - PubMed

Affiliation: Institute of Microelectronics, Tsinghua University, Beijing 100084, China. yinsy@tsinghua.edu.cn.

ABSTRACT
In this paper, we propose a novel 2D-to-3D video conversion method for 3D entertainment applications. 3D entertainment is getting more and more popular and can be found in many contexts, such as TV and home gaming equipment. 3D image sensors are a new method to produce stereoscopic video content conveniently and at a low cost, and can thus meet the urgent demand for 3D videos in the 3D entertaiment market. Generally, 2D image sensor and 2D-to-3D conversion chip can compose a 3D image sensor. Our study presents a novel 2D-to-3D video conversion algorithm which can be adopted in a 3D image sensor. In our algorithm, a depth map is generated by combining global depth gradient and local depth refinement for each frame of 2D video input. Global depth gradient is computed according to image type while local depth refinement is related to color information. As input 2D video content consists of a number of video shots, the proposed algorithm reuses the global depth gradient of frames within the same video shot to generate time-coherent depth maps. The experimental results prove that this novel method can adapt to different image types, reduce computational complexity and improve the temporal smoothness of generated 3D video.

No MeSH data available.