Limits...
Multi-sensor fusion of infrared and electro-optic signals for high resolution night images.

Huang X, Netravali R, Man H, Lawrence V - Sensors (Basel) (2012)

Bottom Line: The framework requires four main steps: (1) inverse filter-based IR image transformation; (2) EO image edge detection; (3) registration; and (4) blending/superimposing of the obtained image pair.Simulation results show both blended and superimposed IR images, and demonstrate that blended IR images have better quality over the superimposed images.Additionally, based on the same steps, simulation result shows a blended IR image of better quality when only the original IR image is available.

View Article: PubMed Central - PubMed

Affiliation: Department of Electrical and Computer Engineering, Stevens Institute of Technology, Hoboken, NJ 07030, USA. xhuang3@stevens.edu

ABSTRACT
Electro-optic (EO) image sensors exhibit the properties of high resolution and low noise level at daytime, but they do not work in dark environments. Infrared (IR) image sensors exhibit poor resolution and cannot separate objects with similar temperature. Therefore, we propose a novel framework of IR image enhancement based on the information (e.g., edge) from EO images, which improves the resolution of IR images and helps us distinguish objects at night. Our framework superimposing/blending the edges of the EO image onto the corresponding transformed IR image improves their resolution. In this framework, we adopt the theoretical point spread function (PSF) proposed by Hardie et al. for the IR image, which has the modulation transfer function (MTF) of a uniform detector array and the incoherent optical transfer function (OTF) of diffraction-limited optics. In addition, we design an inverse filter for the proposed PSF and use it for the IR image transformation. The framework requires four main steps: (1) inverse filter-based IR image transformation; (2) EO image edge detection; (3) registration; and (4) blending/superimposing of the obtained image pair. Simulation results show both blended and superimposed IR images, and demonstrate that blended IR images have better quality over the superimposed images. Additionally, based on the same steps, simulation result shows a blended IR image of better quality when only the original IR image is available.

Show MeSH

Related in: MedlinePlus

Blended images generated with both EO and IR images. (a) Blended result of IR Image 1; (b) Blended result of IR Image 2.
© Copyright Policy
Related In: Results  -  Collection

License
getmorefigures.php?uid=PMC3472830&req=5

f6-sensors-12-10326: Blended images generated with both EO and IR images. (a) Blended result of IR Image 1; (b) Blended result of IR Image 2.

Mentions: During this process, we first blend the transformed IR image with the original IR image, and then we blend the obtained result with the edge-detected image. Here, we fix the blending fraction's values to be 0.8 and 0.01, respectively. Figure 6 shows two blended images that are generated with both EO and IR images. Based on the same steps, Figure 7 shows a blended image that is generated when only original IR image is available, i.e., with the edges extracted from the IR image. Seen from the obtained results, we can clearly see the seamless edge in the blended images and accurately distinguish objects at night via the superimposed edge, especially for objects with a similar temperature, and tiny parts of any object, which are difficult based on traditional object distinguishing approaches. However, in the intended applications, EO images and IR images are captured at different time, which may lead to problems in cases where the situation changes.


Multi-sensor fusion of infrared and electro-optic signals for high resolution night images.

Huang X, Netravali R, Man H, Lawrence V - Sensors (Basel) (2012)

Blended images generated with both EO and IR images. (a) Blended result of IR Image 1; (b) Blended result of IR Image 2.
© Copyright Policy
Related In: Results  -  Collection

License
Show All Figures
getmorefigures.php?uid=PMC3472830&req=5

f6-sensors-12-10326: Blended images generated with both EO and IR images. (a) Blended result of IR Image 1; (b) Blended result of IR Image 2.
Mentions: During this process, we first blend the transformed IR image with the original IR image, and then we blend the obtained result with the edge-detected image. Here, we fix the blending fraction's values to be 0.8 and 0.01, respectively. Figure 6 shows two blended images that are generated with both EO and IR images. Based on the same steps, Figure 7 shows a blended image that is generated when only original IR image is available, i.e., with the edges extracted from the IR image. Seen from the obtained results, we can clearly see the seamless edge in the blended images and accurately distinguish objects at night via the superimposed edge, especially for objects with a similar temperature, and tiny parts of any object, which are difficult based on traditional object distinguishing approaches. However, in the intended applications, EO images and IR images are captured at different time, which may lead to problems in cases where the situation changes.

Bottom Line: The framework requires four main steps: (1) inverse filter-based IR image transformation; (2) EO image edge detection; (3) registration; and (4) blending/superimposing of the obtained image pair.Simulation results show both blended and superimposed IR images, and demonstrate that blended IR images have better quality over the superimposed images.Additionally, based on the same steps, simulation result shows a blended IR image of better quality when only the original IR image is available.

View Article: PubMed Central - PubMed

Affiliation: Department of Electrical and Computer Engineering, Stevens Institute of Technology, Hoboken, NJ 07030, USA. xhuang3@stevens.edu

ABSTRACT
Electro-optic (EO) image sensors exhibit the properties of high resolution and low noise level at daytime, but they do not work in dark environments. Infrared (IR) image sensors exhibit poor resolution and cannot separate objects with similar temperature. Therefore, we propose a novel framework of IR image enhancement based on the information (e.g., edge) from EO images, which improves the resolution of IR images and helps us distinguish objects at night. Our framework superimposing/blending the edges of the EO image onto the corresponding transformed IR image improves their resolution. In this framework, we adopt the theoretical point spread function (PSF) proposed by Hardie et al. for the IR image, which has the modulation transfer function (MTF) of a uniform detector array and the incoherent optical transfer function (OTF) of diffraction-limited optics. In addition, we design an inverse filter for the proposed PSF and use it for the IR image transformation. The framework requires four main steps: (1) inverse filter-based IR image transformation; (2) EO image edge detection; (3) registration; and (4) blending/superimposing of the obtained image pair. Simulation results show both blended and superimposed IR images, and demonstrate that blended IR images have better quality over the superimposed images. Additionally, based on the same steps, simulation result shows a blended IR image of better quality when only the original IR image is available.

Show MeSH
Related in: MedlinePlus