radioelectronicsИзвестия высших учебных заведений России. РадиоэлектроникаJournal of the Russian Universities. Radioelectronics1993-89852658-4794Saint Petersburg Electrotechnical Universityradioelectronics-46Research ArticleТЕЛЕВИДЕНИЕ И ОБРАБОТКА ИЗОБРАЖЕНИЙTELEVISION AND IMAGE PROCESSINGТелевизионная камера видимого и ближнего инфракрасного диапазоновTelevision camera of the visible and near-infrared rangesБарановП. С.BaranovP. S.tv.labs@yandex.ruЧиркуноваА. А.ChirkunovaA. A.aachirkunova@yandex.ruСанкт-Петербургский государственный электротехнический университет "ЛЭТИ" им. В. И. Ульянова (Ленина)РоссияSaint Petersburg Electrotechnical University "LETI"Russian Federation201528082015044756Copyright © Баранов П.С., Чиркунова А.А., 20152015Баранов П.С., Чиркунова А.А.Baranov P.S., Chirkunova A.A.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://re.eltech.ru/jour/article/view/46

Рассмотрены варианты построения мультиспектральной телевизионной камеры видимого и ближнего инфракрасного (RGB&NIR) диапазонов. Разработан макет одноматричной RGB&NIR-камеры. Проведен обзор наиболее перспективных областей применения компактных RGB&NIR-камер.

Variants of design of a multispectral TV camera of visible and near infrared (RGB&NIR) of ranges are considered. The test bench of single image sensor RGB&NIR-camera is developed. The review of the most perspective applications of com.pact RGB&NIR-camera is carried out.

Мультиспектральная камераструктура кодирующих светофильтровRGB&NIR-диапазонThe multispectral cameraRGB&NIR the rangehybrid color filter arrayReferencesGeneralized assorted pixel camera: postcapture control of resolution, dynamic range, and spectru.m / F. Yasuma, T. Mitsunaga, D. Iso, S. K. Nayar // IEEE Trans. on image processing. 2010. Vol. IP-19, № 9. P. 2241-2253.Generalized assorted pixel camera: postcapture control of resolution, dynamic range, and spectru.m / F. Yasuma, T. Mitsunaga, D. Iso, S. K. Nayar // IEEE Trans. on image processing. 2010. Vol. IP-19, № 9. P. 2241-2253.Durini D. High perfomance silicon imaging. Fundamentals and applications of CMOS and CCD image sensor. San Francisco: Woodhead Pub., 2014. 476 p.Durini D. High perfomance silicon imaging. Fundamentals and applications of CMOS and CCD image sensor. San Francisco: Woodhead Pub., 2014. 476 p.Eichenholz J., Dougherty J. Ultracom.pact fully integrated megapixel multispectral imager // Proc. SPIE: Integrated optics: devices, materials, and technologies XIII; ed. by J.-E. Broquin, Ch. M. Greiner. 2009. Vol. 7218. URL: http://www.pixelteq.com./wp-content /uploads/2013/01/PIXELTEQ_Ultracom.pact-Fully-Integrated-MP-MultiSpectral-Imager.pdf (дата обращения: 21.08.2015).Eichenholz J., Dougherty J. Ultracom.pact fully integrated megapixel multispectral imager // Proc. SPIE: Integrated optics: devices, materials, and technologies XIII; ed. by J.-E. Broquin, Ch. M. Greiner. 2009. Vol. 7218. URL: http://www.pixelteq.com./wp-content /uploads/2013/01/PIXELTEQ_Ultracom.pact-Fully-Integrated-MP-MultiSpectral-Imager.pdf (дата обращения: 21.08.2015).Real time megapixel multispectral bioimaging / J. Eichenholz, N. Barnett, Y. Juang et al. // Proc. SPIE: Imaging, manipulation, and analysis of biomolecules, cells, and tissues VIII. 2010. Vol. 7568. URL: http://www.pixelteq.com./wp-content/uploads/2013/01/PIXELTEQ_Real-time-Megapixel-Multispectral-Bioimaging.pdf (дата обращения: 21.08.2015).Real time megapixel multispectral bioimaging / J. Eichenholz, N. Barnett, Y. Juang et al. // Proc. SPIE: Imaging, manipulation, and analysis of biomolecules, cells, and tissues VIII. 2010. Vol. 7568. URL: http://www.pixelteq.com./wp-content/uploads/2013/01/PIXELTEQ_Real-time-Megapixel-Multispectral-Bioimaging.pdf (дата обращения: 21.08.2015).Hyperspectral and multispectral sensors for remote sensing / J. Miller, S. Kullar, D. Cochrane et al. // DALSA teledyne white paper // Proc. SPIE: Multispectral, hyperspectral, and ultraspectral remote sensing technology, techniques, and applications III. 2010. Vol. 7857. URL: http://www.academia.edu/5748427/Hyperspectral_and_multispectral _sensors_for_remote_sensing (дата обращения: 21.08.2015).Hyperspectral and multispectral sensors for remote sensing / J. Miller, S. Kullar, D. Cochrane et al. // DALSA teledyne white paper // Proc. SPIE: Multispectral, hyperspectral, and ultraspectral remote sensing technology, techniques, and applications III. 2010. Vol. 7857. URL: http://www.academia.edu/5748427/Hyperspectral_and_multispectral _sensors_for_remote_sensing (дата обращения: 21.08.2015).Тихонов И. А., Спиридонов И. Н. Инфракрасная визуализация кожных покровов // Биомедицинская радиоэлектроника. 2010. № 9. С. 26-32.Тихонов И. А., Спиридонов И. Н. Инфракрасная визуализация кожных покровов // Биомедицинская радиоэлектроника. 2010. № 9. С. 26-32.Rüfenacht D., Fredembach C., Süsstru.nk S. Automatic and accurate shadow detection using near-infrared in-formation // IEEE Trans. on pattern analysis and machine intelligence. 2014. Vol. PAMI-36, № 8. P. 1672-1678.Rüfenacht D., Fredembach C., Süsstru.nk S. Automatic and accurate shadow detection using near-infrared in-formation // IEEE Trans. on pattern analysis and machine intelligence. 2014. Vol. PAMI-36, № 8. P. 1672-1678.Zhang X., Sim T., Miao X. Enhancing photographs with near infra-red images // IEEE conf. on com.puter vision and pattern recognition (CVPR). 2008. URL: https://www.com.p.nus.edu.sg/~tsim/documents/visnir_camready_smaller.pdf (дата обращения: 21.08.2015).Zhang X., Sim T., Miao X. Enhancing photographs with near infra-red images // IEEE conf. on com.puter vision and pattern recognition (CVPR). 2008. URL: https://www.com.p.nus.edu.sg/~tsim/documents/visnir_camready_smaller.pdf (дата обращения: 21.08.2015).Firmenich D., Brown M. Süsstru.nk S. Multispectral interest points for RGB-NIR image registration // Proc. IEEE int. conf. on image processing (ICIP). 11-14 Sept. 2011, Bru.ssels. URL: http://infoscience.epfl.ch/record/167479/files/FBS11.pdf (дата обращения 21.08.2015).Firmenich D., Brown M. Süsstru.nk S. Multispectral interest points for RGB-NIR image registration // Proc. IEEE int. conf. on image processing (ICIP). 11-14 Sept. 2011, Bru.ssels. URL: http://infoscience.epfl.ch/record/167479/files/FBS11.pdf (дата обращения 21.08.2015).Near-infrared guided color image dehazing / C. Feng, S. Zhuo, X. Zhang et al. // Proc. IEEE 20th int. conf. image processing (ICIP). 15-18 Sept. 2013. URL: http://infoscience.epfl.ch/record/188639/files/poster_cip_2013.pdf (дата обращения: 21.08.2015).Near-infrared guided color image dehazing / C. Feng, S. Zhuo, X. Zhang et al. // Proc. IEEE 20th int. conf. image processing (ICIP). 15-18 Sept. 2013. URL: http://infoscience.epfl.ch/record/188639/files/poster_cip_2013.pdf (дата обращения: 21.08.2015).High-contrast subcutaneous vein detection and localization using multispectral imaging / F. Wang, A. Behrooz, M. Morris, A. Adibi // J. biomed. opt. 2013. Vol. 18, № 5. URL: http://www.ece.gatech.edu/research/photonics/Papers/2013/High-contrast%20subcutaneous%20vein%20detection%20and%20localization%20using%20multispectral%20imaging.pdf (дата обращения: 21.08.2015).High-contrast subcutaneous vein detection and localization using multispectral imaging / F. Wang, A. Behrooz, M. Morris, A. Adibi // J. biomed. opt. 2013. Vol. 18, № 5. URL: http://www.ece.gatech.edu/research/photonics/Papers/2013/High-contrast%20subcutaneous%20vein%20detection%20and%20localization%20using%20multispectral%20imaging.pdf (дата обращения: 21.08.2015).Status of the CMOS Image Sensor Industry 2014 // Report Yole Dévelopement. URL: http://www.yole.fr/iso_upload/Events/PR_Status%20of%20the%20CMOS%20Image%20Sensors%20Industry_YOLE%20DEVELOPPEMENT_January%202014.pdf (дата обращения: 21.08.2015).Status of the CMOS Image Sensor Industry 2014 // Report Yole Dévelopement. URL: http://www.yole.fr/iso_upload/Events/PR_Status%20of%20the%20CMOS%20Image%20Sensors%20Industry_YOLE%20DEVELOPPEMENT_January%202014.pdf (дата обращения: 21.08.2015).Huang H., Liu L., Ngadi M. O. Recent developments in hyperspectral imaging for assessment of food quality and safety // Sensors. 2014. Vol. 14. P. 7248-7276.Huang H., Liu L., Ngadi M. O. Recent developments in hyperspectral imaging for assessment of food quality and safety // Sensors. 2014. Vol. 14. P. 7248-7276.Wu D., Da-Wen Sun. Advanced applications of hyperspectral imaging technology for food quality and safety analysis and assessment: A review. Pt. II: Applications // Innovative Food Science & Emerging Technologies. 2013. Vol. 19. P. 15-28.Wu D., Da-Wen Sun. Advanced applications of hyperspectral imaging technology for food quality and safety analysis and assessment: A review. Pt. II: Applications // Innovative Food Science & Emerging Technologies. 2013. Vol. 19. P. 15-28.Wu D., Da-Wen Sun. Advanced applications of hyperspectral imaging technology for food quality and safety analysis and assessment: A review. Pt. I: Fundamentals // Innovative Food Science & Emerging Technologies. 2013. Vol. 19. P. 1-14.Wu D., Da-Wen Sun. Advanced applications of hyperspectral imaging technology for food quality and safety analysis and assessment: A review. Pt. I: Fundamentals // Innovative Food Science & Emerging Technologies. 2013. Vol. 19. P. 1-14.Multispectral camera. URL: http://www.pixelteq.com./products/spectrocamMultispectral camera. URL: http://www.pixelteq.com./products/spectrocamKing-Hong Chung; Yuk-Hee Chan. Low-com.plexity color demosaicing algorithm based on integrated gradients // J. electron. imaging. 2010. Vol. 19, iss. 2. URL: http://www.eie.polyu.edu.hk/~enyhchan/J-JEI-Low_com.plexity_color_demosaicing_algorithm_based_on_IG.pdf (дата обращения: 21.08.2015).King-Hong Chung; Yuk-Hee Chan. Low-com.plexity color demosaicing algorithm based on integrated gradients // J. electron. imaging. 2010. Vol. 19, iss. 2. URL: http://www.eie.polyu.edu.hk/~enyhchan/J-JEI-Low_com.plexity_color_demosaicing_algorithm_based_on_IG.pdf (дата обращения: 21.08.2015).Prism based 2CCD multi-spectral. URL: http://www.jai.com./en/products /ad-080ge (дата обращения: 21.08.2015).Prism based 2CCD multi-spectral. URL: http://www.jai.com./en/products /ad-080ge (дата обращения: 21.08.2015).Multispectral cameras. URL: http://www.fluxdata.com./multispectral-cameras (дата обращения: 21.08.2015).Multispectral cameras. URL: http://www.fluxdata.com./multispectral-cameras (дата обращения: 21.08.2015).Near infrared (NIR) cameras. URL: http://www.jai.com./en/products/nearinfrared (дата обращения: 21.08.2015).Near infrared (NIR) cameras. URL: http://www.jai.com./en/products/nearinfrared (дата обращения: 21.08.2015).Imec hyperspectral line-scan imager. URL: http://www2.imec.be/content/user/File/Brochures/2015/3_flyer_HSI_Linescan%20sensor.pdf (дата обращения: 21.08.2015).Imec hyperspectral line-scan imager. URL: http://www2.imec.be/content/user/File/Brochures/2015/3_flyer_HSI_Linescan%20sensor.pdf (дата обращения: 21.08.2015).Imec hyperspectral snapshot tiled imager. URL: http://www2.imec.be/content/user/File/Brochures/2015/4_flyer_HSI_Snapshot%20tiled%20sensor.pdf (дата обращения: 21.08.2015).Imec hyperspectral snapshot tiled imager. URL: http://www2.imec.be/content/user/File/Brochures/2015/4_flyer_HSI_Snapshot%20tiled%20sensor.pdf (дата обращения: 21.08.2015).Hyperspectral cameras with USB3 vision. URL: http://www2.imec.be/content/user/File/Brochures/2015/3_flyer_HSI_Linescan%20sensor.pdf (дата обращения: 21.08.2015).Hyperspectral cameras with USB3 vision. URL: http://www2.imec.be/content/user/File/Brochures/2015/3_flyer_HSI_Linescan%20sensor.pdf (дата обращения: 21.08.2015).Imec hyperspectral snapshot tiled imager. URL: http://www2.imec.be/content/user/File/Brochures/2015/4_flyer_HSI_Snapshot%20tiled%20sensor.pdf (дата обращения: 21.08.2015).Imec hyperspectral snapshot tiled imager. URL: http://www2.imec.be/content/user/File/Brochures/2015/4_flyer_HSI_Snapshot%20tiled%20sensor.pdf (дата обращения: 21.08.2015).OV4682 RGB IR, color CMOS 4-megapixel. URL: http://www.ovt.com./products/sensor.php?id=145 (дата обращения: 21.08.2015).OV4682 RGB IR, color CMOS 4-megapixel. URL: http://www.ovt.com./products/sensor.php?id=145 (дата обращения: 21.08.2015).Imec hyperspectral mosaic snapshot imager. URL: http://www2.imec.be/content/user/File/Brochures/2015/IMEC%20HYPERSPECTRAL%20SNAPSHOT%20MOSAIC%20IMAGER%2020150421.pdf (дата обращения: 21.08.2015).Imec hyperspectral mosaic snapshot imager. URL: http://www2.imec.be/content/user/File/Brochures/2015/IMEC%20HYPERSPECTRAL%20SNAPSHOT%20MOSAIC%20IMAGER%2020150421.pdf (дата обращения: 21.08.2015).Designing color filter arrays for the joint capture of visible and near-infrared images / Y. M. Lu, C. Fredembach, M. Vetterli, S. Susstru.nk // Proc. of IEEE int. conf. on image processing (ICIP). 7-10 Nov. 2009, Cairo, Egypt. URL: http://infoscience.epfl.ch/record/142066/files /05414324.pdf?version=1 (дата обращения: 21.08.2015).Designing color filter arrays for the joint capture of visible and near-infrared images / Y. M. Lu, C. Fredembach, M. Vetterli, S. Susstru.nk // Proc. of IEEE int. conf. on image processing (ICIP). 7-10 Nov. 2009, Cairo, Egypt. URL: http://infoscience.epfl.ch/record/142066/files /05414324.pdf?version=1 (дата обращения: 21.08.2015).Sadeghipoor Z., Lu Y. M., Susstru.nk S. Correlation-based joint acquisition and demosaicing of visible and near-infrared images // Proc. of IEEE int. conf. on image processing (ICIP). 11-14 Sept. 2011, Bru.ssels. URL: http://infoscience.epfl.ch/record/168897/files/SadeghipoorICIP11.pdf (дата обращения: 21.08.2015).Sadeghipoor Z., Lu Y. M., Susstru.nk S. Correlation-based joint acquisition and demosaicing of visible and near-infrared images // Proc. of IEEE int. conf. on image processing (ICIP). 11-14 Sept. 2011, Bru.ssels. URL: http://infoscience.epfl.ch/record/168897/files/SadeghipoorICIP11.pdf (дата обращения: 21.08.2015).Simultaneous capturing of RGB and additional band images using hybrid color filter array // D. Kiku, Y. Monno, M.Tanaka, M. Okutomi // Digital photography X // Proc. SPIE. 2014. Vol. 9023. URL: http://www.ok.ctrl.titech.ac.jp/res/MSI/Hybrid/Hybrid.pdf (дата обращения 21.08.2015).Simultaneous capturing of RGB and additional band images using hybrid color filter array // D. Kiku, Y. Monno, M.Tanaka, M. Okutomi // Digital photography X // Proc. SPIE. 2014. Vol. 9023. URL: http://www.ok.ctrl.titech.ac.jp/res/MSI/Hybrid/Hybrid.pdf (дата обращения 21.08.2015).RGB-NIR scene dataset. URL: http://ivrl.epfl.ch/supplementary_material /cvpr11 (дата обращения 21.08.2015).RGB-NIR scene dataset. URL: http://ivrl.epfl.ch/supplementary_material /cvpr11 (дата обращения 21.08.2015).

The authors declare that there are no conflicts of interest present.