Development of an Environmental Monitoring System Based on Spatial Marking and Machine Vision Technologies
https://doi.org/10.32603/1993-8985-2023-26-4-56-69
Abstract
Introduction. The use of available satellite images and aerial photography by unmanned aerial vehicles (UAVs) in the tasks of environmental monitoring is challenged by the imperfection of existing tools. Geographic information systems are characterized by insufficient flexibility to automatically work with heterogeneous sources. The latest models based on artificial intelligence in ecology require preliminary data preparation. The article presents the results of designing a software system for environmental monitoring based on machine vision sensor data, which provides data unification while being flexible both in terms of data sources and methods of their analysis.
Aim. Creation of a generalized software system for coordinated spatial marking of heterogeneous machine vision data for environmental monitoring tasks.
Materials and methods. Software engineering methods, database theory methods, spatial markup methods, image processing methods.
Results. A generalized method for unifying data was developed. The method is based on the analysis of existing open data from remote sensing of the Earth, as well as UAV aerial photography and approaches to environmental monitoring. To implement the method, a flexible architecture of the software system was designed, and a data model for a document-oriented DBMS was developed, which allows storing data and scaling the data analysis procedure.
Conclusion. The existing sources of data and tools for environmental monitoring were analyzed. A generalized method for unifying machine vision data, an architecture, and a data model was created. The method, architecture, and model were successfully implemented as a software system with a web interface
Keywords
spatial marking,
machine vision,
remote sensing of the Earth,
aerial photography,
automation of environmental monitoring
Supporting agencies: The work was carried out with the financial support of the Russian Science Foundation (grant no. 22-76-10042).
About the Authors
M. M. Zaslavskiy
Saint Petersburg Electrotechnical University
Russian Federation
Russian Federation
Mark M. Zaslavskiy – Cand. Sci. (2019), Deputy Head of the Department of Software Engineering and Computer Applications
5 F, Professor Popov St., St Petersburg 197022
The author of more than 15 scientific publications. Area of expertise: spatial markup; artificial intelligence; machine vision; audio processing; automation of evaluation of educational works.
K. E. Kryzhanovskiy
Saint Petersburg Electrotechnical University; Yandex LLC
Russian Federation
Russian Federation
Kirill E. Kryzhanovskiy – Master in "Software engineering"; Junior Software developer of Yandex.
Yandex LLC, 16, Lev Tolstoy St., Moscow 119021
Area of expertise: artificial intelligence; computer vision; orientation and navigation algorithms; unmanned aerial vehicles.
D. V. Ivanov
Saint Petersburg Electrotechnical University
Russian Federation
Russian Federation
Dmitry V. Ivanov – Postgraduate student in "Computer Science and Informatics", assistant of the Department of Software Engineering and Computer Applications
5 F, Professor Popov St., St Petersburg 197022
Author of 3 scientific publications. Area of expertise: drone swarms; spatial markup; artificial intelligence; machine vision.
References
1. Corwin D. L. Climate Change Impacts on Soil Salinity in Agricultural Areas. European J. of Soil Science. 2021, vol. 72, iss. 2, pp. 842–862. doi: 10.1111/ejss.13010
2. Yalew S. G., van Vliet M. T. H., Gernaat D. E. H. J., Ludwig F., Miara A., Park C., Byers E., De Cian E., Piontek F., Iyer G., Mouratiadou I., Glynn J., Hejazi M., Dessens O., Rochedo P., Pietzcker R., Schaeffer R., Fujimori S., Dasgupta S., Mima S., Santos da Silva S. R., Chaturvedi V., Vautard R., van Vuuren D. P. Impacts of Climate Change on Energy Systems in Global and Regional Scenarios. Nature Energy. 2020, vol. 5, no. 10, pp. 794–802. doi: 10.1038/s41560-020-0664-z
3. Streletskiy D. A., Suter L. J., Shiklomanov N. I., Porfiriev B. N., Eliseev D. O. Assessment of Climate Change Impacts on Buildings, Structures and Infrastructure in the Russian Regions on Permafrost. Environmental Research Letters. 2019, vol. 14, no. 2, p. 025003. doi: 10.1088/1748-9326/aaf5e6
4. Dainelli R., Toscano P., Gennaro S. F. Di, Matese A. Recent Advances in Unmanned Aerial Vehicles Forest Remote Sensing – A Systematic Review. Part II: Research Applications. Forests. 2021, vol. 12, iss. 4, p. 397. doi: 10.3390/f12040397
5. Ahmad A., Gilani H., Ahmad S. R. Forest Aboveground Biomass Estimation and Mapping Through High-Resolution Optical Satellite Imagery – A Literature Review. Forests. 2021, vol. 12, iss. 7, p. 914. doi: 10.3390/f12070914
6. McClure E. C., Sievers M., Brown C. J., Buelow C. A., Ditria E. M., Hayes M. A., Pearson R. M., Tulloch V. J. D., Unsworth R. K. F., Connolly R. M. Artificial Intelligence Meets Citizen Science to Supercharge Ecological Monitoring. Patterns. 2020, vol. 1, iss. 7, p. 100109. doi: 10.1016/j.patter.2020.100109
7. Soubry I., Doan T., Chu T., Guo X. A Systematic Review on the Integration of Remote Sensing and Gis to Forest and Grassland Ecosystem Health Attributes, Indicators, and Measures. Remote Sensing. 2021, vol. 13, iss. 16, p. 3262. doi: 10.3390/rs13163262
8. Toth C., Jóźków G. Remote Sensing Platforms and Sensors: A Survey. ISPRS J. of Photogrammetry and Remote Sensing. 2016, vol. 115, pp. 22–36. doi: 10.1016/J.ISPRSJPRS.2015.10.004
9. Makridenko L. A., Volkov S. N., Gorbunov A. V., Salihov R. S., Hodnenko V. P. The First Russian Next Generation High Resolution Earth Remote Sensing Small Satellite Canopus-V No. 1. Voprosy jelektromehaniki. Trudy VNIIJeM [Questions of electromechanics. Proceedings of VNIIEM]. 2017, vol. 156, no. 1, pp. 10–20. (In Russ.)
10. Kirilin A. N., Akhmetov R. N., Anshakov G. P., Storozh A. D., Stratilatov N. R., Tipukhov V. A. Space System of Remote Sensing of the Earth "Resource-P". XL Akademicheskie chtenija po kosmonavtike [XL Academic Readings in Astronautics]. Moscow, 26–29 January 2016, p. 350. (In Russ.)
11. Grigoryev A. A., Baranov M. E. Maintenance of the Software Models of the Spacecraft Communication "Express-AM". Current Problems of Aviation and Cosmonautics. 2018, vol. 2, no. 14, pp. 507–509. (In Russ.)
12. Lokshin B. Express-RV as a Forward-Looking Communications System with Satellites in Highly Elliptical Orbits. Communication Technologies & Equipment. 2019, no. S1, pp. 62–71. (In Russ.)
13. Irons J. R., Dwyer J. L., Barsi J. A. The Next Landsat Satellite: The Landsat Data Continuity Mission. Remote Sensing of Environment. 2012, vol. 122, pp. 11–21. doi: 10.1016/j.rse.2011.08.026
14. Drusch M., Del Bello U., Carlier S., Colin O., Fernandez V., Gascon F., Hoersch B., Isola C., Laberinti P., Martimort P., Meygret A., Spoto F., Sy O., Marchese F., Bargellini P. Sentinel-2: ESA's Optical HighResolution Mission for GMES Operational Services. Remote Sensing of Environment. 2012, vol. 120, pp. 25–36. doi: 10.1016/j.rse.2011.11.026
15. Gao F., Hilker T., Zhu X., Anderson M., Masek J., Wang P., Yang Y. Fusing Landsat and MODIS Data for Vegetation Monitoring. IEEE Geoscience and Remote Sensing Magazine. 2015, vol. 3, iss. 3, pp. 47– 60. doi: 10.1109/MGRS.2015.2434351
16. Roy D. P., Wulder M. A., Loveland T. R. et al. Landsat-8: Science and Product Vision for Terrestrial Global Change Research. Remote Sensing of Environment. 2014, vol. 145, pp. 154–172. doi: 10.1016/j.rse.2014.02.001
17. Donlon C., Berruti B., Buongiorno A., Ferreira M.-H., Féménias P., Frerick J., Goryl P., Klein U., Laur H., Mavrocordatos C., Nieke J., Rebhan H., Seitz B., Stroede J., Sciarra R.The Global Monitoring for Environment and Security (GMES) Sentinel-3 Mission. Remote Sensing of Environment. 2012, vol. 120, pp. 37–57. doi: 10.1016/j.rse.2011.07.024
18. Cao C., Xiong J., Blonski S., Liu Q., Uprety S., Shao X., Bai Y., Weng F. Suomi NPP VIIRS Sensor Data Record Verification, Validation, and Long‐Term Performance Monitoring. J. of Geophysical Research: Atmospheres. 2013, vol. 118, iss. 20, pp. 11664–11678. doi: 10.1002/2013jd020418
19. Morgan J. L., Gergel S. E., Coops N. C. Aerial Photography: a Rapidly Evolving Tool for Ecological Management. BioScience. 2010, vol. 60, no. 1, pp. 47– 59. doi: 10.1525/bio.2010.60.1.9
20. Zhang Y. J. Camera Calibration. 3-D Computer Vision: Principles, Algorithms and Applications. Singapore, Springer Nature Singapore, 2023, pp. 37–65. doi: 10.1007/978-981-19-7580-6_2
21. Hein G. W. Status, Perspectives and Trends of Satellite Navigation. Satellite Navigation. 2020, vol. 1,
22. Petritoli E., Leccese F., Leccisi M. Inertial Navigation Systems for UAV: Uncertainty and Error Measurements. 2019 IEEE 5th Intern. Workshop on Metrology for AeroSpace (MetroAeroSpace). Turin, Italy. 19– 21 June 2019. IEEE, 2019, pp. 1–5. doi: 10.1109/MetroAeroSpace.2019.8869618
23. Tredennick A. T., Hooker G., Ellner S. P., Adler P. B. A Practical Guide to Selecting Models for Exploration, Inference, and Prediction in Ecology. Ecology. 2021, vol. 102, iss. 6, p. e03336. doi: 10.1002/ecy.3336
24. Stupariu M.-S., Cushman S. A., Pleşoianu A.-I., Pătru-Stupariu I., Fürst C. Machine Learning in Landscape Ecological Analysis: A Review of Recent Approaches. Landscape Ecology. 2022, vol. 37, iss. 5, pp. 1227–1250. doi: 10.1007/s10980-021-01366-9
25. Soubry I., Doan T., Chu T., Guo X. A Systematic Review on the Integration of Remote Sensing and Gis to Forest and Grassland Ecosystem Health Attributes, Indicators, and Measures. Remote Sensing. 2021, vol. 13, iss. 16, p. 3262. doi: 10.3390/rs13163262
26. Paramasivam C. R. Merits and Demerits of GIS and Geostatistical Techniques. GIS and Geostatistical Techniques for Groundwater Science. 2019, pp. 17–21. doi: 10.1016/B978-0-12-815413-7.00002-X
27. Ekeanyanwu C. V., Obisakin I. F., Aduwenye P., Dede-Bamfo N. Merging GIS and Machine Learning Techniques: A Paper Review. J. of Geoscience and Environment Protection. 2022, vol. 10, no. 9, pp. 61–83. doi: 10.4236/gep.2022.109004
28. Wong R. F., Rollins C. M., Minter C. F. Recent Updates to the WGS 84 Reference Frame. Proc. of the 25th Intern. Technical Meeting of the Satellite Division of the Institute of Navigation (ION GNSS 2012). Nashville, TN. 17–21 September 2012, pp. 1164–1172.
29. Verma R., Ali J. A Comparative Study of Various Types of Image Noise and Efficient Noise Removal Techniques. Intern. J. of Advanced Research in Computer Science and Software Engineering. 2013, vol. 3, iss. 10, pp. 617–622.
30. Ochotorena C. N., Yamashita Y. Anisotropic Guided Filtering. IEEE Transactions on Image Processing. 2019, vol. 29, pp. 1397–1412. doi: 10.1109/TIP.2019.2941326
31. Hassan M. A., Yang M., Rasheed A., Yang G., Reynolds M., Xia X., Xiao Y., He Z. A Rapid Monitoring of NDVI across the Wheat Growth Cycle for Grain Yield Prediction Using a Multi-Spectral UAV Platform. Plant science. 2019, vol. 282, pp. 95–103. doi: 10.1016/j.plantsci.2018.10.022
32. Copernicus Open Access Hub. Available at: https://scihub.copernicus.eu (accessed 10.07.2023).
33. Earthdata. Available at: https://www.earthdata.nasa.gov (accessed 10.07.2023).
34. Jaramillo D., Nguyen D. V., Smart R. Leveraging Microservices Architecture by Using Docker Technology. SoutheastCon 2016. IEEE, 2016, pp. 1–5. doi: 10.1109/SECON.2016.7506647
35. Requests: HTTP for Humans™. Available at: https://requests.readthedocs.io/ (accessed 10.07.2023).
36. Qin C. Z., Zhan L. J., Zhu A. X. How to Apply the Geospatial Data Abstraction Library (GDAL) Properly to Parallel Geospatial Raster I/O? Transactions in GIS. 2014, vol. 18, iss. 6, pp. 950–957. doi: 10.1111/tgis.12068
37. pallets/flask: The Python micro framework for building web applications. GitHub. Available at: https://github.com/pallets/flask (accessed 10.07.2023).
38. Jose B., Abraham S. Exploring the Merits of Nosql: A Study Based on Mongodb. 2017 Intern. Conf. on Networks & Advances in Computational Technologies (NetACT). Thiruvananthapuram, India. 20–22 July 2017. IEEE, 2017, pp. 266–271. doi: 10.1109/NETACT.2017.8076778
Review
For citations:
Zaslavskiy M.M., Kryzhanovskiy K.E., Ivanov D.V. Development of an Environmental Monitoring System Based on Spatial Marking and Machine Vision Technologies. Journal of the Russian Universities. Radioelectronics. 2023;26(4):56-69. (In Russ.) https://doi.org/10.32603/1993-8985-2023-26-4-56-69
Views:
322
Email this article 