UAV-based Smart Detection Systems Utilizing Artificial Intelligence and Machine Learning

UF/IFAS Precision Agriculture Engineering Program at SWFREC

By Dr. Yiannis Ampatzidis, August 8, 2018

The UF/IFAS ABE Precision Agriculture Engineering program at the Southwest Florida Research and Education Center explores unmanned aerial vehicles (UAV) for agriculture and natural systems, smart sensors and machinery, mechatronics, robotic, artificial intelligence (AI) and machine learning (ML), machine vision, automation, remote sensing, wireless sensor network, and big data applications. Housed at the SWFREC, this program is led by ABE Assistant Professor Dr. Yiannis Ampatzidis

Dr. Ampatzidis works in the area of mechanization and automation of specialty crop production. He focuses on the design, development and testing of sensors and control systems for optimal management of inputs, resources and products.

Dr. Ampatzidis’ current research includes the development of UAV-based smart detection systems utilizing AI and ML. These systems can be used to detect, count and categorize (based on canopy size) individual plants (e.g. citrus trees) using RGB cameras (Fig. 1) and Lidar sensors (Figs. 2 & 3). Additionally, an UAV-based system was developed to early detect, identify and distinguish diseases (e.g. Citrus Black Spot – CBS) and/or disorders using hyperspectral sensors (Figs. 4 & 5).

We have demonstrated that transfer learning (AI approach) can be leveraged when it is not possible to collect thousands of images to train the AI system. Transfer learning is the re-use of a trained neural network to a new problem. For example, transfer learning was used to detect mature (mechanical and robotic harvest project, Fig. 6) and immature (early yield prediction project, Fig. 7) fruit.

Furthermore, several multispectral (e.g. MicaSence RedEdge-M, Sequoia etc.) and thermal (e.g. FLIR Vue Pro R) cameras were evaluated to develop Vegetation Indices (VI) and assess plant health status (Fig. 8).

For more information, follow the Precision Agriculture Engineering program on FacebookTwitter (@PrecAgSWFREC) and subscribe to its channel on YouTube for updates about our program.

Figure 1. Real-Time citrus tree detection using Artificial Intelligence and Machine Learning (a real-time object detection system) on an NVidia Jetson TX2.

Figure 2. DJI Matrice 600 Pro equipped with a Velodyne Lidar system.

Figure 3. Maps developed by a UAV-based Velodyne Lidar system to detect, count and categorize trees.

Figure 4. DJI Matrice 600 Pro equipped with a Resonon hyperspectral camera for early disease detection.

Figure 5. Red marks indicate potential Citrus Black Spot infected citrus trees (generated and detected by a UAV-based hyperspectral camera and AI).

Figure 6. Real-time mature citrus fruit detection using YOLO (a real-time AI object detection system) on an NVidia Jetson TX2 board. These results are achieved by using just 150 pictures to train the AI-based system.

Figure 7. Real-time immature fruit detection using YOLO on an NVidia Jetson TX2 board: a) stitched picture; b) zoom-in.

Figure 8. Vegetation Indices developed to assess citrus trees health status.