CROP AND WEED DETECTION USING IMAGE PROCESSING AND DEEP LEARNING TECHNIQUES

Thesis Overview

<p> <b>1. Introduction&nbsp;</b></p><p>One of the newest and most researched technologies nowadays is deep learning. Deep learning is a technique used to create intelligent systems as similar as possible to human brains. It has made a big impact in all types of domains such as video, audio and image processing (Wason, 2018; Sharma, 2019). On the other hand, agriculture is humanity’s oldest and most essential activity for survival. The growth of population during the last years has led to a higher demand of agricultural products. To meet this demand without draining the environmental resources the agriculture uses, automation is being introduced into this field (Mehta, 2016). The present project aims to merge both concepts by achieving autonomous weed recognition in agriculture; this goal will be reached by using new technologies such as Matlab, FarmBot and Python programming, image processing, deep learning and Artificial Neural Networks (ANNs). These concepts will be explained in more detail throughout this document. This thesis will be developed for Naturbruksförvaltningen, a farming school in Töreboda, Sweden.</p><p> <b>1.1 Background&nbsp;</b></p><p>Robotics and automation have become an emerging subject nowadays; substituting and aiding humans in manual tasks that can become not only tedious and repetitive, but also difficult due to different factors such as precision. In order to go in depth on this technology deep learning has been implemented with the purpose of giving these systems intelligence, making them capable of learning. Examples can be found everywhere, from industries to humankind’s daily life. One of these examples is agriculture, where automation has found solution to some of the challenges faced by farmers on a daily basis such as crop diseases infestations, pesticide control, weed management, lack of irrigation and drainage facilities and lack of storage management (Jha, et al., 2019). As a way to bring this new technology to urban orchards, FarmBot Inc. was created. It is a local startup that is working within advanced precision agriculture through automation and open source technology (Brown, et al., 2017). FarmBot Inc. has developed a series of robots, called FarmBots, to take care of these orchards in an autonomous way while respecting the environment. Naturbruksförvaltningen Sötåsen aims to teach its students how to combine agriculture and technology. To do so, they intend to introduce a FarmBot into their studies and go a step further, not only programming it to do the basic agricultural tasks, but also by including deep learning to make the system capable of differencing on its own whether there are weeds on the orchard or not.&nbsp;</p><p> <b>1.2 Problem description&nbsp;</b></p><p>These last years the combination of automation and computer vision has been introduced into agriculture to reduce human workload. The FarmBot used in this project is one example of that combination. Its functions range from the automation of basic agricultural activities such as watering or seeding, to more advanced and complex tasks such as differencing between crops and weeds. This weed detection system is the focus of this project. It is programmed to take pictures of the crop and process them by a manually activated weed-detection software application from FarmBot where the processing is done based on the colours and location of the elements of the picture. This weed detector is the starting point of this thesis. Why does the weed detector have to be improved? Even if this system seems to be failproof, it is not. There are three main issues that can be considered: firstly, having to manually activate the weed detector application does not reduce the amount of human labour as much as intended. Secondly, basing the detection on colours is not accurate due to the possibility of a change of lighting or the similarity of colours between weed and plants, among other things. Finally, basing the existence of a weed on the location where the FarmBot has previously planted a seed, does not consider a situation where the FarmBot does not necessarily know where all the seeds are located. As a way to solve these issues, this thesis will implement a weed detector software based on deep learning which will be explained in Section 1.3. &nbsp;</p><p> <b>1.3 Aim and objectives</b></p><p>&nbsp;The aim of this project is to implement a different type of weed detection system than the explained in Section 1.2, one that makes use of an ANN to differentiate between crop and weed. In order to achieve this, some objectives need to be set:&nbsp;</p><p>1. Image capture using FarmBot&nbsp;</p><p>2. Image pre-processing with Matlab&nbsp;</p><p>3. ANN training using Matlab&nbsp;</p><p>4. ANN testing&nbsp;</p><p>5. Use the previous pictures to return weed coordinates&nbsp;</p><p>6. Compare ANN performance between the one used by FarmBot and the one used in this project.</p><p>&nbsp;The Matlab system will implement the pre-processing of images and the training of an ANN in order to have a system able to learn from the already processed images and do that processing autonomously. To take those pictures, the FarmBot will be programmed to capture them every certain&nbsp; time and forward those images so that Matlab can retrieve them. The performance evaluation of the different ANNs will be done in order to determine which one is better, if the one originally used by the robot or the one developed in this thesis.&nbsp;</p><p>To achieve these objectives, the robot will be programmed using the User Interface (UI) of FarmBot and the ANN will be trained in Matlab using images retrieved with Python through the FarmBot REST API (Application Programming Interface). The techniques to be used will be computer vision to work with the camera, image processing and deep learning for pattern recognition and ANN training.</p><p> The development of the project will be done considering the following points:&nbsp;</p><p>• Research on the FarmBot programming environment and weed detector&nbsp;</p><p>• Research on computer vision and image processing&nbsp;</p><p>• Research on deep learning and its implementation in Matlab&nbsp;</p><p>• Develop the code to take the pictures on FarmBot&nbsp;</p><p>• Develop the Matlab code with the ANN trained&nbsp;</p><p>• Evaluation and comparison of the ANNs&nbsp;</p><p>To summarize, the project will be considered complete when the neural network achieves an accuracy of more than the 50% when identifying both crop and weeds in an image captured by the FarmBot. Those crops and weeds used for the project are spinach, dandelions and cleavers. &nbsp; <br></p>