Agronomic crops have distinct and uniform morphology, but for roadside invasive species, the high variation in plant morphology and the non-uniform backgrounds will lead to more significant detection errors. Previous studies and discussions supported that computer vision can replace human observers in species detection. However, we still need human observers to create a training dataset. Training, testing, and validation datasets are the essential components of a deep learning model, with the most time-consuming task being image labeling. A larger training dataset can increase model performance, but the size would determine the amount of labor for a single project. Abdulsalam & Aouf suggested that 1,000 images of a particular species are required to achieve high prediction accuracy. Yan & Ryu proposed that the training sizes would differ depending on the mapping species since they only used 400 training samples for corn, but the model could still perform with high accuracy. Compared to the traditional car survey, the AIbased survey method can be conducted by non-experts once the image detection model is trained. Johnsongrass is a common weed along the roads in the United States and is a good model plant for the AI-based survey method. It is native to the Mediterranean and North Africa and was introduced to the United States in the early 1800s . Johnsongrass is a perennial grass and can colonize and spread nearby landscapes through the underground rhizome system . The mature plant can grow up to 2.5 meters in height, and the height of the mature plant can vary based on the local condition . McWhorter reported that the reproductive stage of johnsongrass started around a month after seed emergence, cannabis drying system and the maximum rhizome growth was about 60 meters in 5 months. The flowering part of johnsongrass is a diffuse panicle, which is the primary feature in the image identification process.
The flowering head is orange and purple at the mature stage. Johnsongrass is a weedy relative of the cultivated sorghum , which compete for the same limiting resources, and the presence of johnsongrass will cause yield loss in sorghum or other crop fields . Kansas and Texas are the top two states in cultivated sorghum production regarding planting acreage . As a weedy relative, johnsongrass is widespread in sorghum fields and along the roads around Nebraska, Kansas, and Texas to conduct an automated road survey of johnsongrass, examined the cost-effectiveness, and discussed the potential application of the johnsongrass population map.The performance of the YOLOv2 model in detecting johnsongrass in GSV images was tested using a total of 2,040 test images. Based on the threshold value of 0.6 for presence , the confusion matrix shown in Table 1 was created. The YOLOv2 model achieved a recall value of 85% in the GSV testing dataset. Dang et al. reported similar recall values in their study which the average recall value of the YOLOv3 detection model on 12 different weed species was 87.93%. However, there is still about a 15% chance that johnsongrass could be undetected by our model when johnsongrass is present in the image. In the testing dataset, there were 153 images were classified as FN in the confusion matrix. FN in this project is the image that contained johnsongrass and was identified and labeled by human observers, but the model was not able to output the same result. Individuals that were at pre-flowering stage were not considered as FN since the training dataset only contained mature johnsongrass. The precision for the YOLOv2 model was 0.74, which is lower than the recall . Both the precision and FPR include false positive detection in the calculation. FPR implies that the model could wrongly detect other plant species as johnsongrass, with a 30% chance. Among the group of incorrect detection , FP had twice the number of images compared to FN in the test dataset . Sincemost studies in weed detection were conducted in the crop field, and their models were applied to distinguish weeds from the crop, the precision values were high and were about 85% to 95% . Yan & Ryu applied a CNN model on GSV images to detect roadside crop type, and the results denoted that most crops had detection precision above 90%, but only rice had a 76% precision. The study also reported that the misclassification of rice was more frequent in low-resolution images or if the object was far away from the camera . In our research, the quality of images and the distance between the target object and the camera might contribute to the high FP value. Future work will focus on decreasing the rate of FP in the image where johnsongrass is absent. More CNN models will be tested on the johnsongrass training database to compare the precision and accuracy of different models. More roads with high-resolution images from Google can help improve this survey method’s accuracy. The overall accuracy of the YOLOv2 model was 77.5% for detecting johnsongrass in the GSV images. This index provided an overall evaluation based on total correct detection and the total number of test images. Ringland et al. and Yan & Ryu both conducted image detection models on GSV images to survey different types of crop production along the roads but with different CNN networks from our model. The accuracy of detecting general crops like alfalfa , almond , corn , and rice in the GSV images could reach 92% . An explanation of high accuracy on crops is that major crops always have unique morphology or patterns because of domestication, row spacing, and field layout that might help to increase performance in computer vision. For roadside weedy species like johnsongrass, morphological variations under different environmental conditions were reported in many studies, and the variation could lead to low precision and overall accuracy . There were several challenges in the labeling process, and they can explain most of the incorrect detections. In some annotated training images, the target species were partially occluded by other objects, including other invasive species, traffic signs, and fences. In this case, we could only label either the flowering part or the basal part of johnsongrass. In this project, and for johnsongrass specifically, the panicle part of the plant would be labeled in most cases since we could not differentiate the basal part of johnsongrass from other grass species. The growing stages of the target species were another challenge in the labeling process. The juvenile stage of johnsongrass has no panicles and looks similar to many other grass species. Only the mature johnsongrasses were included in the training dataset, so the model was unlikely to detect individuals at their early vegetative stage.The trained model was applied to 269,489 images collected from Google Street View. In Figure 6, the red points denote the potential location of johnsongrass predicted by the model. The model identified a total of 2,031 images as having johnsongrass. The predicted distribution of johnsongrass suggested that johnsongrass is less widespread in Nevada than in the other states in this study. The location shown on this map is only the prediction. The johnsongrass individual might not be found in that location depending on the growing season since most images were taken 2 to 3 years ago. Deus et al. conducted a Google Street View study that surveyed E. globulus , and their results mentioned that environmental stresses could lead to variability in species abundance in a short period, from one to two years. Recent studies and our results suggested that integrating GSV and a deep learning image detection model can map species on a much larger scale. Yan & Ryu integrated GSV and other deep learning algorithms and produced cropping system maps of Central Valley in California and the state of Illinois. Another roadside crop survey in Thailand covered 572 km of road and examined about 57,000 panoramas . Our study covered more areas , longer roads , growing tray and more panoramas than studies used a similar road survey method . Future research will focus on the survey in other states in the US, and our goal is to survey all the roads in the US.YOLO has been tested in many studies to detect multiple plant species in a single image . Johnsongrass was the only detection target in this study, but other invasive species can be mapped by using our methods. A larger-scale species distribution map can be combined with environmental factors or land use to determine the conditions suitable for spreading the species. An example would be the habitat suitability model, which predicts how well species thrive and spread in a location given environmental conditions . According to Crall et al. , even though the habitat suitability model is a key tool for invasive species risk management, the model requires location data on a large spatial scale. Our method can provide a more prominent presence/absence dataset than the traditional local dataset. Habitat suitability models based on a larger scale can yield a more robust conclusion. As noted, the sampling created by our method is a biased sampling of the environment under which the species can thrive as we only search for species along the roadside habitats. AI-based surveys can provide accurate location data to build and test invasive species dispersal models. The AI-based mapping approach can only detect roadside weedy species. A dispersal model can be applied based on the johnsongrass location map. A typical dispersal model requires two primary parameters, reproduction rate and spread distance, and then for parameterization and calibration, a multiple-time-step map is required . The johnsongrass map was created based on images from a different date. Even though most of the images were taken in recent years, like 2020, a small portion was taken 8 or 10 years ago. Expenses and estimated time for the car survey, the human-based GSV survey, and the AI-based GSV survey were calculated . Expenses for the car survey were calculated based on the same scale as the AI-based survey. The cost breakdown of the car survey was calculated based on regular domestic travel daily expenses. Vehicle rentals and gas estimation are US$ 9,408 and US$ 7,350, respectively, and the accommodation accounts for a more significant portion of the costs, i.e., US$ 25,200 . The total travel time of a car survey per person requires 180 days, estimated based on a daily 750 km drive. Dues et al. conducted a 38-day car survey of 15,000 km of roads in Portugal, and a standard car survey would drive much less than 750 km per day. For the human- and AI-based GSV survey, US$1,890 is required for image purchase from Google, which is US$7 per 1000 images . Labor costs for all three types of surveys were calculated based on the minimum hourly wage in California and the total time spent on each method. In terms of cost, the AI-based GSV survey is 50% less than the human-based GSV survey, and the AI-based GSV survey is only 5.6% of the total cost of the car survey.Compared to car surveys, GSV-based surveys did not require outdoor work and driving, so GSV-based detection can minimize potential worker risks . Studies by Deus et al. and Kotowska et al. reported that the results produced by GSV resemble that produced by the field survey. Compared to the GSV survey by a human observer, the AI-based GSV survey had spent shorter time. Once the detection model is trained, the machine can work 24 hours per day, but on average, a human can process about 6000 ~ 8000 images per day based on our labeling experiences. A more detailed GSV survey might take more time. For example, a human-based GSV survey took 35 hours to examine 2,350 panoramas in Sicily, Italy, to assess invasive species abundance along the roads . As noted, once the training dataset is created, the labor cost of the AI-based GSV surveys is fixed and will not increase as the number of images increases. However, the relation between labor cost and image number in the human-based surveys is linear. As the sampling scales increase, AI-based surveys will outcompete human-based surveys, and the comparison between these two surveys in Table 3 is underestimated. Then in terms of errors, the detection errors by the AI-based model could be consistent, quantified, and improved, while errors in car surveys and humanbased GSV surveys are unable to quantify and inconsistent.