A Two-Stage Self-Supervised Learning Framework for Winter Crop-Weed Image Classification

Summary

This work assesses the application of a two-stage deep learning pipeline for the self-supervised pretraining and supervised fine tuning of winter crop and weed image classification. The experiments on the WinterCropWeedDB dataset are conducted using a single internal split, with Grad-CAM visualizations included.

Abstract

Precision agriculture requires accurate discrimination between winter crops and weeds, but there is a lack of annotated image data for winter cropping systems. This paper investigates a two-stage deep learning approach that integrates self-supervised feature learning with supervised fine-tuning for winter crop and weed image classification. A new winter crop and weed image dataset, WinterCropWeedDB, is proposed and used in this paper, which contains 1,136 high-resolution images of six winter crop species and four weed species collected from agricultural fields in central India. In the first stage of self-supervised learning, an EfficientNet-B3 model is pre-trained using a SimCLR-style self-supervised learning approach with an InfoNCE loss function (temperature τ = 0.5) on the images. The average contrastive loss value reduces from 2.0712 in the first iteration to 1.6835 at the end of pretraining. In the second stage of supervised fine-tuning, the pre-trained EfficientNet-B3 model is fine-tuned with a supervised classifier head on the images and tested on a single internal validation split (30%) of the dataset. The fine-tuned model reaches a maximum validation accuracy of 98.27%, with a macro-averaged F1-score of 0.98. Gradient-weighted class activation mapping (Grad-CAM) and Grad-CAM++ are used on the fine-tuned model to provide a qualitative visualization of the image regions that contribute to class predictions. The experiment outcomes demonstrate the viability of using self-supervised pretraining and supervised fine-tuning for the classification of winter crop and weed images on a region-specific dataset, while also emphasizing the importance of additional testing on independent test sets.

Introduction

Precision agriculture has seen the increasing use of deep learning-based computer vision methods for automated crop and weed classification¹. Convolutional neural networks have been shown to be effective in plant recognition tasks; however, their performance is usually dependent on the availability of large datasets that are annotated in a precise manner². In most agricultural settings, especially in underrepresented cropping systems like winter crops, the process of obtaining extensive labeled image data is time-consuming, laborious, and expensive^3,4. This is a ....

Protocol

Dataset description

The WinterCropWeedDB dataset consists of 1,136 high-resolution (red, green, blue) RGB images of six winter crop species (wheat, chickpea, pea, lentil, mustard, and grass pea) and four weed species (common vetch, lesser canary grass, goosefoot (Chenopodium album), and Euphorbia clementei) taken from winter agricultural fields in the state of Chhattisgarh, India (Figure 1). The images were taken under natural conditions, including varying illumination, growth stages, and background complexity. All images were initially unannotated and used only for self-super....

Results

Overview of prior approaches and workflow positioning

A summary of representative learning approaches applied previously for agricultural crop-weed recognition is given in Table 1. The table presents an overview of supervised, semi-supervised, and self-supervised learning strategies, datasets used, reported results, and the limitations presented. Most earlier works focus on summer cropping systems or object detection tasks and are evaluated on relatively large.......

Discussion

This work examined the use of a two-step deep learning approach consisting of self-supervised pretraining using the SimCLR idea and supervised fine-tuning for winter crop and weed image classification on the WinterCropWeedDB dataset. The results show that the proposed approach can be trained reliably on a region-specific winter agriculture image dataset and tested on one split of the internal validation set. This section presents an analysis of the results, implications, and limitations of this work.

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
CUDA Toolkit	NVIDIA	12.8
cuDNN	NVIDIA	9.1
Graphics processing unit (GPU)	NVIDIA	GeForce RTX 5050 Laptop GPU
Matplotlib	Matplotlib Developers	3.9.2
NumPy	NumPy Developers	1.26.0
Operating system	Microsoft	Linux (WSL2), kernel 6.6.87
Python	Python Software Foundation	3.12.7
PyTorch	PyTorch Foundation	2.1.0 (development build)
scikit-learn	scikit-learn Developers	1.5.1
timm	GitHub repository	1.0.24
Torchvision	PyTorch Foundation	0.25.0 (development build)
WinterCropWeedDB	Mendeley Data, DOI: 10.17632/m4h6zdsh79.1	Version 1