$$\rightleftharpoonup{xx}$$
$$\longleftharp{xx}$$,
$$\longrightharp{xx}$$,
The following protocol illustrates the process of paddy leaf disease detection and treatment recommendation system building and deployment step by step. The steps taken need to be used consecutively to replicate the findings presented in this paper.
Dataset preparation
Paddy leaves stage dataset: This data was based on PaddyNet4. The scene comprises 560 images as classified under four different development stages, namely: Stage 2, Stage 3, Stage 4 and Stage 5. The classification depends on the color and maturity of the paddy leaves, which depend on the concentration of nitrogen, among other factors, and the development of the plant. The presented dataset is important in knowing the right stage of the paddy plant (Figure 1). PaddyNet is accessible publicly via the website of data.mendeley.com/datasets/. Paddy leaves disease dataset: This dataset was available as several separate publicly accessible datasets on Kaggle and combined to create one unified dataset with a total of 6,920 images, each of which belonged to one of six disease types shown on (Figure 2). Treatment dataset: The dataset was critically developed by retrieving and integrating information of various publicly available datasets and contains data on various management practices of paddy diseases at four growth phases21. The main objective of this dataset is to come up with recommendations to prevent and control the paddy disease. The data is one of the resources that are vital towards fighting the diseases in the paddy plants by selecting the right kind of pesticides19,22,23,24,25
Data augmentation
The data augmentation was used to enhance the training dataset and reduce overfitting. The augmentation methods that have been employed in the current study are rotation (up to 30), width and height moving (up to 20%), zooming (up to 20%), shearing, and horizontal flipping. The use of these augmentation methods assists the model in acquiring the essentials of the model and enhances the generalization of the model when it is tested on unseen data. Data augmentation was done on the training dataset, whereas the validation and test datasets were left as they were in order to review the fair performance of the system. The size of the original dataset had to be augmented with data because the size of the original dataset is rather small, and the approach was needed to enhance the generalization and minimize overfitting by introducing the model to the differences in orientation, scale, and position of leaf images.
Image preprocessing
The stage and disease data collected were passed through a number of preprocessing steps to make the data consistent and enhance the performance of the models. All the pictures from both datasets were scaled to 128 × 128 × 3 to ensure a workable fit in any deep learning architecture. The pixel values were scaled to the interval [0,1] so as to hasten convergence when training. Moreover, labels of diseases and their stages were transformed into numbers to evaluate and train the model more powerfully. These preprocessing methods helped to extract features better, a better classification rate, and the robustness of the model to different environmental conditions. The size of the image used, 128 × 128, was compromised between computation and feature loss. Concrete Smaller image size minimizes training time and computational cost, and does not eliminate information (visual features) to classify the disease and stage when utilizing deep learning models.
Dataset splitting
The processed data were divided into three subsets (Table 1): Training Set (80%), Validation Set (10%), Test Set (10%). To avoid an unbalanced distribution of classes in all three subsets, stratified random sampling was used to ensure that both the stage dataset (560 images) and disease dataset (6,920 images) had an equal distribution of classes. There was no data leakage among the subsets.
Model training
In the classification of the stage of the growth of paddy leaves and diseases, two steps model training method was used.
Stage classification model
The classification method of leaf growth stage has four classes and a relatively low data size, thus the architecture of the Convolutional Neural Network (NN) was not very expanded but considered lightweight. A simpler architecture can be used to provide reduced over-fitting and computational power, at the expense of having high accuracy in the case of classification using color and texture features. The new CNN structure will have three convolutional layers using the ReLU activation functions. The convolutional layers are alternated with the max-pooling layers (2×2) to downscale the spatial dimensions, as well as to identify the prominent features. The feature maps undergo a convolutional layer, after which they are flattened, and the fully connected dense layer with dropout regularization is run to discourage overfitting. Lastly, multi-class classification of the four growth stages is done with the help of a softmax output layer. The Adam optimizer, which has a learning rate of 0.001 and categorical cross-entropy as a loss function, was used to train the model. The number of the batch was set to 10, and the model was trained in 10 epochs. Validation accuracy and validation loss were also used to monitor the performance of the model since they are used to make sure that the model is generalized.
Disease classification model based on ensemble learning
For disease classification, a series of deep learning architectures was adopted to improve prediction accuracy. Initial development of a custom CNN model consisted of four layers, namely, three convolutional layers, three max-pooling layers, a dense fully connected layer, and dropout regularization. MobileNetV2 employs depth-wise separable convolutions, inverted residual blocks, and inverted bottleneck with linear residue. The model has a small size of only 3.4 million parameters and can be used in a resource-constrained environment. The models started using ImageNet activations, and the convolutional layers were frozen so that the learned feature representations are not lost. The softmax classification layer was attached to a fully connected layer that had dropout regularization. All the models were trained during 20 epochs with a batch size of 32, and the optimizer and loss were identical to those of the CNN model. In an attempt to improve on classification, the average voting ensemble learning method was used. The results were the average probabilities predicted by all five models (CNN, VGG16, ResNet50, InceptionV3, and MobileNetV2) to obtain the final class predictions. This combination technique was useful in lessening bias in models and enhancing generalization. The last combination model was more accurate in classifications. The reason why the ensemble approach was selected is that learning of the various feature representations is learned using the same data set by the different deep learning models. The ensemble also decreases model bias and variance and increases the overall prediction issues and overall generalization performance as compared to independent models.
Prediction strategy
In the process of predicting paddy leaf disease and its associated stage of growth, the ensemble-based classification method was utilized. The five deep learning models (CNN, VGG16, ResNet50, InceptionV3, and MobileNetV2) were independently trained to classify the diseases and pooled together by utilizing an average voting ensemble method. This procedure entailed the forecasting of probabilities of each disease category of all models and the average probability distribution, which adds precision to predictability and minimizes bias in the model. The final prediction was taken as the disease class with the most avg probability. Further, a different CNN-based stage classification model was trained and used to forecast the growth-stage of the paddy leaf. Both classification models would take in input images of size 128x 128 then normalize the pixel values and then use the models to make an inference. The final predictions of both leaf stage and disease were then mapped onto an existing set of treatment recommendations to offer valid treatment in order to manage the disease.
Example Output: "Leaf stage: 3, Disease: Tungro"
Treatment recommendation
To make the paddy leaf disease classification more practical, a treatment recommendation system was integrated into the application based on Streamlit. The system takes the forecasted disease and leaf stage to offer specific treatment recommendations to make sure that the disease is managed successfully. The model of ensemble disease classification (including CNN, VGG16, ResNet50, InceptionV3, and MobileNetV2) classifies the disease, whereas the model of classifying the leaf stage is another model. The modeled stage of the leaf and disease are compared with the data of treatment that has recommended doses (Preventive measures and Application of Pesticide).
Methodological framework and contribution
The methodology of the presented system is a pipeline of multi-stages, which combines image classification and decision support. The framework consists of five large steps, namely, promotion of images and augmentation, predicting leaf growth stage, using deep learning models trained on the images to classify the disease, combining the prediction results to obtain an ensemble decision, and recommending the treatment based on the type and growth stage of an illness. The methodological contribution of this work is geared at the implementation of many prediction models and a treatment recommendation module into one decision support pipeline. The system is based not just on the classification of diseases but also on incorporating the information of the stage of plant growth so as to provide treatment recommendations at a particular stage. The system is more relevant to the process of agricultural decision-making in the real world because of this stage-conscious recommendation strategy. The input image will first undergo preprocessing before being fed through the stage classification model and disease classification models. The disease model results are averaged using an ensemble technique, and the disease prediction and growth stage are carried to the end to obtain correct treatment recommendations based on the identification of relevant treatment records in the treatment database.
Validation strategy and overfitting prevention
In order to provide stringent validation of the developed models, the data set was split into three disjunctive subsets: training set (80%), validation set (10%), and test set (10%). The models were trained using the training set, tuned using the validation set, and evaluated their final performance using the test set only. The test set remained absolutely independent to have an impartial analysis of the model. The following methods were used to guarantee that the model does not overfit and enhance its ability to generalize. To increase the level of diversity in the dataset, first, data augmentation methods (rotation, zooming, shifting, shearing, and horizontal flipping) were used. Second, dropout regularization was used in the fully connected layers of the CNN models to reduce overfitting. Third, pretrained models (VGG16, ResNet50, InceptionV3, and MobileNetV2) were transferred using the frozen convolutional layers technique to preserve the learned features and minimize the chances of overfitting caused by the small size of the dataset. Lastly, validation loss and accuracy were used to monitor model performance during training to be able to ensure that the model is not overfitting to the training data. These are the validation and regularization schemes that ensure that the model proposed generalizes very well to unseen data and offers good performance.
Proposed system
The suggested one is a deep learning solution to the paddy leaf disease management, which incorporates the leaf stage forecasting and classification of the disease. It starts with the preprocessing of paddy leaf images in terms of resizing them to a common height and the pixel values. The leaf stage of growth is predicted using a neural network, and an average voting technique is used to use an ensemble of deep learning models (CNN, VGG16, ResNet50, InceptionV3, MobileNetV2) to classify diseases. The system uses the forecasted stage and disease to prescribe the right doses of herbicides or pesticides to be used in the treatment. The workflow of the proposed system is shown in (Figure 3). The previous subsections (A through I) explain each of the individual elements of this pipeline, that is why more description is not done here. The Streamlit-based application developed in this study is a prototype decision support tool intended to demonstrate the practical applicability of the proposed model. The system allows users to upload paddy leaf images and receive disease predictions, growth stage information, and treatment recommendations. Large-scale usability testing and field deployment with farmers will be conducted as part of future work to evaluate the system’s real-world effectiveness and usability.