1.

INTRODUCTION

In many developing countries, food security and self-sufficiency have been declared as one of their national goals. Among many challenges to support such goals, food phase classification task is fundamentally potential to help the estimation of yield before harvest. Food crops, such as rice has been extensively cultivated to feed the world growing population. In Indonesia, with a population of more than 260 million people depending on rice as the staple food, rice self-sufficiency are also an important goal. Plant phase monitoring has been conducted by BPS-Statistics Indonesia based on conventional field surveys, namely Kerangka Sample Area (KSA)¹. In 2017, KSA firstly introduced the rice plants phase monitoring data. The field survey data of KSA is built on the powerful statistical method for sampling and estimation. Despite its high quality estimation, there are at least two major disadvantages of KSA field survey (1) cost-sensitive, as it requires the increase of sample size for larger area estimation which would depend on more human and capital resources, (2) lack of consideration to individual variations in non-sample areas, given that the estimation is merely based on the selected samples.

On the other hand, current development in remote sensing data and technology provides the potential of a more scalable, inexpensive and real-time benefit. Over the past decade, a huge amount of worldwide coverage remote sensing images has been publicly accessible. In particular, Landsat 8 satellite provides high-resolution multi-spectral datasets which include rich information of agricultural vegetation growth which freely available. This enable us to analyze the vegetation growth from the past to the present and then predict changes over time^2,3,4.

In this paper, we focus on the classification of rice plant phase in Banyuwangi Regency, Indonesia as our case study. In order to select the finest multitemporal features, we suggest a supervised random forest technique to create features engineering. Based on the comprehensive evaluations, our contributions in this paper lays on two major points. First, we provide insight to capture the temporal autocorrelation to improve the accuracy of the model⁵. Second, we propose a machine learning-based classification model that is not susceptible to multicollinearity. Machine learning based method has gained powerful achievement in many supervised and unsupervised task^6,7.

2.

METHODOLOGY

2.1

Rice phenologycal and spectral pattern

Rice has several growing phases in one planting season, namely land preparation (before planting), early vegetative, late vegetative, early generative (reproductive), final generative (ripening), and harvesting. During land preparation, paddy fields are flooded with the lots of water. Then in the initial vegetative phase, the puddle decreases and the greenness of the rice begins to be planted. The greenness of rice plant will increase in the final vegetative phase, then the yellow panicles begin to grow when entering the generative phase. During the harvest phase, the land becomes dry, brown and open^8,9.

Figure 1.

Phenology Rice Plant Phase

The venology of rice plant phase causes a spectral pattern of vegetation indexes from remote sensing in one planting period. The greenness of the plants is illustrated by the NDVI index and EVI, the level of standing water is indicated by NDWI, whereas open land is obtained from NDBI.

During the land preparation phase, the EVI value is still low and NDWI is high, then the value of EVI starts to rise in the early vegetative phase and NDWI starts to fall. In the final vegetative phase, the EVI value rises steeply and reaches the maximum EVI peak, then the value drops again in the initial generative and final generative phases to harvest. At harvest time NDWI is very low and NDBI peaks occur as shown in Figure 2. Figure 2 is obtained from the extraction of certain samples.

Figure 2.

Examples of Spectral Patterns EVI, NDVI, NDWI, NDBI From 21 Landsat Periods in 2018. (a) Lowland Rice Plants, (b) Non-Rice In Paddy Fields, (c) Forests (Non Paddy Fields), and (d) Houses ( Non Paddy Fields)

In other types of food plants, this spectral pattern can be different. For example in corn or soybeans, variations in the value of NDWI will not be as high as rice plants. In non-paddy fields such as houses or forests, the values of EVI, NDVI,

NDWI, and NDBI tend to be stable. The different spectral patterns when puddles occur, the greenness begin to high, and the land become dry associated with each stage of rice plant phase, allows features of Landsat 8 imagery can be used to distinguish between rice and non-rice classes and rice plant phase classes.

2.4

Classification Approach

Rice is usually planted in the paddy fields both irrigated, non-irrigated, or rainfed paddy fields. The paddy fields in one year are usually planted 1 or 2 times rice plants, sometimes 3 times, interspersed with other food crops if 1 or 2 times. Rice is also possible to be planted in the moor with three assumptions, which are planted with an upland system, land use conversion because the moorings are near the paddy fields, and there is a narrow paddy field mixed with moor so that it is not mapped as a small-scale paddy field (1: 100000) and recorded as a moor grid.

The rice planting methods that allow the same location to be planted alternately with other food crops make two important criteria that the rice plant phase classification model must have are (1) the ability to distinguish between types of rice plants and other plants and (2) the ability to distinguish rice plant phases in one planting period based on rice phenology. These criteria require the classes of rice plant phase model should consist of the rice plant phase, non-rice in paddy fields, and non-paddy fields. An extensive experiment will be conducted to investigate (1) features that distinguish rice and non-rice classes, (2) features that distinguish the rice plant phase class. The classification stages are detailed as follows, as shown in Figure 3.

Figure 3.

Stages of Rice Plant Phase Classification

• Step 1: Download Landsat-8 from November 2017 until December 2018 from google earth engine (GEE). Landsat 8 is downloaded with certain cloud cover criteria and has been done radiomatric and geometric corrections. Extract band 1 untul band 7 from the multitemporal Landsat-8 at the coordinates of the KSA sample and add the KSA observations as label data. Calculating vegetation index features, vegetation index variants, maximum and minimum EVI. Extracting and manipulating data is using R software

• Step 2: Explorating vegetation index patterns and vegetation index variants in 4 periods of each class, whether the vegetation index pattern according to theory of rice phenology in one planting period

• Step 3: Splitting data into training and testing data. making a classification model using training data

• Step 4: Evaluating prediction accuracy using testing data

• Step 5: Explorating feature importance

3.

RESULT

We plot EVI, NDVI, NDBI, and NDWI from 4 period in each class to check whether the vegetation index pattern is according to phenology rice plant phase in one planting period or is not. The Landsat-8 period is 16 days and the KSA period is 1 month, so there can be many combination of different classes between periods. For example when the period t is a early generative class, observations can occur in the early generative range of the end, middle, or beginning. If the observations are in the middle generative range, then the t-1 period is also the early generative class, the t-2 period is the late vegetative class, and the t-3 is also the late class. If the observations are in the beginning generative range, then the t-1 period is the late vegetative, the t-2 is thelate vegetative, and the t-2 is the early vegetative. This can cause different temporal spectral patterns in the same class, so the data becomes non-linear and machine learning is needed to automatically determine class boundaries for the classification process. The overall temporal spectral pattern of each class can be seen in Figure 2

As shown in Figure 4, in the initial vegetative class, the NDWI value in period t is quite high while EVI, NDVI, and NDBI are still low. In this early vegetative class, the period t-1 is likely to be land preparation, period t-2 and t-3 is harvest. It is seen that the NDBI value at t-3 is the highest compared to other periods, the EVI, NDVI, and NDWI values are also the lowest. This plot in the early vegetative class showed a data pattern in accordance with the phenology theory of the rice plant phase.

Figure 4.

Spectral Pattern NDVI, EVI, NDBI, NDWI From 4 Period In Each Class

Variation in vegetation index from 4 periods is quite high in the rice plant phase class and relatively stable for non rice in rice fields rather class and non rice fields class. From this plot it is estimated that the variance value of the vegetation index will be a differentiator of the rice plant phase class with non-paddy in rice fields class and non rice fields.

In the rice plant phase, it seems that the value of the vegetation index between classes looks different. For example, in the late vegetative class, the value of EVI and NDVI is higher than early vegetative, but the NDWI and NDBI values are lower. It is estimated that in the rice plant phase class, the value of the vegetation index will be more differentiating between classes than the vegetation index varian from 4 period. Furthermore, the classification model is carried out using 17 features, that is band 1 to 7 period t band, vegetation index (EVI, NDVI, NDBI, NDWI) period t, vegetation index variants, maximum EVI, and minimum EVI. Accuracy and misclassification of random forest models can be seen in Table 2.

Table 2.

Accuracy of Classification Model

Random forest is an ensemble machine learning that is not sensitive to multicollinearity because each tree is produced using only part of the randomly selected features ¹⁴. Therefore we can classify using several combinations of features to see the effect of a combination of features on the accuracy model and misclassification. For example when classification is only using EVI features, the model accuracy of 0.236. Misclassification occurs between rice and non-rice classes or between rice phase classes. When the classification was using EVI, NDVI, NDWI, and NDBI, the model accuracy increased to 0.491. Misclassification decreases especially in the non-rice fields class which is classified as non-rice in rice fields and the early vegetative class which is classified as another rice plant phase.

The accuracy of random forest model if use one temporal fitur (band 1 until band 7 and vegetation index), on the other hand we doesn’t use fitur vegetation index variants, maximum EVI and minimum EVI from 4 period, the accuracy model only 0.573. The accuracy if we use temporal fitur is 0.727. So There is increasing of accuracy model if we use temporal features compare to the model if we doesn’t use.

The highest accuracy is in class 8 (non rice fields) and the lowest accuracy is in class 5 (bare / land preparation). In class 5, all samples were classified as wrong. This indicates that the features used in the random forest have not been able to capture the class 5 pattern. This class 5 consists of fallow conditions after harvest (open and dry land) and land preparation conditions (flooded with water). As shown in Table 2, the class 5 is still classified incorrectly into the initial vegetative class, harvest, and not rice fields. The hallmark of the early vegetation class is the amount of standing water and the characteristics of the harvest class is open dry land. Both are similar to conditions in fallow class. Experiments using other features that are able to distinguish bare/land preparation from the early vegetative and harvesting need to be done to improve the accuracy of the model.

The accuracy of the early generative class is also still low, 28.57%. This early generative class is still misclassified into the early vegetative class, late vegetative, non-rice in paddy fields, and non rice fields. This misclassification occurs from all of the rice plant phase class become non rice in rice fields class and non rice field class. On the other hand, non rice in rice fields are still misclassified as harvest classes. This misclassification if it has not been able to be lowered will cause over or under estimate of agricultural yields. If many sample from the non rice in paddy field class is predicted as a harvest, then the estimated crop will over estimate. If many sample from the rice plant phase class is predicted as not rice or non-paddy fields, then the estimation of agricultural yields will under estimate.

To provide an overview of important features, a random forest model is maded into two models with different classes. In model A, there are 3 classes that are used, namely rice, non-rice in rice fields, and non-rice fields. We call model A the model that distinguishes rice vs. non-rice classes. Model B uses 7all classes of rice phase, early vegetative, late vegetative, early generative, late generative, harvest, fallow/land preparation, and puso. We call model B as the model that distinguishes the rice plant phase class. Important features on both models can be seen in Figure 5.

Figure 5.

Important features from random forest model that (a) distinguish rice and non-rice classes, (b) distinguish the rice plant phase class

Important features that can distinguish between rice and non-rice classes are vegetation index variant from 4 period and band 1. Vegetation index values in the paddy class are more varied than non paddy classes. NDWI values in the paddy class ranged from 0.0 to -0.6 and in the non paddy class ranged from -0.4 to -0.6. This NDWI shows the existence of standing water, so that the value of NDWI is high during the initial planting and decreases until harvesting. EVI values in the paddy class ranged from 0.1 to 0.7 and in the non paddy class ranged from 0.3 to 0.6. This EVI shows the level of greenness, so in rice classes, the value is low when the initial planting, rises to the maximum peak between the late vegetative phase with the early generative and decreases until harvesting. In non rice classes, there are a number of samples such as horticultural crops in paddy fields, forests, and shrubs that have high EVI values in the range of 0.6 and for 4 periods also remain in that range. There are also some samples such as houses and buildings that have low EVI values in the range of 0.3 and for 4 periods remain in that range.

Something surprising, band 1 was also a distinguishing feature of rice and non-rice classes. Band 1 of this Landsat 8 image is an aerosol or coastal band that is usually applied to marine monitoring. there are two assumptions why band 1 is one of the best differentiating features between rice and non-rice classes. First, maybe this is because some KSA samples are in non-rice fields such as ponds and lakes, so band 1 is useful for recognizing that sample. Second, this shows that band 1 can be explored as an index for plant monitoring.

An important feature of model A shows that if we are going to differentiate between paddy and non-paddy land, or to classify rice plants in a large area that allows observations of non-rice plants, it is better to use variations of vegetation indexes over several periods rather than EVI. If we classify rice and non-rice using only EVI, there is a possibility of misclassification, especially in samples with high EVI values such as non-rice plants in rice fields, forests, shrubs, fields covered with grass, etc.

Model B is used when the classification of rice plants is carried out on land that is always planted with rice. It is assumed that there are no non-rice plants planted on the land. From this model B we get information about the features that distinguish between phases in the rice class. The important features that differentiate between classes are minimum EVI from 4 period, band 5, band 6 EVI, and maxkimum EVI from 4 period. The EVI index, which is usually used to plant phase classification manually, turns out to be at number 4. The values of band 5 and band 6 have been used to calculate the NDBI value, but it turns out the original values of bands 5 and 6 are better at distinguishing the rice plant phase class. It is estimated that bands 5 and 6 can still be used to a new index that captures the value of open and dry land.

Based on the 5 best features of differentiation between rice phase classes, the characteristics of the early vegetative class are those 5 features are of low value. The characteristics of late vegetative class are the minimum EVI value is low, band 6 value is medium, and band 5, EVI, and the maximum EVI value is high. The characteristics of early generative class is the medium value of band 6 and the high value of other 4 features. The characteristics of harvest class is the scores are low on EVI, medium on band 5, and high on minimum EVI, band 6, and maximum EVI.

4.

CONCLUSIONS

In this study, we performed extensive experiments using supervised random forest method to investigate best multitemporal features for the classification of rice plant phase. From the results, several important points can be summarized as insight for improving the classification model of rice plant phase, including (1) classification accuracy increases when temporal features are included in the model, (2) the highest accuracy is in class 8 (non-paddy fields) because the spectral pattern of this class is not quite varied, (3) the lowest accuracy is in class 5 (bare / land preparation), (4) we need to do further experiments with other features that can distinguish bare/land preparation from the early vegetative and harvesting to improve the accuracy of the model in class bare/land preparation and other features that can distinguish generative from late vegetative to improve the accuracy of the model in early generative class, (5) features engineering have an important role for accuracy improvement because different classification classes require different set of features.