Automated Delineation of Smallholder Farm Fields Using Fully Convolutional Networks and Generative Adversarial Networks

doi:10.11947/j.JGGS.2022.0402

Abstract

Abstract:

Accurate boundaries of smallholder farm fields are important and indispensable geo-information that benefits farmers, managers, and policymakers in terms of better managing and utilizing their agricultural resources. Due to their small size, irregular shape, and the use of mixed-cropping techniques, the farm fields of smallholder can be difficult to delineate automatically. In recent years, numerous studies on field contour extraction using a deep Convolutional Neural Network (CNN) have been proposed. However, there is a relative shortage of labeled data for filed boundaries, thus affecting the training effect of CNN. Traditional methods mostly use image flipping, and random rotation for data augmentation. In this paper, we propose to apply Generative Adversarial Network (GAN) for the data augmentation of farm fields label to increase the diversity of samples. Specifically, we propose an automated method featured by Fully Convolutional Neural networks (FCN) in combination with GAN to improve the delineation accuracy of smallholder farms from Very High Resolution (VHR) images. We first investigate four State-Of-The-Art (SOTA) FCN architectures, i.e., U-Net, PSPNet, SegNet and OCRNet, to find the optimal architecture in the contour detection task of smallholder farm fields. Second, we apply the identified optimal FCN architecture in combination with Contour GAN and pixel2pixel GAN to improve the accuracy of contour detection. We test our method on the study area in the Sudano-Sahelian savanna region of northern Nigeria. The best combination achieved F1 scores of 0.686 on Test Set 1 (TS1), 0.684 on Test Set 2 (TS2), and 0.691 on Test Set 3 (TS3). Results indicate that our architecture adapts to a variety of advanced networks and proves its effectiveness in this task. The conceptual, theoretical, and experimental knowledge from this study is expected to seed many GAN-based farm delineation methods in the future.

Key words: field boundary; contour detection; fully convolutional neural networks; generative adversarial networks

Qiuyu YAN, Wufan ZHAO, Xiao HUANG, Xianwei LYU. Automated Delineation of Smallholder Farm Fields Using Fully Convolutional Networks and Generative Adversarial Networks[J]. Journal of Geodesy and Geoinformation Science, 2022, 5(4): 10-22.

Figures/Tables 16

Fig.1

Fig.2

Tab.1

Fig.3

Fig.4

Tab.2

Fig.5

Tab.3

Fig.6

Tab.4

Fig.7

Tab.5

Fig.8

Fig.9

Fig.10

Tab.6

References 40

[1]	LOWDER S K, SKOET J, RANEY T. The number, size, and distribution of farms, smallholder farms, and family farms worldwide[J]. World Development, 2016, 87: 16-29. DOI: 10.1016/J.WORLDDEV.2015.10.041. doi: 10.1016/J.WORLDDEV.2015.10.041
[2]	GARCÍA-PEDRERO A, GONZALO-MARTÍN C, LILLO-SAAVEDRA M. A machine learning approach for agricultural parcel delineation through agglomerative segmentation[J]. International Journal of Remote Sensing, 2017, 38(7): 1809-1819. DOI: 10.1080/01431161.2016.1278312. doi: 10.1080/01431161.2016.1278312
[3]	ROBERTS L G. Machine perception of three-dimensional solids[D]. Cambridge: Massachusetts Institute of Technology, 1963.
[4]	SOBEL I E. Camera models and machine perception[D]. Stanford: Stanford University, 1970.
[5]	CANNY J. A computational approach to edge detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1986, PAMI-8(6): 679-698. doi: 10.1109/TPAMI.1986.4767851
[6]	BERGADO J R, PERSELLO C, GEVAERT C. A deep learning approach to the classification of sub-decimetre resolution aerial images[C]// Proceedings of 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS). Beijing:IEEE, 2016: 1516-1519. DOI: 10.1109/IGARSS.2016.7729387. doi: 10.1109/IGARSS.2016.7729387
[7]	WALDNER F, DIAKOGIANNIS F I, BATCHELOR K, et al. Detect, consolidate, delineate: scalable mapping of field boundaries using satellite images[J]. Remote Sensing, 2021, 13(11): 2197. DOI: 10.3390/RS13112197. doi: 10.3390/RS13112197
[8]	LONG Jiang, LI Mengmeng, WANG Xiaoqin, et al. Delineation of agricultural fields using multi-task BsiNet from high-resolution satellite images[J]. International Journal of Applied Earth Observation and Geoinformation, 2022, 112: 102871. DOI: 10.1016/J.JAG.2022.102871. doi: 10.1016/J.JAG.2022.102871
[9]	GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]// Proceedings of the 27th International Conference on Neural Information Processing Systems. Montreal: ACM, 2014: 2672-2680.
[10]	XIE C, WANG J, ZHANG Z, et al. Adversarial examples for semantic segmentation and object detection[C]// Proceedings of IEEE international conference on computer vision. Venice, Italy:IEEE, 2017:1369-1378.
[11]	HONG Y, HWANG U, YOO J, et al. How generative adversarial networks and their variants work: an overview[J]. ACM Computing Surveys, 2017, 52(1): 10. DOI: 10.1145/3301282. doi: 10.1145/3301282
[12]	LIN D Y, FU K, WANG Y, et al. MARTA GANs: unsupervised representation learning for remote sensing image classification[J]. IEEE Geoscience and Remote Sensing Letters, 2017, 14(11): 2092-2096. DOI: 10.1109/LGRS.2017.2752750. doi: 10.1109/LGRS.2017.2752750
[13]	PERSELLO C, TOLPEKIN V A, BERGADO J R, et al. Delineation of agricultural fields in smallholder farms from satellite images using fully convolutional networks and combinatorial grouping[J]. Remote Sensing of Environment, 2019, 231: 111253. DOI: 10.1016/J.RSE.2019.111253. doi: 10.1016/J.RSE.2019.111253
[14]	COMANICIU D, MEER P. Mean shift: a robust approach toward feature space analysis[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(5): 603-619. DOI: 10.1109/34.1000236. doi: 10.1109/34.1000236
[15]	CHEN P C, PAVLIDIS T. Segmentation by texture using a co-occurrence matrix and a split-and-merge algorithm[J]. Computer Graphics and Image Processing, 1979, 10(2): 172-182. DOI: 10.1016/0146-664X(79)90049-2. doi: 10.1016/0146-664X(79)90049-2
[16]	ADAMS R, BISCHOF L. Seeded region growing[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1994, 16(6): 641-647. DOI: 10.1109/34.295913. doi: 10.1109/34.295913
[17]	MARTIN D R, FOWLKES C C, MALIK J. Learning to detect natural image boundaries using local brightness, color, and texture cues[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 26(5): 530-549. DOI: 10.1109/TPAMI.2004.1273918. doi: 10.1109/TPAMI.2004.1273918 pmid: 15460277
[18]	MUELLER M, SEGL K, KAUFMANN H. Edge- and region-based segmentation technique for the extraction of large, man-made objects in high-resolution satellite imagery[J]. Pattern Recognition, 2004, 37(8): 1619-1628. DOI: 10.1016/J.PATCOG.2004.03.001. doi: 10.1016/J.PATCOG.2004.03.001
[19]	TIWARI P S, PANDE H, KUMAR M, et al. Potential of IRS P-6 LISS IV for agriculture field boundary delineation[J]. Journal of Applied Remote Sensing, 2009, 3(1): 033528. DOI: 10.1117/1.3133306. doi: 10.1117/1.3133306
[20]	TURKER M, KOK E H. Field-based sub-boundary extraction from remote sensing imagery using perceptual grouping[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2013, 79: 106-121. DOI: 10.1016/J.ISPRSJPRS.2013.02.009. doi: 10.1016/J.ISPRSJPRS.2013.02.009
[21]	DROZDZAL M, VORONTSOV E, CHARTRAND G, et al. The importance of skip connections in biomedical image segmentation[C]// Proceedings of the First International Workshop on Deep Learning and Data Labeling for Medical Applications. Athens: Springer, 2016: 179-187. DOI: 10.1007/978-3-319-46976-8_19. doi: 10.1007/978-3-319-46976-8_19
[22]	CHRIST P F, ELSHAER M E A, ETTLINGER F, et al. Automatic liver and lesion segmentation in CT using cascaded fully convolutional neural networks and 3D conditional random fields[C]// Proceedings of the 19th International Conference on Medical Image Computing and Computer-Assisted Intervention. Athens: Springer, 2016: 415-423. DOI: 10.1007/978-3-319-46723-8_48. doi: 10.1007/978-3-319-46723-8_48
[23]	MAGGIORI E, TARABALKA Y, CHARPIAT G, et al. Fully convolutional neural networks for remote sensing image classification[C]// Proceedings of 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS). Beijing:IEEE, 2016: 5071-5074. DOI: 10.1109/IGARSS.2016.7730322. doi: 10.1109/IGARSS.2016.7730322
[24]	YANG J M, PRICE B, COHEN S, et al. Object contour detection with a fully convolutional encoder-decoder network[C]// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 193-202.
[25]	XIE S N, TU Z W. Holistically-nested edge detection[C]// Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago: IEEE, 2015: 1395-1403.
[26]	SHELHAMER E, LONG J, DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640-651. DOI: 10.1109/TPAMI.2016.2572683. doi: 10.1109/TPAMI.2016.2572683 pmid: 27244717
[27]	ZUO Z C, ZHANG W, ZHANG D Y. A remote sensing image semantic segmentation method by combining deformable convolution with conditional random fields[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(6): 718-726. DOI: 10.11947/j.AGCS.2019.20170740. doi: 10.11947/j.AGCS.2019.20170740
[28]	LIN W J, LI Y, ZHAO Q H. High-resolution remote sensing image segmentation using minimum spanning tree tessellation and RHMRF-FCM algorithm[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(1): 64-74. DOI: 10.11947/j.AGCS.2019.20170585. doi: 10.11947/j.AGCS.2019.20170585
[29]	RADFORD A, METZ L, CHINTALA S. Unsupervised representation learning with deep convolutional generative adversarial networks[C]// Proceedings of the 4th International Conference on Learning Representations. San Juan: ICLR, 2016.
[30]	ZHU L, CHEN Y S, GHAMISI P, et al. Generative adversarial networks for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(9): 5046-5063. DOI: 10.1109/TGRS.2018.2805286. doi: 10.1109/TGRS.2018.2805286
[31]	SUN Q Q, BOURENNANE S. Unsupervised feature extraction based on improved Wasserstein generative adversarial network for hyperspectral classification[C]// Proceedings of SPIE 11059, Multimodal Sensing:Technologies and Applications. Munich: SPIE, 2019: 110590L. DOI: 10.1117/12.2527466. doi: 10.1117/12.2527466
[32]	YANG H J, LI Y, YAN X F, et al. ContourGAN: image contour detection with generative adversarial network[J]. Knowledge-Based Systems, 2019, 164: 21-28. DOI: 10.1016/J.KNOSYS.2018.09.033. doi: 10.1016/j.knosys.2018.09.033
[33]	LI J N, LIANG X D, WEI Y C, et al. Perceptual generative adversarial networks for small object detection[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 1951-1959.
[34]	LUCIO D R, LAROCA R, SEVERO E, et al. Fully convolutional networks and generative adversarial networks applied to sclera segmentation[C]// Proceedings of 2018 IEEE 9th International Conference on Biometrics Theory, Applications and Systems. Redondo Beach: IEEE, 2018: 1-7.
[35]	LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]// Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 3431-3440.
[36]	RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]// Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich: Springer, 2015: 234-241. DOI: 10.1007/978-3-319-24574-4_28. doi: 10.1007/978-3-319-24574-4_28
[37]	ZHAO H S, SHI J P, QI X J, et al. Pyramid scene parsing network[C]// Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 6230-6239. DOI: 10.1109/CVPR.2017.660. doi: 10.1109/CVPR.2017.660
[38]	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas:IEEE, 2016: 770-778.
[39]	BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495. DOI: 10.1109/TPAMI.2016.2644615. doi: 10.1109/TPAMI.2016.2644615 pmid: 28060704
[40]	YUAN Y H, CHEN X L, WANG J D. Object-contextual representations for semantic segmentation[C]// Proceedings of the 16th European Conference on Computer Vision. Glasgow: Springer, 2020: 173-190. DOI: 10.1007/978-3-030-58539-6_11. doi: 10.1007/978-3-030-58539-6_11

Tile	Train	Test
No.1	TR1	-
No.2	-	TS1
No.3	TR2	-
No.4	TR3	-
No.5	-	TS2
No.6	-	TS3

Tiles	Precision	Recall	F1 score
TS1	0.746	0.542	0.602
TS2	0.734	0.543	0.601
TS3	0.748	0.548	0.611
Average	0.743	0.544	0.605

Tiles	Precision	Recall	F1 score
TS1	0.714	0.572	0.623
TS2	0.715	0.560	0.613
TS3	0.729	0.567	0.624
Average	0.719	0.566	0.620

Tiles	Precision	Recall	F1 score
TS1	0.689	0.622	0.651
TS2	0.704	0.627	0.661
TS3	0.714	0.617	0.658
Average	0.702	0.622	0.657

Tiles	Precision	Recall	F1 score
TS1	0.703	0.662	0.682
TS2	0.715	0.665	0.689
TS3	0.727	0.653	0.688
Average	0.715	0.660	0.686