A Deep Double-Channel Dense Network for Hyperspectral Image Classification

doi:10.11947/j.JGGS.2021.0404

Abstract

Abstract:

Hyperspectral Image (HSI) classification based on deep learning has been an attractive area in recent years. However, as a kind of data-driven algorithm, the deep learning method usually requires numerous computational resources and high-quality labelled datasets, while the expenditures of high-performance computing and data annotation are expensive. In this paper, to reduce the dependence on massive calculation and labelled samples, we propose a deep Double-Channel dense network (DDCD) for Hyperspectral Image Classification. Specifically, we design a 3D Double-Channel dense layer to capture the local and global features of the input. And we propose a Linear Attention Mechanism that is approximate to dot-product attention with much less memory and computational costs. The number of parameters and the consumptions of calculation are observably less than contrapositive deep learning methods, which means DDCD owns simpler architecture and higher efficiency. A series of quantitative experiences on 6 widely used hyperspectral datasets show that the proposed DDCD obtains state-of-the-art performance, even though when the absence of labelled samples is severe.

Key words: 3D Double-Channel dense layer; Linear Attention Mechanism; Deep Learning (DL); hyperspectral classification

Kexian WANG,Shunyi ZHENG,Rui LI,Li GUI. A Deep Double-Channel Dense Network for Hyperspectral Image Classification[J]. Journal of Geodesy and Geoinformation Science, 2021, 4(4): 46-62.

Figures/Tables 25

Fig.1

Fig.2

Fig.3

Tab.1

Tab.2

Fig.4

Fig.5

Fig.6

Fig.7

Tab.3

Tab.4

Tab.5

Tab.6

Tab.7

Fig.8

Tab.8

Tab.9

Tab.10

Tab.11

Tab.12

Tab.13

Fig.9

Fig.10

Fig.11

Fig.12

References 41

[1]	LI Zhaokui, HUANG Lin, HE Jinrong. A multiscale deep middle-level feature fusion network for hyperspectral classification[J]. Remote Sensing, 2019, 11(6):695. doi: 10.3390/rs11060695
[2]	HONG Yongsheng, GUO Long, CHEN Songchao, et al. Exploring the potential of airborne hyperspectral image for estimating topsoil organic carbon: effects of fractional-order derivative and optimal band combination algorithm[J]. Geoderma, 2020, 365:114228. doi: 10.1016/j.geoderma.2020.114228
[3]	ROCHA A D, GROEN T A, SKIDMORE A K. Spatially-explicit modelling with support of hyperspectral data can improve prediction of plant traits[J]. Remote Sensing of Environment, 2019, 231:111200. doi: 10.1016/j.rse.2019.05.019
[4]	DAI Yuchao, ZHANG Jing, HE Mingyi, et al. Salient object detection from multi-spectral remote sensing images with deep residual network[J]. Journal of Geodesy and Geoinformation Science, 2019, 2(2):101-110. DOI: 10.11947/j.JGGS.2019.0211. doi: 10.11947/j.JGGS.2019.0211
[5]	MELGANI F, BRUZZONE L. Classification of hyperspectral remote sensing images with support vector machines[J]. IEEE Transactions on Geoscience and Remote Sensing, 2004, 42(8):1778-1790. doi: 10.1109/TGRS.2004.831865
[6]	DU Qian, CHANG C I. A linear constrained distance-based discriminant analysis for hyperspectral image classification[J]. Pattern Recognition, 2001, 34(2):361-373. doi: 10.1016/S0031-3203(99)00215-0
[7]	EDIRIWICKREMA J, KHORRAM S. Hierarchical maximum-likelihood classification for improved accuracies[J]. IEEE Transactions on Geoscience and Remote Sensing, 1997, 35(4):810-816. doi: 10.1109/36.602523
[8]	CHAN J C W, PAELINCKX D. Evaluation of Random Forest and Adaboost tree-based ensemble classification and spectral band selection for ecotope mapping using airborne hyperspectral imagery[J]. Remote Sensing of Environment, 2008, 112(6):2999-3011. doi: 10.1016/j.rse.2008.02.011
[9]	HAM J, CHEN Y C, CRAWFORD M M, et al. Investigation of the random forest framework for classification of hyperspectral data[J]. IEEE Transactions on Geoscience and Remote Sensing, 2005, 43(3):492-501. doi: 10.1109/TGRS.2004.842481
[10]	ZHU Zexuan, JIA Sen, HE Shan, et al. Three-dimensional Gabor feature extraction for hyperspectral imagery classification using a memetic framework[J]. Information Sciences, 2015, 298:274-287. doi: 10.1016/j.ins.2014.11.045
[11]	BAU T C, SARKAR S, HEALEY G. Hyperspectral region classification using a three-dimensional Gabor filterbank[J]. IEEE Transactions on Geoscience and Remote Sensing, 2010, 48(9):3457-3464. doi: 10.1109/TGRS.2010.2046494
[12]	TANG Yuanyan, LU Yang, YUAN Haoliang. Hyperspectral image classification based on three-dimensional scattering wavelet transform[J]. IEEE Transactions on Geoscience and Remote Sensing, 2015, 53(5):2467-2480. doi: 10.1109/TGRS.2014.2360672
[13]	CAO Xiangyong, XU Lin, MENG Deyu, et al. Integration of 3-dimensional discrete wavelet transform and Markov random field for hyperspectral image classification[J]. Neurocomputing, 2017, 226:90-100. doi: 10.1016/j.neucom.2016.11.034
[14]	LIU Ziwei, LUO Ping, WANG Xiaogang, et al. Deep learning face attributes in the wild[C]// Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV). Santiago: IEEE, 2015: 3730-3738.
[15]	WANG L, ZHANG C, LI R, et al. Scale- aware neural network for semantic segmentation of multi-resolution remote sensing images[J]. Remote Sensing, 2021, 13(24):5015. doi: 10.3390/rs13245015
[16]	LI R, ZHENG S, ZHANG C, et al. ABCNet: Attentive bilateral contextual network for efficient semantic segmentation of Fine-Resolution remotely sensed imagery[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 181:84-98. doi: 10.1016/j.isprsjprs.2021.09.005
[17]	GONG Jianya, JI Shunping. Photogrammetry and deep learning[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(6):693-704. DOI: 10.11947/j.AGCS.2018.20170640. doi: 10.11947/j.AGCS.2018.20170640
[18]	FAN Dazhao, ZHANG Yongsheng. Satellite image matching method based on deep convolutional neural network[J]. Journal of Geodesy and Geoinformation Science, 2019, 2(2):90-100. DOI: 10.11947/j.JGGS.2019.0210. doi: 10.11947/j.JGGS.2019.0210
[19]	SUN Long, WU Tao, SUN Guangcai, et al. Object detection research of SAR image using improved faster region-based convolutional neural network[J]. Journal of Geodesy and Geoinformation Science, 2020, 3(3):18-28. DOI: 10.11947/j.JGGS.2020.0302. doi: 10.11947/j.JGGS.2020.0302
[20]	HE Hao, WANG Shuyang, WANG Shicheng, et al. A road extraction method for remote sensing image based on encoder-decoder network[J]. Journal of Geodesy and Geoinformation Science, 2020, 3(2):16-25. DOI: 10.11947/j.JGGS.2020.0202. doi: 10.11947/j.JGGS.2020.0202
[21]	ZUO Zongcheng, WANG Wen, ZHANG Dongying. A remote sensing image semantic segmentation method by combining deformable convolution with conditional random fields[J]. Journal of Geodesy and Geoinformation Science, 2020, 3(3):39-49. DOI: 10.11947/j.JGGS.2020.0304. doi: 10.11947/j.JGGS.2020.0304
[22]	ZHENG Zhuo, ZHONG Yanfei, MA Ailong, et al. FPGA: fast patch-free global learning framework for fully end-to-end hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(8):5612-5626. doi: 10.1109/TGRS.36
[23]	JIA Zhuang, LU Wenkai. An end-to-end hyperspectral image classification method using deep convolutional neural network with spatial constraint[J]. IEEE Geoscience and Remote Sensing Letters, 2021, 18(10):1786-1790. doi: 10.1109/LGRS.2020.3008051
[24]	SUN Hao, ZHENG Xiangtao, LU Xiaoqiang. A supervised segmentation network for hyperspectral image classification[J]. IEEE Transactions on Image Processing, 2021, 30:2810-2825. doi: 10.1109/TIP.83
[25]	PAN Bin, XU Xia, SHI Zhenwei, et al. DSSNet: a simple dilated semantic segmentation network for hyperspectral imagery classification[J]. IEEE Geoscience and Remote Sensing Letters, 2020, 17(11):1968-1972. doi: 10.1109/LGRS.8859
[26]	ZHAO Wenzhi, DU Shihong. Spectral-spatial feature extraction for hyperspectral image classification: a dimension reduction and deep learning approach[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(8):4544-4554. doi: 10.1109/TGRS.2016.2543748
[27]	LEE H, KWON H. Going deeper with contextual CNN for hyperspectral image classification[J]. IEEE Transactions on Image Processing, 2017, 26(10):4843-4855. doi: 10.1109/TIP.2017.2725580
[28]	CHEN Yushi, JIANG Hanlu, LI Chunyang, et al. Ghamisi. Deep feature extraction and classification of hyperspectral images based on convolutional neural networks[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(10):6232-6251. doi: 10.1109/TGRS.2016.2584107
[29]	HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE, 2016: 770-778.
[30]	HUANG Gao, LIU Zhuang, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu: IEEE, 2017: 2261-2269.
[31]	ZHONG Zilong, LI J, LUO Zhiming, et al. Spectral-spatial residual network for hyperspectral image classification: a 3-D deep learning framework[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(2):847-858. doi: 10.1109/TGRS.2017.2755542
[32]	WANG Wenju, DOU Shuguang, JIANG Zhongmin, et al. A fast dense spectral-spatial convolution network framework for hyperspectral images classification[J]. Remote Sensing, 2018, 10(7):1068. doi: 10.3390/rs10071068
[33]	HAUT J M, PAOLETTI M E, PLAZA J, et al. Visual attention-driven hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(10):8065-8080. doi: 10.1109/TGRS.36
[34]	MA Wenping, YANG Qifan, WU Yue, et al. Double-branch multi-attention mechanism network for hyperspectral image classification[J]. Remote Sensing, 2019, 11(11):1307. doi: 10.3390/rs11111307
[35]	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]// Proceedings of the 15th European Conference on Computer Vision (ECCV). Munich: Springer, 2018: 3-19.
[36]	FU Jun, LIU Jing, TIAN Haijie, et al. Dual attention network for scene segmentation[C]// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach: IEEE, 2019: 3141-3149.
[37]	LI Rui, ZHENG Shunyi, DUAN Chenxi, et al. Classification of hyperspectral image based on double-branch dual-attention mechanism network[J]. Remote Sensing, 2020, 12(3):582. doi: 10.3390/rs12030582
[38]	LI R, DUAN C, ZHENG S, et al. MACU-Net for semantic segmentation of fine-resolution remotely sensed images[J]. IEEE Geoscience and Remote Sensing Letters, 2021.
[39]	LI R, ZHENG S, DUAN C, et al. Multistage attention resu-net for semantic segmentation of fine-resolution remote sensing images[J]. IEEE Geoscience and Remote Sensing Letters, 2021.
[40]	WANG L, LI R, WANG D, et al. Transformer meets convolution: a bilateral awareness network for semantic segmentation of very fine resolution urban scene images[J]. Remote Sensing, 2021, 13(16):3065. doi: 10.3390/rs13163065
[41]	LI R, ZHENG S, ZHANG C, et al. Multiattention network for semantic segmentation of fine-resolution remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021.

Layer name		Kernel size	Group	Output size
Input		-	-	(9×9×200,1)
3D-CNN+BN+ReLU		(1×1×7)	1	(9×9×97,24)
Linear Attention Mechanism		-	-	(9×9×97,24)
Linear Attention Mechanism		-	-	(9×9×97,24)
Concatenate		-	-	(9×9×97,48)
3D two- way dense layer	3D-CNN+ BN+ReLU	(1×1×1)	3	(9×9×97,48)
	3D-CNN+ BN+ReLU	(3×3×3)	3	(9×9×97,12)
	3D-CNN+ BN+ReLU	(3×3×3)	3	(9×9×97,12)
	3D-CNN+ BN+ReLU	(1×1×1)	3	(9×9×97,48)
	3D-CNN+ BN+ReLU	(3×3×3)	3	(9×9×97,12)
Concatenate		-	-	(9×9×97,72)
3D-CNN+BN+ReLU		(3×3×97)	3	(9×9×1,60)
Global Average Pooling		-	-	(1×60)

No.	Class	Total number	Train	Val	Test
1	Alfalfa	46	3	3	40
2	Corn-notill	1428	42	42	1344
3	Corn-mintill	830	24	24	782
4	Corn	237	7	7	223
5	Grass-pasture	483	14	14	455
6	Grass-trees	730	21	21	688
7	Grass-pasture-mowed	28	3	3	22
8	Hay-windrowed	478	14	14	450
9	Oats	20	3	3	14
10	Soybean-notill	972	29	29	914
11	Soybean-mintill	2455	73	73	2309
12	Soybean-clean	593	17	17	559
13	Wheat	205	6	6	193
14	Woods	1265	37	37	1191
15	Buildings-Grass-Trees	386	11	11	364
16	Stone-Steel-Towers	93	3	3	87
	Total	10249	307	307	9635

No.	Class	Total number	Train	Val	Test
1	Asphalt	6631	33	33	6565
2	Meadows	18649	93	93	18463
3	Gravel	2099	10	10	2079
4	Trees	3064	15	15	3034
5	Painted metal sheets	1345	6	6	1333
6	Bare Soil	5029	25	25	4979
7	Bitumen	1330	6	6	1318
8	Self-Blocking Bricks	3682	18	18	3646
9	Shadows	947	4	4	939
	Total	42776	210	210	42356

No.	Class	Total number	Train	Val	Test
1	Water	65971	65	65	65841
2	Trees	7598	7	7	7584
3	Meadows	3090	3	3	3084
4	Bricks	2685	3	3	2679
5	Soil	6584	6	6	6572
6	Asphalt	9248	9	9	9230
7	Bitumen	7287	7	7	7273
8	Tiles	42826	42	42	42742
9	Shadows	2863	3	3	2857
	Total	148152	145	145	147862

No.	Class	Total number	Train	Val	Test
1	Dense urban fabric	1262	37	37	1188
2	Mineral extraction sites	204	6	6	192
3	Non-irrigated arable land	614	18	18	578
4	Fruit trees	150	4	4	142
5	Olive groves	1768	53	53	1662
6	Coniferous forest	361	10	10	341
7	Densesclerophyllous vegetation	5035	151	151	4733
8	Sparce sclerophyllous vegetation	6374	191	191	5992
9	Sparcely vegetated areas	1754	52	52	1650
10	Rocks and sand	492	14	14	464
11	Water	1612	48	48	1516
12	Coastal water	398	11	11	376
	Total	20024	595	595	19429