Satellite Image Matching Method Based on Deep Convolutional Neural Network

doi:10.11947/j.JGGS.2019.0210

Abstract

Abstract:

This article focuses on the first aspect of the album of deep learning: the deep convolutional method. The traditional matching point extraction algorithm typically uses manually designed feature descriptors and the shortest distance between them to match as the matching criterion. The matching result can easily fall into a local extreme value, which causes missing of the partial matching point. Targeting this problem, we introduce a two-channel deep convolutional neural network based on spatial scale convolution, which performs matching pattern learning between images to realize satellite image matching based on a deep convolutional neural network. The experimental results show that the method can extract the richer matching points in the case of heterogeneous, multi-temporal and multi-resolution satellite images, compared with the traditional matching method. In addition, the accuracy of the final matching results can be maintained at above 90%.

Key words: image matching; deep learning; convolutional neural network; satellite image

Dazhao FAN,Yang DONG,Yongsheng ZHANG. Satellite Image Matching Method Based on Deep Convolutional Neural Network[J]. Journal of Geodesy and Geoinformation Science, 2019, 2(2): 90-100.

Figures/Tables 24

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Tab.1

Fig.7

Fig.8

Fig.9

Tab.2

Fig.10

Tab.3

Tab.4

Fig.11

Fig.12

Fig.13

Fig.14

Fig.15

Fig.16

Fig.17

Fig.18

Fig.19

Fig.20

References 33

[1]	LI Deren . Towards Geo-Spatial Information Science in Big Data Era[J]. Acta Geodaetica et Cartographica Sinica, 2016,45(4):379-384.
[2]	LI Deren, ZHANG Liangpei, Xia Guisong . Automatic Analysis and Mining of Remtoe Sensing Big Data[J]. Acta Geodaetica et Cartographica Sinica, 2014,43(12):1211-1216.
[3]	YAN Guoping, HE Junfeng . Extended Laplacian of Gaussian Operator for Edge Detection[J]. Journal of Huazhong University of Science and Technology (Nature Science Edition), 2006,34(10):21-23.
[4]	ZHAO Wanjin, GONG Shengrong, LIU Chunping , et al. Adaptive Harris Corner Detection Algorithm[J]. Computer Engineering, 2008,34(10):212-214, 217.
[5]	LOWE D G . Distinctive Image Features from Scale-invariant Keypoints[J]. International Journal of Computer Vision, 2004,60(2):91-110.
[6]	KE Yan, SUKTHANKAR R . PCA-SIFT: A More Distinctive Representation for Local Image Descriptors [C]//Proceedings of the IEEE Computer Society Computer Vision and Pattern Recognition. Washington, DC: IEEE, 2004: 506-513.
[7]	BAY H, TUYTELAARS T, VAN GOOL L . SURF: Speeded up Robust Features[M] //LEONARDIS A, BISCHOF H, PINZ A. Computer Vision-ECCV 2006. Berlin: Springer, 2006: 404-417.
[8]	CALONDER M, LEPETIT V, STRECHA C , et al. BRIEF: Binary Robust Independent Elementary Features[M] //DANⅡLIDIS K, MARAGOS P, PARAGIOS N. Computer Vision-ECCV 2010. Berlin: Springer, 2010: 778-792.
[9]	CALONDER M, LEPETIT V, OZUYSAL M , et al. BRIEF:Computing a Local Binary Descriptor Very Fast[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012,34(7):1281-1298.
[10]	XU Yunxi, CHEN Fang . Recent Advances in Local Image Descriptor[J]. Journal of Image and Graphics, 2015,20(9):1133-1150.
[11]	DONG Yang, FAN Dazhao, JI Song , et al. The Purification Method of Matching Points Based on Principal Component Analysis[J]. Acta Geodaetica et Cartographica Sinica, 2017,46(2):228-236.
[12]	FISCHER Philipp, ALEXEY Dosovitskiy, THOMAS Brox . Descriptor Matching with Convolutional Neural Networks: A Comparison to SIFT[J]. arXiv: Computer Vision and Pattern Recognition, 2014.
[13]	ŽBONTAR J, LECUN Y . Computing the Stereo Matching Cost with a Convolutional Neural Network [C]//IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA: IEEE, 2015: 1592-1599.
[14]	ZAGORUYKO S, KOMODAKIS N . Learning to Compare Image Patches via Convolutional Neural Networks [C]//IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA: IEEE, 2015: 4353-4361.
[15]	HAN Xufeng, LEUNG T, JIA Yangqing , et al. MatchNet: Unifying Feature and Metric Learning for Patch-based Matching [C]//IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA: IEEE, 2015: 3279-3286.
[16]	PAL S K, MITRA S . Multilayer Perceptron, Fuzzy Sets, Classification[J]. IEEE Transactions on Neural Networks, 1992,3(5):683-697.
[17]	TRAPPEY A J C, HSU F C, TRAPPEY C V , et al. Development of a Patent Document Classification and Search Platform Using a Back-propagation Network[J]. Expert Systems with Applications, 2006,31(4):755-765.
[18]	SZEGEDY C, LIU Wei, JIA Yangqing , et al. Going Deeper with Convolutions [C]//IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA: IEEE, 2015: 1-9.
[19]	ŽBONTAR J, LECUN Y . Stereo Matching by Training A Convolutional Neural Network to Compare Image Patches[J]. The Journal of Machine Learning Research, 2016,17(1):2287-2318.
[20]	COLLOBERT R, KAVUKCUOGLU K, FARABET C . Torch7: A Matlab-like Environment for Machine Learning [C]//Neural Information Processing Systems Workshop. [s. l]: BigLearn, 2011.
[21]	CHEN Tianqi, LI Mu, LI Yutian , et al. MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems[R]. Nanjin: Neural Information Processing Systems, Workshop on Machine Learning Systems, 2015.
[22]	CHETLUR Sharan, WOOLLEY Cliff, VANDERMERSCH Philippe , et al. cuDNN: Efficient Primitives for Deep Learning[J]. arXiv preprint arXiv: 1410. 0759, 2014.
[23]	XU Wei . Towards Optimal One Pass Large Scale Learning with Averaged Stochastic Gradient Descent[J]. arXiv preprint arXiv: 1107.2490, 2011.
[24]	BROWN M, HUA Gang, WINDER S . Discriminative Learning of Local Image Descriptors[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011,33(1):43-57.
[25]	DU Yulong, LI Jianzeng, ZHANG Yan , et al. Saliency Detection Based on Deep Cross CNN and Non-interaction GrabCut[J]. Computer Engineering and Applications, 2017,53(3):32-40.
[26]	PABLO F Alcantarilla, ADRIEN Bartoli, JESUS Nuevo . KAZE Features [C]//European Conference on Computer Vision, 2012.
[27]	PABLO F Alcantarilla, JESUS Nuevo, ADRIEN Bartoli . Fast explicit diffusion for accelerated features in nonlinear scale spaces [C]//British Machine Vision Conference, 2013.
[28]	ETHAN Rublee, VINCENT Rabaud, KURT Konolige , et al. ORB: An efficient alternative to SIFT or SURF [C]//IEEE International Conference on Computer Vision, 2012.
[29]	STEFAN Leutenegger, MARGARITA Chli, ROLAND Y . Siegwart. BRISK: Binary robust invariant scalable keypoints [C]//IEEE International Conference on Computer Vision, 2011.
[30]	EDWARD Rosten, T Drummond . Machine learning for high-speed corner detection [C]//European Conference on Computer Vision, 2006.
[31]	ELMAR Mair, GREGORY D Hager, DARIUS Burschka , et al. Adaptive and generic corner detection based on the accelerated segment test [C]//European Conference on Computer Vision, 2010.
[32]	HARRIS C, STEPHENS M . A Combined Corner and Edge Detector [C]// Proceedings of the 4th Alvey Vision Conference. Manchester: Springer, 1988: 147-151.
[33]	SHI Jianbo, TOMASI . Good features to track [C]//IEEE Conference on Computer Vision and Pattern Recognition, 1994: 593-600.

Dataset name	Data Sources	Training set size	Test set size
GFG	30-meter resolution Google satellite imagery 50-meter resolution GaoFen4 satellite imagery	200,000 feature point pairs 50% correct match pair 50% mismatched pair	100,000 feature point pairs 50% correct match pair 50% mismatched pair
THD	5-meter resolution TianHui satellite three-line array imagery	200,000 feature point pairs 50% correct match pair 50% mismatched pair	100,000 feature point pairs 50% correct match pair 50% mismatched pair

Dataset name	TPR	FPR95
THD Training set	99.516	0.024
THD Test set	99.100	0.164

Serial number	Feature detection	Feature description	Descriptor matching
1	sift	sift	ANN
2	surf	surf	ANN
3	KAZE	KAZE	BF
4	AKAZE	AKAZE	BF
5	ORB	ORB	BF-Hamming
6	BRISK	BRISK	BF
7	FAST	SIFT	ANN
8	AGAST	SIFT	ANN
9	Harris	SIFT	ANN
10	Shi-Tomasi	SIFT	ANN

Serial number	Comparative experimental method	Feature points of Google image	Feature points of GaoFen4 image	The matching number of traditional method	The matching number of traditional method and RANSAC	The matching correct rate of traditional method and RANSAC/(%)	The matching number of BSS- 2chDCNN	The matching number of BSS-2ch DCNN and RANSAC	The matching correct rate of BSS-2ch DCNN and RANSAC/(%)
1	sift	10905	831	23	5	0.00	320	32	93.75
2	surf	21837	6615	140	5	20.00	4017	124	100.00
3	KAZE	2659	608	5	4	50.00	122	14	92.86
4	AKAZE	2293	491	186	5	0.00	118	20	95.00
5	ORB	5000	2042	13	4	25.00	294	100	100.00
6	BRISK	8021	333	436	7	14.29	117	12	100.00
7	FAST+sift	65178	2812	991	14	21.43	1682	65	98.46
8	AGAST+sift	71291	3305	1167	12	50.00	1993	87	98.85
9	Harris+sift	1924	961	0	0	0.00	43	26	100.00
10	Shi-Tomasi+sift	8781	9115	0	0	0.00	3913	296	100.00