High-resolution Remote Sensing Image Semi-global Matching Method Considering Geometric Constraints of Connection Points and Image Texture Information

doi:10.11947/j.JGGS.2021.0408

Abstract

Abstract:

Dense matching of remote sensing images is a key step in the generation of accurate digital surface models. The semi-global matching algorithm comprehensively considers the advantages and disadvantages of local matching and global matching in terms of matching effect and computational efficiency, so it is widely used in close-range, aerial and satellite image matching. Based on the analysis of the original semi-global matching algorithm, this paper proposes a semi-global high-resolution remote sensing image that takes into account the geometric constraints of the connection points and the image texture information based on a large amount of high-resolution remote sensing image data and the characteristics of clear image texture.
The method includes 4 parts: ① Precise orientation. Aiming at the system error in the image orientation model, the system error of the rational function model is compensated by the geometric constraint relationship of the connecting points between the images, and the sub-pixel positioning accuracy is obtained; ② Epipolar image generation. After the original image is divided into blocks, the epipolar image is generated based on the projection trajectory method; ③ Image dense matching. In order to reduce the size of the cost space and calculation time, the image is pyramided and then semi-globally matched layer by layer. In the matching process, the disparity map expansion and erosion algorithm that takes into account the image texture information is introduced to restrict the disparity search range and better retain the edge characteristics of the ground objects; ④ Generate DSM. In order to test the matching effect, the weighted median filter algorithm is used to filter the disparity map, and the DSM is obtained based on the principle of forward intersection. Finally, the paper uses the matching results of WordView3 and Ziyuan No.3 stereo image to verify the effectiveness of this method.

Key words: geometric constraints; image texture information; semi-global matching

Jingguo LYU,Xingbin YANG,Danlu ZHANG,Shan JIANG. High-resolution Remote Sensing Image Semi-global Matching Method Considering Geometric Constraints of Connection Points and Image Texture Information[J]. Journal of Geodesy and Geoinformation Science, 2021, 4(4): 97-112.

Figures/Tables 21

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Tab.1

Fig.6

Fig.7

Tab.2

Tab.3

Fig.8

Fig.9

Fig.10

Fig.11

Fig.12

Fig.13

Fig.14

Fig.15

Fig.16

Tab.4

Tab.5

References 27

[1]	SCHARSTEIN D, SZELISKI R. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms[J]. International Journal of Computer Vision, 2002, 47(1-3):7-42. doi: 10.1023/A:1014573219977
[2]	KOLMOGOROV V, MONASSE P, TAN P. Kolmogorov and Zabih’s graph cuts stereo matching algorithm[J]. Image Processing on Line, 2014(4):220-251.
[3]	FELZENSZWALB P F, HUTTENLOCHER D P. Efficient belief propagation for early vision[C]// Proceedings of 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington, DC, USA:IEEE, 2004:261-268.
[4]	HIRSCHMULLER H. Accurate and efficient stereo processing by semi-global matching and mutual information[C]// Proceedings of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego, CA, USA:IEEE, 2005:807-814.
[5]	D’ANGELO P. Improving semi-global matching:cost aggregation and confidence measure[J]. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2016,XLI- B1:299-304.
[6]	WENZEL K. Dense image matching for close range photogram-metry[D]. Stuttgart:University of Stuttgart, 2016.
[7]	ROTHERMEL M, WENZEL K, FRITSCH D, et al. SURE:photogrammetric surface reconstruction from imagery[C]// Proceedings of LC3D Workshop. Berlin:[s.n.], 2012.
[8]	KUSCHK G, D’ANGELO P, QIN R, et al. DSM accuracy evaluation for the ISPRS commission I image matching benchmark[J]. The International Archives of the Photogrammetry,Remote Sensing and Spatial Information Sciences, 2014,XL- 1:195-200.
[9]	D’ANGELO P. Evaluation of ZY-3 for DSM and ortho image generation[J]. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2013, XL- 1/W1:57-61.
[10]	GHUFFAR S. Satellite stereo based digital surface model generation using semi global matching in object and image space[J]. ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences, 2016(3-1):63-68.
[11]	BETHMANN F, LUHMANN T. Semi-global matching in object space ISPRS-international Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2015,XL- 3/W2:23-30.
[12]	LUHMANN T, BETHMANN F, HASTEDT H. Dense pointclouds from combined nadir and oblique imagery by object-based semi-global multi-image matching[D]. Oldenburg:Jade University of Applied Sciences, 2017.
[13]	HIRSCHMULLER H. Stereo processing by semiglobal matching and mutual information[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 30(2):328-341. doi: 10.1109/TPAMI.2007.1166
[14]	TATAR N, SAADATSERESHT M, AREFI H, et al. Quasi-epipolar resampling of high resolution satellite stereo imagery for semi global matching[J]. ISPRS-International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2015, XL-1- W5:707-712.
[15]	ZHANG Chunsen, LÜ Peiyu, GUO Bingxuan. Based GCP-SGM algorithm and its application in scene 3D reconstruction of archaeological excavation site[J]. Geomatics and Information Science of Wuhan University, 2015, 40(12):1575-1581.
[16]	ZHANG Yanfeng. Pixel-wise dense matching of aerial images with multi-conditional constraints[D]. Beijing:Chinese Academy of Surveying and Mapping, 2014.
[17]	ROTHERMEL M. Development of A SGM-based multi-view reconstruction framework for aerial imagery[D]. Stuttgart:University of Stuttgart, 2016.
[18]	GONG K, FRITSCH D. A detailed study about digital surface model generation using high resolution satellite stereo imagery[J]. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2016(3-1):69-76.
[19]	YAN Li, FEI Liang, CHEN Changhai, et al. A multi-view dense matching algorithm of high resolution aerial images based on graph network[J]. Acta Geodaetica et Cartographica Sinica, 2016, 45(10):1171-1181.
[20]	LIU Jun, ZHANG Yongsheng, WANG Donghong. Precise positioning of high spatial resolution satellite images based on RPC models[J]. Acta Geodaetica et Cartographica Sinica, 2006, 35(1):30-34.
[21]	LI Deren, ZHANG Guo, JIANG Wanshou, et al. SPOT-5 HRS satellite imagery block adjustment without GCPS or with single GCP[J]. Geomatics and Information Science of Wuhan University, 2006, 31(5):377-381.
[22]	MORGAN M, KIM K O, JEONG S, et al. Epipolar resampling of space-borne linear array scanner scenes using parallel projection[J]. Photogrammetric Engineering & Remote Sensing, 2006, 72(11):1255-1263.
[23]	ZHANG Guo, PAN Hongbo, JIANG Wanshou, et al. Epipolar resampling and epipolar geometry reconstruction of linear array scanner scenes based on RPC model[J]. Remote Sensing for Land & Resources, 2010, 22(4):1-5.
[24]	HIRSCHMULLER H, SCHARSTEIN D. Evaluation of cost functions for stereo matching[C]// Proceedings of 2007 IEEE Conference on Computer Vision and Pattern Recognition. Minneapolis, MN, USA:IEEE, 2007:1-8.
[25]	HUMENBERGER M, ENGELKE T, KUBINGER W. A census-based stereo vision algorithm using modified semi-global matching and plane fitting to improve matching quality[C]// Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition workshops. San Francisco, CA, USA:IEEE, 2010:77-84.
[26]	SPANGENBERG R, LANGNER T, ADFELDT S, et al. Large scale semi-global matching on the CPU[C]// Proceedings of 2014 IEEE Intelligent Vehicles Symposium. Dearborn, MI, USA:IEEE, 2014:195-201.
[27]	YUAN Xiuxiao, CAO Jinshan. Theory and method of precise object positioning of high resolution satellite imagery[M]. Beijing: Science Press, 2012:182-183.

Image resolution 652×665	Initial parallax search range/pixel	Calculation time of cost matrix/s	Calculation time of cost accumulation in 8 directions/s	Calculation time of optimal parallax selection /s	Total time/s
Our method	32	0.14	0.93	0.085	1.15
tSGM	326	0.26	2.51	0.087	2.86
SGM	652	0.61	3.59	0.11	4.31

Error	Left image		Right image
Error	Line	Column	Line	Colum
Maximum	0.3506	0.6534	0.3567	0.6903
Average	0.1693	0.3379	0.1761	0.3474
Standard deviation	0.1025	0.1690	0.1007	0.1803

Experimental area	Maximum absolute error	Average absolute error	Standard deviation error
Flat area	0.6371	0.2820	0.1825
Urban area	0.7343	0.2973	0.3091

Experimental area	Elevation accuracy/m
Experimental area	Average error	Absolute mean error	Root mean squared errors
Plain	-1.6	2.71	3.69
Hilly	-1.71	3.22	4.66
Mountainous	-1.18	3.96	5.62

Stereopair of Wordview3 12984×12928	Input	Module function	Output	Memory cost/G	Calculation time/min
Our method	Stereopair after block adjustment	Dense matching and forward intersection	Dense point cloud	5.46	11
ENVI	Original Stereopair	Block adjustment,dense matching and forward intersection	Dense point cloud and DSM	5.69	9
ERDAS	Stereopair after block adjustment	Dense matching and forward intersection	Dense point cloud	4.47	19