Integration the Low Cost Camera Images with the Google Earth Dataset to Create a 3D Model

As known Close range photogrammetry represents one of the most techniques to create precise 3D model. Metric camera, digital camera, and Laser scanning can be exploited for the photogrammetry with variety level of cost that may be high. In this study, the cost level is taken in to consideration to achieve balance between the cost and the obtained accuracy. This study aims to detect potential of low cost tools for creating 3D model in terms of obtained accuracy and details and comparing it with corresponding studies. Smart phone camera is the most available for everyone; this gave the motivation for use in this study. In addition, Google Earth was used to integrate the 3D model produced from all sides including the roof. Then, two different types of the mobile camera were used in addition to the DSLR camera (Digital Single Lens Reflex) for comparison and analysis purposes. Thus, this research gave flexibility in work and low cost resulting from replacement the metric camera with the smart camera and the unmanned aerial vehicle (UAV) with Google Earth data. Mechanism of the work can be summarized in four steps. Firstly, photogrammetry planning to determine suitable baselines from object and location of targets that measured using GPS and Total station devices. Secondly, collect images using close range photogrammetry technique. Thirdly, processing step to create the 3D model and integrated with Google Earth images using the Agi Photoscan software. Finally, Comparative and evaluation stage to derive the accuracy and quality of the model obtained from this study using statistical analysis method. Regarding this Study, University of Baghdad, central library was selected as the case study. The results of this paper show that the low cost 3D model resulted from integrating phone and Google Earth images gave suitable result with mean accuracy level reached to about less than 5 meters compared with DSLR camera result, this may be used for several applications such as culture heritage and architecture documentation.


Introduction
Close range photogrammetry represents one of the most accurate and low cost technique used for 3D documentation. In photogrammetry, there are many techniques that deal with images to get 2D information used one image also use two images or more to obtain 3D model. It could not get the 3D information from the video or still camera directly without the software to process the image and create model of the area captured. The camera has possibility to get 2D information with high quality images [1]. The incensement possibility of the modern phone in technique drove the collect these phones in group called ''smart phone'', this led to the question if this group capable to use it as a low cost camera in photogrammetry [2]. There is several purpose of the 3D information, the first one is to show the design in 3D space that can assessment of the surface also determine the cost, value, area and test the feasibility of the building. Many years age photogrammetry extraction measurement with high accuracy by dealing with images [3]. Currently the 3D model plays a fundamental role in heritage sites by store the 3D information using photos. Moreover, the 2d images that are required to create 3d model can be obtained from another free source in addition to phone like as Google Earth /Map datasets. For cover mapping or regional land use Google Earth that provide high spatial resolution in free image ,with the development of satellite sensors this led to provide detailed information about the surface targets (size and shape) and the relationship with targets near this surface [4].
One of the popular methods which used in photogrammetry to derive 3D measurement that deal with the principle of parallax based of different photographs that captured from different station point and explanation the shafts in features images this called structure from motion (SFM) [5]. Several investigators studied and discussed the extraction of information from images, Tomasi and Kanade (1992) [6] in their studies which extract 3D information using image sequences by affine factorization method while Debevec et al. (1996) [7] suggested a method that deal with the poses of camera as approximate value and used a fewer images to refines the model. Sun et al. (2013) [8] was investigate algorithms used one or more images of human face from several poses to derive the model of this face based on corresponding points in 2D feature and the depth of this point. Kheyfets (2017) [9] Presented a 3D model of a complex objects (architectural arch & glo-boid worm gear) by using programming language (Auto Lisp). This achieved by writing three programs in AutoCAD package for design, study surface and formation of 3D model. Thus therefore the author recommended for using these programs to solve geometric modelling with complicated tasks. Alidoost et al. (2019) [10] proposed a novel approach for extract 3D reconstruction from single 2D image by using Convolution Neural Network (CNNs). The proposed network able to obtain information and interpret to detect and reconstruction of building. Their experimental data showed the reasonable performance in predicting height value but the accuracy of proposed method was degraded; therefore authors suggested applying their method over different cities.
Dai and Nießner (2019) [11] proposed a novel approach to great 3D mesh model delivered from unstructured range scans by using a graph neural network to generate completed 3D mesh. They applied their method on 8 classes (chairs, tables, sofas, dressers, lamps, boats, cars and airplanes) also compared the results mesh with mesh delivered from state-of-the-art approaches. They concluded from their approach that their meshes sharper and cleaner than other meshes. Burdziakowski and Tysiac (2019) [12] used two different types of data to created model by combined these data. Terrestrial laser scanning and close range photogrammetry techniques were applied on ship hull as a building model. They concluded that this proposed method it was reasonable to obtain high accuracy in engineering measurements of 3D model combined photos and laser scanner data. Boboc et al. (2018) [13] investigated the accuracy of 3D reconstruction delivered from smart phone for cultural heritage. They used Tango constructor application which is available in smart phone to provide 3D model and compared with photogrammetry -based 3D using commercial software. Their results showed that Tango-based method able to provide morphometric data and efficient way for 3D reconstruction additionally fast collection and generation of 3D model but limitation the accuracy of 3D model was influenced. Bruno and Roncella (2019) [14] assessed the actual accuracy of 3D model delivered from Google Street view panoramas. The authors used three different case studies with different geometric characteristics to test and evaluate the accuracy of model. Their search results experimental significant variability due to geometry of image and GPS measurements, therefore they recommended to improve the accuracy of 3D model by using the observation delivered from GPS as ground control in orientation solution. Any user can get aerial images; Google earth offered this image over any area. Verhoeven (2009) [15] used the Balloon and Kite as a platform to derive the image was used in cultural heritage, Kite used when the wind speed is higher while if the wind speed in an area is low the Balloon was used. Smith et al. (2009) [16] used kite aerial photogrammetry with small topographic map. In the last time unmanned aerial vehicle (UAVs) present a practical challenge, Li et al. (2018) [17] enhanced the 3D reconstruction delivered from UAV by using a loop-shooting aided technology. This technology redounded to process the missing details in 3D model which caused by airflow or distribution during capture the image by UAV. The results obtained during their search proved the high accuracy in 3D model additionally simplifies the process of refining the 3D model. Lubis et al. (2017) [18] presented a tool of 3D modelling of buildings in Google Earth and GIS for Batam City, Tunjuk Island, Indonesia. Authors depend on combination method between Google Sketch Up and GIS tool based on data that delivered by researchers or by a professional body, or by user groups and volunteers. The aim of their research to support users with decision making regarding planning or design of Batam City and to provide knowledge about the size, location, and design of the a certain building. This research lacks an assessment of the accuracy obtained from the 3D model due to use an open source data with different level of accuracy, so it cannot be used in applications of precise surveying contrary to this research, also Palanirajan et al. (2019) [19] were proposed a new algorithms to generate 3D building model by using images delivered from UAV as an alternative approach from aerial laser scanners and they get a reliable and complete point cloud of the building dependence on an optimal flight plan. This technique provide a high resolution images with a flexible data source the investigator used this techniques to create the 3D modelling of man mad structures applied the SFM method on UAVs techniques, however the use of UAVs is limited because of their need for security approvals and is very expensive, therefore the aim of this paper is to replace data that obtained from UAVs with the data from Google Earth and integrate this with data delivered from low cost camera to build 3D model.

Study Area and Data Set
The location of the work that applied the proposed\ Baghdad\ Iraq the dimension of this location approximate (64m,58m) width and length receptivity Figure 1 show the study area approach is the central library of the university of Baghdad in Aljadria.

Figure 1. The Central library/ University of Baghdad
The data which used in this paper were collected by different sensors smart phone camera (I phone 6 plus and Galaxy note 3) with resolution (1.01, 1.15 m) receptivity. Additionally, the DSLR camera with the resolution (1.02 m) was used for comparing purpose between the models delivered from different sensors. Finally, the Google Earth was exploded to complete the model with the roof.

Research Methodology
Three-dimensional reconstruction of object delivered from images required several steps that must be accomplished to get the 3D model; this will be summarized in this methodology and flowchart. The first step is to configure the Ground Control Points (GCPs) around the study area this achieved using geodesy receivers (Topcon GR5) based on the static GNSS positioning technique with two hours periodic time. The next step involved the observing points on the building or model, these points used for processing and assessment the new approach therefore we design target 10×10 cm Figure 2 show the design target, these target distributed throughout the building to get more accurate points, this step was performed using Total Station (Topcon) instrument . It afterword, the role of images collect comes that requires the determination of the distance from building to user (H) when the image was captured from this point or station also determine the distance between these station (B) to get the best overlap. Subsequently, the images were captured by low cost camera (phone), which used different mobile and covered the object with sequence images from the same points. At the same time, DSLR camera (Digital Single Lens Reflex) was used to capture the images from these stations to assess the photos delivered from smart phone and answer the question if can use these smart phone as a close range photogrammetry, at the end of this step the survey practical has ended. After collect the images from different positions by two different smart phone and digital camera, the photos must be process to extract the 3D model using Agi Photoscan software, it is an important environment to extract a precise 3D model Figure 3 show the interface of the software which used. There are some processes in this software, these processes consist of the alignment of the photos after loaded it's in Photoscan, at this stage the points will detect, the positions of camera will estimate and the match between photos will achieve. To get more detailed Photoscan could produce extra dense point cloud this option available in Photoscan environment from workflow menu. After then the markers creation on the points which measured by total station device (not all the points because some points used as a check points for the accuracy assessment), the next step the software create the three-dimensional mesh based on dense point cloud Figure 4 explain the distribution of marker on the selected object. Finally the texture of an object was built also are getting three-dimensional model, all these steps in one were chunk in Photoscan software, these procedure applied three times on images (two for smart phone and another one for digital camera). In spite of the building height, we were able to get a model for the object from all sides without the roof. Thus, the approach, which proposed in this paper, is summarized by taking data from Google Earth to integrate these data with the model delivered from images in the first chunk Figure 5 show one of the image delivered from Google Earth. Thereafter, another chunk was created to process the images of Google Earth including the same process of the first chunk. This result from this process is applied to get texture for the roof of the object. In this time, there are two data belong to the same object and to get reconstructed 3D model with extra details, therefore these two chunks (two data) were merged. It is worth to mention that Photoscan provide option to do this processes, which explain in Figure  6 contains all these options.

Figure 6. Present the tab workflow of the menu bar
To assess the accuracy of this approach several points were measured coordinate from Photoscan and compared the results with the original coordinate measured by total station, and determine the validity of smart phone used as a close range photogrammetry. Figure 7 explains all these steps in a flowchart.

Visual Analysis
In this step, the results of the Agi Photo Scan process regarding the two kinds of mobile can be viewed to be visually analyzed. The three main steps to create the 3D model from the mobiles and DSLR camera, which mentioned in the work methodology, can be seen in the Figures 8 and 9 respectively.  Figures 8 and 9, it can be noticed the difference between them. It can noticed the distortion in Figure 9 due to incomplete of the model in this side, which is pointed to by a circle. This distortion resulted from the insufficient number of covered images of the study area determined according to the program's ability to process within a specified number of DSLR camera images (raw image) depending on the specifications of the computer used. Due to lightness of mobile images (jpg extension) and speed in processing, this led to an increase in the number of images taken nearly two times than the raw image of the DSLR camera, which is lead to eliminate the distortion previously mentioned as shown in Figure 8. The problem of incomplete 3D model can also be seen in the final model (texture model) in Figure 10 which displays the 3D texture model for the both types of mobile and DSLR camera. Moreover, this research aims to integrate the 3D model produced from the images of mobile with images of Google Earth to get a complete model from all sides, including the roof. Accordingly, the Agi Photoscan software was used to integrate within the process steps mentioned previously in the study methodology. Thus, final 3D model resulted from integrating between mobile and Google Earth images can be seen in Figure 11.

Statistical Analyses
Regarding this study, the reference markers, which distributed on the whole body of the Central Library from all sides as shown in Figure 12, were measured as a reference points using GPS and Total station techniques. Some of the reference markers were exploited for evaluation purpose by calculated the differences between the references markers and the corresponding positions of the created 3D models resulted from each mobile and DSLR camera images as illustrated in Figure 13. These individual differences are also called residuals, and the Root Mean Square Error serves to aggregate them into a single measure of predictive power, this measures how much error there is between two data sets. In other words, Root Mean Square Error compares a predicted value and an observed or known value. Findings of this statistical test show that the DSLR camera delivered most accurate than mobile camera, where the RMSE of the DSLR camera are (0.4), (0.8), (0.3) m in x, y, z direction respectively while the RMSE in x, y, z direction regarding the mobile camera are (2.0), (7.7), (2.1) m.  It is worth to mention that we use another test to understand the behaviour of the data Outlier Identification using IQR (Inter Quartile) in Real-Time Data , we need to examine how we can compute efficiently in large data bases therefore these descriptive statistics are very useful for understanding the distribution of the data. In this test, some of parameters (min, max, median or mean, Q1, Q3) were calculated; these parameters showed that there are no outliers in these data below listed in Table 1.

Conclusion
This study presents a complete methodology for creating the 3D model using integration method between mobile and Google Earth images. Moreover, this study provides a full impression of the ability of integrating the mobile images with the Google Earth data to produce a 3D model depending on visual and statistical analysis approach.
Where the results of this study showed the possibility of producing a completed 3D model and high quality compared with the 3D DSLR camera model, while the position accuracy of the 3D mobile less than the accuracy of the DSLR model.
Consequently, the resultants of RMSE in 3 directions were nearly 0.9 m regarding DSLR camera and 8 m relating to mobile camera. Thus, this study contributed to give a clear practical indication of the possibility of using this methodology in various engineering applications that are in line with the obtained accuracy.