Next Article in Journal
Evaluating Geo-Tagged Twitter Data to Analyze Tourist Flows in Styria, Austria
Next Article in Special Issue
Developing Versatile Graphic Map Load Metrics
Previous Article in Journal
Building Change Detection Using a Shape Context Similarity Model for LiDAR Data
Previous Article in Special Issue
A Simplified Method of Cartographic Visualisation of Buildings’ Interiors (2D+) for Navigation Applications
Article

VisWebDrone: A Web Application for UAV Photogrammetry Based on Open-Source Software

1
Engineering Department, School of Science and Technology, University of Trás-os-Montes e Alto Douro, 5000-801 Vila Real, Portugal
2
Centre for the Research and Technology of Agro-Environmental and Biological Sciences, CITAB, Universidade de Trás-os-Montes e Alto Douro, UTAD, 5000-801 Vila Real, Portugal
3
Centre for Robotics in Industry and Intelligent Systems (CRIIS), INESC Technology and Science (INESC-TEC), 4200-465 Porto, Portugal
*
Author to whom correspondence should be addressed.
ISPRS Int. J. Geo-Inf. 2020, 9(11), 679; https://0-doi-org.brum.beds.ac.uk/10.3390/ijgi9110679
Received: 24 September 2020 / Revised: 27 October 2020 / Accepted: 13 November 2020 / Published: 15 November 2020
(This article belongs to the Special Issue Geovisualization and Map Design)

Abstract

Currently, the use of free and open-source software is increasing. The flexibility, availability, and maturity of this software could be a key driver to develop useful and interesting solutions. In general, open-source solutions solve specific tasks that can replace commercial solutions, which are often very expensive. This is even more noticeable in areas requiring analysis and manipulation/visualization of a large volume of data. Considering that there is a major gap in the development of web applications for photogrammetric processing, based on open-source technologies that offer quality results, the application presented in this article is intended to explore this niche. Thus, in this article a solution for photogrammetric processing is presented, based on the integration of MicMac, GeoServer, Leaflet, and Potree software. The implemented architecture, focusing on open-source software for data processing and for graphical manipulation, visualization, measuring, and analysis, is presented in detail. To assess the results produced by the proposed web application, a case study is presented, using imagery acquired from an unmanned aerial vehicle in two different areas.
Keywords: open-source; web application; photogrammetry; unmanned aerial vehicles; MicMac; GeoServer; Leaflet; Potree open-source; web application; photogrammetry; unmanned aerial vehicles; MicMac; GeoServer; Leaflet; Potree

1. Introduction

Currently, a wide range of geodetic procedures and methods to record the actual state of objects on the Earth surface is available [1]. Photogrammetry is one of these methodologies, which is considered to be the art, science, and technology of acquiring geometric information related to physical objects through processes of recording, measuring, and interpreting photographs and patterns [2]. These processes include automated interest-point detection [3], Structure from Motion (SfM) algorithms [4], and dense image-matching techniques [5]. To improve and speed up photogrammetric processing, several commercial and open-source libraries and pieces of software have been developed [6,7]. Among the most used commercial software, Pix4Dmapper (Pix4D SA, Lausanne, Switzerland), Agisoft Metashape (Agisoft LLC, St. Petersburg, Russia), and DroneDeploy (DroneDeploy, San Francisco, CA, USA) stand out. Some examples of open-source software and libraries include MicMac [6], VisualSFM [8], Bundler [9], Python Photogrammetry Toolbox [10], and Open Drone Map [11], which are based on SfM techniques [12]. Despite the availability of several solutions for photogrammetric processing, the cost and the performance associated with each solution could be a deciding factor [13]. Regarding the cost of commercial software, prices vary, DroneDeploy being the best option from an economic point of view. In contrast, open-source solutions are free and can be a feasible solution for several users. Despite the high costs associated with commercial software, it is important to consider that software providers make all of needed functionalities available off-the-shelf in a single software package, whereas free and open-source software (FOSS) usually offer a single category of functionalities [14]. In this way, the development of a global and complete application involves the use of different open-source libraries, which has become a great challenge for developers [15].
In the last decade, Internet access through devices (computers, smartphones, and tablets) has increased significantly [16]. For this reason, many organizations are considering the Internet as the main platform for software solutions, creating software as a service (SaaS). This type of platform presents several advantages and it is worth highlighting its ability to maintain and update the applications without distributing and installing any software on client computers [17].
Considering the commercial photogrammetric SaaS, Pix4D Cloud, DroneDeploy, and 4DMapper stand out. Some free photogrammetric web-interfaces (e.g., ARC 3D Webservice, MiaW, Culture 3D Cloud Platform, CMP SfM web service, 3DNOW) are available. However, there is a lack of open-source solutions that offer a complete and integrated solution for the processing and visualization of aerial imagery acquired from unmanned aerial vehicles (UAVs).
As a result, considering the aforementioned advantages, the aim of the proposed research is to present VisWebDrone (visual web platform for drone-based imagery analysis), a web application for UAV photogrammetry based on open-source technologies. The proposed solution will be maintained as a free and open-source platform, available on GitHub, relying on the users contribute. The web application is composed of several pieces of open-source software that are completely integrated, combining the data-processing stage with the graphical presentation of the results. The data-processing stage is performed using MicMac, while the graphical presentation of the results is provided by Leaflet and Potree. Several Graphical User Interface (GUI) solutions, integrating MicMac, have already been developed. Among the solutions developed, the CEREMA (Centre d’études et d’expertise sur les risques, l’environnement, la mobilité et l’aménagemen) has released the AperoDeDenis, a GUI for MicMac that is under development. Another GUI, the InterfaceMicmac, developed by IGN (Institut national de l’information géographique et forestière) was discontinued. Recently, Alessio Calantropio of Politecnico di Torino developed the GEMINI (Graphically Enhanced MicMac’s New Interface) solution, which is focused on the quality assessment of the achieved results [18]. Although all GUI solutions presented applied the abstracting command line provided by MicMac software, none of them has available graphical solutions for the manipulation and visualization of the achieved results, using photogrammetric processing (e.g., visualization of orthophoto mosaics and point clouds). VisWebDrone intends to fill this gap, providing the appropriate tools for this purpose.
Following this introductory section, Section 2 describes the VisWebDrone structure and architecture, presenting the modules and software integrated into the web application. Section 3 is dedicated to the demonstration of the implemented solutions presented in Section 2. Then, to exemplify and test the web application functionalities, two study areas are presented in Section 4. In Section 5, the results are presented and discussed. Finally, in Section 6 the conclusions of the present research are addressed.

2. System Architecture

The conception of a web application for manipulation, analysis, and visualization of UAV-based imagery outputs requires four modules. Module 1 is responsible for the photogrammetric processing, receiving as inputs the images collected by UAVs and producing as outputs an orthophoto mosaic and a 3D dense point cloud. Next, Module 2 is used to store the orthophoto mosaic in a map server. This module receives as input the orthophoto, generated in Module 1, and makes available several web services for geographical data publishing. Module 3 is responsible for orthophoto mosaic visualization, using a Web Map Service (WMS) layer as input and produces a web-mapping page for orthophoto visualization as output. Finally, Module 4 deals with dense point cloud visualization and analysis. This module receives the point cloud data, provided by Module 1, and builds a webpage for its visualization. Figure 1 presents the flowchart of the modules.

2.1. Module 1—Photogrammetric Processing

Among the different photogrammetric packages freely available, OpenDroneMap (ODM) and MicMac were considered the most suitable options for the development of the proposed web application. This choice is mainly because both integrate tools allowing the generation of orthophoto mosaics and point clouds, in the same pipeline.
ODM is a community-based open-source toolkit, developed for processing and analyzing aerial imagery acquired by UAVs. This command-line-based program runs on all major operating systems through the use of Docker container (Docker, Inc., Palo Alto, CA, USA) and is distributed under the GPLv3 license [19].
MicMac is associated with the acronym “Multi Images Correspondances par Méthodes Automatiques de Corrélation—Multi Image Matches by Automatic Correlation Methods” and its development began in 2005, by the French National Institute of Geographic and Forestry Information (IGN) and the French National School of Geographic Sciences (ENSG) [6]. MicMac is an open-source software distributed under the CeCILL-B license. Initially, it was provided as an Extensible Markup Language (XML) framework, allowing the processing parametrization to be defined by users. However, in 2010, the XML framework was discontinued, giving way to a simplified command-line interface that has been contributing to an increase of the use of this open-source software. MicMac is a versatile software, which can be used in different types of objects, from small objects acquired from the ground perspective to buildings acquired from an aerial perspective. Even imagery form regional areas, such as cities or natural places, acquired from manned aircraft or satellites, can be used [20].
Both ODM and MicMac are free and open-source solutions. However, the accuracy of 3D point clouds is vital in VisWebDrone, allowing the execution of accurate three-dimensional measurements. For this purpose, a preliminary comparative test was performed between both solutions. In ODM, the depthmap-resolution parameter controls the density of the point cloud by setting the resolution of the depthmap images. Therefore, a higher value for this parameter (1200) was defined, according to the ODM community suggestions. Regarding the MicMac parameters, the C3DC tool with BigMac parameter was applied, which presents a higher preset of 1pt/4px. Considering the results, the point cloud data generated by MicMac presents a higher density of points (12,462,884 points), as well as a better quality and definition when compared to ODM results (10,851,334 points). Figure 2 depicts the dense point clouds generated using the same imagery dataset in both solutions. Regarding the performance of both software in the production of orthophoto mosaic, the results presented were good and met the expectations. Therefore, for the reasons presented above, MicMac was selected to perform photogrammetric processing.

2.2. Module 2—Orthophoto Mosaic Storage in a Web GIS Server

For the task of storing raster generated data in Module 1, two web Geographic Information System (GIS) servers stand out as some of the most used in FOSS communities, the MapServer and the GeoServer.
MapServer was founded in 1996, at the University of Minnesota. It is an open-source web-mapping project, designed for developing interactive and interoperable web-mapping applications. MapServer offers support for several Open Geospatial Consortium (OGC) standards, such as WMS, Web Feature Service (WFS), and Web Coverage Service (WCS), and is compatible with a wide variety of vector and raster formats. This web server runs as a common gateway interface (CGI) application with Apache and Microsoft web servers [21].
GeoServer, in turn, was founded in 2001, belonging to the Lime Group, the Open Planning Project, and Refractions Research. This platform plays a crucial role in the management, storage and presentation of geospatial data over the Internet. Users can upload data into the GeoServer using a web administration tool, a database, or gsconfig—a Python library for manipulating a GeoServer instance via the GeoServer RESTConfig API. As a web server, it integrates an interface where users can search for layers, consult data provided by database or manage the web services used to share raster and vector data [22]. GeoServer serves spatial data based on OGC standards: WMS for the display of maps as images, WFS for vector data, WCS for raster data, Web Processing Service (WPS) for invoking geospatial processing services, and WFS-T for edition that allows the creation, deletion, and updating of features. GeoServer is a RESTful server where every resource could be requested using a Universal Resource Identifier (URI), which is dependent on the web service defined. Regarding WMS requests, maps could be provided in several formats, such as Geotiff, KML, and Portable Network Graphics (PNG). On the other hand, in WFS requests, the vector data could be provided in formats such as Geography Markup Language (GML), GeoJSON, and Shapefile [23].
Regarding the comparison between both web GIS servers presented in this subsection, they are both similar in the supported OGC standards. However, GeoServer offers WFS-T, which is essential to edit processing results, allowing feature creation, deletion, and updating. MapServer, in contrast, does not offer this important service. Moreover, another important advantage of GeoServer is that it has a web administration tool available that optimizes the user experience which makes it possible to search for layers, consult data provided by database, or manage the web services used to share raster and vector data [24]. For the above-mentioned reasons, GeoServer was elected as the web GIS server to store orthophoto mosaics.

2.3. Module 3—Orthophoto Mosaic Visualization

Web-mapping Application Programming Interfaces (APIs) are function libraries that are usually based on JavaScript. These APIs include tools that facilitate data fusion, provided from different resources, through their acquisition, manipulation, and visualization [25]. To visualize orthophoto mosaics, generated in the Module 1 of the proposed web application, two frequently used web-mapping APIs were analyzed—the OpenLayers and the Leaflet. The selection of these particular web-mapping APIs is because of good support and documentation provided for the developers, which contributes to the increase in number of communities that are using these APIs.
OpenLayers is a JavaScript library used for web map creation. This solution implements a JavaScript API for building rich web-based geographic applications, which is one of the mostly used open-source technologies. It also implements industry-standard methods for geographic data access, such as the OGC, WMS, and WFS standards [26].
Leaflet, in turn, is similar to OpenLayers, being a JavaScript library developed to create web maps. This library includes a wide variety of functions allowing the creation of simple web maps or complex and interactive web maps offering users changing/edition possibilities [27].
Considering the characteristics of each web-mapping API, it can be noted that Leaflet API has much shorter code than OpenLayers for doing the same process, which makes it faster. Moreover, Leaflet supports OpenStreetMap as well as other map providers. For these reasons, Leaflet was considered the most feasible API to load and visualize orthophoto mosaics.

2.4. Module 4—3D Point Cloud Visualization

To allow the 3D point cloud visualization in VisWebDrone, point cloud viewers were analyzed. Despite the lack of FOSS software for this purpose, two solutions were identified: Potree and Plasio technologies [28].
Potree is a web-based point cloud renderer allowing the visualization of billions of points. It simplifies the dissemination of point cloud data, once it is presented in a web page, without requiring the existence of additional software [29]. Potree depends on WebGL technology, which is a JavaScript API compatible with HTML5. WebGL is a variation of OpenGL but is adapted to web browsers, allowing the Graphics Processing Unit (GPU) rendering capabilities to be applied to web pages. Potree also depends on three.js library which consists of a rendering library used to manipulate scene graphs and draw calls [30].
Plasio is also based on WebGL technology, being a web-based point cloud viewer that supports LASer (LAS) files and its compressed version LASer zip (LAZ). To use it for displaying single files, its development team is working on a multi-resolution indexing and rendering stack applied to large point clouds [31].
Potree was the chosen software for the point cloud visualization, once it proved to be the most robust platform, offering a wide set of tools for point cloud visualization, manipulation, and analysis.

2.5. Software Integration

For the web application development, a Representational State Transfer (REST) architecture was implemented, which enables communications between client and server sides, performed through HyperText Transfer Protocol (HTTP) methods [32]. The client side is associated with the web application interface, responsible for enabling users to upload images on a system and define the processing parameters. Users can select a complete photogrammetric processing pipeline that generates the orthophoto mosaic and the point cloud, or select each one individually. The defined processing pipeline, in turn, is sent as a request for the server side, outlining the processing sequence that will be done by MicMac. This process is executed through several PHP and Ajax scripts. Once data processing performed by MicMac is complete, it is possible to obtain the orthophoto mosaic that will be stored on a GeoServer instance, using the gsconfig Python library. GeoServer has a set of web services available that can be used to share raster data in web applications. In this case, the WMS is used through the request of Leaflet. The point cloud is rendered in the web page, using Potree. This process is illustrated in Figure 3.
The web application interface intends to promote a user-friendly experience, presenting a simple, intuitive, and clean design. VisWebDrone is composed of a section to select and upload images collected by UAVs (Figure 4a), a section to select the type of output to process (Figure 4b)—orthophoto mosaic, point cloud, or both—and another section to visualize the results of photogrammetric processing (Figure 4c).

3. Software Implementation

This section presents MicMac, GeoServer, Leaflet, and Potree functionalities and the way they were implemented in the VisWebDrone solution.

3.1. MicMac Implementation

MicMac implements photogrammetry based on SfM algorithms, consisting of two successive steps. The first step is related to the detection and matching of automatic tie points while the second step is associated with the bundle adjustment that corresponds to the computation of viewpoints relative locations (camera positions) [33]. These steps are integrated in MicMac tools, presented in Figure 4. MicMac is very flexible, offering several tools that can be selected to define a complete and appropriate workflow. The workflow used for imagery processing (Figure 5) is based on the following steps:
  • Tie points are identified using the Tapioca tool. This procedure is based on the identification of common features present in the images. Considering the higher computation time on large sets of images, the MulScale mode can reduce the processing time, after the first computation of tie points is performed at a low resolution.
  • Tapas is then used for calculating the relative orientation of images, using the identified tie points, by computing the position and the distortion model of the camera optics;
  • AperiCloud is used to produce a Polygon File (PLY) with the position of cameras;
  • The next step could depend on the usage of ground control points (GCPs). If nonexistent CenterBascule is used, if GCPs are considered to be SaisieAppuisInitQT, and GCPBascule tools can be applied, CenterBascule transforms a relative orientation into an absolute one, using embedded Global Positioning System (GPS), while SaisieAppuisInitQT tool is applied to perform the GCPs marking and then GCPBascule executes the same process as CenterBascule but, in this case, also using GCPs. The web application documentation will explain how users can select the images and the GCP to be marked. The web application activates the MicMac external tool to perform the GCP marking.
  • Campari is responsible for bundle adjustment using tie points and GCPs;
  • ChgSysCo is used to define the final coordinate system of orientation; an XML file that specifies the European Petroleum Survey Group (EPSG) code associated with the coordinate system is used in this step.
  • To obtain the point cloud data, theC3DC tool, provided by Culture 3D Cloud project, was applied [34]. This tool computes an automatic matching, generating a point cloud, in PLY format, from a set of oriented images. The Pims2Mnt tool was responsible for the merging of the individual depth maps into a global Digital Surface Model. To produce the orthophoto mosaic, the Tawny tool was used. The orthophoto mosaic was generated in TIF format and is based on individual orthophotos [35].

3.2. GeoServer and Leaflet Integration

The communication between Leaflet (client side) and GeoServer (server side) is performed through the use of GetMap requests that are used to load the geospatial data, stored in server, into the web-mapping page (Figure 6). The specifications of GetMap requests are defined by Leaflet library, using the WMS standard that is based on HTTP methods [36].

3.3. Potree Implementation

In Potree, a wide set of tools for navigating, measuring, and analyzing the point cloud data are available, contributing to enhancing this powerful open-source solution. The Point-Picking tool is used for navigation and interaction operations, such as in zooming to a point or in the creation of measurements. The Navigation tool offers several navigation modes to be adapted to all situations. The first mode, OrbitControls, offers a set of controls that allows orbiting around a target or pivot. The second mode, FirstPersonControls, allows a fly- or walk-through-like experience, which could be very useful to navigate through enclosed spaces or vast landscapes. The last mode, EarthControls, is very similar to the mode provided by Google Earth. The left mouse button allows the drag and drop movements, and the right mouse button allows users to rotate around the clicked locations. The Clip-Boxes tool allows the catchment of areas of interest using a box for selection, highlighting only the defined area. The Measurement tool allows the measurement of distances, areas, heights, and angles. The performed measures could be saved in JSON or DXF formats [37]. The Height Profile tool is used to define several segments, to obtain the heights in the defined locations. Finally, the Annotations tool allows the creation of annotations in point cloud data, highlighting some specific interesting points.

4. Case Study

To explore and test the web application functionalities, two study areas, with data collected from UAV, were selected and are presented in this section. The data were collected on the campus of the University of Trás-os-Montes e Alto Douro, Vila Real, Portugal. The first study area is related to a set of agronomical trials, while the second includes building areas (Figure 7). The idea behind this selection is to evaluate the performance of the solution in rural environments, where geometric entities are scarce and in semi-urban environments, where the existence of these geometric structures has the potential to improve the performance of these solutions, as they are easily recognizable. Thus, the performance of the proposed application can be tested in most common application cases.

4.1. Data Acquisition

The images were acquired using a commercial unmanned aerial system (UAS), the DJI Phantom 4 (DJI, Shenzhen, China). It is a cost-effective solution that is prepared to execute manual or fully automatic flights, using different configurations through a set of user-defined waypoints. DJI Phantom 4 is a multi-rotor UAV, equipped with an RGB sensor with 1/2.3’’ CMOS that has a resolution of 12.4 MP. The focal length of the camera is 3.6 mm (35 mm equivalent: 20.0 mm). The images collected are 4000 × 3000 pixels, and latitude, longitude, and altitude information were provided by the on-board Global Navigation Satellite System (GNSS) receiver.
For the first study area, 89 images were acquired in a single-grid configuration at a flight height of 40 m, with a longitudinal overlapping between images of 80% and a lateral overlapping of 70%, while in the second study area a double-grid flight configuration was planned at a height of 80 m (324 images acquired), giving a longitudinal overlapping between images of 90% and lateral overlapping of 75%.
To correct the errors of the UAV GNSS receiver, GCPs were acquired in the PT-TM06/ETRS89 coordinate system, using the Mobilemapper 100 GNSS receiver (Ashtech Inc., CA, USA) in RTK mode and were used for imagery alignment during photogrammetric processing. In the first and second study areas, 8 and 5 GCPs were acquired, respectively.

4.2. Data Processing, Aligment, and Visualization

The data acquired in the two study areas were processed using MicMac. However, to allow a comparison with commercial solutions, the photogrammetric software Pix4Dmapper Pro was also used. Thus, it was possible to compare the processing time and to assess the quality of the orthorectified mosaic and the point cloud data. The photogrammetric processing in both solutions was performed locally in the same workstation equipped with two Intel(R) Xeon(R) CPU E5-2680 v4, two NVIDIA Quadro M4000 GPUs, 128 GB RAM and a 1 TB SDD in the Windows 10 Enterprise operating system.

4.2.1. Data Processing Using Pix4Dmapper Pro

The photogrammetric processing using the Pix4Dmapper Pro (version 4.5.6) was based on three main processes. In the first process (Initial Processing), the digital photo camera was calibrated, and images were oriented. Considering the relative orientation, tie points were automatically measured through the stereo pairs in overlapping areas. On the next step, GCPs were entered and measured automatically, with a manual correction in the position of these points. Having concluded the relative orientation, the aerial triangulation step was performed. The triangulation was an automatic step, where tie points were transformed into geodetic coordinates system. In the second process (Point Cloud and Mesh), the point cloud was generated, combining the filtering and smoothing steps to determine point cloud densification, avoiding noisy and erroneous points. In the third process (Digital Surface Model (DSM), Orthomosaic and Index), the generation of DSM and the orthophoto mosaic included several sections allowing the definition of the data format and the specifications of outputs.

4.2.2. Data Processing Using MicMac

Initially, the ExifTool software was used to read the metadata associated with the images collected by UAV. These data provided important information, such as the focal length of the camera and the absolute and the relative altitude and the GNSS coordinates, which were used to locate each image in a basemap presented in a web application.
Following the initial procedure, tie points identification was performed using the Tapioca tool. This tool required the identification of two mandatory arguments, the directory/path of images and the image size, in pixels. After the identification of tie points, these were used by the Tapas tool to calculate the relative orientation of images and the position and distortion model of the camera’s optics. The Tapas tool required, as mandatory arguments, the identification of calibration model (FraserBasic) and the directory/path of images. On the next process, a PLY file with the position of the cameras was generated, using the AperiCloud tool. The following step consisted of the use of SaisieAppuisInitQT for GCPs marking, following by the GCPBascule tool, to transform the relative orientation into an absolute one, based on GCP data. Then, the Campari tool was applied to execute the bundle adjustment made using camera positions and tie points. ChgSysCo was then used to define the final coordinate system. For this purpose, an XML file that identified the EPSG code parameters was created. The defined code was the EPSG: 3763 (ETRS89/ PT-TM06), which is an official coordinate system of Portugal and the same as the GCP. After this step, the C3DC tool was used to perform an automatic match, generating a high-density point cloud, in PLY format, from a set of oriented images. Finally, photogrammetric processing was concluded by the Tawny tool, which was used to produce the orthophoto mosaic in a TIF format.

4.2.3. Visualization of Data Using Leaflet and Potree

On successful completion of data processing, the orthophoto mosaic was uploaded into the GeoServer using the gsconfig Python library. This library allowed the automatic configuration of layer style, as well as the definition of the final coordinate system. Once this procedure was concluded, the communication was implemented between the server (GeoServer) and the Leaflet using the WMS service. On the Leaflet library, the coordinates of the base map were always updated, according to the coordinates associated with the orthophoto mosaic.
Regarding to Potree, the PotreeConverter tool was applied to transform the point cloud in PLY format into a WebGL HTML file optimized to visualize and manipulate the data.

4.2.4. Data Alignment

To evaluate the point cloud accuracy, the mean error and the Root Mean Square Error (RMSE) was calculated as in Equation (1), using Microsoft Excel. RMSE measures the difference between forecast and corresponding observed values, giving a relatively high weight to large errors. In this way, the RMSE is most feasible when large errors are particularly undesirable [38]. Usually, photogrammetry projects require a good accuracy and the existence of lower errors. Therefore, accuracy evaluation through RMSE is essential.
R M S E   = i = 1 n e i 2 n
where ei, represents the error of each point in a given direction (X, Y, Z), and n the number of Check Points (CPs).

5. Results and Discussion

5.1. Orthophoto Mosaic and Point Cloud Results

Considering the first study area, the orthophoto mosaic generated (Figure 8a) was created with a Ground Sampling Distance (GSD) of 1.78 cm, making it possible to visualize the surveyed area with great detail. Regarding the second study area, a lower spatial resolution was obtained on the orthophoto mosaic (Figure 8b) with a GSD of 3.70 cm. Considering the orthophoto mosaic observations, problems in transition areas were not noticeable in the region of interest, as it is not presenting displacement areas, and verifying a good balance in brightness levels in both cases. However, outside the region of interest, where a fewer number of images were acquired, some radiometric or coloring variations are more noticeable close to the borders. These issues were already observed in other studies [39].
Considering the point cloud data, in first study area (Figure 9a), a total of 12,462,884 points were generated. Regarding the point cloud data produced in second study area (Figure 9b), a total of 15,637,246 points were generated. Both cases high-quality dense point clouds were generated. However, some points from dense tall vegetation were not estimated in the second study area.

5.2. Data Alignment and Photogrammetric Processing Time Evaluation

The mean error and RMSE in X, Y, and Z directions were calculated in for both study areas. In this way, several points were identified in the point cloud data and the coordinates of their centers were obtained. These coordinates were then compared with the corresponding GCPs, obtained in the field with the GNSS equipment. The results provided by MicMac were also compared with those obtained by Pix4Dmapper Pro. Table 1 presents the comparative results.
Considering first study area, the accuracy obtained in the processing carried out by MicMac is lower than the GSD, respectively RMSEX = 1.18 cm, RMSEY = 0.90 cm, and RMSEZ = 2.15 cm. Regarding Pix4Dmapper Pro results, the RMSE is slightly lower in all directions, respectively RMSEX = 0.92 cm, RMSEY = 0.67 cm, and RMSEZ = 1.92 cm. However, these results prove the effectiveness of MicMac in photogrammetric processing, since the results are similar to those obtained using a commercial solution and the errors are below the GSD. Regarding the second study area, MicMac results continue to be comparable to Pix4Dmapper Pro, again with a slight advantage for the commercial solution. However, MicMac results respect the requirements by a wide margin. In fact, MicMac achieved individual RMSE of RMSEX =1.95 cm, RMSEY = 1.55 cm and RMSEZ = 2.29 cm, which are in line with those obtained by Pix4Dmapper Pro, respectively RMSEX = 1.66 cm, RMSEY = 1.43 cm, and RMSEZ = 2.17 cm.
According to the literature related to absolute accuracy evaluation [40,41,42], an error of 1–2 times the GSD horizontally and 1–3 times the GSD vertically could be expected, which means that the accuracy obtained with MicMac is in line with these expected values. In this study, we purposely selected mixed areas, where anthropogenic objects occupied rural zones, to assess the global behavior of the analyzed solutions. However, in other studies, in which more homogeneous areas were analyzed, MicMac presented good accuracies, in some cases even better than commercial solutions [43,44,45].
A comparative evaluation of photogrammetric processing time executed by Pix4Dmapper Pro and MicMac was also carried out. The execution time of the two solutions for carrying out the two most relevant tasks are presented in Table 2. Photogrammetric processing was performed with high-quality parameters in both pieces of software, contributing to higher computational costs. Regarding the Pix4Dmapper Pro software, the processing time was faster, since it used resources from both GPU and CPU. On the other hand, MicMac only used the CPU resources, which contributed to a longer processing time. Moreover, the MicMac configurations allow extremely detailed tie point detection, which also contributes to time spent in photogrammetric processing [39,45].
In Table 2, the photogrammetric processing time is proportional to the image number included on the web application. However, the platform works perfectly with a higher quantity of images.
The higher processing times (approximately twice the time), obtained by MicMac, can be considered to be a disadvantage of this solution. However, this software is free and open-source, offering high-quality results with good accuracies that are in line with the accuracy thresholds defined in the literature.
During the photogrammetric processing, Pastis/Tapioca and Apero/tapas were the slowest processes of the pipeline. Several algorithms, such as OriRedTieP, are in development and could optimize and improve the photogrammetric workflows.

6. Conclusions

In the upcoming years, web-based software, services, and applications are expected to increase. The developed web application, presented in this paper, is based solely on open-source technologies, demonstrating its ability in the production of feasible solutions, as long as integration is performed correctly. In the present study, the integration of MicMac, GeoServer, Leaflet, and Potree was successfully performed.
The proposed application is freely available and can be employed for multiple uses in academic or professional contexts. The target users can have little or no experience into the photogrammetric processing of UAV-based imagery since the interface is intuitive and minimal interactions are needed. Moreover, public entities can benefit from the use of this web application, since it can be installed on a server and multiple users can access to it through a web browser (or using Docker for a single device), minimizing the costs in the hardware needed to locally conduct the photogrammetric processing.
An example of potential users are police forces responsible for road traffic accident reconstruction, which can use UAVs to acquire imagery from accident scenes and to process them in a quicker way, and to focus on the final products to produce accurate sketches and to digitally preserve evidence for future usage. The application allows not only visualization of the orthophoto mosaics and point clouds but also analysis of the results by enabling the users to perform several measurements.
The web application was tested by analyzing the performance of photogrammetric processing. The orthophoto mosaic demonstrated a good balance in brightness. The point cloud, in turn, presented a high density of points, demonstrating good quality. Considering the RMSE results obtained by MicMac, the thresholds of 1–2 times the GSD horizontally and 1–3 times the GSD vertically were respected. Moreover, MicMac proved the ability to provide accurate results, even when compared with other popular commercial solutions, such as Pix4Dmapper Pro.
Considering the integration of the results obtained in Leaflet and Potree, this process was executed with success being completely automatic, demonstrating a good quality and performance for the visualization of data. The web application development offers the possibility of being continuously updated. In this way, there is a continuous intention to integrate new open-source solutions and new products.

Author Contributions

Conceptualization—Nathalie Guimarães and Joaquim J. Sousa; Methodology— Nathalie Guimarães, Luís Pádua, and Joaquim J. Sousa; Validation—Nathalie Guimarães and Joaquim J. Sousa; Formal Analysis— Nathalie Guimarães, Luís Pádua, Telmo Adão, and Joaquim J. Sousa; Resources—Nathalie Guimarães, Luís Pádua, Telmo Adão, Jonáš Hruška, Emanuel Peres, and Joaquim J. Sousa; Data Curation—Nathalie Guimarães and Luís Pádua; Writing-Original Draft Preparation—Nathalie Guimarães, Luís Pádua, and Joaquim J. Sousa; Writing-Review and Editing—Nathalie Guimarães, Luís Pádua, and Joaquim J. Sousa; Visualization—Nathalie Guimarães, Luís Pádua, and Joaquim J. Sousa; Supervision—Emanuel Peres and Joaquim J. Sousa; Project Administration—Joaquim J. Sousa; Funding Acquisition—Joaquim J. Sousa. All authors have read and agreed to the published version of the manuscript.

Funding

This study was supported by the FCT-Portuguese Foundation for Science and Technology for Nathalie Guimarães (UI/BD/150727/2020). This research was also funded by the European Regional Development Fund (ERDF) through the Operational Programme for Competitiveness and Internationalisation—COMPETE 2020—under the PORTUGAL 2020 Partnership Agreement, of project “REVEAL—Drones for supporting traffic accident evidence acquisition by Law Enforcement Agents” (Nº 33113). Financial support was also provided by the FCT to Luís Pádua (SFRH/BD/139702/2018). This work was also supported by National Funds by FCT-Portuguese Foundation for Science and Technology, under the project UIDB/04033/2020.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Bartoš, K.; Pukanská, K.; Sabová, J. Overview of Available Open-Source Photogrammetric Software, its Use and Analysis. Int. J. Innov. Educ. Res. 2014, 2, 62–70. [Google Scholar]
  2. Aber, J.S.; Marzolff, I.; Ries, J. Small-Format Aerial Photography: Principles, Techniques and Geoscience Applications; Elsevier: Amsterdam, The Netherlands, 2010; ISBN 978-0-08-093283-5. [Google Scholar]
  3. Trujillo, L.; Olague, G. Automated Design of Image Operators that Detect Interest Points. Evol. Comput. 2008, 16, 483–507. [Google Scholar] [CrossRef] [PubMed]
  4. Cooper, S.D.; Roy, D.P.; Schaaf, C.B.; Paynter, I. Examination of the Potential of Terrestrial Laser Scanning and Structure-from-Motion Photogrammetry for Rapid Nondestructive Field Measurement of Grass Biomass. Remote Sens. 2017, 9, 531. [Google Scholar] [CrossRef]
  5. Yan, L.; Fei, L.; Chen, C.; Ye, Z.; Zhu, R. A Multi-View Dense Image Matching Method for High-Resolution Aerial Imagery Based on a Graph Network. Remote Sens. 2016, 8, 799. [Google Scholar] [CrossRef]
  6. Rupnik, E.; Daakir, M.; Pierrot Deseilligny, M. MicMac—A free, open-source solution for photogrammetry. Open Geospatial Data Softw. Stand. 2017, 2, 14. [Google Scholar] [CrossRef]
  7. Gonçalves, J.A.; Henriques, R. UAV photogrammetry for topographic monitoring of coastal areas. ISPRS J. Photogramm. Remote Sens. 2015, 104, 101–111. [Google Scholar] [CrossRef]
  8. Wu, C. VisualSFM: A Visual Structure from Motion System. Available online: http://ccwu.me/vsfm/ (accessed on 14 September 2020).
  9. Snavely, N.; Seitz, S. Bundler User’s Manual. Available online: https://www.cs.cornell.edu/~snavely/bundler/bundler-v0.4-manual.html (accessed on 9 September 2020).
  10. Rahaman, H.; Champion, E. To 3D or Not 3D: Choosing a Photogrammetry Workflow for Cultural Heritage Groups. Heritage 2019, 2, 112. [Google Scholar] [CrossRef]
  11. Open Drone Map, C. Open Drone Map Documentation. Available online: https://www.opendronemap.org/ (accessed on 7 September 2020).
  12. Westoby, M.J.; Brasington, J.; Glasser, N.F.; Hambrey, M.J.; Reynolds, J.M. ‘Structure-from-Motion’ photogrammetry: A low-cost, effective tool for geoscience applications. Geomorphology 2012, 179, 300–314. [Google Scholar] [CrossRef]
  13. Swain, N.R.; Latu, K.; Christensen, S.D.; Jones, N.L.; Nelson, E.J.; Ames, D.P.; Williams, G.P. A review of open source software solutions for developing water resources web applications. Environ. Model. Softw. 2015, 67, 108–117. [Google Scholar] [CrossRef]
  14. Steiniger, S.; Hunter, A.J.S. The 2012 free and open source GIS software map—A guide to facilitate research, development, and adoption. Comput. Environ. Urban Syst. 2013, 39, 136–150. [Google Scholar] [CrossRef]
  15. Gkatzoflias, D.; Mellios, G.; Samaras, Z. Development of a web GIS application for emissions inventory spatial allocation based on open source software tools. Comput. Geosci. 2013, 52, 21–33. [Google Scholar] [CrossRef]
  16. Choi, A.J. Internet of Things: Evolution towards a hyper-connected society. In Proceedings of the 2014 IEEE Asian Solid-State Circuits Conference (A-SSCC), KaoHsiung, Taiwan, 10–12 November 2014; pp. 5–8. [Google Scholar]
  17. Delipetrev, B.; Jonoski, A.; Solomatine, D.P. Development of a web application for water resources based on open source software. Comput. Geosci. 2014, 62, 35–42. [Google Scholar] [CrossRef]
  18. Calantropio, A.; Deseilligny, M.P.; Rinaudo, F.; Rupnik, E. Evaluation of photogrammetric block orientation using quality descriptors from statistically filtered tie points. ISPRS Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2018, XLII-2, 185–191. [Google Scholar] [CrossRef]
  19. Groos, A.R.; Bertschinger, T.J.; Kummer, C.M.; Erlwein, S.; Munz, L.; Philipp, A. The Potential of Low-Cost UAVs and Open-Source Photogrammetry Software for High-Resolution Monitoring of Alpine Glaciers: A Case Study from the Kanderfirn (Swiss Alps). Geosciences 2019, 9, 356. [Google Scholar] [CrossRef]
  20. MicMac documentation MicMac, Apero, Pastis and Other Beverages in a Nutshell! Available online: https://github.com/micmacIGN/Documentation/blob/master/DocMicMac.pdf (accessed on 27 September 2019).
  21. Kropla, B. Beginning MapServer: Open Source GIS Development; Apress: New York, NY, USA, 2006; ISBN 978-1-4302-0053-6. [Google Scholar]
  22. Iacovella, S. GeoServer Beginner’s Guide: Share Geospatial Data using Open Source Standards; Packt Publishing: Birmingham, UK, 2017; ISBN 978-1-78829-407-2. [Google Scholar]
  23. Steiniger, S.; Hunter, A.J.S. Free and Open Source GIS Software for Building a Spatial Data Infrastructure. In Geospatial Free and Open Source Software in the 21st Century; Bocher, E., Neteler, M., Eds.; Springer: Berlin/Heidelberg, Germany, 2012; pp. 247–261. ISBN 978-3-642-10594-4. [Google Scholar]
  24. Ballatore, A.; Tahir, A.; McArdle, G.; Bertolotto, M. A comparison of open source geospatial technologies for web mapping. Int. J. Web Eng. Technol. 2011, 6, 354. [Google Scholar] [CrossRef]
  25. Bloch, J. How to Design a Good API and Why It Matters. In Companion to the 21st ACM SIGPLAN Symposium on Object-oriented Programming Systems, Languages, and Applications; ACM: New York, NY, USA, 2006; pp. 506–507. [Google Scholar]
  26. Orchard, L.M. (Ed.) Professional JavaScript Frameworks: Prototype, YUI, Ext JS, Dojo and MooTools; Wrox Programmer to Programmer; Wiley: Indianapolis, IN, USA, 2009; ISBN 978-0-470-38459-6. [Google Scholar]
  27. Crickard, P. Leaflet.js Essentials; Packt Publishing: Birmingham, UK, 2014; ISBN 978-1-78355-482-9. [Google Scholar]
  28. Martinez-Rubi, O.; Verhoeven, S.; Meersbergen, M.V.; Schütz, M.; Oosterom, P.V.; Goncalves, R.; Tijssen, T. Taming the beast: Free and open-source massive point cloud web visualization. Capturing Real. 2015. [Google Scholar] [CrossRef]
  29. Schutz, M.; Wimmer, M. High-quality point-based rendering using fast single-pass interpolation. In Proceedings of the 2015 Digital Heritage, Granada, Spain, 28 September–2 October 2015; 2015; pp. 369–372. [Google Scholar]
  30. Discher, S.; Richter, R.; Döllner, J. A scalable webGL-based approach for visualizing massive 3D point clouds using semantics-dependent rendering techniques. In Proceedings of the 23rd International ACM Conference on 3D Web Technology—Web3D ’18, Poznań, Poland, 20–22 June 2018; pp. 1–9. [Google Scholar]
  31. Verma, U.; Butler, H. Plasio. Available online: https://github.com/verma/plasio (accessed on 19 November 2019).
  32. Masse, M. REST API Design Rulebook: Designing Consistent RESTful Web Service Interfaces; O’Reilly Media: Sebastopol, CA, USA, 2011; ISBN 978-1-4493-1990-8. [Google Scholar]
  33. Galland, O.; Bertelsen, H.S.; Guldstrand, F.; Girod, L.; Johannessen, R.F.; Bjugger, F.; Burchardt, S.; Mair, K. Application of open-source photogrammetric software MicMac for monitoring surface deformation in laboratory models: Photogrammetry in laboratory models. J. Geophys. Res. Solid Earth 2016, 121, 2852–2872. [Google Scholar] [CrossRef]
  34. Alliez, P.; Forge, F.; de Luca, L.; Pierrot-Deseilligny, M.; Preda, M. Culture 3D Cloud: A Cloud Computing Platform for 3D Scanning, Documentation, Preservation and Dissemination of Cultural Heritage. Available online: https://hal.archives-ouvertes.fr/hal-02096629 (accessed on 23 September 2020).
  35. Duarte, L.; Teodoro, A.C.; Moutinho, O.; Gonçalves, J.A. Open-source GIS application for UAV photogrammetry based on MicMac. Int. J. Remote Sens. 2017, 38, 3181–3202. [Google Scholar] [CrossRef]
  36. Veenendaal, B.; Brovelli, M.A.; Li, S. Review of Web Mapping: Eras, Trends and Directions. ISPRS Int. J. Geo-Inf. 2017, 6, 317. [Google Scholar] [CrossRef]
  37. Schuetz, M. Potree: Rendering Large Point Clouds in Web Browsers; Technische Universität Wien: Vienna, Austria, 2016. [Google Scholar]
  38. Willmott, C.; Matsuura, K. Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance. Clim. Res. 2005, 30, 79–82. [Google Scholar] [CrossRef]
  39. Labbé, S.; Gómez-Candón, D.; El-Natour, G.; Dorado, J.; Fernández-Quintanilla, C.; López-Granados, F.; Peña-Barragán, J.M.; de Castro, A.I.; Torres-Sánchez, J.; Rabatel, G. Automatic mosaicking of very high spatial resolution UAV multispectral images for precision agriculture: Test of MICMAC freeware. In Proceedings of the First RHEA International Conference on Robotics and associated High-technologies and Equipment for Agriculture, RHEA-2012, Pisa, Italy, 19–21 September 2012; pp. 269–274. [Google Scholar]
  40. Barry, P.; Coakley, R. Accuracy of UAV Photogrammetry Compared With Network RTK GPS; Baseline Surveys: Cork, Ireland, 2018; Section 5. [Google Scholar]
  41. Liu, Y.; Zheng, X.; Ai, G.; Zhang, Y.; Zuo, Y. Generating a High-Precision True Digital Orthophoto Map Based on UAV Images. ISPRS Int. J. Geo-Inf. 2018, 7, 333. [Google Scholar] [CrossRef]
  42. Ruzgienė, B.; Berteška, T.; Gečyte, S.; Jakubauskienė, E.; Aksamitauskas, V.Č. The surface modelling based on UAV Photogrammetry and qualitative estimation. Measurement 2015, 73, 619–627. [Google Scholar] [CrossRef]
  43. Jaud, M.; Passot, S.; Le Bivic, R.; Delacourt, C.; Grandjean, P.; Le Dantec, N. Assessing the Accuracy of High Resolution Digital Surface Models Computed by PhotoScan® and MicMac® in Sub-Optimal Survey Conditions. Remote Sens. 2016, 8, 465. [Google Scholar] [CrossRef]
  44. Mendes, T.; Henriques, S.; Catalao, J.; Redweik, P.; Vieira, G. Photogrammetry with UAV’s: Quality assessment of open-source software for generation of ortophotos and digital surface models. In Proceedings of the VIII edição da Conferência Nacional de Cartografia e Geodesia (VIII CNCG), Reboleira, Portugal, 29–30 October 2015; pp. 29–30. [Google Scholar]
  45. Moutinho, Ó. Uma Alternativa Open Source Para Fotogrametria Com RPAS. Master’s Thesis, Faculdade de Ciências da Universidade do Porto, Porto, Portugal, 2015. [Google Scholar]
Figure 1. Flowchart of the four modules needed for the conception of the web application.
Figure 1. Flowchart of the four modules needed for the conception of the web application.
Ijgi 09 00679 g001
Figure 2. 3D dense point clouds produced by free and open-source photogrammetric processing software: (a) OpenDroneMap, (b) MicMac.
Figure 2. 3D dense point clouds produced by free and open-source photogrammetric processing software: (a) OpenDroneMap, (b) MicMac.
Ijgi 09 00679 g002
Figure 3. Architecture of the proposed web application.
Figure 3. Architecture of the proposed web application.
Ijgi 09 00679 g003
Figure 4. VisWebDrone functionalities and interface visualization: (a) upload of imagery; (b) type of output to process; and (c) visualization and analysis of the results.
Figure 4. VisWebDrone functionalities and interface visualization: (a) upload of imagery; (b) type of output to process; and (c) visualization and analysis of the results.
Ijgi 09 00679 g004
Figure 5. MicMac workflow.
Figure 5. MicMac workflow.
Ijgi 09 00679 g005
Figure 6. GeoServer and Leaflet integration.
Figure 6. GeoServer and Leaflet integration.
Ijgi 09 00679 g006
Figure 7. Location of the study sites.
Figure 7. Location of the study sites.
Ijgi 09 00679 g007
Figure 8. Orthophoto mosaics produced in MicMac in first study area (a), and second study area (b).
Figure 8. Orthophoto mosaics produced in MicMac in first study area (a), and second study area (b).
Ijgi 09 00679 g008
Figure 9. Point cloud visualization, using Potree in first study area (a) and second study area (b).
Figure 9. Point cloud visualization, using Potree in first study area (a) and second study area (b).
Ijgi 09 00679 g009
Table 1. Comparative results of photogrammetric processing in Pix4Dmapper Pro and MicMac in the two study areas.
Table 1. Comparative results of photogrammetric processing in Pix4Dmapper Pro and MicMac in the two study areas.
Study AreaDirectionPix4DmapperMicMac
Mean (cm)RMSE (cm)Mean (cm)RMSE (cm)
1X−0.0760.9230.2001.183
Y−0.0020.669−0.3400.904
Z0.0251.9161.8002.145
2X0.3571.662−0.1741.947
Y−0.6361.4260.2921.552
Z0.3792.1690.6202.290
Table 2. Photogrammetric processing time in Pix4Dmapper and MicMac.
Table 2. Photogrammetric processing time in Pix4Dmapper and MicMac.
TaskFirst Study Area (hh:mm:ss)Second Study Area (hh:mm:ss)
MicMacPix4DmapperMicMacPix4Dmapper
OrthophotoMosaic Generation01:00:0500:08:1803:36:0000:32:50
Point Cloud Densification01:53:0001:32:1304:36:0004:00:05
Total Time02:53:0501:40:3108:12:0004:32:55
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Back to TopTop