Next Article in Journal
Spatiotemporal Modeling of Urban Growth Predictions Based on Driving Force Factors in Five Saudi Arabian Cities
Previous Article in Journal
A Supervised Approach to Delineate Built-Up Areas for Monitoring and Analysis of Settlements
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Occlusion-Free Visualization of Important Geographic Features in 3D Urban Environments

State Key Laboratory of Remote Sensing Science, Beijing Normal University, Beijing 100875, China
*
Author to whom correspondence should be addressed.
ISPRS Int. J. Geo-Inf. 2016, 5(8), 138; https://0-doi-org.brum.beds.ac.uk/10.3390/ijgi5080138
Submission received: 28 March 2016 / Revised: 26 July 2016 / Accepted: 28 July 2016 / Published: 8 August 2016

Abstract

:
Modern cities are dense with very tall buildings, which often leads to features of interest (FOIs, e.g., relevant roads and associated landmarks) being occluded by clusters of buildings. Thus, from any given point of view, users can see only a small area of the city. However, it is currently an important technical problem to maintain the visibility of FOIs while preserving the urban shapes and spatial relationships between features. In this paper, we present a novel automatic visualization method to generate occlusion-free views for FOIs in real time. Our method integrates with three effective cartographic schemes: route broadening, building displacement, and building scaling, using an optimization framework A series of distortion energies are presented to preserve the urban resemblance, considering the view position and the urban features based on spatial cognition to maintain spatial and temporal coherence. Our approach can be used to visualize large urban environments at interactive framerates in which the visibility of the occluded FOIs is maximized while the deformation of the landscape’s shape is minimized. Using this approach, the visual readability of such 3D urban maps can be much improved.

1. Introduction

Modern cities are often dense with very tall buildings that occlude roads and neighboring landmarks. Traveling in such unfamiliar urban areas is a difficult task. With the help of car navigation systems or maps, users may plan their routes and arrive at their destinations. Currently, most of these systems are equipped with digital maps. Although 2D maps can provide an overview of features of interest (FOIs), their inability to preserve a realistic egocentric view results in lower intuitiveness and a reduction in the readability of such urban maps for the following reasons: First, interpretation of a map can require considerable mental effort. A user must locate his/her position and orient the map properly [1]. Second, a correspondence between the map and the environment must be established [2,3]. Humans accomplish this by identifying and matching landmarks in a scene with their map representations [4]. Third, the realism of the 3D environment is lost. A 2D map cannot provide depth information; thus, it fails to yield a visual match to the real landscapes people see in their daily lives [5].
With the rapid development of photogrammetry, computer vision, scanners, and 3D modeling technologies, it is currently possible to construct detailed and easy to use 3D models in a cost-efficient manner [6,7]. Compared with 2D maps, a 3D representation of urban environments provides depth information. Thus, it yields a better visual representation and maintains the realism of the urban surroundings. However, occlusions by urban objects are very common, particularly for cities packed with skyscrapers. When perspective projection is used, distant objects are inevitably occluded by those nearer the viewpoint [8]. As for car navigation applications, the route to be traveled is likely to be occluded by any surrounding high buildings. Therefore, users cannot obtain the spatial information concerning the route and, consequently, will not be able to make good advance preparations for upcoming turns or intersections. It is challenging to eliminate occlusions of important geographic features while simultaneously preserving the global layout of a 3D map [9]; Hirono et al. [5]. Moreover, rendering such an occlusion-free view consumes considerable computational resources.
To address these problems, an optimization method is presented to automatically eliminate occlusions of FOIs in urban areas while also constraining distortions to be as small as possible (see Figure 1). The occlusion-free problem is formulated as a quadratic programming problem. Route broadening, building displacement, and building scaling are integrated into the optimization framework. We assume that landmarks associated with the route of interest are already available and that they are integrated into the same optimization framework for visualization alongside the route. Our method can maximize the visibility of occluded FOIs while minimizing deformations to the landscape.
The main contributions of this paper can be summarized as follows:
(1)
Present a novel interactive visualization framework that integrates three operators to avoid large distortions during disocclusion.
(2)
Propose an optimization approach that includes appropriate measures of resemblance and occlusion of FOIs.
(3)
Implement a deformable model for urban areas that considers buildings and streets.

2. Related Work

2.1. Visualization of FOI

Carpendale et al. [10] and Möser et al. [11] introduced deformation techniques into 3D environments to magnify a region of interest. An occlusion-free visualization tool can be useful for navigation, in games and in monitoring operations. Here, the term navigation includes car-based navigation, mobile navigation and computer navigation systems. Users will grasp their location, surroundings and FOIs quickly and conveniently. Occlusion-free techniques are also useful in 3D game maps. Moreover, using monitors and occlusion-free visualization, we could select some FOIs to locate and catch criminals quickly. However, the distortions introduced by occlusion-removal methods are always a major concern. Takahashi et al. [12] developed a system for creating panoramic maps that enhances the visibility of important roads and landmarks. Later, they proposed an occlusion-free route animation for car navigation in mountainous areas [9]. In their system occlusions are eliminated by rearranging onscreen landmarks. This approach also requires the geographical landmarks to be sampled from different viewpoints on the hemisphere that covers the terrain to take all occlusion situations into account. Therefore, the number of landmarks is usually large, and the data are difficult to make smaller without introducing artifacts. It may be tedious to generate occlusion-free views at interactive framerates if the number of landmarks is large.
In this paper, we mainly focus on disoccluding the FIOs in urban environments. A positive definite quadratic programming is formulated to remove FOI occlusions. A multi-perspective rendering combines what is observed from several viewpoints into a single image. Despite the incongruity of these views, effective multi-perspective images are still able to preserve spatial coherence. By combining multi-perspective rendering and cartographic generalization techniques, Grabler et al. [13] developed a system for automatically generating tourist maps that highlights the most salient objects, such as streets and landmarks. This system renders buildings in multi-perspective views to increase the visibility of the appropriate routes and associated landmarks. This multi-perspective rendering technique can increase the visibility of the FOIs and landmarks, but it is a time-consuming process, and it causes large distortions of other objects in 3D urban environments. Degener et al. [2] introduced a space-deformation based camera model tailored to display short routes by showing the entire path contiguously, as observed from an egocentric perspective within a single image. However, the large distortions of the resulting zigzagging and twisted routes often caused misleading navigation for users. Later Degener and Klein [14] presented a variational approach to generate panoramic maps in mountain areas. This approach can provide a good overview and it avoids the occlusion of important geographical features; however, it also requires considerable computation. Glander and Döllner [15,16] created untextured cell blocks to replace the original 3D building models to realize dynamic landmark highlighting according to a virtual camera distance. They achieved a smooth visual effect, but the approach weakens important details of city images. Cui et al. [17] developed an automatic approach to compute a curved ray camera that shows routes in such a way that occlusion-free views can be obtained when the computed cameras are connected. However, the perturbation of spatial relationships caused by this camera-centric approach is a major concern. Hirono et al. [5] presented a method to enhance map readability that can identify common design criteria often employed in hand-drawn illustrative maps created by cartographic artists. This method employed fixed orientation, relative position, and scale limits to maintain consistency in the arrangement of geographic features and formulated these as hard constraints, while introducing minimum displacement and occlusion avoidance as a nuanced approach to enhance the reality and readability of 3D urban maps, respectively. The challenge here is to find a deformation of 3D map geometry that meets five design criteria. They formulated this problem as a constrained optimization problem by employing linear programming techniques, but their method can only generate static occlusion-free maps.

2.2. Differences between the Proposed Method and Related Methods

Recently, deformation methods [5,18,19] have been proposed to alleviate occlusions in urban areas. Deformation methods clear the line of sight by deforming the occluding objects and the surroundings. Deng et al. [18] presented an automatic method that enabled an interactive context-aware visualization of the relevant urban features. Hirono et al. [5] proposed an optimization method based on common aesthetic criteria to disocclude landmarks for urban environments. However, this optimization does not support the interactivity required for various tasks. Qu et al. [19] presented a focus+context method that features bird’s-eye views with broadened routes and enlarged landmarks. In this paper, we present a visualization method to generate occlusion-free views for FOIs automatically. The method employs three disocclusion operators: road broadening, building downscaling, and building displacement. Our method provides an optimum compromise among the occlusion operators that minimizes distortions while enhancing the visibility of FOIs. The generated 3D views provide a good overview of the FOIs while maintaining a high resemblance between the disoccluded cityscape and its original appearance. The previous works most related to our approach are those of Qu et al. [19], Hirono et al. [5] and Deng et al. [18]. Here, we will discuss the differences between our proposed method and these existing methods:
(1)
Differences between the proposed method and the method of Deng et al. [18].
Deng et al.’s [18] method employed four disocclusion operators—viewpoint elevation, road shifting, building scaling, and building displacement—to visualize urban FOIs. The use of the viewing height disocclusion operator is very controversial, because this solution is both trivial and not particularly desirable. In an interactive environment, changing the viewpoint is a concern, and such changes constitute the major limitation of Deng’s method. Users should be able to maintain control of the viewpoint in an interactive application. Therefore, in this paper we integrate only the route broadening, building displacement, and building scaling disocclusion operators, and use them within an optimization framework to delete FOI occlusions. Moreover, spatial and temporal constraints are employed to keep the disocclusion views both spatially and temporally consistent. Deng et al.’s work [18] applied sequential quadratic programming to solve the optimization problem using an iterative approach in which each iteration step requires the gradient and the Hessian of the Lagrange function. Consequently, when the number of visibility functions is large, this method requires a great deal of computation. In contrast, in this paper, we formulate a positive definite quadratic programming method to delete FOI occlusions.
(2)
Differences between the proposed method and the method of Hirono et al. [5].
(i)
We use an object-based view to constrain the deformation of the urban shape rather than maintaining orientations of paired parallel lines, because maintaining the lines increases the computation time. Moreover, our method treats the relevant roads and buildings as unit objects to constrain their positions, which reduces the time cost. This approach is quite different from that of Hirono et al. [5].
(ii)
We formulate the occlusion-free animation problem as a quadratic programming problem and decompose the matrix during preprocessing, making it possible to generate occlusion-free views from frame to frame at interactive framerates. The optimization method used in [5] preserves the resemblance but does not provide the high system interactivity required in various tasks due to the high computational load involved in optimization.
(3)
Differences between the proposed method and the method of Qu et al. [19].
We integrate route broadening, building displacement, and building scaling into one optimization framework and design a series of distortion energies to preserve the urban resemblance. The result from this optimization framework can maximize the visibility of the occluded FOIs, while also maintaining a strong resemblance to the original views. Qu et al. [19] adopted three steps to generate context-aware views; they first enlarged roads with a seam-carving technique and then used a grid-based zooming technique to expand the blocks and landmarks. Finally, for the buildings still blocking the route, they zoom out the buildings to reveal the route. Although they use the optimal method for each step, their method does not include a global optimization. Moreover, they disocclude a route and its associated landmarks only from a single 45-degree bird’s-eye view, whereas our method has no such constraints.

3. Interactive Occlusion-Free 3D Visualization

Mapmakers use a variety of cartographic generalization techniques including simplification, displacement, deformation, and selection to improve the clarity of maps and to emphasize the most important spatial information, while preserving spatial relationships among map objects [4,19,20]. In this work, we apply a series of generalization techniques, including route broadening, building displacement, and building scaling through an optimization framework to eliminate occlusions from surrounding urban objects.
Route broadening: Aiming to emphasize the relevant routes and improve their clarity, cartographers usually broaden the road. Thus, we widen the relevant route to emphasize it and reinforce the aesthetic effect (Figure 2b).
Building displacement: We move the context objects of FOIs to eliminate occlusions. The spatial pattern and adjacency of buildings should be maintained during the displacement operation (Figure 2c). To retain the distribution patterns, a proximity graph is constructed to represent the building clusters in each city block. The graph connects all the buildings in a block as a whole during the building displacement process.
Building scaling: Similar to building displacement, scaling the context objects of FOIs also helps to eliminate occlusions of FOIs caused by objects in the foreground (Figure 2d).
When only one of these techniques is adopted, the resulting deformation of the 3D urban scene is large. In this paper, we propose an optimization approach that integrating all three of the above techniques using energy functions (Figure 2e).

3.1. Problem Statement

The problem studied here can be described as follows:
Given a viewpoint, all objects I0 in the 3D urban environment and FOIs F0I0, the goal is to obtain the corresponding deformed objects I of I0 and the features of interest F of F0 that minimize the constrained optimization problem:
min Ed(I, F)
s.t. Eo(I, F) = 0
where Ed(I, F) is the deviation from I and F to I0 and F0, respectively, and Eo(I, F) is the occlusion of F.
Subject to the following constraints:
Ed(I, F) ≥ 0
Eo(I, F) ≥ 0
If Ed(I, F) = 0 then I = I0 and if Eo(I, F) = 0, then F is visible. Thus, the solution to Equation (1) should be the optimal view to minimize the distortions while maintaining the visibility of F.
The occlusion-free views are implemented in Cartesian coordinates in which the X-axis and Y-axis are located in the ground plane while the Z-axis is perpendicular to the horizontal plane. Furthermore, the buildings can be approximated as a combination of a number of rectangular boxes, roads are treated as rectangular strips, and their boundaries are composed of polylines. In our formulation, we assume that all features are aligned at the same elevation and that the terrain does not occlude roads and landmarks. If a city is located in a mountainous area, the method of Deng et al. [21] is first employed to eliminate occlusions of the FOIs caused by terrain. Then, the presented method in this paper is applied to remove FOI occlusions caused by neighboring objects such as buildings. We elaborate the formulation of Ed(I, F) and Eo(I, F) in the following sections.

3.2. Disocclusion Operators

As illustrated in Figure 3, for a route point PR occluded by the building B from the viewpoint V, the occlusion of PR can be eliminated by displacing the building B to B0 and broadening the road from { P R ,   P R ¯ } to the new position { P R ,   P R ¯ } in the horizontal direction. Since simply using one or two disocclusion operators to reveal the occluded target would introduce a larger distortion, for disocclusion we combine all three operators (i.e., route broadening, building displacement, and building scaling) to minimize the distortion. The building displacement and downscaling, which are absent in [19], are effective for the elimination of occlusions in the horizontal and vertical directions. The relation between the maximum building displacement pB and the maximum road point displacement for broadening P R ¯ , as well as the relation between the maximum amount the building needs to be lowered (∆h) and the maximum viewpoint height change ∆H, can be deduced from simple similar triangle relationships in the following two figures respectively, as follows:
Δ P B Δ P R ¯ = H h H < 1

3.3. Occlusion Measurement

The majority of the urban objects are roads and buildings. Based on the above geometrical abstraction, roads can be treated as several rectangular strips. As shown in Figure 4a, the visibility of a road is determined by the visibility of the segments closest to the viewpoint, that is to say, as long as that segment is visible, the corresponding road is visible. Similarly, visibility of the building is related to the visibility of the segments on the footprint closest to the viewpoint (Figure 4b). Thus, the visibility of both roads and buildings can be attributed to visibility of the segments. Next, we focus on the method for measuring the visibility of a segment on the ground plane.
As shown in Figure 5, the segment AB on the ground and the viewpoint V form a triangle, ABV, which is not an infinite plane. Obviously, all of the lines of sight from V to the segment AB belong to the triangle ABV, called the view plane. Figure 4 further shows the condition where the segment AB is occluded by a building. Apparently, only if the view plane and the building boundary have intersections, the road should be occluded by the building. Moreover, the longer the segment on the edge above the view plane is, the more seriously the road is occluded by the building. The edge KK seriously occludes the road whose segment above the view plane is K′T. Therefore. we can use the length of K′T to measure the visibility of the segment. When the length of K′T = 0, the road is visible.
We define the displacement vector as T = (Tx, Ty) and the scaling vector as S = (Sx, Sy, Sz). Since the scaling operation is relative to the center of the building, namely, Sx, Sy, Sz ∈ [0,1]. After a building is moved and zoomed, the new vertices K′ on the building can be computed by Equation (4):
K ( x k ,   y k ,   z k ) = O + T + S K O = ( x 0 + T x + S x x k 0 ,   y 0 + T y + S y x y 0 ,   z 0 + T z + S z x z 0 )
where O(xo, yo, zo) is the center of the building, K(xk, yk, zk) represents the vertices on the footprint of the building nearest to the road, and KO(xk0, yk0, zk0) is the vector from the center of the building to the vertices K.
The parametric equation of the line segment K′K is:
{ x = x k y = y k z = t i + z k   ( 0 < t i < H i )
where Hi is the height of building i.
According to the two endpoints of the relevant route and the viewpoint V, it is easy to obtain the equation of the view plane:
a X + b Y + c Z + d = 0
Combining Equations (5) and (6), we obtain:
t i = a c x k b c y k z k d c
Therefore, we acquire the occlusion term from the building to the relevant road, namely, the length of K′T:
E o ( i ) = { ( H i t i ) ,   o < t i H i 0 ,   o t h e r
Then the constraint in Equation (1) can be rewritten as follows:
E o ( i ) 0   ( i = 1 , ... , n )
where n is the number of buildings in the city block.
To reduce the number of constraints and simplify them, we search the building set Bocc, which occludes the FOIs at each frame, and restrict these occlusion terms to zero. Therefore, Equation (9) can be rewritten as follows:
Eo(i) = 0 (iBocc)

3.4. Deviation Measurement

At the base of the urban spatial pattern, we quantify the distortions so as to maintain the similarity of the block before and after the optimization.

3.4.1. Urban Pattern Extraction

Zhang et al.’s method [12] is adopted to extract the urban pattern. To extract the urban spatial pattern, we consider a range of scales from a global aspect to local features. Globally, the city can be subdivided into hierarchical regions according to the city morphology [22]. We extract the distribution pattern of the buildings as well as local alignment groups in the pattern. For the distribution pattern, we construct a proximity graph to represent the clusters and relations among buildings in each block [12]. For the local alignment, we extract the alignment groups based on the gestalt rules [12,23] and associate the groups with the graph.

3.4.2. Displacement and Scale Deviation

To maintain the spatial similarity of the 3D urban environment before and after the optimization, we take the deviations into account and constrain them to the minimum.
(1) 
Global constraint for the relative positions among adjacent buildings
The relative spatial positions among adjacent buildings should be maintained. The sizes of all buildings are zoomed out as consistently as possible. Therefore, we obtain this energy term to measure the relative deviation and size scale from the global optimization:
E b l o c k S e t s = i , j B ω t T i T j 2 + ω s T ( S i S j ) 2
where B is the set of all buildings, and Ti and Tj represent the displacement vectors of buildings i and j, respectively. Si and Sj are the scaling vectors of i and j, respectively, ωt is the weight of the building displacement, and ωs = (ωsx, ωsy, ωsz) represents the scaling weights along the X-axis, Y-axis, and Z-axis. In our experiment, we set ωt = 0.01 and ωs = (1.2, 1.2, 2.4). ωij is defined as the weight on the relative deviation from buildings i and j. In general, the greater the distance between two buildings is, the smaller the value of ωij is; thus, we use the thermonuclear function to represent the weight varying with the distance:
ω i j = exp ( d i j t )
where dij is the square of the Euclidean distance between the footprint centers of buildings i and j, t is the sensitivity factor for adjusting ωij, which changes with dij, and t =1.0 × 106.
(2) 
Constraints of buildings in the same block
Based on the urban morphology theory [24] and gestalt psychology [25], people often tend to regard buildings located in the same block as a whole. Thus, the relative deviation between buildings in the same block should be constrained to be as consistent as possible:
E b l o c k = b k K i , j b k ω i j ( ω t T i T j 2 + ω s T ( S i S j ) 2 )
where K is the set of all blocks, and other parameters have the same meaning as those in Equation (11).
(3) 
Constraint of the deviation of the buildings
The deviation of the deformed buildings is determined by their displacement and scaling sizes:
E b l d = i B ω t T i 2 + ω s T ( I S i ) 2
where I =(1.0, 1.0, 1.0) is a unit vector, and other parameters have the same meaning as those in Equation (11).
(4) 
Road constraint
A road can be divided into rectangular strips consisting of vertices V = {v10, v11, v20, v21, …, vn0, vn1} distributed on both sides of the road as well as along the central axis (Figure 6). In the vertex subscript, the first bit represents the road serial number, and the second indicates whether the vertex is on the left side or the right. Consequently, we have:
v i 0 + v i 1 2 = p i
If one of the segments along the central axis of the road is vivi + 1, then the lines parallel with vivi + 1 satisfy Equation (16):
n i T p + c = 0 ,   p = ( x , y ) T
where ni is the normalized vector perpendicular to vivi+1, and c is a constant related to the distance from c to the central axis. Obviously, the segments on both sides of this central axis satisfy Equation (17):
n i T ( v ( i + 1 ) v ( i ) ) = 0
The deviation of the road is measured by its location and shape. The location deviation can be represented by the square of the distance between the center of the deformed road and the original road:
E p o s ( V ) = v i 0 , v i 1 V v i 0 + v i 1 2 p i 2
The shape deviation includes the degree of imbalance between the road segment and the central axis:
E s hp ( V ) = v ( i + 1 ) , v i V n i T ( v ( i + 1 ) v ( i ) ) 2
The shape deviation also includes the change to the road width:
E wid ( V ) = v i 0 , v i 1 V ( v i 0 v i 1 ) ω r d ( v i 0 v i 1 ) 2
where ωrd is the scale factor of the road widening operation. For the relevant roads, we set ωrd =1.5; otherwise, ωrd =1.0.
From Equations (18)–(20), we obtain the deviation term of the road (Equation (21)):
E r o a d ( x ) = E p o s ( V ) + η s h p E s h p ( V ) + E w i d ( V )
where Eshp is weighted by η s h p , which is set to 10.0 to ensure that the shape of the road is not changed.
(5) 
Constraint of the relative positions between the road and surrounding buildings
To measure the deviation from the road to its neighboring buildings, we search the set Φ = { v i ,   ( v i   v ( j + 1 ) ) } , including the road and its neighboring buildings. Here, vi is the footprint center of building i, and Φ = { v i ,   ( v i   v ( j + 1 ) ) } is the road near i. Thus, we can obtain the relative deviation from the road and its neighboring buildings by Equation (22):
E b l d _ r d = Φ i Φ v i p R j ( v j ' ,   v ( j + 1 ) ' ) v i + p R ( v j ,   V ( J + 1 ) ) 2
where R J ( v j ,   v ( j + 1 ) ) = ( 1 t ) v j + t v ( j + 1 ) . ,   t [ 0 , 1 ] are the road vertices nearest to building i.
(6) 
Deviation of spatial and temporal coherence constraints
The deviation from spatial and temporal consistency can be determined by the relative deviation of the location and size of the building between the current frame and previous one. Therefore, we obtain:
E T e m p = i B ω t T i T i 2 + ω s S i S i 2
where Ti and Si are the displacement and scale vector of building i, respectively. The definitions of ωt and ωs are the same as in Equation (11).

3.5. Implementation

From the construction of the deviation terms, we can observe that each deviation is in a positive definite quadratic form. Therefore, Equation (1) can be rewritten as:
min   E d = 1 2 X T G X + r T X s . t . E o ( I , F ) = 0
where G and r are the matrix and vector of the deviations of the city block, respectively, which can be constructed and factored during preprocessing.
Figure 7 illustrates the pipeline of our interactive FOI visualization scheme. To reveal the FOIs, the system extracts the urban pattern during preprocessing and constructs the deviation matrix G in Equation (24). Then, it implements the Cholesky decomposition [26]. At the first frame during rendering of the environment, the system solves the quadratic programming problem described in Equation (24) to optimize all the relevant routes and buildings. When navigating in an urban environment, dynamically widening the relevant road may mislead the users; therefore, the roads are not further enlarged after the first frame. We find that Eo is linear when the road vertices are constant. Therefore, Equation (24) can be rewritten as:
min   E d = 1 2 X T G X + r T X s . t . h i ( X ) = a i T X b i = 0 ( i = 1 ,    2 , ... , l )
The deviation term is:
E d = E b l o c k S e t s + E b l o c k + E b l d + E r o a d + E b l d _ r d + E T e m p
The solution to the quadratic programming problem in Equation (25), which contains only linear constraints, is fast enough to generate occlusion-free views at interactive framerates.

4. Experimental Results

4.1. Study Data

To validate the proposed method we implemented it on a laptop with a Core i3 CPU running at 2.30 GHz with 2GB of RAM. The method was then tested on two real urban datasets. One is a region of Hong Kong Island, and the other is the entire Vancouver downtown peninsula. In total, there are 1017 buildings in the Hong Kong city data. Hong Kong is densely populated with tall buildings along roads. The shortest building is 48.3 m in height, while the highest one is 374 m. The buildings in downtown Vancouver are lower, with an average height of 28.8 m. The highest building in Vancouver is 170 m, but the urban scene scale is much larger than the Hong Kong scene; it contains 1643 buildings. In the two urban scenes described above, relevant routes and landmarks are often occluded. From a bird’s-eye view, some landmarks are small, or may not even be discernible.

4.2. Result

In traditional 3D views, when users find their relevant route occluded, they can either adjust the viewpoint to another angle or raise the viewpoint to clear the occlusion. Figure 8a shows a z-shaped route in the Hong Kong scene that is occluded by surrounding buildings. Figure 8c shows a 3D landscape viewed from a very high viewpoint. This view is not only difficult to control but also weakens the perspective effect and loses the spatial context of the urban environment. Without changing the viewpoint, our method eliminates the occlusion by integrating building displacement and scaling with the widening of the relevant route through the optimization framework. As shown in Figure 8b, it appears that our method eliminates the occlusion from the buildings effectively while maintaining good spatial similarity with the original view.
Serious occlusions often appear at some intersections. Figure 9a shows a horizontal road interacting with many vertical roads. The intersection segments of the roads are seriously occluded. In our occlusion-free view (Figure 9b), the intersections are completely visible. When navigating in a city, users can comprehend more spatial information about the crossroads and, thus, plan their travel in advance.
An effective 3D navigation view should provide rich visual representations of FOIs to help users quickly identify their locations and determine the most relevant route to their destination. In Figure 10a, the relevant route (in blue) and associated landmarks (with textures) are occluded by the surrounding buildings, while Figure 10b shows the generated occlusion-free view. The relevant route and its associated landmarks are visible without occlusions while the spatial relationships are maintained both before/after the deformation. The 3D views generated by our method give users a good overview of the city; meanwhile, the whole route as well as the associated landmarks remain visible. Therefore, our method increases the readability of 3D urban maps and emphasizes the FOIs in the scene while minimizing the deformations of other parts.
Another important issue to be validated is that the approach should retain the spatial-temporal coherence of the 3D urban scene during the animation. In the two supplemental videos, there are no “jumps” between frames; they are smooth over time. As for the spatial coherence, the distorted view often incurs natural illusions of 3D shapes because the parallel relationships between a pair of line features are fully maintained. As for the temporal coherence, the proposed approach can fully eliminate local vibration effects during navigation. However, in some extreme conditions (e.g., a very tall building is very close to the relevant road or a very tall building is surrounded by the relevant route), the building may be over-scaled and, consequently, appear too small. Figure 11a shows some buildings (red rectangle) surrounded by the relevant road network. To reveal the surrounding roads, these buildings are over-scaled (Figure 11b).
The experiment provided quantitative results for the visualization processes. It demonstrates that the maximum displacement distance is less than 20 meters by Equation (25). We note that the maximum scaling along the Z-axis is typically larger than that along the X- and Y-axes. This occurs because scaling along the Z-axis is an efficient way to reduce occlusions. In these examples, we use the predefined parameters values introduced in Section 3. However, users can adjust them to obtain more preferable results. This adjustment will be introduced in Section 5.
To better reflect the global impact of the presented method, we provide the quantitative results of the three disocclusion operators (Table 1) and the statistical results of the occlusion measurement and deviation measurement applied on the various 3D scenes (Table 2).

4.3. Performance

The efficiency of our approach depends on the size of the city and the number of buildings occluding the FOIs. The size of the city determines the dimension of the quadratic programming problem, while the number of occluded buildings determines the number of constraints. In the two urban scenes shown above, the viewpoints were kept at 500 m in height with a view angle of 30 degrees, and the number of buildings rendered in the Hong Kong region and Vancouver are 1017 and 1643, respectively. Table 3 shows the statistical results.
From Table 2, it can be noted that when more FOIs are selected, the performance naturally drops—primarily due to the additional constraints introduced to the optimization. However, our system can still achieve between 10.1 and 5.4 frames per second (fps) in Hong Kong while disoccluding the complex relevant roads. To accelerate the rendering speed of the occlusion-free 3D urban environment, longer relevant routes are decomposed into several segments (e.g., for a car navigation application). We only eliminate occlusions in the current segment where the car is moving and the next several upcoming segments, but ignore occlusions in the rest of the segments. Experiments show that by dividing long routes, our technique can achieve a framerate in the Hong Kong environment of 9.3–15.6 fps, while in the larger Vancouver environment, it can achieve 6.8–14.8 fps. Figure 12 shows the result obtained by using this decomposition technique to process the relevant route shown in Figure 1b. Compared with Figure 1b, the visibility of the entire route in Figure 11 is reduced, but the route segments (highlighted in pink) toward which the car (shown in dot) is driving are disoccluded (and meanwhile, the framerates increase).

4.4. Weight Adjustment

In our method, there are three weight parameters that are closely related to the deformation, namely, the road scale factor ωrd, the building displacement weight ωt and the scaling weight ωs = (ωsx, ωsy, ωsz). Although we have provided the recommended values, users might want to alter these parameters themselves. In most 3D urban environments, we set ωsx = ωsy = ω, ωsz = 2ω, while ωrd, ωt and ω can be set by users. Our method allows them to adjust ωrd before navigation and to adjust ωt and ω during navigation. Figure 13 shows the views with different ωrd, ωt and ωs values. Adjusting the weights can reveal the route and the associated landmarks with low distortions while maintaining the surrounding objects to provide a nice overview.

5. User Study

We evaluated our method by conducting a user study with 27 subjects with different ages (from 19 to 31), genders (nine females), and backgrounds (from computer science and geoinformatics specialties). The subjects’ task is to match a route with a 2D map using the conventional perspective (CP) views, our 3D occlusion-free (OF) views, or the focus+context (FC) views, respectively, which is similar to the evaluation task presented by Rosen and Popescu [8]. Before the test, the subjects were shown the three types of views with no special technical instructions or suggestions and were given ten minutes to familiarize themselves with our system. A route with multiple turnings is set as the FOI and the viewpoint is set at a position where the FOI was partially occluded. In the test, the subject should navigate the camera to orient themselves and trace the route. Then, they must mark it on the 2D map that is simultaneously displayed on the other screen. For each subject, six different examples in the Hong Kong and Vancouver datasets (three examples for each) were tested, in which each type of view appeared in two examples randomly. For each example, we recorded the total time the subject spent and the accuracy of the mark the user made. In total, we obtained 27 × 6 = 162 records.
Table 4 summarizes the obtained average timings. On average, our OF views resulted in a 96.8% speed improvement compared with the CP views and a 29.9% speed improvement compared with the FC views. Regarding the correctness of the 27 × 2 = 54 cases for each view type, the subjects correctly marked the FOIs on the 2D maps in 88.9% (48/54) of the cases using the OF views, compared with 83.3% (45/54) for the FC views and 79.6% (43/54) for the CP views.
After completing the tasks, the subjects were surveyed concerning which types of views they felt were more suitable for the task, by providing ratings on a scale of 1 (not helpful at all) to 5 (very helpful). The average ratings were 4.6 for the OF views, 3.7 for the FC views, and 2.8 for the CP views. Furthermore, the subjects were asked whether the distortions in the OF and FC views would mislead users, also rated on a 1 (very misleading) to 5 (not at all) scale. In the results, the OF views (4.3 on average) were thought to be slightly more misleading compared with the FC views (4.8 on average). Overall, the results and the feedback reveal a clear viewer preference for our method.

6. Conclusions

This paper proposes an interactive occlusion-free navigation of the FOIs in 3D complex urban environments. A positive definite quadratic program is applied to optimize the energy function that combined route widening with building displacement and building scaling techniques to make the FOIs visible. Considering the urban spatial distribution, our method maintains the spatial similarity of the urban environment before and after the deformation. A series of designed energy terms maintain the consistency of the spatial patterns in urban blocks during occlusion elimination, and the positive definite quadratic program solves the optimization problems quickly by decomposing and calculating the deviation matrix of the block spatial pattern in advance. The generated occlusion-free views for the FOIs can be visualized at interactive framerates; therefore, they are suitable for applications in car navigation systems and location-based service applications.
In some extreme cases (see Figure 10 for an example) in which the route is very complex, the buildings are sometimes over-scaled, which cause a large deformation. Considering the above limitations, future research will focus on accelerating the framerates. We will achieve this by applying urban simplifications, such as the methods of Glander et al. [15] and Zhang et al. [8]. In future work, we will implement the transformation process directly in the GPU [27] to improve the framerate of our system.

Acknowledgments

This work was supported by the National Natural Science Foundation of China under Grant 41371324.

Author Contributions

Liang Zhang and Xiang Xu performed the experiments; Liqiang Zhang contributed analysis tools; Liang Zhang, Liqiang Zhang and Xiang Xu wrote the paper.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Hermann, F.; Bieber, G.; Duesterhoeft, A. Egocentric maps on mobile devices. In Proceedings of International Workshop on Mobile Computing, Monpellier, France, 21–22 July 2003.
  2. Degener, P.; Schnabel, R.; Schwartz, C.; Klein, R. Effective visualization of short routes. IEEE Trans. Vis. Comput. Graph. 2008, 14, 1452–1458. [Google Scholar] [PubMed]
  3. Lu, Y.; Liu, Y. Pervasive location acquisition technologies: Opportunities and challenges for geospatial studies. Comput. Environ. Urban Syst. 2012, 36, 105–108. [Google Scholar] [CrossRef]
  4. Presson, C.C. The development of map-reading skills. Child Dev. 1982, 53, 196–199. [Google Scholar] [CrossRef] [PubMed]
  5. Hirono, D.; Wu, H.; Arikawa, M.; Takahashi, S. Constrained optimization for disoccluding geographic landmarks in 3d urban maps. In Proceedings of the 6th IEEE Pacific Visualization Symposium (PacificVis 2013), Sydney, Australia, 26 February–1 March 2013; pp. 17–24.
  6. Buchroithner, M. True-3D in Cartography: Autostereoscopic and Solid Visualisation of Geodata; Springer: Berlin, Germany, 2012. [Google Scholar]
  7. Zhang, L.; Deng, H.; Chen, D.; Wang, Z. A spatial cognition-based urban building clustering approach and its applications. Int. J. Geogr. Inf. Sci. 2013, 27, 721–740. [Google Scholar]
  8. Rosen, P.; Popescu, V. An evaluation of 3D scene exploration using a multiperspective image framework. Vis. Comput. 2011, 27, 623–632. [Google Scholar] [CrossRef] [PubMed]
  9. Takahashi, S.; Yoshida, K.; Shimada, K.; Nishita, T. Occlusion-Free Animation of Driving Routes for Car Navigation Systems. IEEE Trans. Vis. Comput. Graph. 2006, 12, 1141–1148. [Google Scholar] [CrossRef] [PubMed]
  10. Carpendale, S.; Ligh, J.; Pattison, E. Achieving higher magnification in context. ed. In Proceedings of the 17th Annual ACM Symposium on User Interface Software and Technology, Santa Fe, NM, USA, 24–27 October 2004; pp. 71–80.
  11. Möser, S.; Degener, P.; Wahl, R.; Klein, R. Context aware terrain visualization for wayfinding and navigation. Comput. Graph. Forum 2009, 27, 1853–1860. [Google Scholar] [CrossRef]
  12. Takahashi, S.; Ohta, N.; Nakamura, H.; Takeshima, Y.; Fujishiro, I. Modeling Surperspective Projection of Landscapes for Geographical Guide-Map Generation. Comput. Graph. Forum 2002, 21, 259–268. [Google Scholar] [CrossRef]
  13. Grabler, F.; Agrawala, M.; Sumner, R.W.; Pauly, M. Automatic generation of tourist maps. ACM Trans. Graph. 2008, 27, 3. [Google Scholar] [CrossRef]
  14. Degener, P.; Klein, R. A variational approach for automatic generation of panoramic maps. ACM Trans. Graph. 2009, 28, 1–14. [Google Scholar] [CrossRef]
  15. Glander, T.; Döllner, J. Automated cell based generalization of virtual 3D city models with dynamic landmark highlighting. In Proceedings of 11th ICA Workshop on Generalization and Multiple Representation, Monpellier, France, 20–21 June 2008.
  16. Glander, T.; Döllner, J. Abstract representations for interactive visualization of virtual 3D city models. Comput. Environ. Urban Syst. 2009, 33, 375–387. [Google Scholar] [CrossRef]
  17. Cui, J.; Rosen, P.; Popescu, V.; Hoffmann, C. A curved ray camera for handling occlusions through continuous multiperspective visualization. IEEE Trans. Vis. Comput. Graph. 2010, 16, 1235–1242. [Google Scholar] [PubMed]
  18. Deng, H.; Zhang, L.; Mao, X.; Qu, H. Interactive urban context-aware visualization via multiple disocclusion operators. IEEE Trans. Vis. Comput. Graph. 2016, 22, 1862–1874. [Google Scholar] [CrossRef] [PubMed]
  19. Qu, H.; Wang, H.; Cui, W.; Wu, Y.; Chan, M.-Y. Focus+ context route zooming and information overlay in 3D urban environments. IEEE Trans. Vis. Comput. Graph. 2009, 15, 1547–1554. [Google Scholar] [PubMed]
  20. Dibiase, D.; MacEachren, A.M.; Krygier, J.B.; Reeves, C. Animation and the role of map design in scientific visualization. Cartogr. Geogr. Inf. Sci. 1992, 19, 201–214. [Google Scholar] [CrossRef]
  21. Deng, H.; Zhang, L.; Han, C.; Ren, Y.; Zhang, L.; Li, J. Efficient occlusion-free visualization for navigation in mountainous areas. Comput. Geosci. 2013, 52, 389–397. [Google Scholar] [CrossRef]
  22. Li, Z.; Yan, H.; Ai, T.; Chen, J. Automated building generalization based on urban morphology and Gestalt theory. Int. J. Geogr. Inf. Sci. 2004, 18, 513–534. [Google Scholar] [CrossRef]
  23. Nan, L.; Sharf, A.; Xie, K.; Wong, T.-T.; Deussen, O.; Cohen-Or, D.; Chen, B. Conjoining Gestalt rules for abstraction of architectural drawings. ACM Trans. Graph. 2011, 30, 185. [Google Scholar] [CrossRef]
  24. Patricios, N.N. Urban design principles of the original neighbourhood concepts. Urban Morphol. 2002, 6, 21–32. [Google Scholar]
  25. Regnauld, N. August. Recognition of building clusters for generalization. In Proceedings of the 7th International Symposium on Spatial Data Handling, Delft, the Netherlands, 12–16 August 1996; Volume 1, pp. 185–198.
  26. Press, W.H.; Teukolsky, S.A.; Vetterling, W.T.; Flannery, B.P. Numerical Recipes in C: The Art of Scientific Computing (Second Edition); Cambridge University England EPress: Cambridge, UK, 1992. [Google Scholar]
  27. Wang, Z.; Zhang, L.; Fang, T.; Mathiopoulos, T.; Tong, X.; Qu, H.; Xiao, Z.; Li, F.; Chen, D. A Multiscale and Hierarchical Feature Extraction Method for Terrestrial Laser Scanning Point Cloud Classification. IEEE Trans. Geosci. Remote Sens. 2015, 53, 2409–2425. [Google Scholar] [CrossRef]
Figure 1. Animation snapshots in navigating Hong Kong city. (a) Ordinary perspective view. Corresponding occlusion-free view with spatial-temporal coherence. The route (in blue) is occluded by surrounding buildings in (a) while it is clearly visible in (b).
Figure 1. Animation snapshots in navigating Hong Kong city. (a) Ordinary perspective view. Corresponding occlusion-free view with spatial-temporal coherence. The route (in blue) is occluded by surrounding buildings in (a) while it is clearly visible in (b).
Ijgi 05 00138 g001
Figure 2. Disocclusion of the relevant route (blue): (a) the original view; (b) road broadening; and (c) building displacement. Some buildings are moved too far. (d) The buildings are zoomed out. (e) Combined road widening, building displacement, and building scaling to remove occlusions. Compared to (bd), the view in (e) not only eliminates occlusions but also retains a resemblance to (a).
Figure 2. Disocclusion of the relevant route (blue): (a) the original view; (b) road broadening; and (c) building displacement. Some buildings are moved too far. (d) The buildings are zoomed out. (e) Combined road widening, building displacement, and building scaling to remove occlusions. Compared to (bd), the view in (e) not only eliminates occlusions but also retains a resemblance to (a).
Ijgi 05 00138 g002
Figure 3. Disocclusion operators.
Figure 3. Disocclusion operators.
Ijgi 05 00138 g003
Figure 4. Visibility analysis for a road and building. (a) If segment AB is visible from the viewpoint, the whole road ABCD is visible; and (b) similarly, if the segments AD and DC are visible from the viewpoint, the building is visible.
Figure 4. Visibility analysis for a road and building. (a) If segment AB is visible from the viewpoint, the whole road ABCD is visible; and (b) similarly, if the segments AD and DC are visible from the viewpoint, the building is visible.
Ijgi 05 00138 g004
Figure 5. Visibility measurement for the line segment AB.
Figure 5. Visibility measurement for the line segment AB.
Ijgi 05 00138 g005
Figure 6. Road representation.
Figure 6. Road representation.
Ijgi 05 00138 g006
Figure 7. The framework of the presented method.
Figure 7. The framework of the presented method.
Ijgi 05 00138 g007
Figure 8. A z-shaped road in Hong Kong: (a) original perspective view; (b) occlusion-free view; and (c) the original perspective view from a very high camera height.
Figure 8. A z-shaped road in Hong Kong: (a) original perspective view; (b) occlusion-free view; and (c) the original perspective view from a very high camera height.
Ijgi 05 00138 g008
Figure 9. A road in Vancouver: (a) original perspective view with occluded intersections; and (b) occlusion-free view showing clear intersections.
Figure 9. A road in Vancouver: (a) original perspective view with occluded intersections; and (b) occlusion-free view showing clear intersections.
Ijgi 05 00138 g009
Figure 10. A route and some associated landmarks in Vancouver. (a) The route (blue) and its associated four landmarks (red rectangles) are occluded by other buildings; and (b) the same route and the landmarks after disocclusion by our optimization method.
Figure 10. A route and some associated landmarks in Vancouver. (a) The route (blue) and its associated four landmarks (red rectangles) are occluded by other buildings; and (b) the same route and the landmarks after disocclusion by our optimization method.
Ijgi 05 00138 g010
Figure 11. (a) An occlusion-free route network in Vancouver city; and (b) when the route is very complex, the buildings shown in the red rectangle may be over-scaled.
Figure 11. (a) An occlusion-free route network in Vancouver city; and (b) when the route is very complex, the buildings shown in the red rectangle may be over-scaled.
Ijgi 05 00138 g011
Figure 12. Animation snapshots when navigating in Vancouver. The route segments closest to the car (highlighted in pink) are disoccluded.
Figure 12. Animation snapshots when navigating in Vancouver. The route segments closest to the car (highlighted in pink) are disoccluded.
Ijgi 05 00138 g012
Figure 13. Occlusion-free views with different weights: (a) original perspective view; (b) occlusion-free view with predefined weights; (c,d) occlusion-free views with different ωt values; (e,f) occlusion-free views with different ωs values; and (g,h) occlusion-free views with different ωrd values.
Figure 13. Occlusion-free views with different weights: (a) original perspective view; (b) occlusion-free view with predefined weights; (c,d) occlusion-free views with different ωt values; (e,f) occlusion-free views with different ωs values; and (g,h) occlusion-free views with different ωrd values.
Ijgi 05 00138 g013
Table 1. Quantitative results of the three operators.
Table 1. Quantitative results of the three operators.
FiguresMax. Scale_xMax. Scale_yMax. Scale_z
Figure 80.9530.7970.302
Figure 90.7120.8240.269
Figure 100.7360.8010.325
Figure 110.6270.7450.125
Max. Scale_x, Max. Scale_y, Max. Scale_z: maximum scaling along the X-, Y-, and Z-axes, respectively.
Table 2. The occlusion and deviation measurements values for the scenes shown in the figures.
Table 2. The occlusion and deviation measurements values for the scenes shown in the figures.
FiguresThe Total Occlusion Measurement ValueThe Total Deviation Measurement Value
Figure 11127.81854.9
Figure 81201.31984.1
Figure 91236.61991.4
Figure 101382.92011.2
Figure 111514.72179.1
Table 3. Statistics for the experimental examples.
Table 3. Statistics for the experimental examples.
Figure#{Occ.}#{Rd}Opt. TimeFPS0FPS
Figure 1.40–8521116–30321.5–28.310.1–6.4
Figure 7.33711328.812.7
Figure 8.1082634327.57.6
Figure 9.2282879.87.7
Figure 10.10840100710.35.2
#{Occ.}: the number of occluded buildings; #{Rd.}: the number of road segments; Opt. time: time cost for optimization (in ms); FPS0: framerates at runtime in the original perspective views; and FPS: framerates at runtime in the occlusion-free views.
Table 4. Timings and accuracies for FOI identification with the CP, OF, and FC views.
Table 4. Timings and accuracies for FOI identification with the CP, OF, and FC views.
Example123456
Time (CP)74.1 s49.8 s98 s105 s109 s239 s
Acc. (CP)83.3%80.0%85.7%83.3%77.8%70.0%
Time (OF)36.3 s26.1 s52.8 s61.5 s69.9 s92.7 s
Acc. (OF)100%87.5%90.9%81.8%87.5%88.9%
Time (FC)45.3 s40.0 s79.5 s76.5 s77.8 s133 s
Acc. (FC)87.5%88.9%88.9%80.0%80.0%75.0%

Share and Cite

MDPI and ACS Style

Zhang, L.; Zhang, L.; Xu, X. Occlusion-Free Visualization of Important Geographic Features in 3D Urban Environments. ISPRS Int. J. Geo-Inf. 2016, 5, 138. https://0-doi-org.brum.beds.ac.uk/10.3390/ijgi5080138

AMA Style

Zhang L, Zhang L, Xu X. Occlusion-Free Visualization of Important Geographic Features in 3D Urban Environments. ISPRS International Journal of Geo-Information. 2016; 5(8):138. https://0-doi-org.brum.beds.ac.uk/10.3390/ijgi5080138

Chicago/Turabian Style

Zhang, Liang, Liqiang Zhang, and Xiang Xu. 2016. "Occlusion-Free Visualization of Important Geographic Features in 3D Urban Environments" ISPRS International Journal of Geo-Information 5, no. 8: 138. https://0-doi-org.brum.beds.ac.uk/10.3390/ijgi5080138

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop