Social Media Data in Urban Design and Landscape Research: A Comprehensive Literature Review

Abstract

Social media data have been widely used in natural sciences and social sciences in the past 5 years, benefiting from the rapid development of deep learning frameworks and Web 2.0. Its advantages have gradually emerged in urban design, urban planning, landscape architecture design, sustainable tourism, and other disciplines. This study aims to obtain an overview of social media data in urban design and landscape research through literature reviews and bibliometric visualization as a comprehensive review article. The dataset consists of 1220 articles and reviews works from SSCI, SCIE, and A&HCI, based on the Web of Science core collection, respectively. The research progress and main development directions of location-based social media, text mining, and image vision are introduced. Moreover, we introduce Citespace, a computer-network-based bibliometric visualization, and discuss the timeline trends, hot burst keywords, and research articles with high co-citation scores based on Citespace. The Citespace bibliometric visualization tool facilitates is used to outline future trends in research. The literature review shows that the deep learning framework has great research potential for text emotional analysis, image classification, object detection, image segmentation, and the expression classification of social media data. The intersection of text, images, and metadata provides attractive opportunities as well.

1. Introduction

Urban design and landscapes are constantly generating new public needs in the midst of social change, particularly the growing emphasis on spatial perception and public participation. Specifically, the objectives of urban design and landscape research have focused on environmental issues and quality of life, the nature of cities, and how urban forms can best fit public needs [1]. There is an academic consensus that planning and design should consider public voices in the renewal and design of urban spaces or landscapes [2,3]. More importantly, with the breakthrough progress in deep learning technology and 5G information technology, multi-source big data, such as social media research, have ushered in new opportunities and challenges in the fields of geographic science, urban planning, urban design, and landscape architecture [4].

Social media is an interactive digital media technology based on Web 2.0. Its purpose is to facilitate the sharing of information, ideas, and professional interests among individuals or groups [5]. Social media data have the characteristics of diverse data sources, large amounts and varieties of data, and strong data spontaneity. Social media data promote more democratic planning and more meaningful public participation [6] and can lead to higher satisfaction with respect to the daily use of urban spaces or landscape spaces. The interests and activities of users on social media can reflect not only socioeconomic and political life but also personal views, interests, needs, and behaviors [7]. For example, social-media-based opinion analysis is used in the business sector to obtain timely consumer opinions on specific topics or products in order to create higher brand value. Meanwhile, the government uses social media to track, simulate, and predict public tendencies. Predicting results can help to formulate policies that are more suitable for public sentiment (emotion) and opinions. Content analysis of social media images can be used to predict public visual perception, assess landscape value, etc.

According to statistics, the most active social media platforms in 2021 [8] included Facebook, Twitter, Flickr, Instagram, Sina Weibo, etc. These mainstream social media studies on urban design and landscape will play a key role in public perception and interaction with the natural environment and in shaping conservation and environmental management [8]. However, limited by the web crawler technology and the application programming interface (API) platform of social media platforms, which is not fully open (rate limits, fee models), as well as the limitations on the frequency, total amount, and objects of data acquisition, research in this field is objectively limited [9]. However, the difficulty of data acquisition, incomplete coverage of user metadata, and the difficulty of processing text and image data are the greatest constraints that restrict social media data from assisting in urban design. Social media data have increasingly become a hotspot for urban spatial analysis and urban design research.

Literature reviews of the urban design and landscape fields have rarely been conducted in previous review studies based on social media data. At the same time, there is also a lack of systematic analyses of the intersection of location-based social media (LBSM), natural language processing (NLP), and computer vision (CV), especially since there is a large gap regarding the questions of which urban problems can be solved and future trends using the deep learning framework.

For this paper, a combination of systematic reviews and bibliometric software were used to perform a comprehensive literature review. The main research objectives are to summarize the research themes, research questions and methods, and limitations of social-media-data-based urban design and landscape research, and to extrapolate future research trends. This paper mainly focuses on the systematic review of the main research directions of LBSM, NLP, and CV, discussing which urban and landscape research questions are addressed by metadata, text, and images, respectively. LBSM can act as a proxy for real cities and provide a better understanding of urban spaces’ underlying structures and dynamics [10,11]. NLP technology is mainly derived from word processing technology, which can assess 140 characters of social media data, including clustering topics [12], views on major events [13], and sentiment/emotion analysis [14,15,16]. CV is an important branch of the computer field mainly used to process image information, and spontaneous images represent the public’s visual experiences, perceptions, and expectations [17,18]. To take this analysis a step further, we use Citespace bibliometrics to analyze co-citations, timelines, and burst keywords, and the results will aid in the analysis of existing research and future research trends [19,20]. Specifically, this paper attempts to assess the effectiveness and applicability of social media data in the field of urban design and landscape and to identify future application patterns, development prospects, and urgent limitations. At the urban design and landscape level, such a comprehensive literature review could enable advanced public participation and promote the greater application of science in local government in order to meet the public needs of residents and tourists.

2. Methods

2.1. Data Collection

The literature data source for this paper is the Web of Science core collection (WOS) dataset, and the retrieval time was 3 August 2022. The indexed databases include SCIE, SSCI, and A&HCI. In the core collection, TS = (“social media” or “social network”) and TS = (“urban” or “landscape”). Document type = (article or review), and “SCIE”, “SSCI”, and “A&HCI” were set as citation indices. The keywords “Social media” and “Social networks” are different names for social media research, while urban design and landscape studies are both interdisciplinary sciences, with research literature focusing on architecture, urban planning, physical geography, geography, and environmental science. For this reason, the research areas include “Urban studies”, “Environment sciences and Ecology”, “Architecture”, “Geography” and “Physical Geography”, with a total of 1220 articles as of 3 August 2022.

2.2. Systematic Reviews

We discuss the research progress in the three major branches of LBSM, text mining, and image processing, and which urban and landscape research questions are addressed by metadata, text, and images, respectively. This paper identifies the potential of, and research gaps in, the directions of the fields of LBSM, NLP, and CV, especially deep learning methods. On this basis, social media data exert their advantages for simulation, evaluation, and spatial design, as well as cross-disciplinary research using traditional urban design methods or other types of big data.

The research object of LBSM is mainly social media metadata, and the research content targets user information, check-in information, timestamps, and geographic information as the research objects. Individual data are regarded as data points with spatiotemporal information in LBSM research. Space, time, and identity are regarded as the most important elements of LBSM, and much of this literature has concentrated on spatial issues. LBSM research brings great opportunities and challenges to GIS science, spatiotemporal databases, and spatiotemporal analysis [21]. The analysis of geotagged photographic data and user-posted content is widely considered to be the future research direction of LBSM [22,23].

NLP and CV are shorthand for the intersection of social media data and computer research, which primarily represents computer processing for social media text and images. NLP and CV research is also the main focus of systematic reviews, covering the manual processing of text and images, statistics, machine learning methods, and deep learning methods. In addition to the research methods, the urban and landscape research questions they address are also summarized.

2.3. Bibliometric Visualization

To further explore the literature trends and future directions, the bibliometric visualization method is used for our analysis. Bibliometric visualization is used to show the changes in, and characteristics of, the research literature in a discipline or profession as it develops [24]. The goal of bibliometric visualization is to summarize data in order to present the state of the intellectual structure and emerging trends of a research topic or field [25]. As for the bibliometric analysis software, Citespace is a form of open-source visual text mining literature analysis software.

Specifically, the version of Citespace is version 6.1 R2 in our study, which also provides the visualization of co-citations by the references, clustering timelines, and burst keyword analysis. The main goal of this tool is to facilitate the analysis of emerging trends in a knowledge domain. The visualization method of Citespace is a tree rings, i.e., a geospatial map.

The clustering algorithm of the Citespace visualization software is mainly based on the use of noun terms to detect research focal points in the discipline, which has the great advantage of enabling the visualization mapping of the timeline and burst words. Citespace, by conducting a quantitative analysis of the relevant literature, forms a corresponding knowledge map and provides the latest research progress, frontiers, evolution paths, and future development trends, aiming to provide a reference and basis for relevant research.

Compared with other related reviews, this study actively ignores the analysis of factors such as the institution, country, or journal and focuses more on the analysis of keywords, burst words, clusters, etc. The latter are highly relevant to the research regarding urban design and landscape.

3. Results

3.1. Publication Growth Trend

According to statistics, urban design and landscape research based on social media data can be roughly divided into three stages (Figure 1): From 2009 to 2012, the research was in the initial stage, and the number of published papers was between two and five per year. From 2013 to 2016, the publication of papers was in a slow state of growth, with between 20 and 60 papers published annually. Since 2017, the number of papers has surged, showing a rapid growth trend. The number of papers published each year ranges between 100 and 400. It is foreseeable that the growth rate will continue to grow rapidly.

Exploratory stage (2009–2012): Research from this period falls into two main categories. One is social hot topics or public perceptions of the community environment based on text mining, and the other is the qualitative analysis that explores the role of social media data in perceiving the urban environment.

Initial Development (2013–2016): During this period, the relationship between LBSM and urban space was gradually used as the research context, and the analysis of images or text was appropriately incorporated, but the research methods of image content or text content were mainly based on statistics or manual approaches.

Rapid Development (2017–present): In this stage, with the advancement of machine learning and deep learning, the complexity of the research framework, the scope of urban space research questions, and the accuracy of image recognition and text processing models have all been greatly improved.

3.2. Main Research Directions

3.2.1. Location-Based Social Media and Spatial Analysis

LBSM is the largest research branch of current social media data research. Check-in geodata, user profiles, and timestamps have become the most important metadata elements of social media. Approaches that extract location-based geo-information include latitude and longitude geotagging, place names, and time zones. Using geotagging to extract locations is the main approach [26]. Almost all mainstream social media allow users to share geographic information with timestamps when generating text, photos, videos.

The research perspectives mainly include urban evolution and spatial identification, demographic structure statistics, and crowd activity patterns. The early research (2009–2016) mainly focused on urban evolution research and spatial recognition. With the great improvement of GIS spatial analysis capabilities and the processing capacity of big data, the scale of LBSM research has gradually developed towards refinement. Nowadays, (2017–present), most of the research on the spatiotemporal behavior of population groups is the main focus, and this type of research can also reflect the changes in urban spaces and population structures.

Urban space evolution and identification research: Social media data can reflect the mechanisms of urban natural evolution and boundary division [27]. These experiments usually take months or days as the time-slicing unit of social media data. García Palomares et al. [28] collected data on active Twitter users in Madrid at different times of the day and calculated the activity dynamics of areas with different urban functions at different times of the day. LBSM data can also identify urban public open spaces [29] and urban commercial districts [30] by dynamically detecting urban vitality. More specifically, Pablo Martí et al. [29] identified user preferences in urban public squares using the number of check-ins through Foursquare, the number of visitors, the total number of shared images, and the total number of user tips and likes. Moreover, social media data can also be combined with night-light images, POI data, heat maps, and other multi-source big data to classify urban functional areas [31,32,33], and it can be combined with subway records, taxi GPS tracks, and other mobile data to identify urban central areas [34]. Wu et al. [35] used Weibo check-in data and POI data (divided into housing HPOI, consumption CPOI, transportation TPOI, and other OPOIs) to study Shenzhen’s urban vitality. Strictly speaking, most of the research on urban space evolution and identification is closely related to the research on crowd activity patterns. This type of research is characterized by the large scale of the urban space or landscape and low data accuracy requirements, and most studies require a combination of multiple types of data, such as POI data, demographic yearbooks, and other geographic information.

Demographic structure research: Due to its immense user group and user personal information, social media data can be used to analyze the urban population structure and even to quickly conduct auxiliary statistics on the urban population. Steiger et al. [36] collected one year’s worth of Twitter data in the London core area and analyzed the spatial, temporal, and semantic clustering of urban population activities through topic semantic similarity assessment and spatial autocorrelation. They demonstrated that LBSM data can be used for demographics. At the same time, some scholars have analyzed the population distribution in terms of gender differences [23,37,38,39] and age groups [40], as well as ethnicity and education level [41]. In addition to this, LBSM data can also be used to discover the spatiotemporal behavior of different populations within a specific geographic range. These studies based on LBSM have implications for research engaging with demographics, user gender statistics, social equity in public space, and changing user (tourist and resident) trends in the urban design and landscape area.

Crowd activity patterns research: Existing studies on urban crowd activity patterns based on LBSM data can be divided into several categories. Firstly, there is the activity patterns dynamic analysis of crowd activities based on check-in data, which is characterized by real-time analysis, with spatial and temporal granularity. After data preprocessing, the spatial analysis performed on the ArcGIS platform mainly includes kernel density estimation (KDE) analysis [42,43,44], geographical distribution measurements [30], hexagonal grids [45,46], hotspots [47], and 3D visualization [48]. This type of research is usually based on spatiotemporal information [49]. Lloyd and Cheshire [50] collected Twitter data about retail stores and used adaptive KDE to visualize the flow of people related to retail stores, an approach that is less biased and smoother than traditional KDE methods. Steiger et al. [51] proposed a self-organizing map (Geo-SOM and Geo-H-SOM) for the visualization of human activity patterns. Secondly, there are studies that investigate spatiotemporal activity characteristics and influencing factors of urban space. Such studies usually use metadata information to measure the extent to which the rhythm of human activities is affected by time and spatially show the cluster distribution, core-edge, and distance decay of crowd activities [52]. Zhang and Zhou [53], as well as Lyu and Zhang [54], used multiple linear regressions (statistical analysis) to measure the influencing factors of parks through social media data, and the effects of these findings on planning programming decisions for urban design and landscape analysis were transparent. Thirdly, there are studies that aim to discover the urban renewal trends by identifying the characteristics of population preference and current land use. Most of these studies properly combine the CV [55] and NLP [56,57] methods, and the metadata reflect human behavior patterns [58], while images and text reflect subjective feelings.

However, current research based solely on metadata undoubtedly has limitations in terms of public perception or emotional tendencies. At the same time, the privatization of user information and lack of geo-information are the main reasons why LBSM research is questioned. Many studies opt to retain geo-tagged data only after the data cleaning process, and a large number of images or texts are cleaned at the beginning. With the development of deep learning frameworks, the potential information of text, images, and shared videos on social media can be deeply mined. Overall, LBSM research, combined with the study of text and image, is a future research development trend.

3.2.2. Natural Language Processing and Text Mining

In addition to the spatiotemporal information of LBSM data, social media also include text sharing at the semantic information level, that is, text information with a maximum of 140 characters [59]. The text mining analysis of social media has the potential to be used to understand public opinion, as well as emotional changes. Specifically, approaches to the application of social media text data in urban space and landscape research can be divided into three categories: word frequency and content analysis, topic models and clusters, and sentiment/emotional classification.

Word frequency statistics constitute the most basic and effective method. Most word frequency statistics rely on computational methods [60]. Kim et al. [61] selected the High Line Park as a research case and conducted text mining on Twitter texts over one year. The research included text content analysis (word frequency, tags, topics) and network analysis. At the same time, the word cloud is also an effective word frequency visualization method which is commonly used in NLP research, and the word size relates to the number of texts received [62,63,64,65,66]. Word cloud visualization is performed on keywords using language libraries and analysis software, such as WordArt and Python, and refers to the visual analysis of keywords with a high text occurrence frequency [63]. In addition, the Word2Vec model can extract and identify crowd activity reviews using feature words, also known as a feature learning technique. Each word is represented in the input as a fixed-length feature vector using Word2Vec [67,68]. Kim et al. [69] introduced Word2Vec and the dynamic topic model to capture the voices of citizens. In this study, social issues raised by citizens regarding transportation, the environment, and culture were identified as factors that can effectively contribute to public decision making in urban design. Word2vec has also inspired the variant models, such as Doc2Vec [70] and Place2vec [71], that incorporate POI into urban design research. The embedding-model-processed social media text data are vectorized in these variant models.

Moreover, word frequency analysis is generally accompanied by a topic model and clustering algorithm [72]. Unsupervised methods for the automatic discovery of topics allow machines to read and summarize core themes and concepts. Such NLP tasks are examples of topic modeling and topic clustering, such as latent Dirichlet analysis (LDA) [73] and the density-based spatial clustering of applications with noise (DBSCAN) [74]. The LDA model and its variants are the most frequently used methods in urban and landscape studies, including the exploration of variations in social media platforms used across a whole city [75,76,77], impacts of social events or public opinion [77,78,79,80], and reviews of tourist attractions [81]. On the urban design scale, Song et al. [82] obtained the review data from TripAdvisor about Bryant Park in New York and explored the experiences and emotional changes of residents and tourists through review information. The research used the topic classification model LDA (latent Dirichlet allocation) to analyze the topic and obtained five (T0-T4) topic clusters. However, the LDA topic model relies on subjective human qualitative interpretations and requires domain knowledge of urban design or landscapes [83].

Opinion mining and sentiment/emotion analysis aim to extract the sentiment orientation from given texts [84]. In NLP, sentiment/emotion analysis is the task of treating opinions, sentiment polarity, and subjectivity in the text [85]. Emotion is a feeling or reaction such as “Happy”, “Fear”, or “Sad”, including 6, 9, 15, and 28 classes and other different classification standards [86]. Sentiments represent a polarity of emotion [87], and positive/negative and neutral valence annotation is an active area [88]. In urban design and landscape studies, as a social listening approach, sentiment/emotion analysis often provides insight into the public’s subjective emotional disposition toward the urban space or landscape space, which is expressed in design-related decision making based on public spatial positive and negative reviews [89,90]. Sentiment/emotion mapping is an intuitive way of visualizing spatial distribution and spatial intensity. Most studies on emotion have established a consensus that, while negative emotional tendencies are rare, they are the more valuable for urban design [91]. Traditionally, manual sentiment labeling has been an effective way to understand public responses [92]. Moreover, emotional words or emotional dictionaries are beneficial to the output of accurate results. Emotional dictionaries contain information about the emotion or polarity expressed by words. The main emotional dictionaries in our dataset include the Circumplex Model (i.e., 28 emotional words) [93], SentiStrength lexicon [94], HowNet dictionary [95], NRC Emolex [96], VADAR [12,97,98], AFINN lexicon dictionary [99,100,101], and so on. Using emotional dictionaries is probably the easiest way to express and assess public emotion [102]. However, the accuracy of this method decreases as the complexity of the sentence structure increases.

Furthermore, there are also studies using deep learning text mining methods, such as CNN (convolutional neural network), RNN (recurrent neural network), LSTM (long short-term memory), GRU (gated recurrent units), Transformer BERT (bidirectional encoder representations from transformers), and its optimization frameworks. According to the Stanford University SQuAD2.0 dataset on NLP performance, the performance of BERT and its optimized models in NLP tasks has greatly exceeded human performance (EM = 86.831, F1 = 89.452) [103]. However, for now, the RNN frameworks still occupy a large proportion of NLP tasks in urban space and landscape field. Sun et al. [104] used Word2vec and the LSTM model to measure visitor satisfaction with peri-urban green and open spaces based on Sina Weibo. In addition to the original LSTM model, there are many optimized LSTM models, such as BiLSTM (bidirectional LSTM) and BiLSTM-CNN [105]. Gong et al. [106] used the variant ALSTM (activity LSTM) model of LSTM deep learning, which they developed to analyze Yelp comments. The ALSTM model can more accurately identify text messages on social media and determine the daily activities of residents. As for BERT model, Wang et al. [107] constructed a BERT model for text classification tasks to measure the residents’ perceptions of festivals, and it also combined the cluster model and LDA model, as mentioned earlier. In the text classification task, BERT achieves nearly 90% accuracy based on social media text data [108]. The BERT model also has an obvious advantage in sentiment analysis tasks [109]. Furthermore, optimized BERT and BERT-CNN models can even achieve about a 10% higher accuracy than LSTM (or Bi-LSTM) in sentiment analysis tasks [110,111,112]. However, BiLSTM models can achieve significantly higher results than the BERT model using a small dataset [113]. In conclusion, the deep learning method seems to improve the accuracy of NLP tasks.

3.2.3. Computer Vision and Image Processing

State-of-the-art models for the tasks of automatic image classification, image recognition, and image segmentation via deep learning CV are considered important new tools for studying cities and landscapes [114]. In terms of the research progress, it has developed from image-associated tags, labeling, and ratings to image classification and image content analysis. In terms of the analytical approaches, they have gradually developed from manual tagging or classification to interdisciplinary work, with CV tasks for image processing tasks. In the past, urban and landscape studies using social media images mainly focused on the public perception of urban spaces, cultural ecosystem services, landscape preferences, landscape features, and urban spatial identity, which are classic considerations in urban and landscape design.

Traditionally, image analysis tasks mainly consist of manual image tag classification and manual content analysis. Image tags include a prominent image and title, a description, and proprietary tags, which are, strictly speaking, essential metadata types [115,116]. Firstly, manual tag categorization, or image content analysis, is an effective method for small data, and images are often categorized by landscape features in scenes [117,118,119]. On the other hand, activities and headshots (facial expressions) can also be labeled [120]. In the visualization approach, using image tags, the word cloud in the NLP domain can show the most prominent image tags with geo-information [121], and the DBSCAN clustering algorithm is also a suitable visualization approach [122]. Secondly, for image content analysis, the objective coding approach of manual content analysis is a scientific statistical and classification method [123]. Manual image content criteria and experiences are highly relied upon [117]. To further complement the image content criteria, online questionnaires/surveys [124], automatic REST API analysis [125], and filtering models [126] provide complementary standards. In addition, Google Cloud Vision has particular relevance for the extraction of text from images, facial recognition, and the recognition of image objects, and it is a pre-trained machine learning algorithm. Wartmann et al. [127] used Google Cloud Vision to define a tranquil landscape space, making it possible to find a peaceful landscape space in Scotland. Google Cloud Vision is often used in conjunction with tools such as Python scripting [128], R package [129,130,131], or RoogleVision [132].

However, manual image content analysis and image processing methods based on machine learning have a limited ability to analyze large quantities of social media image data and an insufficient understanding of the complex urban scenes described. With the advent of deep learning, CV tasks have been revolutionized. CV tasks applied to social media image data are only a small part of the vast field of CV. Image classification, image segmentation, and object detection are primarily tasks that apply to social media image data. Figure 2 shows the urban research questions and the main deep learning frameworks used in existing studies that can be addressed by CV tasks. As for CV deep learning frameworks [133], the breakthrough progress in CNNs [134,135], RNN, RBMs (restricted Boltzmann machines), GAN (generative adversarial networks), Transformer, and YOLO (You Only Look Once, version 1 to version 7) has led to their dominance in CV tasks such as image classification, image segmentation, and object detection.

Image classification involves treating the entire image as a whole and assigning a specific label to it. Since the release of the ImageNet dataset and ImageNet large-scale visual recognition challenge (ILSVRC) in 2010, image classification has been one of the most widely researched topics. In ILSVRC, the image classification task is to determine the category to which the object in the image belongs out of 1000 categories (the number of ImageNet categories), mainly using the top-five error rate evaluation method. That is, each image is given five prediction results, and as long as one out of five meets the true category, it is considered the correct classification. In urban design and landscape studies, the introduction of a deep learning image classification frameworks can answer the major research questions of resident/tourist behavior and perception, the identification of urban landscape features, landscape assessment, and natural disaster assessment. ResNet-50 [17,136], ResNet-101 [137], ResNet-152 [138,139], VGGNet [18,114,140,141,142], Densenet-161 [143], and Inception v3 [144] are the deep learning frameworks used in these studies, with VGG-16 and ResNet being the most commonly used. Essentially, the deep learning approach straddles the limits of traditional approaches in terms of accuracy and data volume for the purpose of image content analysis. It reflects research questions that are inextricably linked to subjective user/public perception, which has important implications for urban studies or tourism perception analysis. Furthermore, due to the limited amount of user-generated content (UGC) data shared in urban and landscape spaces, many studies have resorted to image databases for the purpose of pre-training or transfer learning to validate datasets, mainly CIFAR-10, CIFAR-100, ImageNet, and Places365.

Object detection tasks refer to the detection and localization of objects (cars, humans, buildings, etc.) using boundaries in the form of rectangular boxes and the scale of every instance of each object category. The typical technical route is: (1) image segmentation, (2) object detection, (3) object recognition, and (4) object tracking. Deep learning models such as YOLO, Mask-RCNN, Fast-RCNN, Faster-RCNN DeepLab, and SWIN-Transformer can overcome the limit of the number of recognized objects [145]. As regards the current positions of these techniques, Transformer and YOLO have the highest framework box AP (box average precision) in vision, with the SWIN-Transformer framework reaching 63.3 box AP. In urban design and landscape studies, Song et al. [146] used YOLO v3 to perform the object detection of “Vehicle”,” Bicycle”, and” Pet” in High Line Park and the Atlanta Beltline. YOLO can also be used to predict multiple objects in the same image [147,148]. Moreover, facial recognition and expression detection are subparts of object detection, and the main object being detected is the human face from portrait social media images. Ashkezari-Toussi et al. [149] used the Dlib-ml toolkit, based on the iBUG 300-W Facial Land Mark dataset, to detect emotional states, gender, and age using the portrait images on Flickr, which is a novel approach for analyzing urban emotions using machine learning.

Image segmentation is the division of an image into subparts or sub-objects. The main frameworks include Mast-RCNN, SegNet [150], PSPNet [151], DeepLab V3 [152], etc. Strictly speaking, image segmentation technology shows a more intuitive ability in street-view studies and will not be expanded on in this study. Although few existing studies have used image segmentation models for social media images, the segmentation models of street-view images have provided a great deal of inspiration for urban spatial quality assessment. At the same time, since the image segmentation task is inextricably linked with the object detection task, the combination of the two CV tasks still has potential.

3.3. Bibliometric Visualization Results

3.3.1. Co-Citation Literature Analysis

Co-citation analysis is possible when two articles appear in the bibliography of a third article, giving the two articles a co-citation relationship [153,154]. Co-cited references can be aggregated into coupled multiple reference clusters, each representing a common research theme [155]. Co-citation reference statistics indicate the most representative research themes. Using Citespace to visualize co-citation reference articles, the co-citation clustering of 10 keywords was detected after pruning (Figure 3). These words are cultural ecosystem services, KDE, land use classification, urban park, human mobility, Sina-Weibo, recreation, urbanization gradient, energy, and social network analysis.

Table 1. Top 15 co-citation reference articles in urban design and landscape.

Author	Article Title	Year	Count
E. Rozas et al. [117]	Using social media photos to explore the relation between cultural ecosystem services and landscape features across five European sites	2018	53
H. Tenkanen et al.	Instagram, Flickr, or Twitter: Assessing the usability of social media data for visitor monitoring in protected areas	2017	49
Van Zanten et al.	Continental-scale quantification of landscape values using social media data	2016	46
Y Liu et al.	Social Sensing: A New Approach to Understanding Our Socioeconomic Environments	2015	46
V. Heikinheimo et al. [118]	User-Generated Geographic Information for Visitor Monitoring in a National Park: A Comparison of Social Media Data and Visitor Survey	2017	43
Z. Hamstead et al.	Geolocated social media as a rapid indicator of park visitation and equitable park access	2018	43
D. Richards et al. [130]	Using image recognition to automate assessment of cultural ecosystem services from social media photographs	2018	39
M. Donahue et al.	Using social media to understand drivers of urban park visitation in the Twin Cities, MN	2018	39
K. Tieskens et al. [119]	Aesthetic appreciation of the cultural landscape through social media: An analysis of revealed preference in the Dutch river landscape	2018	37
T. Shelton et al. [46]	Social media and the city: Rethinking urban socio-spatial inequality using user-generated geographic information	2015	32
Y. Chen et al.	Delineating urban functional areas with building-level social media data: A dynamic time warping (DTW) distance based k-medoids method	2017	30
N. Yoshimura et al.	Demand and supply of cultural ecosystem services: Use of geotagged photos to map the aesthetic value of landscapes in Hokkaido	2018	30
A. Hausmann et al.	Social Media Data Can Be Used to Understand Tourists’ Preferences for Nature-Based Experiences in Protected Areas	2018	30
P. Tenerelli et al.	Crowdsourcing indicators for cultural ecosystem services: A geographically weighted approach for mountain landscapes	2016	30
A. Dunkel [116]	Visualizing the perceived environment using crowdsourced photo geodata	2015	30

shows the top 15 co-citation reference articles with the year and co-citation frequency count in the urban design and landscape field. The research questions include landscape features, landscape values, delineating urban areas, public monitoring and behavioral patterns, public perception, cultural ecosystem services, aesthetic appreciation, and socio-spatial inequality. It can be seen that geotagged metadata from social media have important implications for spatial analysis. Text mining technology is used in social media text processing through text clustering, text topic anlaysis, content analysis, sentiment classification, etc. Social media image content analysis and tag clustering analysis can be used for landscape assessment or public perception research. This corresponds to the studies on social media using LBSM, NLP, and CV, some of which are also cited in the systematic review section. The majority of these co-citation reference publications were published between 2015 and 2018, and the main research methods are also dominated by statistics and machine learning methods, which provide a strong theoretical foundation for deep learning methods.

3.3.2. Timeline Graph Analysis Based on Keywords

The timeline analysis of keywords focuses on revealing the keyword trends of literature clustering over the time change process, which can effectively reflect the development of social media data in the field of urban design and landscape. The timeline analysis of keywords of the articles in the urban design and landscape field was carried out. Figure 4 shows a timeline view of the dynamic keywords in 10 clusters [19]. At the top of Figure 4 are the periods in years and cluster labels obtained by the LSI algorithm, with the time slice set as 1 year and the period set as 2009 to 2022. The x axis is the period and the y axis is the clustering partition.

The keywords in the same cluster are on the same level. When the keywords appear frequently, it can indicate that these keywords are hot topics in the period. By clustering keywords, we found 10 hot topics: Cluster# 0—cultural ecosystem service from 2009 to 2022, Cluster# 1—social media from 2010 to 2022, Cluster# 2—subject well-being from 2010 to 2022, Cluster# 3—sentiment analysis from 2011 to 2022, Cluster# 4—social network analysis from 2009 to 2020, Cluster# 5—social capital from 2011 to 2022, Cluster# 6—urban planning from 2019 to 2022, Cluster# 7—citizen engagement from 2015 to 2020, Cluster# 8—data mining from 2011 to 2020, and Cluster# 9—smart cities from 2013 to 2022. The timeline analysis consists of 453 nodes and 1668 links. The longest-lasting clusters are “cultural ecosystem service”, “social network analysis”, and “urban planning”, representing a complete research lineage in these three clusters, which continue to be studied with great enthusiasm. Relatively speaking, the clusters of “social network analysis” and “citizen engagement” lasted for a relatively short period. Moreover, it is worth mentioning that Cluster#3, sentiment analysis, and Cluster#4, social network analysis, show a high percentage of links with other clusters. This may also support the argument that NLP plays a significant supporting role in the field of LBSM and CV research.

3.3.3. Burst Keywords Based on Keywords

The outbreak of keywords can reflect changes in research topics and focal points in a certain field and can also further explain the content of the timeline view [156]. In terms of the duration of the burst keywords in Citespace (Figure 5), community and diversity are the longest-lasting keywords, and they are both concentrated in the period from around 2010–2017, which also validates our initial judgment on the exploratory stage and initial development stage.

In terms of intensity, social networks and social network analysis, diversity, spatial analysis, geography, information, and participation are all very strong keywords. The latest burst keywords are experience, perception, strategy, urban park, opportunity, and exposure. This means that urban design and landscape research using social media data has changed from spatial analysis to the analysis of subjective emotion, visual perception, and public experiences. This is due to the transformation of research questions on topics ranging from the spatial functionality to the spatial experiential, based on burst keywords ranging from the community, scale, and spatial analysis to experience and perception. On the other hand, we can observe the shift in research methods from spatial analysis intersecting with geography or physical geography to perception analysis in the field of artificial intelligence. The burst of keywords validates our conclusion in the previous section on NLP and CV trends, namely, that developments in NLP and CV have facilitated the recognition of images and text.

4. Discussion

4.1. Bibliometric Discussion

The bibliometric visualization used in this paper is intended only as a method for the further interpretation of the overall research trends in LBSM, NLP, and CV. In terms of co-citation references, the studies with the highest number of co-citations are mainly those based on LBSM and CV, and the publication dates are also mostly concentrated in the period between 2017 and 2018. This proves that LBSM and CV have formed a close coupling in the research field of urban design and landscapes. As for NLP research, its strengths are more evident in the fields of disaster and social opinion analysis.

Timeline analysis and burst keywords represent the transition of social media data in urban design and landscape research from objective analysis, such as spatial analysis, diversity, and spatial networks, to user subjective analysis, such as perception, sentiment/emotion, and experience. Research topics have expanded from GIS-based spatial and network analysis and social network analysis to include topics such as visual perception and emotional experience, which are inextricably linked to technological developments.

4.2. LBSM Research Prospects

The advantage of LBSM research is that metadata have three major data elements: volunteered geographic information, timestamps, and user information (gender, age, attribution, etc.). Given this characteristic, we believe that the future research directions of LBSM can be divided into three categories:

• Urban space evolution and identification research

Combining other big data types (GPS data, mobile phone signaling data, street-view data, etc.) with social media data for mutual verification, this can mitigate the disadvantage of social media data with missing data on specific population structures.

• Demographic structure statistical research.

Increase the statistics and distribution of demographic data in small areas such as city parks or urban landmarks, and the disadvantage of insufficient user information from social media data can be improved by combining online questionnaires, demographic yearbooks, and mobile phone user information.

• Crowd activity pattern research.

Higher geographic accuracy, smaller time granularity, and the analysis of population activity patterns and spatiotemporal characteristics on multiple scales in the study area. On the other hand, nowadays, more comprehensive research at the intersection of CV and NLP fields is imperative for facing more complex social problems.

4.3. Deep Learning Methods for NLP and CV Tasks

With respect to traditional methods, the greater use statistical methods and evaluation models from the social sciences to assist in understanding users’ visual perceptions and affective tendencies can be expected. Manual text analysis or image classification by more than two researchers would be more scientific approaches. Although manual methods are time-consuming and labor-intensive, it is undeniable that manual methods still perform well for small samples. Furthermore, given that most social media data are only 20% to 40% geotagged, NLP and CV research are more suitable for the urban and landscape issues reflected in social media data without geographic information.

Table 2. Main methods and features of research using social media data for NLP and CV tasks.

Main Data Source	Tasks	Main Method	Features
Sharing Texts Twitter TripAdvisor Facebook Weibo Google Reviews	Text mining	1. Word Frequency 2. TD-IDF 3. Word2Vec 4. LDA Topic 5. DBSCAN and K-means …	1. Effective content analysis method, commonly using word cloud for visualization. 2. TF-IDF is a statistical method used to evaluate the importance of a word to a text set. 3. Word2Vec mainly includes skip-gram and a continuous bag of words (CBOW). 4. LDA is an unsupervised learning model for discovering implicit topic information in text mining tasks. 5. DBSCAN is a density-based clustering algorithm, and the K-means clustering algorithm is suitable for spherical distribution datasets.
	Sentiment (emotion) analysis	1. Human Performance 2. Emotion Lexicons 3. Machine Learning 4. RNN-LSTM, Bi-LSTM, LSTM-CNN 5. Transformer-BERT …	1. Suitable for small data samples with short text. 2. Emotional polarity classification, emotional words are provided by different emotional lexicons. 3. Supervised learning training, mostly used to analyze sentiment polarity. 4. Still performs well using short texts for emotion detection. 5. BERT performed better than LSTM in most cases and showed a better performance for long sentences due to the self-attention module.
Sharing Images Twitter Flickr Panoramio Facebook Instagram	Image classification	1. Human Performance 2. Machine Learning 3. Google Cloud vision 4. Based on CNN 5. Emerging frameworks	1. Concentrate on the tags and scores for ranking references, suitable for small data samples. 2. The recognition rate of traditional methods such as Random Forest, K-means, and SVM, in the case of small samples, is close to the deep learning method. 3. Image label detection based on machine learning. 4. ResNet-50, ResNet-101, ResNet-152, VGGNet, Dense-net-161, Inception v3. Suitable for big data and small data samples, with a high accuracy. Places365 and ImageNet datasets can be used as transfer learning databases for deep learning. 5. The Top1 accuracy rate of the Transformer, EfficientNet, and Conv + Transformer model can be over 90%.
	Object Detection	1. R-CNN 2. YOLO V1-V7 3. Mask-RCNN 4. PSPNet …	1. R-CNN, Fast R-CNN, and Faster R-CNN train a linear regression model to predict the edge box offset. 2. YOLO object detection has the ability of real-time prediction, and objects can be detected using YouTube video content. 3. Mask-RCNN algorithm is composed of Faster-RCNN and the semantic segmentation algorithm mask branch (FCN). 4. PSPNet provides efficient global context priors for pixel-level scene parsing.
	Semantic segmentation
	Facial emotion recognition	1. Manual recognition 2. EmoDetect 3. Google Cloud vision	1. Manual emotion classification and emotion indexes. 2. Extracted to describe the expression. Face landmarks are illustrated on the face. 3. Detecting 8 emotions, including happy, sad, etc.

shows the main research methods and features of research using social media data in the fields of NLP and CV. These methods are used in many fields, such as spatial analysis, ecological assessment, landscape aesthetics, public perception, and sentiment analysis for urban and landscape. In the field of NLP deep learning methods, we believe that, in terms of both research methods and research accuracy, NLP research has a high-level of performance in the fields of content analysis, topic modeling, and sentiment classification. As repeatedly emphasized before, in addition to NLP deep learning frameworks, the deep integration of NLP with LBSM and CV makes this field exciting. As for the CV deep learning method, first of all, the research based on image classification has greater potential, which includes the capacity of the classifier that can classify scenes and crowd activities at the same time. The current research mainly focuses on scene classification and image expression classification that can be combined with object detection to examine people’s public activities. Secondly, the object detection tasks used in previous research focus on single moving objects, and multi-element hybrid detection in urban spaces and landscape spaces can be used to explore the activity patterns or spatial relationships between footpaths, lanes, and bicycle lanes. Thirdly, semantic segmentation models are mostly used in the study of street views, and first-person perspective data provide new ideas for measuring the positional relationships and proportions of various landscape elements in urban spaces.

5. Conclusions

The conclusions are summarized as follows. The research on social media data in the field of urban design and landscapes has entered a period of high development since 2017. Based on the visualization results of Citespace, the current hot topics of research include green space or urban parks, cultural ecosystem services, human activity, spatial-temporal patterns, etc. Based on the research trend, the research focal points have gradually moved from urban or landscape spatial analysis and social network analysis to public perception, experience, and human emotion/sentiment issues. NLP will be used more as an adjunct method of study, with a more significant trend towards combining CV and LBSM.

In terms of research methods, deep learning frameworks for text content and sentiment analysis, image content and detection analysis, social science statistics, and location-based spatiotemporal dynamic analysis will be the mainstream research methods in the future. As for the research questions regarding urban design and landscapes, the research content is still based on people’s spatial cognition and subjective feelings. Specifically, this includes urban space evolution, urban space optimization, demographics, user sentiments, public perception, landscape feature extraction, landscape evaluation, etc. Combined with the dynamic social behavior of people, we can derive research questions on landscape spatial vitality, the dynamic identification of urban functional areas, urban imagery monitoring, and urban dynamic management.

However, the use of social media data is still in its infancy, and the social media data also have many limitations at present, such as the limitations of data acquisition, data volume, and user privacy. Many researchers have questioned the application of these data in the field of urban and landscape research [157,158,159]. The focus of the research questions mainly includes the following aspects:

• Data acquisition is difficult and has many restrictions.
• The age and occupation of users are unevenly distributed, and user information is incomplete.
• Geotagged data account for about 20–30% of the total data, which makes the generalizability of LBSM spatial analysis results questionable.
• A high processing difficulty and low accuracy due to a large amount of text data, images, and video data.
• The contribution of the deep learning outputs in the NLP and CV fields on the urban design or landscape level in spatial analysis.

Based on the existing computer technology, social science technology, and spatial analysis methods, we suggest that the limitations of social media data can be alleviated in three ways. First, this can be achieved by supplementing metadata with traditional urban design methods, such as map marker methods, observation counting methods, and interview methods. Second, when studying specific urban issues, the data should be combined with other types of big data, such as land use functions, GPS data, demographic data, etc. Finally, the analysis of text and images plays a great role in the research problems related to public preferences, such as urban imagery, public perception, and public emotion survey. The deep learning frameworks can overcome the problems of data processing and low accuracy. It should be noted that some doubts and limitations are the defects of current computer technology. For example, it is difficult to automatically add geo-tags to non-geo-tagged image data through image content recognition and use machine learning to simulate the activity trajectories of missing age groups by learning the social habits of young people. One thing is certain: the development of artificial intelligence will also bring immeasurable potential to the application of social media data in the field of urban design and landscapes.