Showing posts with label NLP. Show all posts
Showing posts with label NLP. Show all posts

Wednesday, September 02, 2026

EPB: Collaborations and Topics Over Decade

When Environment and Planning B: Urban Analytics (EPB) turned 50 we wrote a short commentary about how the topics in the journal have evolved over the years. In the latest issue of the journal we (Na Jiang, Joana Barros, Seraphim Alvanides, Jinlin Wu and myself) revisited this, while at the same time explored how collaborations and topics have been evolving along with how this links to the new Editorial Advisory Board which has recently been selected to reflect the journal’s thematic areas of focus and the breadth of expertise needed to guide it into the future. 

If you wish to find out more, we encourage you to read the commentary (a link is provided at the bottom of this post). However, to give a sense of some of the things we explore, below we show some of the images relating to the commentary ranging from our workflow to some of our results from the analysis of authors, paper titles and abstracts, and how through topic modeling and community detection we can explore how the collaboration networks and topics have changed over time. 

The workflow.
 
Visualization of the EPB author network.

Evolution of the largest 10 communities by decade: (a) The Number of Active Members; (b) The Number of Papers Published; (c) Michael Batty’s Collaboration Network.

Topics evolution overtime.
Full reference: 

Crooks, A.T., Jiang, N., Barros, J., Alvanides, S., & Wu, J. (2026). Environment and Planning B: Collaborations and Topics Over Decades, Environment and Planning B, 53(6), 1189-1199. https://doi.org/10.1177/23998083261474805. (pdf)

Wednesday, May 27, 2026

New Paper: Exploring Fear in Urban Environments

In the past we have written about how we have used social media to study a plethora of topics with respect the the form and function of cities among many other things. But one thing we have not explored is fear and more specifically fear of crime and how this can be mined through geosocial media

This has now changed with a new paper entitled "Exploring Fear in Urban Environments: Place and Space Analysis of Social Media Data" which has recently been published in Applied Geography.  In this paper, Ying Zhou and myself extract fear related posts from social media and examine the places and spaces where people experience fear, as well as the factors that contribute to it in New York City. 

We do this by utilizing Natural Language Processing (NLP) techniques for sentiment and text analysis, including a RoBERTa-based emotion classification model and the BERTopic model for topic modeling. The former model narrowed the raw data to those with the dominant emotion of fear, and the latter analyzed space- and place-related features that contribute to the fear sentiment. Then, the selected social media data were analyzed using spatial clustering methods (i.e., Hotspot Analysis (Getis-Ord Gi*) and Local Moran’s I) and compared with urban crime data for weekly trends and spatial patterns. As such the paper has the following research objectives:
  1. exploring places where people expressed fear through social media; 
  2. making comparisons between safety-related fear and crime from the perspective of both time and space; 
  3. extracting urban environmental and social features that lead to fear.

If this sounds of interest, and you wish to find out more with respect to our findings, below you can read the abstract to the paper, see some of the figures which describe our research methodology and results while at the bottom of the post you can find a link to the paper itself. Finally the code we utilized in the paper can be found at https://osf.io/y7xfc/overview.

Abstract:

One goal of creating livable cities is to enhance public safety. While previous research in urban studies has focused on correlations between physical environments and crime, it has typically relied on criminal statistics. However, fear of crime is an emotional response to perceived risks rather than a direct reflection of crime levels, so it cannot be analyzed solely by crime data. Additionally, urban planning today has gradually shifted its focus from a top-down to a bottom-up approach, making it essential to understand and foster spaces where residents feel safe. This research examines the spaces and places where people experience fear, as well as the factors that contribute to it, in New York City. We utilized social media data to gather people’s expressions of the city and identified posts expressing fear emotion using the RoBERTa-based model and a rule-based classifier. Then, the selected social media data and crime were compared temporally by weekly trends and spatially by clustering methods (i.e., Hotspot Analysis (Getis-Ord Gi*) and Local Moran’s I). The results show that their temporal and spatial patterns partially have limited alignment. To delve into the origins of fear, we extend our analysis by adopting BERTopic to identify topics and summarize them into themes (e.g., places, transportation, people, others) to understand the bottom-up emergence of fear, thereby informing a people-centered approach to research on urban issues. 

Keywords: Social media; Natural language processing; Sentiment analysis; Urban environment.

Methodology framework.

An example of textual analysis on fear-related tweets: from machine-generated topics to human-interpreted themes describing fear in NYC.

Weekly trends comparison between safety-related fear and violent crime.

Clustering features analysis by the method of hotspot analysis (Getis-Ord Gi∗).

Full Reference: 

Zhou, Y. and Crooks, A.T. (2026), Exploring Fear in Urban Environments: Place and Space Analysis of Social Media Data, Applied Geography, 192: 104051 (pdf)

Tuesday, April 22, 2025

Mapping the Invisible

Readers might of noticed that recently we have been exploring the use of street view images to explore cities or how we can utilize geosocial media to understand the form of function of cities, but one thing we have not explored is the role of smell and how it shapes peoples perceptions of urban spaces. However, in a new paper recently published in the Annals of the American Association of Geographers with Qingqing Chen, Ate Poorthuis we do just that. The paper is entitled "Mapping the Invisible: Decoding Perceived Urban Smells Through Geosocial Media in New York City" In the paper we use text mining techniques to tease out smell related information from over 56 million geolocated tweets which are then assigned to specific small categories (e.g., nature, food, waste) resulting in a new smellscape map for New York city. 

If this sounds of interest, below you can read the abstract to our paper, see our workflow and resulting smellscape map. While the the analysis steps, along with the smell dictionary used, are documented in the research code compendium at  https://figshare.com/s/8418d47cdc5c539b78ab. Finally at the bottom of the page, you can find the full reference and a link to the paper. 

Abstract:

Smells can shape people’s perceptions of urban spaces, influencing how individuals relate themselves to the environment both physically and emotionally. Although the urban environment has long been conceived as a multisensory experience, research has mainly focused on the visual dimension, leaving smell largely understudied. This article aims to construct a flexible and efficient bottom-up framework for capturing and classifying perceived urban smells from individuals based on geosocial media data, thus, increasing our understanding of this relatively neglected sensory dimension in urban studies. We take New York City as a case study and decode perceived smells by teasing out specific smell-related indicator words through text mining techniques from a historical set of geosocial media data (i.e., Twitter/X). The data set consists of more than 56 million data points sent by more than 3.2 million users. The results demonstrate that this approach, which combines quantitative analysis with qualitative insights, can not only reveal “hidden” places with clear spatial smell patterns, but also capture elusive smells that might otherwise be overlooked. By making perceived smells measurable and visible, we can gain a more nuanced understanding of smellscapes and people’s sensory experiences within the urban environment. Overall, we hope our study opens up new possibilities for understanding urban spaces through an olfactory lens and, more broadly, multisensory urban experience research. 

Key Words: geosocial media, multisensory urban experiences, network analysis, New York City, smellscape, text mining, urban smells.

A framework of deriving perceived smells.

An overview of research workflow.

An overview of the six dominant overlapping smells across New York City using the weaving mapping method. The weaving map uses the concept of strands to represent attributes. Each strand here represents one specific smell category, with the intensity of the color changing based on the density of that smell category within each neighborhood (i.e., grid cells).

Full Reference: 

Chen, Q., Poorthuis A. and Crooks, A.T., (2025), Mapping the Invisible: Decoding Perceived Urban Smells through Geosocial Media in New York City, Annals of the American Association of Geographers, 115(6), 1444-1464. Available at https://doi.org/10.1080/24694452.2025.2485233. (pdf)

Thursday, February 06, 2025

From print to perspective: A mixed-method analysis of the convergence and divergence of COVID-19 topics in newspapers and interviews

In previous posts we have noted how one can explore urban issues through newspapers, while at the same time we have used social media to explore trends in vaccinations. In a recently published paper in PLOS Digital Health entitled "From print to perspective: A mixed-method analysis of the convergence and divergence of COVID-19 topics in newspapers and interviews" with Qingqing Chen, Adam Sullivan, Jennifer Surtees, Laurene Tumiel-Berhalter and myself, we thought we would explore how COVID-19 was reported in newspapers and how this varied from interviews. 

The rationale behind this was that the COVID-19 pandemic has led to diverse experiences influenced by public health measures like lockdowns and social distancing. To explore these dynamics, we introduce a novel ’big-thick’ data approach that integrates extensive U.S. newspaper data with detailed interviews. By employing natural language processing (NLP) and geoparsing techniques, we identify key topics related to the pandemic and vaccinations both in newspapers and personal narratives from interviews, and compare the (spatial) convergences and divergences between them. 

We found that both sources converge to highlight the profound impacts of the pandemic on daily life. However, newspapers provide a macro-level perspective, predominately covering policy, public health efforts and economics, while interviews reveal the nuanced impacts at the micro-level, focusing on personal experiences, emotion and concerns. An intriguing finding is the pronounced concern regarding the reliability of news information from interviews. By showcasing both convergences and divergences in identified topics, our study enhances the understanding of key issues that both disseminated to and resonate with the public, contributing to the development of more effective communication strategies for future public health crises.

If this sounds of interest, below you can read the abstract to the paper, see some of the figures which include our workflow and some of the results. At the bottom of the post you can see the full reference and a link to the actual paper. While at https://figshare.com/s/339b1c0d059c189dd6a4?file=44583661 you can find the code we used for our analysis. 

Abstract:

In the face of the unprecedented COVID-19 pandemic, various government-led initiatives and individual actions (e.g., lockdowns, social distancing, and masking) have resulted in diverse pandemic experiences. This study aims to explore these varied experiences to inform more proactive responses for future public health crises. Employing a novel “big-thick” data approach, we analyze and compare key pandemic-related topics that have been disseminated to the public through newspapers with those collected from the public via interviews. Specifically, we utilized 82,533 U.S. newspaper articles from January 2020 to December 2021 and supplemented this “big” dataset with “thick” data from interviews and focus groups for topic modeling. Identified key topics were contextualized, compared and visualized at different scales to reveal areas of convergence and divergence. We found seven key topics from the “big” newspaper dataset, providing a macro-level view that covers public health, policies and economics. Conversely, three divergent topics were derived from the “thick” interview data, offering a micro-level view that focuses more on individuals’ experiences, emotions and concerns. A notable finding is the public’s concern about the reliability of news information, suggesting the need for further investigation on the impacts of mass media in shaping the public’s perception and behavior. Overall, by exploring the convergence and divergence in identified topics, our study offers new insights into the complex impacts of the pandemic and enhances our understanding of key issues both disseminated to and resonating with the public, paving the way for further health communication and policy-making.
An overview of the research workflow.

The monthly distribution of collected articles in the United States from January 2020 to December 2021.

An example of identified entities labeled with predefined entity types.

The spatial distribution of newspaper articles by different scales.


The spatial distribution of identified newspaper topics across different regions in New York State.

Ordered rank of identified topics by percentage from interviews.

Full reference:
Chen, Q., Crooks, A.T., Sullivan, A.J., Surtees, J.A. and Tumiel-Berhalter, L. (2025). From Print to Perspective: A mixed-method analysis of the convergence and divergence of COVID-19 topics in newspapers and interviews, PLOS Digital Health. Available at https://doi.org/10.1371/journal.pdig.0000736. (pdf)

Wednesday, October 04, 2023

Leveraging newspapers to understand urban issues

In the past, this blog has explored several aspects of Detroit, such as how well its covered with Volunteered Street View Imagery or how through the use of agent-based models one can explore issues with urban shrinkage. Keeping up with the theme of shrinkage and Detroit but at the same time utilizing our growing interest in natural language processing (especially topic modeling) we (Na (Richard) Jiang, Hamdi Kavak, Wenjing Wang and myself) have a new paper entitled "Leveraging newspapers to understand urban issues: A longitudinal analysis of urban shrinkage in Detroit" published in Environment and Planning B

In the paper, we take 6794 English news articles published by national and local press organizations (e.g., Forbes, The New York Times, Newsweek, The Detroit News) between 1975 to 2021 using the keywords “Detroit”, “shrink” and “decline.” These keywords were selected based on the characteristics of the study area (i.e., Detroit) and the phenomenon of urban shrinkage. With these data we then use BERTopic to detect and classify all collected news articles into certain topics. We chose BERTopic because it captures the semantic relationship among words converting sentences and words to embedding and automatically generates the topic unlike other NLP topic modeling techniques (e.g., LDA). Our topic modeling results identify several insights with respect to Detroit's shrinkage. For example, we can detect the side effects of the 2007-2009 economic recession on Detroit's automobile industry, local employment status, and the housing market. If sounds of interest and you want to find out more, below we provide the abstract, some figures from the paper including the methodology workflow and an example of the resulting topics over time. Finally, at the bottom of the page you can see the full reference and s link to the paper itself.

Abstract 

Today we are awash with data, especially when it comes to studying cities from a diverse data ecosystem ranging from demographic to remotely sensed imagery and social media. This has led to the growth of urban analytics providing new ways to conduct quantitative research within cities. One area that has seen significant growth is using natural language processing techniques on text data from social media to explore various issues relating to urban morphology. However, we would argue that social media only provides limited insights when dealing with longer-term urban phenomena, such as the growth and shrinkage of cities. This relates to the fact that social media is a relatively recent phenomenon compared to longer-term urban problems that take decades to emerge. Concerning longer-term coverage, newspapers, which are increasingly becoming digitized, provide the possibility to overcome the limitations of social media and provide insights over a timeframe that social media does not. To demonstrate the utility of newspapers for urban analytics and to study longer-term urban issues, we utilize an advanced topic modeling technique (i.e., BERTopic) on a large number of newspaper articles from 1975 to 2021 to explore urban shrinkage in Detroit. Our topic modeling results reveal insights related to how Detroit shrinks. For example, side effects of 2007 to 2009 economic recessions on Detroit’s automobile industry, local employment status, and the housing market. 

Key Words: Natural Language Processing, Topic Modeling, Newspapers, Urban Shrinkage, Urban Analytics.

 

 Vacancy status change from 1970 to 2010 for city of Detroit and surrounding area.
Topic modeling work flow.
Topics over time (a) urban, (b) population, (c) shrinkage, (d) economy, (e) job, (f) house.

Full Reference:

Jiang, N., Crooks, A.T., Kavak, H. and Wang, W. (2023), Leveraging Newspapers to Understand Urban Issues: A Longitudinal Analysis of Urban Shrinkage in Detroit, Environment and Planning B. Available at https://doi.org/10.1177/23998083231204695. (pdf)

Wednesday, March 22, 2023

AAG 2023 Presentations

At this years Association of American Geographers (AAG) Annual Meeting we have a number of presentations ranging from how one can leverage newspaper articles to study cities over time, to that of how people may chose to become vaccinated. These presentations build on the great work of students and postdocs here at the University at Buffalo and link to our interests in urban analytics, machine learning and agent-based modeling. Below we just give a glimpse at these topics (along with their abstracts) and if you are interested in finding out more please reach out to us.

First up is a presentation with Qingqing Chen and Boyu Wang entitled "Community resilience to wildfires: A network analysis approach utilizing human mobility data."  In this presentation we explore how we can quantify a communities resilience to wildfires utilizing human mobility through network analysis methods. 

Abstract 

Natural disasters, such as earthquakes, floods, and wildfires, have been a long-standing concern to societies at large. With growing attention being paid to sustainable and resilient communities, such concern has been brought to the forefront of resilience studies. However, the definition of disaster resilience is intricate and can vary across the diverse disciplines that study them (e.g., geography, sociology and political science), making its definition and quantification elusive. Moreover, the vast majority of studies often focus on the immediate response to an event, not the long-term recovery of the area impacted by disasters. Thus to date investigating the resilience of an area or a society over a prolonged period of time has remained largely unexplored. To overcome these issues, we propose a novel approach from a social perspective utilizing network analysis and concepts from disaster science (e.g., the resilience triangle) to quantify the long-term impacts of wildfires, especially on collective human behavior. Taking the Camp and Mendocino Complex wildfires - the most deadly and the largest complex wildfires in California to date, respectively - as case studies, we capture the features of resilience, such as robustness and vulnerability, of communities based on human mobility data from 2018 to 2020. The results show that demographic and socioeconomic characteristics alone only partially capture community resilience, however, by leveraging human mobility data and network analysis techniques, we can enhance our understanding of resilience over space and time, which can provide a new lens to study natural disasters and their long-term impacts on society.

Keywords: Community Resilience, Natural Disasters, Wildfires, Social Network Analysis, Human Mobility, Space and Time.

Full Reference

Chen, Q., Wang, B. and Crooks, A.T. (2023), Community Resilience to Wildfires: A Network Analysis Approach Utilizing Human Mobility Data, The Association of American Geographers (AAG) Annual Meeting, 23rd –27th March, Denver, CO. (pdf)

Next up, moving from mobility to textural data, specifically that of newspapers Na (Richard) Jiang and myself have a presentation entitled "Leveraging Newspapers to Understand Urban Issues: A Longitudinal Analysis of Urban Shrinkage in Detroit". In this work we explore how can leverage Bertopic (a topic modeling technique) on newspaper articles spanning the years 1975 to 2021 to explore urban shrinkage in Detroit. 

 

Abstract 

Today we are awash with data especially when it comes to studying cities from a diverse data ecosystem ranging from demographic to that of remotely sensed imagery and social media. This has led to the growth of geographical data science and urban analytics providing new ways to conduct quantitative research within cities. One area that has seen significant growth is that of using natural language processing techniques on text data from social media to explore various issues relating to urban morphology. However, social media only provides limited insights when dealing with longer-term urban phenomena, such as the growth and shrinkage of cities. This relates to the fact that social media is a relatively recent phenomenon compared to more longer-term urban problems that take decades to emerge. With respect to the longer-term coverage, newspapers which are increasingly becoming digitized provide the possibility to overcome the limitations of social media and provide insights over a timeframe that social media does not. To demonstrate the utilization of newspapers within urban analytics and to study longer-term urban issues, we present an advanced topic modeling technique (i.e., Bertopic) on a large number of newspaper articles spanning the years 1975 to 2021 to explore urban shrinkage in Detroit. Our topic modeling results reveal the insights related to Detroit's shrinkage can be linked to the side effects of economic recessions on Detroit's automobile industry, local employment status, and the housing market. As such, this work demonstrates the potential of utilizing newspaper articles to study long-term issues

Keywords: Natural Language Processing, Topic Modeling, Newspapers, Text Data, Urban Shrinkage, Urban Analytics. 

 Full Reference

Jiang, N., and Crooks A.T. (2023), Leveraging Newspapers to Understand Urban Issues: A Longitudinal Analysis of Urban Shrinkage in Detroit, The Association of American Geographers (AAG) Annual Meeting, 23rd –27th March, Denver, CO. (pdf)

Switching gears slightly, we have another presentation that leverages text data, in this case Yelp reviews to help inform decision making within an agent-based model. This presentation with Boyu Wang is entitled "Do people care about others' opinions of places? Utilizing crowdsourced data and deep learning to model peoples’ review patterns."  We use a geospatial artificial intelligence (GeoAI) technique called aspect-based sentiment analysis to extract and categorize reviewers' opinion aspects on places within urban areas and then use this information to inform an agent-based model of peoples choices to which restaurants to go to.


Abstract  

People's opinions are one of the defining factors that turn spaces into meaningful places. While these opinions are subject to individual differences, they can also be influenced by the opinions from others. Online platforms such as Yelp allow users to publish their reviews on businesses. To understand reviewers' opinion formation processes and the emergent patterns of published opinions, we utilize geospatial artificial intelligence (GeoAI) techniques especially that of aspect-based sentiment analysis methods (a deep learning approach) on a geographically explicit Yelp dataset to extract and categorize reviewers' opinion aspects on places within urban areas. Such data is then used as a basis to inform an agent-based model, where reviewers' (i.e., agents') opinions are characterized by opinion dynamics. The parameters of these models are calibrated using extracted opinion aspects from the Yelp dataset. Such a method moves opinion dynamics models away from theoretical concepts to a more data-driven approach, with a specific emphasis being made on place. Focusing on 10 US metropolitan areas which are spread out across the country, we examine the calibrated influence coefficients for each opinion aspect category (e.g., location, experience, service), to compare reviewers' opinion formation processes across different categories. The results show the emergent patterns of reviewers' opinions and the influence of these opinions on others. As such this work demonstrates how using deep learning techniques on geospatial data can help advance our understanding of place and cities more generally.

Keywords: Agent-Based Modeling, Crowdsourcing, Deep Learning, GeoAI, Opinion Dynamics, Urban Analytics

Full Reference

Wang, B. and Crooks, A.T. (2023), Do People Care About Others' Opinions of Places? Utilizing Crowdsourced Data and Deep Learning to Model Peoples’ Review Patterns, The Association of American Geographers (AAG) Annual Meeting, 23rd –27th March, Denver, CO. (pdf)
Following with the agent-based modeling theme, our final presentation with Fuzhen Yin and Li Yin is entitled "How Information Propagation in Physical, Relational and Cyber Spaces Affects Covid-19 Vaccine Uptake: Evidence from Rural Areas." In this work we explore how people may or not be influenced by others (in physical, relational and cyber spaces) with respect to vaccination uptake.


 
Abstract 
With the advent of information and communication technologies, human dynamics studied in a purely physical space increasingly shift to a cyber and relational context. While researchers increasingly recognize the shift and call for attention to the multi-dimensionality of human dynamics (e.g., Splatial framework). Rarely have studies investigated how the information propagated in hybrid spaces affects people’s decision-making process, such as Covid-19 vaccine uptake. Meanwhile, compared to the urban population, the rural population faces greater digital barriers and has been further left out in human dynamics research. To fill this gap, our study investigates Covid-19 vaccine uptake in a rural county (i.e., Chautauqua) in New York State through agent-based modeling. We first generated a synthetic population to match the demographic characteristics of the census data. Then we created home, work, school, and social media networks to represent hybrid spaces. We defined the opinion dynamics of agents based on the social influence network theory. Next, we calibrated and validated our agent-based model based on real-world vaccine update records. Our research helps to elucidate the information propagation mechanism in hybrid spaces and clarify the decision-making process in the digital age. Furthermore, our method can also shed light on how to overcome data limitations for under-represented populations such as those who live in rural areas.

Keywords: Agent-based modeling, Covid-19, Vaccination, Opinion dynamics, Urban informatics, Rural geography

Full Reference 
Yin, F., Crooks, A.T. and Yin, L. (2023), How Information Propagation in Physical, Relational and Cyber Spaces Affects Covid-19 Vaccine Uptake: Evidence from Rural County, The Association of American Geographers (AAG) Annual Meeting, 23rd –27th March, Denver, CO. (pdf)