New Paper: A Thematic Similarity Network Approach for Analysis of Places Using VGI

Building upon our work on volunteered geographical information (VGI) and ambient geographic information (AGI) and how such data (e.g. social media) can be used to understand place, Xiaoyi Yuan, Andreas Züfle and myself have a new paper entitled: "A Thematic Similarity Network Approach for Analysis of Places Using Volunteered Geographic Information" in the ISPRS International Journal of Geo-Information.  In this paper we use textual data from crowdsourced reviews originating with TripAdvisor and geo-located Twitter data and leverage this unstructured geographical information to comprehend the complexity of places at scale. Specifically we explore the connectedness and relationships of places through thematic (i.e., topical) similarity networks using Manhattan, New York as a case study. If such work sounds of interest to you, below we provide the abstract to the paper in order for you to gain a greater understanding of work, along with some figures that show our workflow and how communities where connected, before presenting some of our results. Finally at the bottom of the post, the full reference and a link to the paper is provided.  For those interested in extending or utilizing this work. The python code for presented in our analysis is available at: https://bitbucket.org/xiaoyiyuan/network_vgi/

Abstract:

The research presented in this paper proposes a thematic network approach to explore rich relationships between places. We connect places in networks through their thematic similarities by applying topic modeling to the textual volunteered geographic information (VGI) pertaining to the places. The network approach enhances previous research involving place clustering using geo-textual information, which often simplifies relationships between places to be either in-cluster or out-of-cluster. To demonstrate our approach, we use as a case study in Manhattan (New York) that compares networks constructed from three different geo-textural data sources --TripAdvisor attraction reviews, TripAdvisor restaurant reviews, and Twitter data. The results showcase how the thematic similarity network approach enables us to conduct clustering analysis as well as node-to-node and node-to-cluster analysis, which is fruitful for understanding how places are connected through individuals’ experiences. Furthermore, by enriching the networks with geodemographic information as node attributes, we discovered that some low-income communities in Manhattan have distinctive restaurant cultures. Even though geolocated tweets are not always related to place they are posted from, our case study demonstrates that topic modeling is an efficient method to filter out the place-irrelevant tweets and therefore refining how of places can be studied.

Keywords: Geo-Textual Data, Volunteered Geographic Information, Crowdsourcing, Similarity Network Analysis, Topic Modeling

Work flow from data input to the construction of the thematic similarity network and analysis (i.e., community detection and unique nodes discovery).

A stylized network demonstrating the process of community detection from a fully-connected similarity network.

Network visualization of all communities from the thematic similarity networks with major communities highlighted. Only the major communities are shown on the map for the sake of clarity. Major communities in Network visualization and mapping for each network are colored the same and thus the legend applies for both.

Two examples of communities with boundary nodes and their respective topics.

Full Reference:

Yuan X., Crooks, A.T. and Züfle, A. (2020), A Thematic Similarity Network Approach for Analysis of Places Using Volunteered Geographic Information, ISPRS International Journal of Geo-Information,  9(6), 385, https://doi.org/10.3390/ijgi9060385. (pdf)

Crowdsourcing Street View Imagery: A Comparison of Mapillary and OpenStreetCam

In the past we have written extensively on Volunteered Geographic Information (VGI) such as OpenStreetMap or Twitter. However, we have not really explored Street View Imagery  (SVI), well not until now. Within the realm of VGI, SVI has emerged in recent years as a novel and rich source of data on cities from which geographic information can be derived.

Perhaps the most well-known example of SVI utilization is that of Google Street View (GSV). While SVI has been traditionally collected by governmental agencies and companies alike, we are now also witnessing the emergence of Volunteered Street View Imagery (VSVI), which relies on a crowdsourced effort to provide geotagged street-level imagery coverage of traversable pathways (e.g., a street or trail). Such imagery, similar to GSV, provides detailed information about the location of objects such as cars, road markings, traffic lights and signs, and allows for the automatic extraction of features at scale. Such imagery can also be mined using machine learning algorithms to automatically derive points of interest (POI) databases (e.g., locations of coffee shops and fire hydrants) without the intervention of the citizen.

To explore VSVI we have just published a new paper entitled: "Crowdsourcing Street View Imagery: A Comparison of Mapillary and OpenStreetCam" in the ISPRS International Journal of Geo-Information. In this paper we examine VSVI data collected from two different platforms: Mapillary and OpenStreetCam (OSC) for four metropoiltan areas in the United States (i.e., Washington (District of Columbia), San Francisco (California), Phoenix (Arizona), and Detroit (Michigan)). Both of these online platforms accept sequences of images captured from mobile devices and uploaded via an app on the device (like those shown in the image to the right). Images are geolocated using the device’s global positioning system (GPS). More specifically the paper examines:

• the level of spatial coverage of each platform in order to assess the overall potential of such platforms to provide adequate coverage of geographic information.
• user contribution patterns in Mapillary and OSC in order to understand how users are contributing to these platforms.

Results from our systematic and quantitative analysis of these two emerging VGI sources indicate that most Mapillary and OSC contributions occurred along control-access highways and local roads, and that the overall coverage in these sources is variable in comparison to an authoritative source (i.e., TIGER). Furthermore, our results showed that while the number of contributors varied across sites, only a few contributors were responsible for producing most of the raw data. User contribution patterns were also different in Mapillary and OSC. Specifically, we found that while patterns in coverage were variable for the different OSC sites, coverage patterns in Mapillary tended to be similar among sites. This finding may be linked to several factors, including differences in mapping practice, or issues with participation inequality, a topic that has been highly researched for other VGI platforms such as OSM, but which is still lacking within VSVI. Lastly, user contributions in Mapillary tended to be higher around 8:00 am, 1:00 pm and 5:00 pm (local time). This finding suggests that VSVI contributions tend to coincide with the morning and afternoon commute, and the lunch hour of the contributors.

If you wish to find out more about this work below we provide the abstract to the paper, a visual flowchart of our workflow and some of our our results. The full reference and link to the paper is provided at the bottom of the post.

Abstract:

Over the last decade, Volunteered Geographic Information (VGI) has emerged as a viable source of information on cities. During this time, the nature of VGI has been evolving, with new types and sources of data continually being added. In light of this trend, this paper explores one such type of VGI data: Volunteered Street View Imagery (VSVI). Two VSVI sources, Mapillary and OpenStreetCam, were extracted and analyzed to study road coverage and contribution patterns for four US metropolitan areas. Results show that coverage patterns vary across sites, with most contributions occurring along local roads and in populated areas. We also found that a few users contributed most of the data. Moreover, the results suggest that most data are being collected during three distinct times of day (i.e., morning, lunch and late afternoon). The paper concludes with a discussion that while VSVI data is still relatively new, it has the potential to be a rich source of spatial and temporal information for monitoring cities.

Keywords: Crowdsourcing; Volunteered Geographic Information; Street View Imagery; Mapillary, OpenStreetCam

Overview of methodology

Spatial distribution of road networks.

Spatial comparison of roads in kilometers.

Full Reference: 

Mahabir, R., Schuchard, R., Crooks, A.T., Croitoru, A. and Stefanidis, A. (2020), Crowdsourcing Street View Imagery: A Comparison of Mapillary and OpenStreetCam, ISPRS International Journal of Geo-Information. 9(6), 341; https://doi.org/10.3390/ijgi9060341 (pdf)

Detecting and Mapping Slums using Open Data

Urban and slum areas in Nairobi (False composite image created
by stacking image bands 7, 6 and 4 from the Landsat 8 satellite.

Turning back to slums, we just published paper entitled "Detecting and Mapping Slums using Open Data: A Case Study in Kenya" in the International Journal of Digital Earth. This work builds and extends our previous research on using new sources of data to explore the slum settlements in 3 cities in Kenya (i.e. Nairobi, Mombasa and Kisumu). Specifically, we examine how the fusion of Volunteered Geographical Information, Social Media, and other open data sources can complement remote sensing imagery in supporting slum detection, mapping and monitoring. 

We do this by using data mining tools (e.g. logistic regression, discriminant analysis and the See5 decision tree), to develop context-sensitive definitions for slums based on location, as well as for testing the generalizability of indicators and derived slum models. The end result is an indicator database for slums using open sources of physical and socio-economic data that can be used to characterize slum settlements. If you wish to know more, below we provide the abstract to the paper along with some of the figures and the full citation with a link to the paper itself.

Abstract:

The worldwide slum population currently stands at over one billion, with substantial growth expected in the coming decades. Traditionally, slums have been mapped using information derived mainly from either physical indicators using remote sensing data, or socio-economic indicators using census data. Each data source on its own provides only a partial view of slums, an issue further compounded by data poverty in less developed countries. To overcome such issues, this paper explores the fusion of traditional with emerging open data sources and data mining tools to identify additional indicators that can be used to detect and map the presence of slums, map their footprint, and map their evolution. Towards this goal, we develop an indicator database for slums using open sources of physical and socio-economic data that can be used to characterize slum settlements. Using this database, we then leverage data mining techniques to identify the most suitable combination of these indicators for mapping slums. Using three cities in Kenya as test cases, results show that the fusion of these data can improve the mapping accuracy of slums. These results suggest that the proposed approach can provide a viable solution to the emerging challenge of monitoring the growth of slums.

Keywords: Slums; Remote Sensing; Socio-economic; Urban sustainability; Data mining; Kenya

Study areas in Kenya

Methodology workflow

Distribution of positive classified cases for slums for (a) logistic regression, (b) discriminant analysis and (c) the See5 decision tree.

Full Reference:

Mahabir, R., Agouris, P., Stefanidis, A., Croitoru, A. and Crooks, A.T. (2018), Detecting and Mapping Slums using Open Data: A Case Study in Kenya, International Journal of Digital Earth. DOI: https://doi.org/10.1080/17538947.2018.1554010. (pdf)

Exodus 2.0: Crowdsourcing Geographical and Social Trails of Mass Migration

Readers of the blog might know we have an interest in volunteered geographic information, social media and Web 2.0 technologies and how they can be used to explore urban systems. Recently however, we turned our focus on how such information and technologies can be used to explore and understand mass migrations.

To this end we recently had a paper published in the Journal of Geographical Systems entitled "Exodus 2.0: Crowdsourcing Geographical and Social Trails of Mass Migration". We adopt the term Exodus 2.0 to refer to this new migration paradigm in the digital age, whereby information is a commodity in the migration process.

Given the nature of migration processes, it is possible to explore them across two key dimensions: geographical and situational. The geographical dimension is associated with the physical migration pathways migrants take from a country of origin to a destination site (often through a number of intermediate “stop” sites). The situational dimension is associated with the social connectivity of moving migrant populations, the conditions on the ground, and the activities that take place as part of migration efforts (including the root conditions, proximate conditions and triggering events).

Factors that potentially cause refugee production and
 mass movement based on identified factors detailed by
Clark (1989) and Zottarelli (1998).

In the paper, we use the ongoing Syrian humanitarian crisis as a case study to to explore how the factors that potentially causes refugee production and mass movement  can be gleamed from new sources of data. Specifically, the potential of crowd-generated data—especially open data, volunteered geographic information and social media content (e.g. OpenStreetMap, Flickr, Twitter and Instagram)  to provide information about migration processes.  Through a series of case studies  we show how such data (when combined with more traditional data sources) offers a new lens to study such the geographical and situational dimensions of mass migration. Finally we discuss  how such data could be used to inform migration modeling. If we have not bored you yet and you are interested in finding out more about this line of inquiry, below we provide the abstract to the paper, some of the figures which go along with our analysis for studying the refugee production and movement. Finally, we also provide the full reference and a link to the paper. 

Abstract:

The exodus of displaced populations is a recurring historical phenomenon, and the ongoing Syrian humanitarian crisis is its latest incarnation. During such mass migration events, information is an essential commodity. Of particular importance is geographical (e.g., pathways and refugee camps) and social (e.g., refugee activities and networking) information. Traditionally, such information had been produced and disseminated by authorities, but a new paradigm is emerging: Web 2.0 and mobile computing technologies enable the involved stakeholder communities to produce, access, and consume migration-related information. The purpose of this article is to put forward a new typology for understanding the factors around migration and to examine the potential of crowd-generated data—especially open data and volunteered geographic information—to study such events. Using the recent wave of migration to Europe from the Middle East and northern Africa as a case study, we examine how migration-related information can be dynamically mined and analyzed to study the migrants’ pathways from their home countries to their destination sites, as well as the conditions and activities that evolve during the migration process. These new data sources can provide a deeper and more fine-grained understanding of the migration process, often in real-time, and often through the eyes of the communities affected by it. Nevertheless, this also raises significant methodological and technical challenges for their future use associated with potential biases, data quality issues, and data processing.

Keywords: Refugees, Forced migration, Humanitarian crisis, Volunteered geographic information, Crowdsourcing, Social media, GIS, Web 2.0.

Cumulative flow (2011–2015) illustrating Syrian forced migration to neighboring countries and other destination countries. Line thickness indicates increasing number of persons migrating.

Retweet network of geolocated Twitter microblogs that are discussing opinions, news and retweeting information related to “refugee” in multiple languages from May to August 2017.

A concept graph illustrating the associations between a keyword related to root factors of mass migration such as poverty (“welfare”) to other keywords, as they appear in our Twitter data corpus. The color of the node refers to specific themes: locations (green), actors (dark red), topics (red), entities and individuals (blue), concepts (white), and events (yellow). Red edges represent active associations between terms; gray edges represent inactive associations between terms.

An agent-based model of migration: top: the spatial environment, where the lines represent migration pathways, and the nodes represent number of migrants. Purple nodes represent final destination sites, red nodes show migrant deaths, and green nodes show migrants en route (source: Hu 2016).

Full Reference: 

Curry, T., Croitoru, A., Crooks, A.T. and Stefanidis, A. (in press), Exodus 2.0: Crowdsourcing Geographical and Social Trails of Mass Migration, Journal of Geographical Systems. DOI: https://doi.org/10.1007/s10109-018-0278-1 (pdf)

Authoritative and VGI in a Developing Country: A Comparative Case Study of Road Datasets in Nairobi

Continuing with our work of volunteered geographic information (VGI), we recently had a paper published in  the ISPRS International Journal of Geo-Information entitled: "Authoritative and Volunteered Geographical Information in a Developing Country: A Comparative Case Study of Road Datasets in Nairobi, Kenya.”

The motivation behind the paper was that while there are numerous studies comparing VGI to authoritative data in the developed world, there are very few that do so in developing world. In order to address this issue in the paper we compare the quality of authoritative road data (i.e. from the Regional Center for Mapping of Resources for Development - RCMRD) and non-authoritative crowdsourced road data (i.e. from OpenStreetMap (OSM) and Google’s Map Maker) in conjunction with population data in and around Nairobi, Kenya.

Results from our analysis show variability in coverage between all these datasets. RCMRD provided the most complete, albeit less current, coverage when taking into account the entire study area, while OSM and Map Maker showed a degradation of coverage as one moves from central Nairobi towards more rural areas. Further information including the abstract to our paper, some figures and full reference is given below.

Abstract:

With volunteered geographic information (VGI) platforms such as OpenStreetMap (OSM) becoming increasingly popular, we are faced with the challenge of assessing the quality of their content, in order to better understand its place relative to the authoritative content of more traditional sources. Until now, studies have focused primarily on developed countries, showing that VGI content can match or even surpass the quality of authoritative sources, with very few studies in developing countries. In this paper we compare the quality of authoritative (data from the Regional Center for Mapping of Resources for Development - RCMRD) and non-authoritative (data from OSM and Google’s Map Maker) road data in conjunction with population data in and around Nairobi, Kenya. Results show variability in coverage between all these datasets. RCMRD provided the most complete, albeit less current, coverage when taking into account the entire study area, while OSM and Map Maker showed a degradation of coverage as one moves from central Nairobi towards rural areas. Furthermore, OSM had higher content density in large slums, surpassing the authoritative datasets at these locations, while Map Maker showed better coverage in rural housing areas. These results suggest a greater need for a more inclusive approach using VGI to supplement gaps in authoritative data in developing nations.

Keywords: Volunteered Geographic Information; Crowdsourcing; Road Networks; Population Data; Kenya  

Road Coverage per km2

Pairwise difference in road coverage. Clockwise from top left: i) RCMRD 2011 versus Map Maker 2014; ii) RCMRD 2011 versus OSM 2011; iii) RCMRD 2011 versus OSM 2014; iv) OSM 2014 versus Map Maker 2014 (Red cells: first layer has higher coverage; Green cells: second layer has higher coverage).

Full Reference:

Mahabir, R., Stefanidis, A., Croitoru, A., Crooks, A.T. and Agouris, P. (2017), “Authoritative and Volunteered Geographical Information in a Developing Country: A Comparative Case Study of Road Datasets in Nairobi, Kenya”, ISPRS International Journal of Geo-Information, 6(1): 24, doi:10.3390/ijgi6010024.

As always any thoughts or comments about this work are welcome.

The study of slums as social and physical constructs: challenges and emerging research opportunities

Conceptual model for integrating social
and physical constructs to monitor,
analyze and model slums.

Continuing our research on slums, we have just had a paper published in the journal Regional Studies, Regional Science entitled "The Study of Slums as Social and Physical Constructs: Challenges and Emerging Research Opportunities". In this open access publication we review past lines of research with respect to studying slums which often focus on one of three constructs: (1) exploring the socio-economic and policy issues; (2) exploring the physical characteristics; and, lastly, (3) those modelling slums. We argue that while such lines of inquiry have proved invaluable with respect to studying slums, there is a need for  a  more  holistic  approach  for  studying  slums  to truly understand  them at the local, national and regional scales. Below you can read the abstract of our paper:

"Over 1 billion people currently live in slums, with the number of slum dwellers only expected to grow in the coming decades. The vast majority of slums are located in and around urban centres in the less economically developed countries, which are also experiencing greater rates of urbanization compared with more developed countries. This rapid rate of urbanization is cause for significant concern given that many of these countries often lack the ability to provide the infrastructure (e.g., roads and affordable housing) and basic services (e.g., water and sanitation) to provide adequately for the increasing influx of people into cities. While research on slums has been ongoing, such work has mainly focused on one of three constructs: exploring the socio-economic and policy issues; exploring the physical characteristics; and, lastly, those modelling slums. This paper reviews these lines of research and argues that while each is valuable, there is a need for a more holistic approach for studying slums to truly understand them. By synthesizing the social and physical constructs, this paper provides a more holistic synthesis of the problem, which can potentially lead to a deeper understanding and, consequently, better approaches for tackling the challenge of slums at the local, national and regional scales."

Keywords: Slums; informal settlements; socio-economic; remote sensing; crowdsourced information; modelling.

Framework for studying and understanding slums.

We hope you enjoy this paper and we wound be interested in receiving any feedback.

Full Reference:

Mahabir, R., Crooks, A.T., Croitoru, A. and Agouris, P. (2016), “The Study of Slums as Social and Physical Constructs: Challenges and Emerging Research Opportunities”, Regional Studies, Regional Science, 3(1): 737-757. (pdf)

A Semester with Urban Analytics

This past semester I gave a new class at GMU entitled "Urban Analytics". In a nutshell the class was about introducing students to a broad interdisciplinary field that focuses on the use of data to study cities. More specifcally the emphasis of the class was to provide students with a understanding of what methods, tools and theory can be used to monitor, analyze and model cities. 

From my past research and also when preparing the class material,  I have come to the realization that to study cities (like many others, you know who you are) that there is no one general model, tool or dataset. Therefore, one needs to maintain a toolbox of specialized tools than can be applied to different aspects of urban problems and questions. 

The toolbox that we used in class included a variety of software such as ArcGIS, QGIS, GeoDa, SANET along with programing and scripting in Python and R to modeling  cities via UrbanSim, NetLogo and MASON. Data we used ranged from crowdsourced (e.g. volunteered geographical information) data such as from OpenStreetMap or Wikipedia, to crowd harvested (ambient geographical information) data such as Twitter and Flickr, as-well as more traditional sources of data such as the US Census.

The Urban Analytics Toolbox

As an introduction to urban analytics, the course had the following objectives:

1. to understand the motivation for the use of data to study cities, including some historical aspects; 
2. to learn about the variety of Urban Analytics research programs across the several disciplines (urban planning, regional science, public policy, geography, computational social science etc.), through a survey of the literature and case studies. 
3. to understand the distinct contribution that Urban Analytics can make by providing specific insights about cities at multiple scales. 
4. to provide the foundations for more advanced work in the area of Urban Analytics. 

As with many of my courses, students were expected to complete a end of semester project. Below is a selection of these projects which explored some aspect of urban life.

I would like to thank the students for participating in this new class. It was a fun trip.

Crowdsourcing A Collective Sense of Place

Following on with our GeoSocial Analysis work, we recently had a paper published in  PLOS ONE entitled "Crowdsourcing A Collective Sense of Place." In the paper we discuss and showcase how one can take a quantitative approach to derive a collective sense of place from Twitter contributions and also from corresponding Wikipedia entries.

To illustrate this we present a brief study three cities, that of New York, Los Angeles and Singapore. Below you read the abstract of the paper, see some images from the paper especially the flowchart describing overall process used to discover platial alignment, along with the full reference to the paper.

Place can be generally defined as a location that has been assigned meaning through human experience, and as such it is of multidisciplinary scientific interest. Up to this point place has been studied primarily within the context of social sciences as a theoretical construct. The availability of large amounts of user-generated content, e.g. in the form of social media feeds or Wikipedia contributions, allows us for the first time to computationally analyze and quantify the shared meaning of place. By aggregating references to human activities within urban spaces we can observe the emergence of unique themes that characterize different locations, thus identifying places through their discernible sociocultural signatures. In this paper we present results from a novel quantitative approach to derive such sociocultural signatures from Twitter contributions and also from corresponding Wikipedia entries. By contrasting the two we show how particular thematic characteristics of places (referred to herein as platial themes) are emerging from such crowd-contributed content, allowing us to observe the meaning that the general public, either individually or collectively, is assigning to specific locations. Our approach leverages probabilistic topic modelling, semantic association, and spatial clustering to find locations are conveying a collective sense of place. Deriving and quantifying such meaning allows us to observe how people transform a location to a place and shape its characteristics.  

Keywords: Online Encyclopedias, Twitter, Data Visualization, Social Media, Semantics.

Flowchart describing overall process used to discover platial alignment.

Statistically significant clusters of recreation and entertainment categories concentrated over Manhattan, NYC

Maps depict significant hotspots for each of the high-level categories for (A) Singapore, (B) London, (C) Los Angeles, and (D) New York City.

Full Reference: 

Jenkins A., Croitoru, A. Crooks, A.T. and Stefanidis, A. (2016), Crowdsourcing A Collective Sense of Place, PLoS ONE 11(4): e0152932. doi:10.1371/journal.pone.0152932

New Paper: Assessing the impact of demographic characteristics on spatial error in VGI features

LISA analysis of positional accuracy for the OSM  data set

Building upon our interest in volunteered geographic information (VGI) and extending our previous paper  "Assessing Completeness and Spatial Error of Features in Volunteered Geographic Information" we have just published the paper with the rather long title "Assessing the impact of demographic characteristics on spatial error in volunteered geographic information features" where we explore how demographics impact on the quality of VGI data

Below is the abstract of the paper: 

The proliferation of volunteered geographic information (VGI), such as OpenStreetMap (OSM) enabled by technological advancements, has led to large volumes of user-generated geographical content. While this data is becoming widely used, the understanding of the quality characteristics of such data is still largely unexplored. An open research question is the relationship between demographic indicators and VGI quality. While earlier studies have suggested a potential relationship between VGI quality and population density or socio-economic characteristics of an area, such relationships have not been rigorously explored, and mainly remained qualitative in nature. This paper addresses this gap by quantifying the relationship between demographic properties of a given area and the quality of VGI contributions. We study specifically the demographic characteristics of the mapped area and its relation to two dimensions of spatial data quality, namely positional accuracy and completeness of the corresponding VGI contributions with respect to OSM using the Denver (Colorado, US) area as a case study. We use non-spatial and spatial analysis techniques to identify potential associations among demographics data and the distribution of positional and completeness errors found within VGI data. Generally, the results of our study show a lack of statistically significant support for the assumption that demographic properties affect the positional accuracy or completeness of VGI. While this research is focused on a specific area, our results showcase the complex nature of the relationship between VGI quality and demographics, and highlights the need for a better understanding of it. By doing so, we add to the debate of how demographics impact on the quality of VGI data and lays the foundation to further work.

The analysis workflow

Full Reference:

Mullen W., Jackson, S. P., Croitoru, A., Crooks, A. T., Stefanidis, A. and Agouris, P., (2014), Assessing the Impact of Demographic Characteristics on Spatial Error in Volunteered Geographic Information Features, GeoJournal. DOI: 10.1007/s10708-014-9564-8

Work featured in IQT Quarterly

Two of our recent papers have been  featured in IQT Quarterly. The first looks at completeness and error in VGI and the second features some of our work on social media and polycentric communities. The papers have been significantly shortened and edited and make easy reading (that's not to say the original papers were difficult to read :). For those not familiar with IQT Quarterly, it  is a publication from In-Q-Tel which: 

"was created to bridge the gap between the technology needs of the U.S. Intelligence Community (IC) and emerging commercial innovation".

Full References:

Stefanidis, A., Cotnoir, A., Croitoru, A., Crooks, A.T., Radzikowski, J. and Rice, M. (2013), Demarcating New Boundaries: Mapping Virtual Polycentric Communities through Social Media Content, IQT Quarterly, 5 (2): 12-14. (pdf)

Jackson, S. P., Mullen W., Agouris, P., Crooks, A. T., Croitoru, A. and Stefanidis, A. (2013), Assessing Completeness and Spatial Error of Features in Volunteered Geographic Information, IQT Quarterly, 5 (2): 22-26. (pdf)

New Publication: GIS and Agent-Based models for Humanitarian Assistance

Inputs to the model

As the readers of the blog know, we have an interest in GIS, agent-based modeling and crowdsourcing. Now we have a paper that combines all these three elements. Its entitled "GIS and Agent-Based models for Humanitarian Assistance" and is published in Computers, Environment and Urban Systems. 

The model itself was written in MASON and uses extensively GeoMASON. Data comes from several different sources (both raster and vector) including OpenStreetMap and LandScan. Below you can read an abstract of the paper and see a movie of one of the scenarios.

"Natural disasters such as earthquakes and tsunamis occur all over the world, altering the physical landscape and often severely disrupting people’s daily lives. Recently researchers’ attention has focused on using crowds of volunteers to help map the damaged infrastructure and devastation caused by natural disasters, such as those in Haiti and Pakistan. This data is extremely useful, as it is allows us to assess damage and thus aid the distribution of relief, but it tells us little about how the people in such areas will react to the devastation. This paper demonstrates a prototype spatially explicit agent-based model, created using crowdsourced geographic information and other sources of publicly available data, which can be used to study the aftermath of a catastrophic event. The specific case modelled here is the Haiti earthquake of January 2010. Crowdsourced data is used to build the initial populations of people affected by the event, to construct their environment, and to set their needs based on the damage to buildings. We explore how people react to the distribution of aid, as well as how rumours relating to aid availability propagate through the population. Such a model could potentially provide a link between socio-cultural information about the people affected and the relevant humanitarian relief organizations."

Full Reference: 

Crooks, A.T. and Wise, S. (2013), GIS and Agent-Based models for Humanitarian Assistance, Computers, Environment and Urban Systems, 41: 100-111. (pdf)

Completeness and Spatial Error of Features in VGI

I have had an interest in volunteered geographic information (VGI) for quite some time (see my publications or blog posts) but only recently have I had an opportunity to look at the spatial error of features within VGI. To this end, our paper entitled "Assessing Completeness and Spatial Error of Features in Volunteered Geographic Information" has just been published in ISPRS International Journal of Geo-Information. Below is the abstract of the paper along with some figures. Further details about the paper can be seen at the bottom of the page.

The assessment of the quality and accuracy of Volunteered Geographic Information (VGI) contributions, and by extension the ultimate utility of VGI data has fostered much debate within the geographic community. The limited research to date has been focused on VGI data of linear features and has shown that the error in the data is heterogeneously distributed. Some have argued that data produced by numerous contributors will produce a more accurate product than an individual and some research on crowd-sourced initiatives has shown that to be true, although research on VGI is more infrequent. This paper proposes a method for quantifying the completeness and accuracy of a select subset of infrastructure-associated point datasets of volunteered geographic data within a major metropolitan area using a national geospatial dataset as the reference benchmark with two datasets from volunteers used as test datasets. The results of this study illustrate the benefits of including quality control in the collection process for volunteered data. 

Keywords: volunteered geographic information (VGI); OpenStreetMap; quality; error; point.

Comparison of OSM, OSMCP, and ORNL data.

Various identified locations of Southwest Early College

Full reference:

Jackson, S. P., Mullen W., Agouris, P., Crooks, A., Croitoru, A. and Stefanidis, A. (2013), Assessing Completeness and Spatial Error of Features in Volunteered Geographic Information, ISPRS International Journal of Geo-Information, 2 (2): 507-530. Download from here.

Natural Disasters and Crowdsourcing: Haiti

Natural disasters such as earthquakes and tsunamis occur all over the world, altering the physical landscape and often severely disrupting people’s daily lives. Recently researchers’ attention has focused on using crowds of volunteers to help map the infrastructure and devastation caused by natural disasters, such as those in Haiti and Pakistan. For example, in the movie below shows the response to the earthquake by the OpenStreetMap community within 12 hours of the earthquake. The white flashes indicate edits to the map (often by tracing satellite/aerial photography).

While this data is extremely useful, as it is allows us to assess damage and thus aid the distribution of relief, but it tells us little about how the people in such areas will react to the devastation, the supply of food, or the reconstruction. To address this, we are exploring how agent-based modeling can be used to explore peoples reactions. To do this we have created a prototype spatially explicit agent-based model, created using crowdsourced geographic information and other sources of publicly available data, which can be used to study the aftermath of a catastrophic event. The specific case modeled here is the Haiti earthquake of January 2010. Crowdsourced data is used to build the initial populations of people affected by the event, to construct their environment, and to set their needs based on the damage to buildings. 

The idea behind the model is to explore how people react to the distribution of aid, as well as how rumors propagating through the population and crowding around aid distribution points might lead to food riots and similar social phenomena. Such a model could potentially provide a link between socio-cultural information of the people affected and relevant humanitarian relief organizations.

The animation above shows one simulation run where there is the spread of  information and agent movement (red dots) around one center (blue dot). While the chart below shows how over time the density of agents around the food station increases as the simulation progresses.

The idea behind such a model is one can take crowdsourced information and fuse it into an agent-based model and see how people will react to the distribution of food centers. For example, the movie below shows how agents find out about four (hypothetical) different food centers and decide whether or not to go to them in a 6 by 8km area of Port-au-Prince.

Spread of information and agent movement (red dots) in a 6 by 8km area of Port-au-Prince.

More details about this model to come......

#Earthquake: Twitter as a Distributed Sensor System

Our work on using social media continues to develop and we have recently had a paper accepted in Transactions in GIS, entitled "#Earthquake: Twitter as a Distributed Sensor System". Below we present our abstract and some of the results.

Social media feeds are rapidly emerging as a novel avenue for the contribution and dissemination of information that is often geographic. Their content often includes references to events occurring at, or affecting specific locations. Within this paper we analyze the spatial and temporal characteristics of the twitter feed activity responding to a 5.8 magnitude earthquake which occurred on the East Coast of the United States (US) on August 23, 2011. We argue that these feeds represent a hybrid form of a sensor system that allows for the identification and localization of the impact area of the event. By contrasting this to comparable content collected through the dedicated crowdsourcing ‘Did You Feel It?’ (DYFI) website of the US Geological Survey we assess the potential of the use of harvested social media content for event monitoring. The experiments support the notion that people act as sensors to give us comparable results in a timely manner, and can complement other sources of data to enhance our situational awareness and improve our understanding and response to such events.

The movie below show geolocated tweets with references to the earthquake through keyword (earthquake or earth and quake) and hashtag search (#earthquake or #quake) for the first hour after the earthquake.

The following images give a glimpse at some of our analysis.

Response pattern as function of distance from epicenter for the first 400 seconds after the earthquake. At the top we see a plot of (reaction time, distance) of all tweets during that period. At the bottom we show the histogram of the number of tweets as a function of distance.

Locations of the 40 tweets in the shaded area of the figure above overlaid over the USGS CDI scale map. Tweet locations are marked as green circles. Color-coding in the graph is ranging from red (high perceived intensity) to yellow (lower perceived intensity). The dashed line shows a distance of approximately 950 km (8.5 degrees of angular distance) from the epicenter.

The movie below gives you an idea of some of the tweet content:





Full reference to this paper is:

Crooks, A.T., Croitoru, A., Stefanidis, A. and Radzikowski, J. (2013), #Earthquake: Twitter as a Distributed Sensor System, Transactions in GIS, 17(1): 124-147. (pdf)

Social Media and the Emergence of Open-Source Geospatial Intelligence

Sample of geolocated tweets referring
Occupy Wall Street.

We have just finished a paper entitled 'Social Media and the Emergence of Open-Source Geospatial Intelligence' for Socio-Cultural Dynamics and Global Security. For those interested below is the abstract:

The emergence of social media has provided the public with an effective and irrepressible real-time mechanism to broadcast information. The great popularity of platforms such as twitter and YouTube, and the substantial amount of content that is communicated through them are making social media an essential component of open-source intelligence. The information communicated through such feeds conveys the interests and opinions of individuals, and reveals links and the complex structure of social networks. However, this information is only partially exploited if one does not consider its geographical aspect. Indeed, social media feeds more often than not have some sort of geographic content, as they may communicate the location from where a particular report is contributed, the geolocation of an image, or they may refer to a specific sociocultural hotspot. By harvesting this geographic content from social media feeds we can transfer the extracted knowledge from the amorphous cyberspace to the geographic space, and gain a unique understanding of the human lansdscape, its structure and organization, and its evolution over time. This new-found opportunity signals the emergence of open-source geospatial intelligence, whereby social media contributions can be analyzed and mined to gain unparalleled situational awareness. In this paper we showcase a number of sample applications that highlight the capabilities of harvesting geospatial intelligence from social media feeds, focusing particularly on twitter as a representative data source.

Selection of geolocated pairs of tweeters and retweeters in Tokyo at the time immediately
following the Sendai earthquake

Full reference:

Stefanidis, A., Crooks, A.T., Radzikowski, J., Croitoru, A. and Rice, M. (2014), Social Media and the Emergence of Open-Source Geospatial Intelligence, in Murdock, D.G., Tomes, R. and Tucker, C. (eds.), Human Geography: Socio-Cultural Dynamics and Global Security, US Geospatial Intelligence Foundation (USGIF), Herndon, VA, pp. 109-123. (pdf)

Harvesting ambient geospatial information from social media feeds

A paper I  recently co-authored with Anthony Stefanidis and Jacek Radzikowski from George Mason University entitled "Harvesting ambient geospatial information from social media feeds" is now available in  GeoJournal. 

 

The abstract for the paper reads as follows: "Social media generated from many individuals is playing a greater role in our daily lives and provides a unique opportunity to gain valuable insight on information flow and social networking within a society. Through data collection and analysis of its content, it supports a greater mapping and understanding of the evolving human landscape. The information disseminated through such media represents a deviation from volunteered geography, in the sense that it is not geographic information per se. Nevertheless, the message often has geographic footprints, for example, in the form of locations from where the tweets originate, or references in their content to geographic entities. We argue that such data conveys ambient geospatial information, capturing for example, people’s references to locations that represent momentary social hotspots. In this paper we address a framework to harvest such ambient geospatial information, and resulting hybrid capabilities to analyze it to support situational awareness as it relates to human activities. We argue that this emergence of ambient geospatial analysis represents a second step in the evolution of geospatial data availability, following on the heels of volunteered geographical information."

Geolocating pairs of tweeters and retweeters

Virtual Geographic Environments

A quick note for a new book entitled "Virtual Geographic Environments" from ESRI Press who write:

"Virtual Geographic Environments, edited by Hui Lin and Michael Batty, collects key papers that define the current momentum in GIS and "virtual geographies." Contributions by leading members of the geospatial community to Virtual Geographic Environments illustrate the cutting edge of GIScience, as well as new applications of GIS with the processing and delivery of geographic information through the Web and handheld devices, forming two major directions to these developments. The four-part organization leads from a primer on VGEs to virtual cities and landscapes, interface design and public participation, and finally mobile and networked VGEs. Current topics, such as crowd sourcing and related services, point to the development of new business models that merge proprietary and nonproprietary systems."

Andrew Hudson-Smith and myself have contributed a chapter entitled "The Renaissance of Geographic Information: Neogeography, Gaming and Second Life". The abstract for our paper is:

"Web 2.0, specifically The Cloud, GeoWeb and Wikitecture are revolutionising the way in which we present, share and analyse geographic data. In this paper we outline and provide working examples a suite of tools which are detailed below, aimed at developing new applications of GIS and related technologies. GeoVUE is one of seven nodes in the National Centre for e-Social Science whose mission it is to develop web-based technologies for the social and geographical sciences. The Node, based at the Centre for Advanced Spatial Analysis, University College London has developed a suite of free software allowing quick and easy visualisation of geographic data in systems such as Google Maps, Google Earth, Crysis and Second Life. These tools address two issues, firstly that spatial data is still inherently difficult to share and visualise for the non-GIS trained academic or professional and secondly that a geographic data social network has the potential to dramatically open up data sources for both the public and professional geographer. With our applications of GMap Creator, and MapTube to name but two, we detail ways to intelligently visualise and share spatial data. This paper concludes with detailing usage and outreach as well as an insight into how such tools are already providing a significant impact to the outreach of geographic information."