Elections and Bots

Continuing our work on bots,  Ross Schuchard and myself have a new paper in PLOS ONE entitled "Insights into elections: An ensemble bot detection coverage framework applied to the 2018 U.S. midterm elections." Our motivation for the work came from the fact that during elections internet-based technological platforms (e.g., online social networks (OSNs), online political blogs etc.) are gaining more power compared to mainstream media sources (e.g., print, television and radio). While such technologies are reducing the barrier for individuals to actively participate in political dialogue, the relatively unsupervised nature of OSNs increases susceptibility to misinformation campaigns, especially with respect to political and election dialogue. This is especially the case for social bots—automated software agents designed to mimic or impersonate humans which are prevalent actors in OSN platforms and have proven to amplify misinformation.  

The issue however is that no single detection algorithm is able to account for the myriad of social bots operating in OSNs. To overcome this issue, this research incorporates multiple social bot detection services to determine the prevalence and relative importance of social bots within an OSN conversation of tweets. Through the lens of the 2018 U.S. midterm elections, 43.5 million tweets were harvested capturing the election conversation which were then analyzed for evidence of bots using three bot detection platform services: Botometer, DeBot and Bot-hunter.

We found that bot and human accounts contributed temporally to our tweet election corpus at relatively similar cumulative rates. The multi-detection platform comparative analysis of intra-group and cross-group interactions showed that bots detected by DeBot and Bot-hunter persistently engaged humans at rates much higher than bots detected by Botometer. Furthermore, while bots accounted for less than 8% of all unique accounts in the election conversation retweet network, bots accounted for more than 20% of the top-100 and top-25 ranking out-degree centrality, thus suggesting persistent activity to engage with human accounts. Finally, the bot coverage overlap analysis shows that minimal overlap existed among the bots detected by the three bot detection platforms, with only eight total bot accounts detected by all (out of a total of 254,492 unique bots in the overall tweet corpus ).

If this research sounds interesting to you, below we provide the abstract to the paper along with some figures outlining our methodology and some of the results. While at the bottom of the post you can see the full reference and there is a link to the paper were you can read more.

Abstract:

The participation of automated software agents known as social bots within online social network (OSN) engagements continues to grow at an immense pace. Choruses of concern speculate as to the impact social bots have within online communications as evidence shows that an increasing number of individuals are turning to OSNs as a primary source for information. This automated interaction proliferation within OSNs has led to the emergence of social bot detection efforts to better understand the extent and behavior of social bots. While rapidly evolving and continually improving, current social bot detection efforts are quite varied in their design and performance characteristics. Therefore, social bot research efforts that rely upon only a single bot detection source will produce very limited results. Our study expands beyond the limitation of current social bot detection research by introducing an ensemble bot detection coverage framework that harnesses the power of multiple detection sources to detect a wider variety of bots within a given OSN corpus of Twitter data. To test this framework, we focused on identifying social bot activity within OSN interactions taking place on Twitter related to the 2018 U.S. Midterm Election by using three available bot detection sources. This approach clearly showed that minimal overlap existed between the bot accounts detected within the same tweet corpus. Our findings suggest that social bot research efforts must incorporate multiple detection sources to account for the variety of social bots operating in OSNs, while incorporating improved or new detection methods to keep pace with the constant evolution of bot complexity.

 

Fig 1. Social bot analysis framework employing multiple bot detection platforms. The framework enables the application of ensemble analysis methods to determine the prevalence and relative importance of social bots within Twitter conversations discussing the 2018 U.S. midterm elections.

 

Fig 3. Cumulative tweet contribution rates for the 2018 U.S. midterm OSN conversation (October 10 – November 6, 2018) from the (a) human (blue) / bot (red) and (b) DeBot (green) / Botometer (pink) / Bot-hunter (orange) account classification perspectives.

Fig 4. Intra-group and cross-group retweet communication patterns of human (blue) and social bot (red) users within the 2018 U.S. midterm election Twitter conversation according to each bot detection classification platform: (a) Combined Bot Sources (b) DeBot (c) Botometer (d) Bot-hunter. The combined bot sources results (shown in gray) classified an account as a bot in aggregate fashion if any of the three detection platforms classified the account as a bot.

Fig 5. Social bot account evidence within the top-N (where, N = 1000 / 500 / 100 / 25) centrality rankings [(a) eigenvector (b) in-degree (c) out-degree (d) PageRank] according to bot classification results from Bot-hunter (orange), Botometer (pink) and DeBot (green).

 

Fig 7. Bot detection coverage analysis for bots detected within the 2018 U.S. midterm election Twitter conversation using the Botometer, Bot-hunter and DeBot bot detection platforms.

 

Full reference:

Schuchard, R.J. and Crooks, A.T. (2021), Insights into Elections: An Ensemble Bot Detection Coverage Framework Applied to the 2018 U.S. Midterm Elections, PLoS ONE, 16(1): e0244309. Available at  https://doi.org/10.1371/journal.pone.0244309. (pdf).

Beyond Words: Comparing Structure, Emoji Use, and Consistency Across Social Media Posts

Continuing our work on Emojis, at the forthcoming International Conference on Social Computing, Behavioral-Cultural Modeling and Prediction and Behavior Representation in Modeling and Simulation (or SBP-BRiMS for short) we (Melanie Swartz, Arie Croitoru and myself) have a paper entitled "Beyond Words: Comparing Structure, Emoji Use, and Consistency Across Social Media Posts." In the paper we introduce and demonstrate a language-agnostic methodology to characterize structures of content and emoji use within a document (in this case a tweet), measure consistency of structures across a set of documents, and cluster documents and users with similar patterns and behavior. Using a corpus of 44 million tweets collected in October and November 2018 related to the 2018 U.S. midterm elections based on keywords, hashtags, and user accounts associated with candidates or political parties we were able to gain insights into the unique or shared structures of communication styles and emoji use of over 3.3 million unique users and user roles such as journalists, bots and others. If this sounds of interest to you, below we provide the abstract to the paper, some tables and figures of the our findings along with the full reference and a link to the paper. Furthermore, if you are interested in extending this work to other areas, Melanie has made the code available at https://github.com/msemoji/.

Abstract

Social media content analysis often focuses on just the words used in documents or by users and often overlooks the structural components of document composition and linguistic style. We propose that document structure and emoji use are also important to consider as they are impacted by individual communication style preferences and social norms associated with user role and intent, topic domain, and dissemination platform. In this paper we introduce and demonstrate a novel methodology to conduct structural content analysis and measure user consistency of document structures and emoji use. Document structure is represented as the order of content types and number of features per document and emoji use is characterized by the attributes, position, order, and repetition of emojis within a document. With these structures we identified user signatures of behavior, clustered users based on consistency of structures utilized, and identified users with similar document structures and emoji use such as those associated with bots, news organizations, and other user types. This research compliments existing text mining and behavior modeling approaches by offering a language agnostic methodology with lower dimensionality than topic modeling, and focuses on three features often overlooked: document structure, emoji use, and consistency of behavior.

Keywords: Data Mining, Social Media, Emojis, User Behavior Modeling.

Emoji attributes

Most common content structures with emojis for non-retweets.

Clusters of users with similar behavior across two factors in non-retweets (left) and retweets (right) Colors indicate cluster assignments.

Full Reference: 

Swartz, M., Crooks, A.T. and Croitoru, A. (2020), Beyond Words: Comparing Structure, Emoji Use, and Consistency Across Social Media Posts, in Thomson, R., Bisgin, H., Dancy, C., Hyder, A. and Hussain, M. (eds), 2020 International Conference on Social Computing, Behavioral-Cultural Modeling & Prediction and Behavior Representation in Modeling and Simulation, Washington DC., pp 1-11. (pdf)

 

Bots Fired: Examining Social Bot Evidence in Online Mass Shooting Conversations

Building upon our work with respect to how bots impact online conversations pertaining to global events and  health,  we have extended this research to see what role bots play in mass shooting events. In our new paper published in Palgrave Communications entitled "Bots Fired: Examining Social Bot Evidence in Online Mass Shooting Conversations" we examine four mass shooting events (i.e., Las Vegas, Sutherland Springs, Parkland, and Santa Fe) and find that social bots participate and contribute to online mass shooting conversations in a manner that is distinguishable from human contributions. Below we provide the abstract, along with some figures from the paper that highlight our methodology and main results. Finally at the bottom of the post, we provide the full reference to the paper. 

Abstract:

Mass shootings, like other extreme events, have long garnered public curiosity and, in turn, significant media coverage. The media framing, or topic focus, of mass shooting events typically evolves over time from details of the actual shooting to discussions of potential policy changes (e.g., gun control, mental health). Such media coverage has been historically provided through traditional media sources such as print, television, and radio, but the advent of online social networks (OSNs) has introduced a new platform for accessing, producing, and distributing information about such extreme events. The ease and convenience of OSN usage for information within society’s larger growing reliance upon digital technologies introduces potential unforeseen risks. Social bots, or automated software agents, are one such risk, as they can serve to amplify or distort potential narratives associated with extreme events such as mass shootings. In this paper, we seek to determine the prevalence and relative importance of social bots participating in OSN conversations following mass shooting events using an ensemble of quantitative techniques. Specifically, we examine a corpus of more than 46 million tweets produced by 11.7 million unique Twitter accounts within OSN conversations discussing four major mass shooting events: the 2017 Las Vegas concert shooting, the 2017 Sutherland Springs church shooting, the 2018 Parkland school shooting and the 2018 Santa Fe school shooting. This study’s results show that social bots participate in and contribute to online mass shooting conversations in a manner that is distinguishable from human contributions. Furthermore, while social bots accounted for fewer than 1% of total corpus user contributors, social network analysis centrality measures identified many bots with significant prominence in the conversation networks, densely occupying many of the highest eigenvector and out-degree centrality measure rankings, to include 82% of the top-100 eigenvector values of the Las Vegas retweet network.

Keywords: Social bots, mass shootings, school shootings, online social networks, computational social science.

 Overview of social bot analysis framework illustrating methodological steps taken to analyze social bots within online social network conversations involving mass shooting events

Overall tweet corpus volumes and suspected social bot contributions for each associated OSN mass shooting 215 event conversation.

Intra-group and cross-group retweet interaction rates among and between human (blue) and suspected social bot (red) user accounts for a one-month period following the (a) Las Vegas, (b) Sutherland Springs, (c) Parkland and (d) Santa Fe shooting events.

Social bot accounts in the top-N, where N = 1000/100/10, (a) eigenvector, (b) in-degree, (c) out-degree and (d) PageRank centrality measurement rankings within OSN mass shooting retweet networks discussing the Las Vegas (red), Sutherland Springs (green), Parkland (blue) and Santa Fe (purple) shooting events.

Full Reference:

Schuchard, R., Crooks, A.T., Croitoru, A. and Stefanidis, A. (2019) Bots Fired: Examining Social Bot Evidence in Online Mass Shooting Conversations, Palgrave Communications, 5: 158. Available at https://doi.org/10.1057/s41599-019-0359-x. (pdf)

Communities, Bots and Vaccinations

Following on from our work on bots and health discussions in relation to online social networks (OSNs), Xiaoyi Yuan, Ross Schuchard and myself have just published a paper entitled "Examining Emergent Communities and Detecting Social Bots within the Polarized Online Vaccination Debate in Twitter" in Social Media + Society. Within the paper we explore the communication patterns of vaccine discussions in Twitter. More specifically we ask three questions:

1. Do vaccine discussions on Twitter show a highly clustered pattern in the sense that users tend to communicate more often with those who have same opinions towards vaccination than those who do not? 
2. If the communication is highly clustered, to what extent do pro-vaccine users reach out to anti-vaccine users and vice versa? 
3. How much do social bots, computer algorithms designed to mimic human behavior and interact with humans in an automated fashion, contribute to the conversation as previous research has shown that social bots can have certain impact on human communication in social media?

In order to answer these questions, we use a variety of machine learning techniques (e.g.  logistic regression, support vector machine (e.g. linear and non-linear kernel), k-nearest neighbors, nearest centroid, and Naïve Bayes) trained with labeled data (which is available at https://github.com/XiaoyiYuan/vaccination_online_discussion) to categorize each user’s vaccination stance. By exploring a combination of opinion groups and retweet networks we discovered that pro- and anti-vaccine users retweet predominantly from their own opinion group, while users with neutral opinions are distributed across communities. In addition, our bot analysis (using the  open-source DeBot detection platform) discovered that 1.45% of the corpus users were identified as likely bots and these produced 4.59% of all tweets within our data set. If you wish to find out more about our paper, below you can read the abstract along with seeing some figures including a sketch of our methodology, a selection of results and a link to the paper.

Abstract:

Many states in the United States allow a “belief exemption” for measles, mumps, and rubella (MMR) vaccines. People’s opinion on whether or not to take the vaccine can have direct consequences in public health. Social media has been one of the dominant communication channels for people to express their opinions of vaccination. Despite governmental organizations’ efforts of disseminating information of vaccination benefits, anti-vaccine sentiment is still gaining momentum. Studies have shown that bots on social media (i.e., social bots) can influence opinion trends by posting a substantial number of automated messages. The research presented here investigates the communication patterns of anti- and pro-vaccine users and the role of bots in Twitter by studying a retweet network related to MMR vaccine after the 2015 California Disneyland measles outbreak. We first classified the users into anti-vaccination, neutral to vaccination, and pro-vaccination groups using supervised machine learning. We discovered that pro- and anti-vaccine users retweet predominantly from their own opinion group. In addition, our bot analysis discovers that 1.45% of the corpus users were identified as likely bots which produced 4.59% of all tweets within our dataset. We further found that bots display hyper-social tendencies by initiating retweets at higher frequencies with users within the same opinion group. The article concludes that highly clustered anti-vaccine Twitter users make it difficult for health organizations to penetrate and counter opinionated information while social bots may be deepening this trend. We believe that these findings can be useful in developing strategies for health communication of vaccination.

Keywords: Anti-vaccine Movement, Twitter, Social Media, Opinion Classification, Bot Analysis

Full Reference:

Yuan, X.,  Schuchard, R.  and Crooks, A.T. (2019), Examining Emergent Communities and Detecting Social Bots within the Polarized Online Vaccination Debate in Twitter, Social Media + Society. https://doi.org/10.1177/2056305119865465 (pdf)

Bot stamina: Examining the influence and staying power of bots in online social networks

Following on with our work on bots we just had a paper published in Applied Network Science entitled "Bot Stamina: Examining the Influence and Staying Power of Bots in Online Social Networks" which is significant extension of a previous conference paper. In the paper we look at thee global Twitter conversions in 2016. Specifically the 2016 U.S. presidential election primary races (February 1–28, 2016), the ongoing Ukrainian conflict (August 1–28, 2016), and the Turkish government’s implementation of censorship (December 1–28, 2016) and the influence of Bots on these conversations.

Tweets were classified as Bots using DeBot and then we explore the relative importance and persistence of social bots in online social networks by looking at retweet networks and centrality rankings (i.e. degree, in-degree, out-degree, eigenvector, betweenness and PageRank). We find through such centrality measurements that even though Bots made up less than 0.3% of the total user population, they displayed a profound level of structural network influence.  If you would like to know more about this work, below we provide the abstract to the paper, along with some figures, including one that describes our methodology, and some initial results. Finally at the bottom of the page we provide the full reference and a link to the paper.

Abstract

This study presents a novel approach to expand the emergent area of social bot research. We employ a methodological framework that aggregates and fuses data from multiple global Twitter conversations with available bot detection platforms and ultimately classifies the relative importance and persistence of social bots in online social networks (OSNs). In testing this methodology across three major global event OSN conversations in 2016, we confirmed the hyper-social nature of bots: suspected social bot accounts make far more attempts on average than social media accounts attributed to human users to initiate contact with other accounts via re-tweets. Social network analysis centrality measurements discover that social bots, while comprising less than 0.3% of the total corpus user population, display a dis-proportionately high profound level of structural network influence by ranking particularly high among the top users across multiple centrality measures within the OSN conversations of interest. Further, we show that social bots exhibit temporal persistence in centrality ranking density when examining these same OSN conversations over time.

Keywords: Social bot analysis, computational social science, social network analysis, online social networks

Full Reference:

Schuchard, R., Crooks, A.T., Stefanidis, A. and Croitoru, A. (2019), Bot Stamina: Examining the Influence and Staying Power of Bots in Online Social Networks, Applied Network Science, 4: 55. Available at https://doi.org/10.1007/s41109-019-0164-x. (pdf)

Bots in Social Networks

Recently our research has started to dig deeper into social media, especially how bots diffuse information in online social networks (OSNs). To this end at the  7th International Conference on Complex Networks and Their Applications we had a paper entitled: Bots in Nets: Empirical Comparative Analysis of Bot Evidence in Social Networks. 

In this paper we present a framework to characterize the pervasiveness and relative importance of bots in various OSNs conversations of three significant global events in 2016. In total, we harvested more than 30 million tweets from the U.S. Presidential Election, the Ukrainian Conflict and Turkish Political Censorship and compared the conversational patterns of bots and humans within each event. The results from this analysis showed that although Twitter participants identified as social bots comprised only 0.28% of all OSN users in this study, they accounted for a significantly large portion of prominent centrality rankings across the three conversations. If you want to know more about this new work, below we provide the abstract to the paper, a selection of figures and tables (including our methodology, some summary information about our data corpus and some of the results). Finally at the bottom of this post we have the full reference and a link to the paper.

Abstract:

The emergence of social bots within online social networks (OSNs) to diffuse information at scale has given rise to many efforts to detect them. While methodologies employed to detect the evolving sophistication of bots continue to improve, much work can be done to characterize the impact of bots on communication networks. In this study, we present a framework to describe the pervasiveness and relative importance of participants recognized as bots in various OSN conversations. Specifically, we harvested over 30 million tweets from three major global events in 2016 (the U.S. Presidential Election, the Ukrainian Conflict and Turkish Political Censorship) and compared the conversational patterns of bots and humans within each event. We further examined the social network structure of each conversation to determine if bots exhibited any particular network influence, while also determining bot participation in key emergent network communities. The results showed that although participants recognized as social bots comprised only 0.28% of all OSN users in this study, they accounted for a significantly large portion of prominent centrality rankings across the three conversations. This includes the identification of individual bots as top-10 influencer nodes out of a total corpus consisting of more than 2.8 million nodes. 

Keywords: bots, online social networks, social network analysis.

Fig. 1. Overall methodology to analyze bot evidence across multiple Twitter OSN conversations.

Table 1. Harvested Twitter Corpus Overview

Fig. 5. Correlation of centrality measures for select centrality comparisons: (a) U.S. Election eigenvector versus betweenness analysis, (b) Ukraine Conflict eigenvector versus betweenness analysis and (c)Ukraine Conflict eigenvector versus degree analysis.

Table 2. Bot density of largest emergent communities.

Full Reference:

Schuchard, R., Crooks, A.T., Stefanidis, A.  and Croitoru, A. (2018), Bots in Nets: Empirical Comparative Analysis of Bot Evidence in Social Networks, in Aiello, L.M., Cherifi, C. Cherifi, H., Lambiotte, R., Lió, P. and Rocha, L.M. (eds.), Volume 2, Proceedings of the 7th International Conference on Complex Networks and Their Applications, Cambridge, United Kingdom, Springer, pp 424-436. (pdf)