.svg)
Throughout the fall of 2022, the Samara Centre for Democracy deployed SAMbot in eight municipal elections across Canada to measure abusive content received by candidates and political parties on Twitter.
SAMbot tracked each of these elections, in their respective regions & time zones, from the evening that nominations closed until the evening of election day.
SAMbot tracks the Twitter mentions of all candidates who possessed a public and active Twitter account as of the end of each election’s respective nomination deadlines.
Since data from each election was collected during different time periods, different lengths, and with different totals of tracked candidates, it is not useful, nor advised, to compare municipal SAMbot data across elections.
SAMbot does not track or store retweets, as counting the same tweet more than once can distort the analysis. SAMbot only evaluates text within a tweet; content such as images, audio, or videos that may spread abuse cannot be evaluated by SAMbot. Twitter data was collected and used in line with Twitter’s acceptable terms of use.
SAMbot is a machine learning bot — a software application that runs automated tasks. SAMbot tracks all English and French tweets sent to candidates. Each message that SAMbot tracked, whether it’s a reply, quote tweet or mention, was analyzed and sorted based on five abuse categories:
Abuse categories are not exclusive. Tweets can possess more than one attribute (for example, a tweet could contain a threat and an identity attack simultaneously).
SAMbot makes a confidence prediction to assess whether a tweet meets an abuse category. When a tweet is evaluated, SAMbot gives the tweet a score from 0% to 100% for each category, based on how certain SAMbot is that the tweet meets that abuse category. If SAMbot assesses a tweet as >=51% likely to meet an abuse category, we determine that the tweet has met the criteria.
If a tweet is scored as >=51% in any of those five abuse categories, it would also be categorized as an abusive tweet. The abusive tweet category serves to aggregate all comments that meet at least one abuse category.
Using machine learning allows us to analyze tweets at a massive scale. SAMbot can evaluate millions of tweets for how likely abusive or toxic they are. However, we willingly sacrifice a degree of precision in the pursuit of scale through our use of machine learning. There is no explicit qualitative analysis conducted by human researchers to evaluate these tweets before they are collected; SAMbot, as a piece of software, conducts all of this sentiment analysis. Thus, SAMbot-related findings are not intended to act as precise findings, but instead to help point us in the direction of where and how online abuse manifests in Canadian political discussions.
SAMbot uses a machine learning model to assess abusive language. These models are ever-evolving which means that each time SAMbot is deployed, it becomes more accurate and informed. In addition to measuring volume of abuse, SAMbot also provides insight into the type of abuse received by candidates (e.g. identity attacks, sexually explicit content, threats).
However, this also means that data collected during different SAMbot election periods must be siloed. Data collected in different periods (say for example, our 2021 federal election data versus our 2022 municipal election data) can not, and should not be compared because the methodology behind how SAMbot evaluates language has changed significantly between periods, thus, the data is incompatible, and will likely be skewed in certain capacities.
Click here for:
This material is part of the Online Abuse in Local Elections: The SAMbot Municipal Report.
How to cite: The Samara Centre for Democracy, 2023 SAMbot Municipal Elections Report, (Toronto: The Samara Centre for Democracy, 2023), http://www.samaracentre.ca/project/online-abuse-in-local-elections-sambot-municipal-report.
.png)
-min.png)
.jpg)
.png)
-min%20(1).jpg)
.jpg)
.jpg)
.jpg)
.jpg)
