United States

This work describes the process of creating a corpus of Twitter conversations annotated for the presence of \textit{counterspeech} in response to toxic speech related to axes of discrimination linked to sexism, racism and homophobia. 
The main novelty is an annotated dataset comprising relevant tweets in their context of occurrence. 
The corpus is made up of tweets and responses captured by different profiles replying to discriminatory content or objectionably couched news. 
An annotation scheme was created to illustrate the relevant dimensions of toxic speech and counterspeech.
An analysis of the collected and annotated data and of the Inter-Annotator Agreement (IAA) that emerged during the annotation process is included. Moreover, we report about preliminary experiments on automatic \textit{counterspeech} detection, based on supervised automatic learning models trained on the new dataset. 
The results highlight the fundamental role played by the context in this detection task, confirming our intuitions about the importance to collect tweets in their context of occurrence.

NAACL 2022

Counter-TWIT: An Italian Corpus for Online Counterspeech in Ecological Contexts

italian twitter

alberto

counterspeech

toxic speech

agreement

language model

annotation

[![](https://assets.underline.io/uploads/markdown_image/1/image/81a3c0317d24f663b49b996875024d45.png)](https://aclanthology.org/volumes/2022.coling-1/)



### THE CONFERENCE WE KNOW AND WE WANT.
**COLING**, the International Conference on Computational Linguistics, is one of the premier conferences for the natural language processing and computational linguistics.

First established in 1965, the biennial COLING conference is held in diverse parts of the globe and attracts participants from both top-ranked research centers and emerging countries. Today, the most important developments in our field are taking place not only in universities and academic research institutes but also in industrial research departments including tech-startups. COLING provides opportunities for all these communities to showcase their exciting discovery.

In fall of 2022, COLING will be held in Gyeongju in a hybrid format. All participants can either present at the venue site or join virtually. As more people get vaccinated, we are happy to provide safer environments for our colleagues. We believe that COLING 2022 will be one of the conferences, free from the pandemic. The hybrid format gives presenters and sponsors a valuable opportunity to promote their companies in both an online and in-person venue. For the first time in a long time, customers can interact with their sponsor's products first-hand. The online venue, too, gives sponsors the chance to network with those unable to attend the in-person session.

The hybrid format ultimately lends itself to greater exposure for our sponsors: COLING2022 will let you reach more potential partners and customers than ever before!

To gain access to this event page you are required to register. Find more information and pay the registration fee on event the organizer’s website **[https://coling2022.org/reg](https://coling2022.org/reg)**

COLING 2022

COLING, the International Conference on Computational Linguistics, is one of the premier conferences for the natural language processing and computational linguistics.

workshop paper

## Welcome to NAACL 2022!
The Annual Meeting of the North American Chapter of the Association for Computational Linguistics (NAACL) is taking place July 10-15, 2022 as a hybrid event, in Seattle, WA and online. We are happy to welcome all of you to conference! 
 
The main conference program features oral presentations, in-person and virtual posters and demo sessions, a plenary session for our best paper presentations and awards, keynote presentations, a plenary panel on the place of linguistics and symbolic structures in NLP, and an Industrial Track panel on Careers in NLP.
 
Posters (including Findings of NAACL 2022) and demos are grouped by areas for both the in-person and the virtual sessions. For the virtual component, the talks will be on Zoom and the posters and the demos will be in GatherTown. The Student Research Workshop and will have an oral session and a poster session.
 
The program also features 6 Tutorials and 26 Workshops.
 
 We wish you a wonderful conference!
 
[The NAACL 2022 Organizing Committee](https://2022.naacl.org/committees/organization/)

2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics

Analyzing ethnic or religious bias is important for improving fairness, accountability, and transparency of natural language processing models. However, many techniques rely on human-compiled lists of bias terms, which are expensive to create and are limited in coverage. In this study, we present a fully data-driven pipeline for generating a knowledge graph (KG) of cultural knowledge and stereotypes. Our resulting KG covers 5 religious groups and 5 nationalities and can easily be extended to include more entities. Our human evaluation shows that the majority (59.2%) of non-singleton entries are coherent and complete stereotypes. We further show that performing intermediate masked language model training on the verbalized KG leads to a higher level of cultural awareness in the model and has the potential to increase classification performance on knowledge-crucial samples on a related task, i.e., hate speech detection.

StereoKG: Data-Driven Knowledge Graph Construction for Cultural Knowledge and Stereotypes

Uses of pejorative expressions can be benign or actively empowering. When models for abuse detection misclassify these expressions as derogatory, they inadvertently censor productive conversations held by marginalized groups. One way to engage with non-dominant perspectives is to add context around conversations. Previous research has leveraged user- and thread-level features, but it often neglects the spaces within which productive conversations take place. Our paper highlights how community context can improve classification outcomes in abusive language detection. We make two main contributions to this end. First, we demonstrate that online communities cluster by the nature of their support towards victims of abuse. Second, we establish how community context improves accuracy and reduces the false positive rates of state-of-the-art abusive language classifiers. These findings suggest a promising direction for context-aware models in abusive language research.

Enriching Abusive Language Detection with Community Context

"Dogwhistles" are expressions intended by the speaker to have two messages: a socially unacceptable "in-group" message understood by a subset of listeners and a benign message intended for the out-group. We take the result of a word-replacement survey of the Swedish population intended to reveal how dogwhistles
are understood, and we show that the difficulty of annotating dogwhistles is reflected in the separability of the space of a sentence-transformer
Swedish BERT trained on general data.

Distributional properties of political dogwhistle representations in Swedish
BERT

Toxic speech is regarded as one of the crucial issues plaguing online social media today. Most recent work on toxic speech detection is constrained to the modality of text and written conversations with very limited work on toxicity detection from spoken utterances or using the modality of speech. In this paper, we introduce a new dataset DeToxy, the first publicly available toxicity annotated dataset for the English language. DeToxy is sourced from various openly available speech databases and consists of over 2 million utterances. We believe that our dataset would act as a benchmark for the relatively new and unexplored SLP task of detecting toxicity from spoken utterances and boost further research in this space. Finally, we also provide strong unimodal baselines for our dataset and compare traditional 2-step cascade and End-to-End (E2E) approaches. Our experiments show that in the case of spoken
utterances, text-based approaches are largely dependent on gold human-annotated transcripts for their performance and also suffer from the problem of keyword bias. However, the presence of speech files in DeToxy helps facilitates the development of E2E speech models which alleviate both the above-stated problems by better capturing speech clues.

DeToxy: A Large-Scale Multimodal Dataset for Toxicity Classification in
Spoken Utterances

A common approach for testing fairness issues in text-based classifiers is through the use of counterfactuals: does the classifier output change if a sensitive attribute in the input is changed? Existing counterfactual generation methods typically rely on wordlists or templates, producing simple counterfactuals that don't take into account grammar, context, or subtle sensitive attribute references, and could miss issues that the wordlist creators had not considered. In this paper, we introduce a task for generating counterfactuals that overcomes these shortcomings, and demonstrate how large language models (LLMs) can be leveraged to make progress on this task. We show that this LLM-based method can produce complex counterfactuals that existing methods cannot, comparing the performance of various counterfactual generation methods on the Civil Comments dataset and showing their value in evaluating a toxicity classifier.

Flexible text generation for counterfactual fairness probing

Lexicons play an important role in content moderation, often being the first line of defense. However, little or no literature exists in analyzing the representation of queer-related words in them. In this paper, we consider twelve well-known English lexicons containing inappropriate words and analyze how gender and sexual minorities are represented in these lexicons. Our analyses reveal that several of these lexicons barely make any distinction between pejorative and non-pejorative queer-related words. We express concern that such unfettered usage of non-pejorative queer-related words may impact queer presence in mainstream discourse. Our analyses further reveal that the lexicons have poor overlap in queer-related words. We finally present a quantifiable measure of consistency and show that several of these lexicons are not consistent in how they include (or omit) queer-related words.

Revisiting Queer Minorities in Lexicons

Despite recent advances in machine learning based hate speech detection, classifiers still struggle with generalizing knowledge to out-
of-domain data samples. In this paper, we investigate the generalization capabilities of deepearning models to different target groups of hate speech under clean experimental settings. Furthermore, we assess the efficacy of three different strategies of unsupervised domain adaptation to improve these capabilities. Given the diversity of hate and its rapid dynamics in the online world (e.g. the evolution of new target groups like virologists during the COVID-19 pandemic), robustly detecting hate aimed at newly identified target groups is a highly relevant research question. We show that naively trained models suffer from a target group specific bias, which can be reduced via domain adaptation. We were able to achieve a relative improvement of the F1-score between 5.8% and 10.7% for out-of-domain target groups of hate speech compared to baseline approaches by utilizing domain adaptation.

Improving Generalization of Hate Speech Detection Systems to Novel Target Groups via Domain Adaptation

Hate Speech has been around in Ethiopia before social media, but with very limited reach. With the coming of social media companies that have no or little business interest to lose in low-resourced languages such as those in Ethiopia, diaspora activists that have nothing or little to lose from engaging in online hate speech, and several technical and institutional challanges, hate speech on social media slowly became mainstream in Ethiopia, tearing societies apart and eventually serving as an animating force for a genocidal war on Tigrayans. In this speech, I will briefly assess the normalization of hate speech in Ethiopia, the factors that led to this, and the role hate speech and social media played during the Tigray war, social media hate speech detection and monitoring, and what should be done going forward. 


Social Media and Hate Speech in time of War: The case of Tigray War

Large pre-trained models are usually fine-tuned on downstream task data, and tested on unseen data.
When the train and test data come from different domains, the model is likely to struggle, as it is not adapted to the test domain. We propose a new approach for domain adaptation (DA), using neuron-level interventions: 
We modify the representation of each test example in specific neurons, resulting in a counterfactual example from the source domain, which the model is more familiar with. The modified example is then fed back into the model.
While most other DA methods are applied during training time, ours is applied during inference only, making it more efficient and applicable. 
Our experiments show that our method improves performance on unseen domains.

IDANI: Inference-time Domain Adaptation via Neuron-level Interventions

Deep learning methods have enabled task oriented
semantic parsing of increasingly complex utterances.
However, a single model is still typically trained and
deployed for each task separately, requiring labeled
training data for each, which makes it challenging to
support new tasks, even within a single business
vertical (e.g., food-ordering or travel booking). In this
paper we describe Cross-TOP (Cross-Schema Task-
Oriented Parsing), a zero-shot method for complex
semantic parsing in a given vertical. By leveraging the
fact that user requests from the same vertical share
lexical and semantic similarities, a single crossschema
parser is trained to service an arbitrary
number of tasks, seen or unseen, within a vertical.
We show that Cross-TOP can achieve high accuracy
on a previously unseen task without requiring any
additional training data, thereby providing a scalable
way to bootstrap semantic parsers for new tasks. As
part of this work we release the FoodOrdering
dataset, a task-oriented parsing dataset in the foodordering
vertical, with utterances and annotations
derived from five schemas, each from a different
restaurant menu.

Downloads

Next from NAACL 2022

StereoKG: Data-Driven Knowledge Graph Construction for Cultural Knowledge and Stereotypes

Similar lecture

Establishing annotation quality in multi-label annotations

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from NAACL 2022

StereoKG: Data-Driven Knowledge Graph Construction for Cultural Knowledge and Stereotypes

Similar lecture

Establishing annotation quality in multi-label annotations

Downloads