United States

An intelligent system is expected to perform reasonable inferences, accounting for both the literal meaning of a word and the meanings a word can acquire in different contexts. A specific kind of inference concerns the connective and, which in some cases gives rise to a temporal succession or causal interpretation in contrast with the logic, commutative one (Levinson, 2000). In this work, we investigate the phenomenon by creating a new dataset for evaluating the interpretation of and by NLI systems, which we use to test three Transformer-based models. Our results show that all systems generalize patterns that are consistent with both the logical and the pragmatic interpretation, perform inferences that are inconsistent with each other, and show clear divergences with both theoretical accounts and humans’ behavior.

NAACL 2022

Pragmatic and Logical Inferences in NLI Systems: The Case of Conjunction Buttressing

multinli

conjunction buttressing

implicature

natural language inference

workshop paper

## Welcome to NAACL 2022!
The Annual Meeting of the North American Chapter of the Association for Computational Linguistics (NAACL) is taking place July 10-15, 2022 as a hybrid event, in Seattle, WA and online. We are happy to welcome all of you to conference! 
 
The main conference program features oral presentations, in-person and virtual posters and demo sessions, a plenary session for our best paper presentations and awards, keynote presentations, a plenary panel on the place of linguistics and symbolic structures in NLP, and an Industrial Track panel on Careers in NLP.
 
Posters (including Findings of NAACL 2022) and demos are grouped by areas for both the in-person and the virtual sessions. For the virtual component, the talks will be on Zoom and the posters and the demos will be in GatherTown. The Student Research Workshop and will have an oral session and a poster session.
 
The program also features 6 Tutorials and 26 Workshops.
 
 We wish you a wonderful conference!
 
[The NAACL 2022 Organizing Committee](https://2022.naacl.org/committees/organization/)

2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics

Prior studies have raised concerns over specificity issues in clinical advice. Lacking specificity — explicitly discussed detailed information — may affect the quality and implementation of clinical advice in medical practice. In this study, we developed and validated a fine-grained annotation schema to describe differ- ent aspects of specificity in clinical advice extracted from the medical research literature. We also presented our initial annotation effort and discussed future directions towards an NLP- based specificity analysis tool for summarizing and verifying the details in clinical advice.


"Devils are in the Details'' — Annotating Specificity of Clinical Advice from Medical Literature

This paper presents a linguistically driven proof of concept for finding potentially euphemistic terms, or PETs. Acknowledging that PETs tend to be commonly used expressions for a certain range of sensitive topics, we make use of distributional similarities to select and filter phrase candidates from a sentence and rank them using a set of simple sentiment-based metrics. We present the results of our approach tested on a corpus of sentences containing euphemisms, demonstrating its efficacy for detecting single and multi-word PETs from a broad range of topics. We also discuss future potential for sentiment-based methods on this task.

Searching for PETs: Using Distributional and Sentiment-Based Methods to Find Potentially Euphemistic Terms

We report results of experiments using BART (Lewis et al., 2019) and the Penn Discourse Tree Bank (Webber et al., 2019) (PDTB) to generate texts with correctly realized discourse relations. We address a question left open by previous research (Yung et al., 2021; Ko and Li, 2020) concerning whether conditioning the model on the intended discourse relation—which corresponds to adding explicit discourse relation information into the input to the model—improves its performance. Our results suggest that including discourse relation information in the input of the model significantly improves the consistency with which it produces a correctly realized discourse relation in the output. We exemplify and argue for the integration of metrics inspired by hypotheses in psycholinguistics into our evaluation

Generating Discourse Connectives with Pre-trained Language Models: Discourse Relations Help Yet Again

Semantic Role Labeling (SRL) has long been referred to as the task of detecting who did what to whom, where, when, and how? Yet most research efforts looked for answers to these questions only at pre-defined syntactical positions in the sentence, disregarding what language users typically understand from discourse and wider document contexts. These implicit arguments are a naturally occurring phenomenon that can be widely beneficial for many natural language understanding challenges. In this work, we propose a method for crowdsourcing implicit semantic arguments on a large scale. To further explore the conceptualization of such arguments by crowd workers, we let them annotate questions and clauses for a given predicate and all its roles. We hope that this dataset will help overcome the data bottleneck for implicit SRL and will allow machines to perform frame semantics on the discourse level.

Looking Beyond Syntax: Detecting Implicit Semantic Arguments

Despite significant progress in Vision-and-Language Navigation (VLN) over the years, existing agents are evaluated on fine-grained instructions that are neither practical nor efficient. In this study, we consider underspecification as a ubiquitous and important phenomenon to bridge the gap between curated instructions and natural human instructions in reality. We introduce a new setting, namely Underspecified vision-and-Language Navigation (ULN). We verify that our instructions are more practical and efficient from the users' perspective. We propose a goal-aware agent with an Exploitation-to-Exploration (E2E) module that estimates step-wise uncertainty. Experimental results show that existing VLN models are still brittle to underspecification. Empirically, our method outperforms the baseline by a large margin in success rate while maintaining a comparable performance on R2R.

ULN: Towards Underspecified Vision-and-Language Navigation

Some of the information that is conveyed in text is not explicitly stated, but instead, is inferred by the reader. While current state-of-the-art language models are very good at recovering explicit information, they still struggle with inferring implicit information from text, and as a result, that information is lost and NLU quality suffers. One such implicit information phenomenon is gapping, a linguistic phenomenon in coordination structures containing VPs, in which a VP that appears in the initial clause is elided from a non-initial clause and can be inferred from the context of the sentence.

Bridging the Gap: Recovering Elided VPs in Coordination Structures

To tackle the rising phenomenon of hate speech, efforts have been made towards data curation and analysis. When it comes to analysis of bias, previous work has focused predominantly on race. In our work, we further investigate bias in hate speech datasets along racial, gender and intersectional axes. We identify strong bias against African American English (AAE), masculine and AAE+Masculine tweets, which are annotated as disproportionately more hateful and offensive than from other demographics. We provide evidence that BERT-based models propagate this bias and show that balancing the training data for these protected attributes can lead to fairer models with regards to gender, but not race.

Analyzing Hate Speech Data along Racial, Gender and Intersectional Axes


The representations in large language models contain multiple types of gender information. We focus on two types of such signals in English texts: factual gender information, which is a grammatical or semantic property, and gender bias, which is the correlation between a word and specific gender. We can disentangle
the model’s embeddings and identify components encoding both types of information with probing. We aim to diminish the stereotypical bias in the representations while preserving the factual gender signal. Our filtering method
shows that it is possible to decrease the bias of gender-neutral profession names without significant deterioration of language modeling capabilities. The findings can be applied to language generation to mitigate reliance on stereotypes while preserving gender agreement in coreferences

Don’t Forget About Pronouns: Removing Gender Bias in Language Models Without Losing Factual Gender Information

We investigate whether it’s feasible to de-gender resumes to mitigate potential gender bias in algorithmic resume screening. Using a corpus of 709k resumes from IT firms, we first train a series of models to classify the self­ reported gender of the applicant, thereby measuring the extent and nature of gendered information encoded in resumes. We then conduct a series of gender obfuscation experiments, where we iteratively remove gendered information from resumes while preserving the job­-relevant content. Finally, we investigate the trade­off between gender obfuscation and the performance of a resume screening algorithm. We find three main results: (1) there is a significant amount of gendered information in resumes. (2) lexicon-­based gender obfuscation method can reduce the amount of gendered information to a large extent. However, after a certain point, the performance of the resume screening algorithm starts suffering. (3) general-­purpose gender debiasing methods for NLP models such as removing gender subspace from embeddings are not effective in obfuscating gender.


Gendered Language in Resumes and its Implications for Algorithmic Bias in Hiring

Detecting and mitigating harmful biases in modern language models are widely recognized as crucial, open problems. In this paper, we take a step back and investigate how language models come to be biased in the first place. We use a relatively small language model, using the LSTM architecture trained on an English Wikipedia corpus. With full access to the data and to the model parameters as they change during every step while training, we can map in detail how the representation of gender develops, what patterns in the dataset drive this, and how the model's internal state relates to the bias in a downstream task (semantic textual similarity). We find that the representation of gender is dynamic and identify different phases during training. Furthermore, we show that gender information is represented increasingly locally in the input embeddings of the model and that, as a consequence, debiasing these can be effective in reducing the downstream bias. Monitoring the training dynamics, allows us to detect an asymmetry in how the female and male gender are represented in the input embeddings. This is important, as it may cause naive mitigation strategies to introduce new undesirable biases. We discuss the relevance of the findings for mitigation strategies more generally and the prospects of generalizing our methods to larger language models, the Transformer architecture, other languages and other undesirable biases.

Downloads

Next from NAACL 2022

"Devils are in the Details'' — Annotating Specificity of Clinical Advice from Medical Literature

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from NAACL 2022

"Devils are in the Details'' — Annotating Specificity of Clinical Advice from Medical Literature

Downloads