Canada

When people think of everyday things like an egg, they typically have a mental image associated with it. This allows them to correctly judge, for example, that &quot;the yolk surrounds the shell&quot; is a false statement. Do language models similarly have a coherent picture of such everyday things? To investigate this, we propose a benchmark dataset consisting of 100 everyday things, their parts, and the relationships between these parts, expressed as 11,720 &quot;X relation Y?&quot; true/false questions. Using these questions as probes, we observe that state-of-the-art pre-trained language models (LMs) like GPT-3 and Macaw have fragments of knowledge about these everyday things, but do not have fully coherent &quot;parts mental models&quot; (54-59% accurate, 19-43% conditional constraint violation). We propose an extension where we add a constraint satisfaction layer on top of the LM&#39;s raw predictions to apply commonsense constraints. As well as removing inconsistencies, we find that this also significantly improves accuracy (by 16-20%), suggesting how the incoherence of the LM&#39;s pictures of everyday things can be significantly reduced.

&lt;iframe src=&quot;https://app.sli.do/event/pk54NpNnxuhWHUa3vSMXH2/embed/polls/15f28102-47dc-4840-ad93-d72f3dff5de4&quot; width=&quot;300&quot; height=&quot;400&quot;&gt;&lt;/iframe&gt;

ACL 2023

Do language models have coherent mental models of everyday things?

mental model

neuro-symbolic

language model

reasoning

## **The Linguistic Construction of Emotional Challenges in a Changing Society**


**Languaging Diversity** is the itinerating conference of the [I-Land Interuniversity Research Centre](https://www.unior.it/index2.php?content_id=14038&content_id_start=1&titolo=i-land-research-centre&parLingua=ENG), an international independent and interdisciplinary center for advanced studies in Linguistics, Social Sciences and the Humanities with a focus on the key notions of identity and diversity, from a linguistic, cultural, and gender studies perspective, in and across cultures. 

Following the six successful events hosted by the Universities of Naples (2013), Catania (2014), Macerata (2016), Cagliari (2017), Antwerp (2018) and Teruel (2019), the seventh edition of the Languaging Diversity Conference (LD2021) will be organized by the University of Lille, France. Due to the COVID-19 ongoing pandemic, the event will be held exclusively online via the virtual conference management platform [Underline](https://underline.io/).


**Focus **

The 7th edition of the Languaging Diversity Conference (LD2021) focuses on the emotional challenges that result from the social innovations of the last two decades, and on the way such experiences are expressed in discourse. Conference convenors invite papers and posters that investigate the intersection of emotion, communication and social change, also within the framework of digital communication. The contributions presented at the LD2021 conference will explore the expression of emotion(s) at the different levels of linguistic description (phonological, morphological, syntactic, lexical, semantic, pragmatic, discursive), and in texts and contexts related to traditional and digital genres in different domains, including (but not restricted to) health, medicine and science, media, politics, the law, academia, the world of corporate and institutional work, education and learning. 


**Conference convenors**

* Giuditta Caliendo, Université de Lille & STL, France
* Maarten Lemmens, Université de Lille & STL, France
* Mégane Lesuisse, Université de Lille & STL, France

 
**Scientific Committee**

* Giuseppe Balirano, Università degli Studi di Napoli ‘l’Orientale’, Italy
* Henry Hernandez Bayter, Université de Lille & STL, France
* Stéphanie Caët, Université de Lille & STL, France
* Giuditta Caliendo, Université de Lille & STL, France
* Bert Cappelle, Université de Lille & STL, France
* Paola Catenaccio, Università degli Studi di Milano, Italy
* Emmanuelle Canut, Université de Lille & STL, France
* Stefania D’Avanzo, Università degli Studi della Campania ‘Luigi Vanvitelli’, Italy
* Ilse Depraetere, Université de Lille & STL, France
* Emilia Di Martino, Università degli Studi ‘Suor Orsola Benincasa’, Italy
* Antonio Fruttaldo, Università degli Studi di Napoli ‘l’Orientale’, Italy
* Dariusz Galasiński, Uniwersytet Wrocławski, Poland
* Kim Grego, Università degli Studi di Milano, Italy
* Siria Guzzo, Università degli Studi di Salerno, Italy
* Geert Jacobs, Universiteit Gent, Belgium
* Samantha Laporte, Université de Lille & STL, France
* Maarten Lemmens, Université de Lille & STL, France
* Mégane Lesuisse, Université de Lille & STL, France
* Jeannette Littlemore, University of Birmingham, UK
* Julien Longhi, Université de Cergy-Pontoise, France
* Corinne Oster, Université de Lille & CECILLE, France
* Paolo Mairano, Université de Lille & STL, France
* Antonella Napolitano, Università del Sannio, Italy
* Maria Cristina Nisco, Università degli studi di Napoli ‘Parthenope’, Italy
* Annick Rivens Mompean, Université de Lille & STL, France 
* Margaret Rasulo, Università degli Studi della Campania ‘Luigi Vanvitelli’, Italy
* Catherine Ruchon,Université Sorbonne Paris Nord -Pléiade, France
* Hans-Jörg Schmid, LMU München, Germany
* Jerome Tessuto, Università degli Studi della Campania ‘Luigi Vanvitelli’, Italy


[**Organizing Committee**](https://ld2021.sciencesconf.org/)

* Giuditta Caliendo, Université de Lille & STL, France
* Justin Duquesnoy, Université de Lille & STL, France
* Océane Foubert, Université de Lille & STL, France
* Lola Marinato, Université de Lille & STL, France
* Maarten Lemmens, Université de Lille & STL, France
* Mégane Lesuisse, Université de Lille & STL, France
* Sayf-Eddine Mohamed, Université de Lille & STL, France

**Keynote Speakers**

**Maïa Ponsonnet**
<iframe width=640 height=360 style="border:0;" scrolling="no" src="https://screencast-o-matic.com/watch/crQUbGV6Kgz" allowfullscreen="true"></iframe>

**Jeannette Littlemore**
<iframe width=640 height=360 style="border:0;" scrolling="no" src="https://screencast-o-matic.com/watch/cr6i2TVX2d0" allowfullscreen="true"></iframe>

**Andrea Scarantino**
<iframe width=640 height=360 style="border:0;" scrolling="no" src="https://screencast-o-matic.com/watch/crQUbEV6KgI" allowfullscreen="true"></iframe>

To access the **Languaging Diversity 2021** conference, please remember to [register.](https://ld2021.sciencesconf.org/registration) 

Languaging Diversity 2021

Languaging Diversity  is the itinerating conference of the I-Land Interuniversity Research Centre, an international independent and interdisciplinary center for advanced studies in Linguistics, Social Sciences and the Humanities with a focus on the key notions of identity and diversity, from a linguistic, cultural, and gender studies perspective, in and across cultures. 

When people think of everyday things like an egg, they typically have a mental image associated with it. This allows them to correctly judge, for example, that "the yolk surrounds the shell" is a false statement. Do language models similarly have a coherent picture of such everyday things? To investigate this, we propose a benchmark dataset consisting of 100 everyday things, their parts, and the relationships between these parts, expressed as 11,720 "X relation Y?" true/false questions. Using these questions as probes, we observe that state-of-the-art pre-trained language models (LMs) like GPT-3 and Macaw have fragments of knowledge about these everyday things, but do not have fully coherent "parts mental models" (54-59% accurate, 19-43% conditional constraint violation). We propose an extension where we add a constraint satisfaction layer on top of the LM's raw predictions to apply commonsense constraints. As well as removing inconsistencies, we find that this also significantly improves accuracy (by 16-20%), suggesting how the incoherence of the LM's pictures of everyday things can be significantly reduced.

<iframe src="https://app.sli.do/event/pk54NpNnxuhWHUa3vSMXH2/embed/polls/15f28102-47dc-4840-ad93-d72f3dff5de4" width="300" height="400"></iframe>

poster

### Welcome to ACL 2023, the 61st Annual Meeting of the Association for Computational Linguistics! 
<br>The conference will be held in Toronto, Canada, July 9-14, 2023.<br><br>
Following the succession of the recent conferences in our field, ACL 2023 will adopt a hybrid format.
While the impact of Covid has considerably diminished in terms of traveling, obtaining visas to Canada
entails a very long process. Moreover, the global economic conditions pose challenges for many individuals to travel to conferences. Recognizing these circumstances, we know many participants may not be
able to attend the conference in person. Therefore, we are committed to providing a great virtual platform
so everyone has the opportunity to interact with other participants and enjoy the conference. Based on the
current registered participants, approxiately 30% have chosen to attend the conference virtually. Whether
you join us in person or virtually, we sincerely hope everyone has a remarkable conference experience.<br>
This General Chair’s message is where I express my gratitude to the many individuals who have made
enormous contributions to the conference over the past year.

Read [**ACL 2023 General Chair's message**](https://docs.google.com/document/d/1WobYM7norbG4dI48s75HfJoD89qgX5a_F-6U8AteLSA/edit?usp=sharing/) in full.

##### **[Conference Handbook](https://2023.aclweb.org/downloads/acl2023-handbook.pdf)**
<br><br><br><br>

The Association for Computational Linguistics (ACL) is the premier international scientific and professional society for people working on computational problems involving human language, a field often referred to as either computational linguistics or natural language processing.

Large Language Models (LLMs) such as GPT-3 have emerged as general-purpose language models capable of addressing many natural language generation or understanding tasks. On the task of Machine Translation (MT), multiple works have investigated few-shot prompting mechanisms to elicit better translations from LLMs. However, there has been relatively little investigation on how such translations differ qualitatively from the translations generated by standard Neural Machine Translation (NMT) models. In this work, we investigate these differences in terms of the literalness of translations produced by the two systems. Using literalness measures involving word alignment and monotonicity, we find that translations out of English (E-X) from GPTs tend to be less literal, while exhibiting similar or better scores on MT quality metrics. We demonstrate that this finding is borne out in human evaluations as well. We then show that these differences are especially pronounced when translating sentences that contain idiomatic expressions.

Do GPTs Produce Less Literal Translations?

Neural QCFG is a grammar-based sequence-to-sequence model with strong inductive biases on hierarchical structures. It excels in interpretability and generalization but suffers from expensive inference. In this paper, we study two low-rank variants of Neural QCFG for faster inference with different trade-offs between efficiency and expressiveness. Furthermore, utilizing the symbolic interface provided by the grammar, we introduce two soft constraints over tree hierarchy and source coverage. We experiment with various datasets and find that our models outperform vanilla Neural QCFG in most settings.

<iframe src="https://app.sli.do/event/pk54NpNnxuhWHUa3vSMXH2/embed/polls/44d3110a-8e84-4821-9d73-aceed752247c" width="300" height="400"></iframe>

Improving Grammar-based Sequence-to-Sequence Modeling with Decomposition and Constraints

Token dropping is a recently-proposed strategy to speed up the pretraining of masked language models, such as BERT, by skipping the computation of a subset of the input tokens at several middle layers. It can effectively reduce the training time without degrading much performance on downstream tasks. However, we empirically find that token dropping is prone to a semantic loss problem and falls short in handling semantic-intense tasks. Motivated by this, we propose a simple yet effective semantic-consistent learning method (ScTD) to improve the token dropping. ScTD aims to encourage the model to learn how to preserve the semantic information in the representation space. Extensive experiments on 12 tasks show that, with the help of our ScTD, token dropping can achieve consistent and significant performance gains across all task types and model sizes. More encouragingly, ScTD saves up to 57% of pretraining time and brings up to +1.56% average improvement over the vanilla token dropping.

<iframe src="https://app.sli.do/event/pk54NpNnxuhWHUa3vSMXH2/embed/polls/646785ef-a691-434e-8904-77d3a099ab19" width="300" height="400"></iframe>

Revisiting Token Dropping Strategy in Efficient BERT Pretraining

Fact verification aims to automatically judge the veracity of a claim according to several pieces of evidence. Due to the manual construction of datasets, spurious correlations between claim patterns and its veracity (i.e., biases) inevitably exist. Recent studies show that models usually learn such biases instead of understanding the semantic relationship between the claim and evidence. Existing debiasing works can be roughly divided into data-augmentation-based and weight-regularization-based pipeline, where the former is inflexible and the latter relies on the uncertain output on the training stage. Unlike previous works, we propose a novel method from a counterfactual view, namely CLEVER, which is augmentation-free and mitigates biases on the inference stage. Specifically, we train a claim-evidence fusion model and a claim-only model independently. Then, we obtain the final prediction via subtracting output of the claim-only model from output of the claim-evidence fusion model, which counteracts biases in two outputs so that the unbiased part is highlighted. Comprehensive experiments on several datasets have demonstrated the effectiveness of CLEVER.

<iframe src="https://app.sli.do/event/532fnPQGAZvefKDC1fZsuL/embed/polls/2519e0ea-4028-45cb-bcc3-cb23abe30546" width="300" height="400"></iframe>

Counterfactual Debiasing for Fact Verification

Language models (LMs) are pretrained on diverse data sources---news, discussion forums, books, online encyclopedias. A significant portion of this data includes facts and opinions which, on one hand, celebrate democracy and diversity of ideas, and on the other hand are inherently socially biased. Our work develops new methods to (1) measure media biases in LMs trained on such corpora, along social and economic axes, and (2) measure the fairness of downstream NLP models trained on top of politically biased LMs. We focus on hate speech and misinformation detection, aiming to empirically quantify the effects of political (social, economic) biases in pretraining data on the fairness of high-stakes social-oriented tasks. Our findings reveal that pretrained LMs do have political leanings which reinforce the polarization present in pretraining corpora, propagating social biases into hate speech predictions and media biases into misinformation detectors. We discuss the implications of our findings for NLP research and propose future directions to mitigate unfairness.

<iframe src="https://app.sli.do/event/532fnPQGAZvefKDC1fZsuL/embed/polls/45895f18-0222-4f96-a94a-1a1ae6eb09b7" width="300" height="400"></iframe>

From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models

User simulators are agents designed to imitate human users; recent advances have found that Task-oriented Dialogue (ToD) systems optimized toward a user simulator could better satisfy the need of human users. However, this might result in a sub-optimal ToD system if it is tailored to only one \textit{ad hoc} user simulator, since human users can behave differently. In this paper, we propose a framework called MUST to optimize ToD systems via leveraging Multiple User SimulaTors. The main challenges of implementing MUST fall in 1) how to adaptively determine which user simulator to interact with the ToD system at each optimization step, since the ToD system might be over-fitted to some specific user simulators, and simultaneously under-fitted to some others; 2) how to avoid catastrophic forgetting of the adaption for a simulator that is not selected for several consecutive optimization steps.To tackle these challenges, we formulate MUST as a Multi-armed bandits (MAB) problem and provide a method called MUST$_{\mathrm{adaptive}}$ that balances \textit{i}) the \textit{boosting adaption} for adaptive interactions between different user simulators and the ToD system and\textit{ii}) the \textit{uniform adaption} to avoid the catastrophic forgetting issue.With both automatic evaluations and human evaluations, our experimental results on MultiWOZ show that the dialogue system trained by MUST achieves a better performance than those trained by a single user simulator. It also has a better generalization ability when testing with unseen user simulators.

<iframe src="https://app.sli.do/event/532fnPQGAZvefKDC1fZsuL/embed/polls/8a6b9f60-373d-40e9-86b7-025065db126d" width="300" height="400"></iframe>

One Cannot Stand for Everyone! Leveraging Multiple User Simulators\\ to train Task-oriented Dialogue Systems

Long-form question answering systems provide rich information by presenting paragraph-level answers, often containing optional background or auxiliary information. While such comprehensive answers are helpful, not all information is required to answer the question (e.g. users with domain knowledge do not need an explanation of background). Can we provide a concise version of the answer by summarizing it, while still addressing the question? We conduct a user study on summarized answers generated from state-of-the-art models and our newly proposed extract-and-decontextualize approach. We find a large proportion of long-form answers (over 90%) in the ELI5 domain can be adequately summarized by at least one system, while complex and implicit answers are challenging to compress. We observe that decontextualization improves the quality of the extractive summary, exemplifying its potential in the summarization task. To promote future work, we provide an extractive summarization dataset covering 1K long-form answers and our user study annotations. Together, we present the first study on summarizing long-form answers, taking a step forward for QA agents that can provide answers at multiple granularities.

<iframe src="https://app.sli.do/event/532fnPQGAZvefKDC1fZsuL/embed/polls/78962a9e-8d78-4edf-b726-37954b80c784" width="300" height="400"></iframe>

Concise Answers to Complex Questions: Summarization of Long-form Answers

Fine-grained emotion classification (FEC) is a challenging task. Specifically, FEC needs to handle subtle nuance between labels, which can be complex and confusing. Most existing models only address text classification problem in the euclidean space, which we believe may not be the optimal solution as labels of close semantic (e.g., afraid and terrified) may not be differentiated in such space, which harms the performance. In this paper, we propose HypEmo, a novel framework that can integrate hyperbolic embeddings to improve the FEC task. First, we learn label embeddings in the hyperbolic space to better capture their hierarchical structure, and then our model projects contextualized representations to the hyperbolic space to compute the distance between samples and labels. Experimental results show that incorporating such distance to weight cross entropy loss substantially improve the performance on two benchmark datasets, with around 3% improvement compared to previous state-of-the-art, and could even improve up to 8.6% when the labels are hard to distinguish. Code is available at https://github.com/dinobby/HypEmo.

<iframe src="https://app.sli.do/event/532fnPQGAZvefKDC1fZsuL/embed/polls/2d0cc541-fed6-45ea-a43a-386ed52f3a6d" width="300" height="400"></iframe>

Label-Aware Hyperbolic Embeddings for Fine-grained Emotion Classification

Methods to generate text from structured data have advanced significantly in recent years, primarily due to fine-tuning of pre-trained language models on large datasets. However, such models can fail to produce output faithful to the input data, particularly on out-of-domain data. Sufficient annotated data is often not available for specific domains, leading us to seek an unsupervised approach to improve the faithfulness of output text. Since the problem is fundamentally one of consistency between the representations of the structured data and text, we evaluate the effectiveness of cycle training in this work. Cycle training uses two models which are inverses of each other: one that generates text from structured data, and one which generates the structured data from natural language text. We show that cycle training, when initialized with a small amount of supervised data (100 samples in our case), achieves nearly the same performance as fully supervised approaches for the data-to-text generation task on the WebNLG, E2E, WTQ, and WSQL datasets. We perform extensive empirical analysis with automated evaluation metrics and a newly designed human evaluation schema to reveal different cycle training strategies' effectiveness of reducing various types of generation errors.Our code is publicly available at https://github.com/Edillower/CycleNLG.

<iframe src="https://app.sli.do/event/532fnPQGAZvefKDC1fZsuL/embed/polls/80679978-e161-41be-bf61-19620b4ef9a1" width="300" height="400"></iframe>

Faithful Low-Resource Data-to-Text Generation through Cycle Training

Natural language understanding (NLU) tasks face a non-trivial amount of ambiguous samples where veracity of their labels is debatable among annotators. NLU models should thus account for such ambiguity, but they approximate the human opinion distributions quite poorly and tend to produce over-confident predictions. To address this problem, we must consider how to exactly capture the degree of relationship between each sample and its candidate classes. In this work, we propose a novel method with deep model compression and show how such relationship can be accounted for. We see that more reasonably represented relationships can be discovered in the lower layers and that validation accuracies are converging at these layers, which naturally leads to layer pruning. We also see that distilling the relationship knowledge from a lower layer helps models produce better distribution. Experimental results demonstrate that our method makes substantial improvement on quantifying ambiguity without gold distribution labels. As positive side-effects, our method is found to reduce the model size significantly and improve latency, both attractive aspects of NLU products.

<iframe src="https://app.sli.do/event/532fnPQGAZvefKDC1fZsuL/embed/polls/32bf6552-ecc5-4200-be92-d175e1706eee" width="300" height="400"></iframe>

Downloads

Next from ACL 2023

Do GPTs Produce Less Literal Translations?

Similar lecture

How to do things with emotional expressions