United Arab Emirates

Accurate alignment between languages is fundamental for improving cross-lingual pre-trained language models (XLMs). Motivated by the natural phenomenon of code-switching (CS) in multilingual speakers, CS has been used as an effective data augmentation method that offers language alignment at word- or phrase-level, in contrast to sentence-level via parallel instances. Existing approaches either use dictionaries or parallel sentences with word-alignment to generate CS data by randomly switching words in a sentence. However, such methods can be suboptimal as dictionaries disregard semantics, and syntax might become invalid after random word switching. In this work, we propose EntityCS, a method that focuses on Entity-level Code-Switching to capture fine-grained cross-lingual semantics without corrupting syntax. We use Wikidata and the English Wikipedia to construct an entity-centric CS corpus by switching entities to their counterparts in other languages. We further propose entity-oriented masking strategies during intermediate model training on the EntityCS corpus for improving entity prediction. Evaluation of the trained models on four entity-centric downstream tasks shows consistent improvements over the baseline with a notable increase of 10% in Fact Retrieval. We release the corpus and models to assist research on code-switching and enriching XLMs with external knowledge.

EMNLP 2022

EntityCS: Improving Zero-Shot Cross-lingual Transfer with Entity-Centric Code Switching

entity knowledge

crossingual transfer

code switching

## Welcome to EMNLP 2022!
I am delighted to welcome you to EMNLP 2022! I believe this conference has been complicated beyond any precedent. Over the past year, it’s been thrilling to see the organization team approach each new puzzle with creativity and enthusiasm. We hope that those participating in Abu Dhabi as well as those joining remotely will leave the conference feeling newly inspired by the program and newly connected to our ever-growing community. Following EMNLP 2021 and major NLP conferences since, EMNLP 2022 is “hybrid,” serving both virtual and in-person participants.

Our key innovations for EMNLP 2022 include:

* EMNLP 2022 is “hybrid” in a second sense, as well: we allowed both direct and rolling review paper submissions, building on the pilot experiment of EMNLP 2021, which considered a small number of ARR submissions. 
* Familiar from NAACL but new to EMNLP, we’ve added an industry track.
* During the conference, “portals” will link virtual poster sessions to in-person conference participants during poster sessions each day.
* The first ACL-family conference in the United Arab Emirates.

 *Message from Noah A. Smith, University of Washington and Allen Institute for AI, Seattle, Washington, USA* 
***EMNLP 2022 General Chair***
 
[![](https://assets.underline.io/uploads/markdown_image/1/image/9eec7d4a287ee18c278b08229290aa83.png)](https://drive.google.com/file/d/1OlPv6QBeo62VVTughj2jkiLeyHd1WnUt/view)
 
[![](https://assets.underline.io/uploads/markdown_image/1/image/a3db7a768409f05192210d98601edb25.png)](https://emnlp2022.rocket.chat/)

To access this site you need to register. Please register [here](https://2022.emnlp.org/registration/).

Register here

Welcome!
EMNLP 2022 will take place in Abu Dhabi from December 7th to December 11th, 2022. And it will be held in hybrid mode, both online and offline.

findings / work in progress

The goal of this paper is to present a methodology that allows social scientists and domain experts in Latin America to explore  social biases present in word embeddings, and to define the kinds of biasses they want to explore. In particular, we investigate how domain experts and social scientists can transfer their domain knowledge through representations of lexical resources that they can manipulate intuitively, resorting to their expertise to assess both hypothesized and emergent biases.

Lexical methods for bias exploration from a Latin American perspective

In this paper, we present a new approach to overcome the problem of language resources that share significant amount of linguistic resource in the Ometo language family. The dataset is collected from religious domain consisting of four Ometo (Wolaita, Gamo, Gofa, and Dawuro) languages paired with English sentence. Among the experiments, the Wolaita, Dawuro and Gamo paired with English sentence combination for training and Gofa for testing gives highest BLEU score 4.5 than the other combinations. The BLEU score shows a promising result despite the language differences, the morphological richness, and complexity of the Ometo languages which has high impact on the performance of the Ometo-English machine translation.

Low Resourced Multilingual Neural Machine Translation for Ometo-English

In spite of many proposals to solve the neural machine translation problem of low-resource languages, it continues to be difficult for languages with few resources. The issue becomes even more complicated when few resources cover only a single domain. In our attempt to combat this issue, we propose a new approach to improve NMT for low-resource languages. The proposed approach using the transformer model shows 5.3, 5.0, and 3.7 BLEU score improvement for Gamo-English, Gofa-English, and Dawuro-English language pairs, respectively. We discuss our contributions and envisage future steps in this challenging research area.

Improving neural machine translation for low-resource languages using related language resources

Word embeddings powered the early days of neural network-based NLP research. Their effectiveness in small data regimes makes them still relevant in low-resource environments. However, they are limited in two critical ways: linearly increasing memory requirement based on the number of tokens and out-of-vocabulary token handling. In this work, we present a distillation technique of word embeddings into a CNN network using contrastive learning. This method allows embeddings to be regressed given the characters of a token. Low resource languages are the primary beneficiary of these distilled embeddings and hence, we show the effectiveness of such a model on Amharic, Semitic languages that is spoken in Ethiopia.

DistillEmb: Distilling word embeddings via contrastive learning

This paper introduces the Forgotten Realms Wiki (FRW) data set and uses it for domain specific natural language generation. The data set was extracted from the Forgotten Realms Fandom wiki dedicated to the eponymous Dungeons and Dragons (D\&D) game setting. The FRW data set is constituted of 11 sub-data sets in varying degrees of pre-processing including plain text, annotated text, linked graphs, info-box dictionary, and several embedding models. This is the first data set of this size for the D\&D domain. We then present a pairwise similarity comparison benchmark and perform D\&D domain specific natural language gener- ation using the corpus and evaluate the named entity classification with respect to the lore of Forgotten Realms.

Synthesis and Evaluation of a Domain-specific Large Data Set for Dungeons \& Dragons

Automatic personality detection has many applications in different fields. In this paper, since there is no substantial study on Persian speakers' personality detection, we present an efficient and high-performance personality detection system in Persian language. We collected text data from 3781 Twitter users, embedded them using fasttext and fed them to a neural network which comprises LSTM and attention layers. Finally, our model demonstrated significant improvement in at least two traits. Also, we used LIME to interpret how a derivative of our model predicts individual personality.

MBTI Personality Prediction Approach on Persian Twitter

Hate and offensive speech on social media is targeted to attack an individual or group of community based on protected characteristics such as gender, ethnicity, and religion. Hate and offensive speech on social media is a global problem that suffers the community especially, for an under-resourced language like Afaan Oromo language. One of the most widely spoken Cushitic language families is Afaan Oromo. Our objective is to develop and test a model used to detect and classify Afaan Oromo hate speech on social media. We developed numerous models that were used to detect and classify Afaan Oromo hate speech on social media by using different machine learning algorithms (classical, ensemble, and deep learning) with the combination of different feature extraction techniques such as BOW, TF-IDF, word2vec, and Keras Embedding layers. To perform the task, we required Afaan Oromo datasets, but the datasets were unavailable. By concentrating on four thematic areas of hate speech, such as gender, religion, race, and offensive speech, we were able to collect a total of 12,812 posts and comments from Facebook. BiLSTM with pre-trained word2vec feature extraction is an outperformed algorithm that achieves better accuracy of 0.84 and 0.88 for eight classes and two classes, respectively.

Afaan Oromo Hate Speech Detection and Classification on Social Media

Manglish (Malaysian Colloquial English) is a type of creole\textbackslash{}footnote\{According to Wikipedia, a creole language is a stable natural language developed from a mixture of different languages.\} widely used in Malaysia. Like other creoles, Manglish is under-resourced and ignored in NLP literature. Manglish is made up of English as source language but also draws parts of its grammar and vocabulary from its parent languages such as Malay, Chinese (including mandarin and dialects) and Tamil. Due to the severe lack of resources, this work leverages on related-resources to train language models on Manglish. We look into two main categories of related resources: (1) available corpora and (2) existing pre-trained language models. For corpora, Singlish (Singaporean Colloquial English) and various code-switched corpora are used; as for pre-trained language models, existing models such as Singlish and other code-switched LMs involving its `parent languages' are exploited. Given that Manglish is made up of resource-rich parent languages, i.e. English and Chinese, efficient processing of Manglish can allow us to tap into language technologies developed for these highly resourced languages.

Exploiting Available Resources for the Training of Manglish Language Models

Machine translation for Sranantongo, a low-resource Creole language spoken predominantly in Surinam, is virgin territory. In this study we create a general purpose machine translation system for Sranantongo to and from Dutch. In order to facilitate this research, we introduce the SRNcorpus, a collection of parallel Dutch-Sranantongo and monolingual Sranantongo data. We experiment with a wide range of proven machine translation methods. Our best models show the feasibility of machine translation for Sranantongo.

Towards a general purpose machine translation system for Sranantongo

Word sense disambiguation (WSD) plays an important role, in increasing the performance of NLP applications such as information retrieval, question answering, and machine translation. The manual disambiguation process by humans is tedious, prone to errors, expensive, and time-consuming. Low-resource languages such as Amharic do not have WordNet which makes the WSD task challenging. Previous works try to identify and develop Amharic word sense disambiguation using different approaches but still, there are no easy and automatic WSD tools.  Recent research uses handcrafted rules or directly fetching the meaning of an ambiguous word from the synset list in WordNet, but did not learn different representations from data automatically.. 
For this study, we collect the WSD dataset and employ contextual embeddings, namely AmRoBERTa to automatically disambiguate words to their proper sense. We employ CNN, Bi-LSTM, and BERT for the classification task and attain 90\%, 88\%, and 93\%  accuracy respectively. Based on our findings, BERT has better accuracy results than others.
When we use the masking technique of the pre-trained contextual embedding to find the correct sense, it attains 70\textbackslash{}\% accuracy. However, when we use the pre-trained contextual model and glosses to disambiguate the correct sense with embedding similarity, it achieves 71\textbackslash{}\% accuracy. The main contributions of our work are the release of new datasets and exploring the use of pre-trained contextual embeddings for the Amharic word sense disambiguation model.

Downloads

Next from EMNLP 2022

Lexical methods for bias exploration from a Latin American perspective

Similar lecture

EntityCS: Improving Zero-Shot Cross-lingual Transfer with Entity-Centric Code Switching

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from EMNLP 2022

Lexical methods for bias exploration from a Latin American perspective

Similar lecture

EntityCS: Improving Zero-Shot Cross-lingual Transfer with Entity-Centric Code Switching

Downloads