Indonesia

Most work on large language models (LLMs) has focused on what we call &quot;vertical&quot; scaling: making LLMs even better for a relatively small number of high-resource languages. We address &quot;horizontal&quot; scaling instead: extending LLMs to a large subset of the world&#39;s languages, focusing on low-resource languages. Our Glot500-m model is trained on more than 500 languages, many of which are not covered by any other language model. But how do we know that the model has actually learned these 500 languages? Broad low-resource evaluation turns out to be a difficult problem in itself and we tried to innovate in several ways. One issue we were not able to solve is that parts of our evaluation standard cannot be distributed due to copyright restrictions. We also find that attributing good/bad performance to the so-called curse of multilinguality is naive and there is in fact also a &quot;boon of multilinguality&quot;. We have released Glot500-m and are in the process of making our training corpus Glot500-c publicly available. 

IJCNLP-AACL 2023

Glot500: Creating and Evaluating a Language Model for 500 Languages

### Welcome to ACL 2023, the 61st Annual Meeting of the Association for Computational Linguistics! 
 The conference will be held in Toronto, Canada, July 9-14, 2023. 
Following the succession of the recent conferences in our field, ACL 2023 will adopt a hybrid format.
While the impact of Covid has considerably diminished in terms of traveling, obtaining visas to Canada
entails a very long process. Moreover, the global economic conditions pose challenges for many individuals to travel to conferences. Recognizing these circumstances, we know many participants may not be
able to attend the conference in person. Therefore, we are committed to providing a great virtual platform
so everyone has the opportunity to interact with other participants and enjoy the conference. Based on the
current registered participants, approxiately 30% have chosen to attend the conference virtually. Whether
you join us in person or virtually, we sincerely hope everyone has a remarkable conference experience. 
This General Chair’s message is where I express my gratitude to the many individuals who have made
enormous contributions to the conference over the past year.

Read [**ACL 2023 General Chair's message**](https://docs.google.com/document/d/1WobYM7norbG4dI48s75HfJoD89qgX5a_F-6U8AteLSA/edit?usp=sharing/) in full.

##### **[Conference Handbook](https://2023.aclweb.org/downloads/acl2023-handbook.pdf)**

ACL 2023

The Association for Computational Linguistics (ACL) is the premier international scientific and professional society for people working on computational problems involving human language, a field often referred to as either computational linguistics or natural language processing.

Most work on large language models (LLMs) has focused on what we call "vertical" scaling: making LLMs even better for a relatively small number of high-resource languages. We address "horizontal" scaling instead: extending LLMs to a large subset of the world's languages, focusing on low-resource languages. Our Glot500-m model is trained on more than 500 languages, many of which are not covered by any other language model. But how do we know that the model has actually learned these 500 languages? Broad low-resource evaluation turns out to be a difficult problem in itself and we tried to innovate in several ways. One issue we were not able to solve is that parts of our evaluation standard cannot be distributed due to copyright restrictions. We also find that attributing good/bad performance to the so-called curse of multilinguality is naive and there is in fact also a "boon of multilinguality". We have released Glot500-m and are in the process of making our training corpus Glot500-c publicly available. 

keynote

### Welcome to IJCNLP-AACL 2023!
I am honored and happy to welcome you to IJCNLP-AACL 2023! The conference hosts both the 13th
International Joint Conference on Natural Language Processing (IJCNLP) and the 3rd Conference of
the Asia-Pacific Chapter of the Association for Computational Linguistics (AACL). 
The joint conferences of IJCNLP and AACL have been organized by alternating leadership in the
Asia-Pacific region, in odd years by Asian Federation of Natural Language Processing (AFNLP) and
in even years by AACL. The conference is organized by AFNLP this year. 

Considering the current situation that we are not yet fully over the coronavirus pandemic, the
conference is offered in a hybrid mode, off-line in Nusa Dua, Bali, Indonesia, and on-line with the help
of Underline, during November 1-4, 2023. For those attending off-line, I hope that you will find time to
explore the exciting local area, which remains quite warm and pleasant at this time of the year. For
those attending on-line, I hope that the attendance is not much hampered, sometimes even enhanced
and comfortable, though the on-line arrangements have been made without ample time. 

Message from Jong C. Park, Korea Advanced Institute of Science and Technology (KAIST) 
IJCNLP-AACL 2023 General Chair 
###### [Conference Handbook](https://drive.google.com/file/d/1S2JSXa105xJxIkNBfhYmGH20Om6wARBx/view?usp=sharing)

In order to acces this event page, you need to register [here](http://www.ijcnlp-aacl2023.org/)

The 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics

Machine translation models are still inappropriate for translating chats, despite the popularity of translation software and plug-in applications. The complexity of dialogues poses significant challenges and can hinder cross-lingual communication. Instead of pursuing a flawless translation system, a more practical approach would be to issue warning messages about potential mistranslations to reduce confusion. However, it is still unclear how individuals perceive these warning messages and whether they benefit the crowd. This paper tackles to investigate this research question and demonstrates the warning messages' contribution to making chat translation systems effective.

An Investigation of Warning Erroneous Chat Translations in Cross-lingual communication

Machine Learning models in the field of Information Extraction for Scientific Publications
require high-quality labeled data. The large
amount of easily accessible LATEX source code
is a treasure trove of high-quality labeled data.
However, existing datasets comprised of document collections and PDF extraction tools have
limitations: (1) The hierarchical structure of
papers is lost because labeling is done in terms
of pages rather than documents; (2) The reading order is not extracted, which potentially
muddles the extracted contextual structure; (3)
Papers included in the datasets are not likely
to be up-to-date.To address these challenges,
we propose LATEX Rainbow, a framework that
bridges LATEX to PDF that can automatically annotate and extract semantic and layout information from LATEX source code. This framework
extends existing annotation methods by taking
into account the properties of different existing
approaches. It can produce token-level semantic structure annotations, preserve the paper’s
reading order, and extract the table of contents,
i.e., the article’s section structure. LATEX Rainbow enables anyone to extend their datasets
with the latest documents. The project is opensourced on GitHub1
for community contributions and use.


LaTeX Rainbow: Universal LaTeX to PDF Document Semantic & Layout Annotation Framework

Text classification is an important technique in
natural language processing for categorizing
text into appropriate domains. With the increasing amount of textual data, robust text classification is in high demand. This paper focuses on
single-label classification of text for scholarly
articles, aiming to analyze a large number of
papers. Inspired by the successful Fusion-inDecoder method used in question-answering
tasks, we propose an accurate method suitable
for long articles. We evaluate the effectiveness of our method through experiments on
single-label classification with scholarly articles, demonstrating its high F1 scores.

Leveraging the Fusion-in-Decoder for Label Classification

This study tackles the challenge of generating
appropriate academic titles based on the paper’s
abstract. We approach this task as a high-level
text summarization problem and introduce an
innovative post-processing method that combines a predictive model with a set of linguistic
rules to enhance the quality of the title generation. We start by evaluating three Natural
Language Generation models (BART, T5, Flan
T5), by identifying the top-performing model
and by configuring it to generate diverse titles.
We then conduct experiments employing various post-processing strategies -using SciBERT
and linguistic rules- to select the best title out
of all machine-generated options. Finally, we
assess our title selection methods in relation to
human evaluations.

Enhancing Academic Title Generation Using SciBERT and Linguistic Rules

Keeping track of all relevant recent publications and experimental results for a research area is a challenging task. Prior work has demonstrated the efficacy of information extraction models in various scientific areas. Recently, several datasets have been released for the yet understudied materials science domain. However, these datasets focus on sub-problems such as parsing synthesis procedures or on subdomains, e.g., solid oxide fuel cells. In this resource paper, we present MuLMS, a new dataset of 50 open-access articles, spanning seven sub-domains of materials science. The corpus has been annotated by domain experts with several layers ranging from named entities over relations to frame structures. We present competitive neural models for all tasks and demonstrate that multi-task training with existing related resources leads to benefits.

MuLMS: A Multi-Layer Annotated Text Corpus for Information Extraction in the Materials Science Domain

We introduce a comprehensive end-to-end pipeline designed to extract complete bibliography section from English scientific articles in digital-born PDF format and further split them into individual citations. At the heart of our pipeline lies the utilization of Languageindependent Layout Transformer (LiLT), a multimodal model that combines text and layout features to enhance the accuracy and robustness of bibliography extraction. By considering both text and visual structure, LiLT significantly improves the identification of bibliographic sections within scientific articles. To split the extracted full bibliography into individual citations, we employ a custom fine-tuned version of SciBERT, a Transformer-based model that excels at handling complex formatting variations common in scholarly bibliography. Having such end-to-end pipeline in-house allows us to bypass reliance on third-party black box tools, such as GROBID, offering greater control and transparency in the bibliography extraction process. Another highlight of our pipeline is its extensibility, as it can be seamlessly adapted to multilingual and image-based PDFs, hence allowing its utility across a wide range of scholarly content. When evaluated on an in-house dataset of digital-born English PDF articles published at Elsevier, we achieved an F1-score of 94.6%, a notable 3.1% improvement over GROBID, which is a well-regarded tool for bibliography parsing in the industry.

An End-to-End Pipeline for Bibliography Extraction from Scientific Articles

Citation graphs represent the citation relations between papers, and they are commonly used by researchers to identify relevant papers. However, citation graphs do not always represent how papers are related to each other. To make more effective use of citation graphs to discover relevant papers, we can consider identifying the functions of citations and label each edge in the citation graphs with its function. This paper proposes a method to identify the functions of citations automatically. The proposed model utilizes language models, e.g., SciBERT, to identify the description of citation functions. However, the language models are limited in terms of the number of input tokens; thus, the entire citing paragraph cannot be processed at once. To overcome this problem, we analyzed the distribution of the descriptions of citation functions in the citing paragraphs and determined the focusing part in identifying the citation functions. Experiments conducted on scientific paper data demonstrated the effectiveness of the proposed method.

On the Use of Language Models for Function Identification of Citations in Scholarly Papers

In this talk, I will present our journey of large-scale multimodal pretrained (generative) models across various modalities (text, images, videos, audio, layouts, etc.) and enhancing important aspects such as unification, efficiency, interpretability, and evaluation. We will start by discussing early cross-modal vision-and-language pretraining models (LXMERT). We will then look at early unified models (VL-T5) to combine several multimodal tasks (such as visual QA, referring expression comprehension, visual entailment, visual commonsense reasoning, captioning, and multimodal translation) by treating all tasks as text generation. We will also look at recent advanced unified models (with joint objectives and architecture, as well as newer unified modalities during encoding and decoding) such as textless video-audio transformers (TVLT), vision-text-layout transformers for universal document processing (UDOP), and composable any-to-any text-audio-image-video multimodal generation (CoDi). Second, we will discuss interpretable and controllable multimodal generation via LLM-based planning and programming, such as layout-controllable image generation via visual programming (VPGen), consistent multi-scene video generation via LLM-guided planning (VideoDirectorGPT), and open-domain, open-platform diagram generation (DiagrammerGPT). I will conclude with important evaluation aspects of multimodal generation models, based on fine-grained skill and social bias evaluation (DALL-Eval), as well as interpretable and explainable visual programs (VPEval).

Multimodal Generative LLMs: Unification, Interpretability, Evaluation

Katie Zacarian is the CEO and co-founder of Earth Species Project. Earth Species Project is a non-profit organization with the mission of decoding animal communication. To achieve this mission, ESP collaborates broadly across many scientific disciplines, bringing together AI researchers, biological scientists, ethicists, philosophers, conservationists, and more. With so many disciplines of research partnering with the organization, ESP’s listening to animals hinges almost entirely on its ability to listen to all of the various collaborators contributing to the work. In this talk, Zacarian talks about the challenges, and joys of listening
deeply to the science and expertise outside of our own disciplines–and eventually–outside of our species.

Can We Hear You? Models for Listening in the Animal Kingdom and Scientific Communities

In this talk, I will present the efforts of Natural Language Processing (NLP) research and product development in Indonesia. In about two decades ago, the NLP researches in Indonesia was not well developed, whether in method or NLP data sources. The number of NLP researchers itself is quite low, compared to the population number of Indonesian people. There was even no available research publication for any regional language in Indonesia.
Since then, there have been several attempts to push NLP research in Indonesia, including the forming of Indonesian Association for Computational Linguistics, the participation of Indonesia researchers as part of several international conference committees, and NLP courses in universities which motivate students to conduct NLP researches. Some obstacles are still exist until now including the research fund support from the government; but the number of Indonesian NLP researchers are now increasing, and so is the NLP related start ups in Indonesia.

Downloads

Next from IJCNLP-AACL 2023

An Investigation of Warning Erroneous Chat Translations in Cross-lingual communication

Similar lecture

Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from IJCNLP-AACL 2023

An Investigation of Warning Erroneous Chat Translations in Cross-lingual communication

Similar lecture

Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages

Downloads