Netherlands

HCOMP 2020

Human Computation Applications

## Welcome to NAACL 2022!
The Annual Meeting of the North American Chapter of the Association for Computational Linguistics (NAACL) is taking place July 10-15, 2022 as a hybrid event, in Seattle, WA and online. We are happy to welcome all of you to conference! 
 
The main conference program features oral presentations, in-person and virtual posters and demo sessions, a plenary session for our best paper presentations and awards, keynote presentations, a plenary panel on the place of linguistics and symbolic structures in NLP, and an Industrial Track panel on Careers in NLP.
 
Posters (including Findings of NAACL 2022) and demos are grouped by areas for both the in-person and the virtual sessions. For the virtual component, the talks will be on Zoom and the posters and the demos will be in GatherTown. The Student Research Workshop and will have an oral session and a poster session.
 
The program also features 6 Tutorials and 26 Workshops.
 
 We wish you a wonderful conference!
 
[The NAACL 2022 Organizing Committee](https://2022.naacl.org/committees/organization/)

NAACL 2022

2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics

technical paper

The 8th AAAI Conference on Human Computation and Crowdsourcing (HCOMP 2020) will be held October 26–28th virtually at The Netherlands Institute for Sound and Vision in Hilversum, The Netherlands.

While artificial intelligence (AI) and human-computer interaction (HCI) represent traditional mainstays of the conference, HCOMP believes strongly in fostering and promoting broad, interdisciplinary research. Our field is particularly unique in the diversity of disciplines it draws upon and contributes to, including human-centered qualitative studies and HCI design, social computing, artificial intelligence, economics, computational social science, digital humanities, policy, and ethics. We promote the exchange of advances in human computation and crowdsourcing not only among researchers, but also engineers and practitioners, to encourage dialogue across disciplines and communities of practice.

This year, we especially invite works that will generate new insights into the quality (or qualities) of human-annotated datasets, including elements such as reliability and replicability of human computation and crowdsourcing experiments, novel metrics for aggregation of results, and holistic approaches to deal with bias, fairness and interpretability.

HCOMP believes strongly in fostering and promoting broad, interdisciplinary research. Our field is particularly unique in the diversity of disciplines it draws upon and contributes to, including human-centered qualitative studies and HCI design, social computing, artificial intelligence, economics, computational social science, digital humanities, policy, and ethics. 


While most crowd work seeks consistent answers, creative domains often seek more diverse input. The typical crowd mechanisms for controlling quality may stifle creativity, yet removing them altogether could just produce noise. Schemas and metadata provide two mechanisms for embedding existing knowledge into task environments. Schemas are expert-derived patterns designed to structure how people think through a problem. Metadata, on the other hand, illustrate a range of creative input that fits within the structure of a schema. To understand the relative effects of schemas and metadata, we conducted a study where crowd workers are asked to generate creative interpretations for a set of placemaking examples. Crowd workers were guided either by schema plus metadata, schema alone, or neither. We found that showing schema along with crowd-produced metadata helped workers contribute interpretations that are both more on-topic and diverse, compared to using the schema alone or no schema. We discuss the implications on how crowds can creatively build on insights shared by others.

Schema and Metadata Guide the Collective Generation of Relevant and Diverse Work

Combining decisions made by multiple individuals had
been shown to improve the quality of the final decision
in various tasks. However, simple aggregation rules,
such as the majority vote, often fail. We aim to develop
and asses methods that can successfully identify
the optimal decision in those challenging cases, using
Machine-Learning (ML) techniques, operating on data
augmented by meta-cognitive features. Specifically, we
developed several decision-aggregation methods, using
different classification procedures, and relying on features
extracted from meta-cognitive information. To test
our approach, we collected 2928 responses for different
complex problem-solving tasks, which we used to engineer
features and compose the training data for our ML
models. Initial results demonstrate a highly encouraging
potential for our methods to increase the likelihood
of determining the optimal decision compared to popular
aggregation methods.

Feature-Engineering from Meta-Cognitive Data for Machine-Learning based Decision-Aggregation

Machine learning (ML) based systems are frequently used to assist human decision makers, in scenarios ranging from medical diagnostics, to bail decision-making. To understand and moderate the effects of using ML based decision aids in practice, it is necessary to understand how human decision makers interact with the machine decision aids. In this talk, we focus on two aspects of this human-machine interaction: (i) human perceptions of algorithmic fairness, and (ii) machine-assisted human decision-making.
In our study of algorithmic fairness, we take a descriptive, human-centred approach. Using tools designed to assist judges with making bail decisions as a case study, we study how people reason about the fairness of using certain input features for making bail decisions, and propose ML algorithms that account for these moral considerations.
When exploring the problem of machine-assisted decision-making, we go beyond studying the accuracy and fairness of machine decision aids. We focus on understanding how ML-based decision aids influence the decisions of the human decision makers that they are assisting, and on identifying the factors which determine and moderate the magnitude of this influence.

Accounting for Human Factors in Machine-Assisted Decision-Making

With the increasing prevalence of digitalization, the digital transformation is also reaching the building sector, which opens new opportunities for social and economic develop-ment. Therefore, the smart building concept is expanding from intelligent buildings and energy-efficient buildings, which aims to not only control devices and appliances in-sides, but also integrate virtual power plants and adapt to grid's conditions. This broad definition of the term is intended to illustrate the diversity of the challenges related to the en-ergy transition in buildings. In this thesis, a series of innova-tive approaches are applied to analyze the impacts of smart buildings in the energy system and tries to maximize the ben-efits of smart buildings by means of Natural Language Pro-cessing (NLP) and Explainable Artificial Intelligence (XAI) with human computation. Apart from the social trend mining for smart buildings, this thesis focuses on optimizing energy consumption behaviors in smart buildings based on user in-teraction and engaging the public to apply smart building ser-vices. Furthermore, a big data platform is established to inte-grate the impact analysis of above functions with correspond-ing interactive visualization for better reliability and inter-pretability of policymaking.

The Role of Energy Efficient Smart Buildings in the Transformation of the Energy System: a Data Science Analysis from Social and Technical Perspectives

Downloads

Next from HCOMP 2020

Schema and Metadata Guide the Collective Generation of Relevant and Diverse Work

Similar lecture

Applications

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from HCOMP 2020

Schema and Metadata Guide the Collective Generation of Relevant and Diverse Work

Similar lecture

Applications

Downloads