United States

The Abstraction and Reasoning Challenge (ARC) is a set of tasks where one must induce a program from a few given input-output examples, and apply it to a new input. Although humans can easily solve most tasks, ARC is challenging for state-of-the-art algorithms. We hypothesize that humans use intuitive program induction, and interpret ARC tasks by constructing &quot;natural programs&#39;&#39; in language.

We experimentally study &quot;natural programs&#39;&#39; by formulating a two-player game: A participant solves and then communicates the program to another participant using natural language; the second participant must solve the task using the description alone. We find at least 361 out of 400 tasks can be solved from a natural language description, demonstrating that natural language is sufficient in transmitting these natural programs. 

We compared the natural language programs to computer programs constructed using two separate state-of-the-art program synthesis approaches, and conducted a study on leveraging natural language annotations to improve performance of program-synthesis tools.

CogSci 2021

Language annotated Abstraction and Reasoning Corpus

### Looking for Cogsci 2023? Please [click here](https://underline.io/events/405/reception) to switch to 2023 edition.


**Dear Colleagues,**

Welcome to the 43rd Annual Conference of the Cognitive Science Society! 

This year’s conference brings together the latest research from all Cognitive Science and also highlights this year’s theme **Comparative Cognition: Animal Minds.** 

The program features four plenary speakers: Friederike Range (University of Veterinary Medicine, Vienna), Onur Güntürkün (Ruhr University Bochum), Julia Fischer (University of Göttingen), and Erich Jarvis (The Rockefeller University). The keynotes and invited symposia showcase the broad spectrum of ideas that encompass the conference theme of comparative cognition. 

Further, the program includes the Rumelhart Prize keynote presentation by Susan Goldin-Meadow (University of Chicago), and symposia to celebrate the winner of the Elman Prize: Wei Ji Ma (New York University), the Heineken Award winner Robert Zatorre (McGill University), and the Robert J. Glushko Dissertation Prizes session.

The program committee for CogSci 2021 received 1,025 submissions, including 710 full papers, 275 member abstracts, 8 publication-based talks, as well as 15 proposals for symposia, and 17 for workshops and tutorials. After a rigorous review process, the committee selected 152 papers for oral presentation and inclusion in the conference proceedings (14.8%), 332 papers for poster presentation and inclusion in the proceedings (32.4%), and 175 papers for poster presentation with inclusion of abstracts in the proceedings (17.1%) We also selected 275 submitted member abstracts. In addition, we accepted 6 publication-based talks, 12 symposia, and 9 workshops and tutorials to make for an exciting and inclusive program.

We hope that you enjoy the conference this year!<br><br>

**Your Program Chairs,**
![](https://assets.underline.io/uploads/markdown_image/1/image/99984908caba5e6453e5fc5ba34f3507.jpg)

**Learn more about the Vienna Cognitive Science Hub at the University of Vienna, hosts of CogSci 2021**

<iframe width="560" height="315" src="https://www.youtube.com/embed/464h1m3iU68" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

To review the proceedings for the conference or review the searchable program, please visit the [website](https://cognitivesciencesociety.org/program/).


The Cognitive Science Society brings together researchers from around the world who hold a common goal: understanding the nature of the human mind. The mission of the Society is to promote Cognitive Science as a discipline, and to foster scientific interchange among researchers in various areas of study, including Artificial Intelligence, Linguistics, Anthropology, Psychology, Neuroscience, Philosophy, and Education. The Society is a non-profit professional organization and its activities include sponsoring an annual conference and publishing the journals Cognitive Science and TopiCS.

The Abstraction and Reasoning Challenge (ARC) is a set of tasks where one must induce a program from a few given input-output examples, and apply it to a new input. Although humans can easily solve most tasks, ARC is challenging for state-of-the-art algorithms. We hypothesize that humans use intuitive program induction, and interpret ARC tasks by constructing "natural programs'' in language.

We experimentally study "natural programs'' by formulating a two-player game: A participant solves and then communicates the program to another participant using natural language; the second participant must solve the task using the description alone. We find at least 361 out of 400 tasks can be solved from a natural language description, demonstrating that natural language is sufficient in transmitting these natural programs. 

We compared the natural language programs to computer programs constructed using two separate state-of-the-art program synthesis approaches, and conducted a study on leveraging natural language annotations to improve performance of program-synthesis tools.

poster

Social robotics has shown expansive growth in areas related to companionship/assistance for older adults. Critically, everyday interactions with artificial agents often involve spoken language in the context of a shared visual environment. Therefore, language interfaces for these applications must account for the distinctive nature of visually-situated communication revealed by psycholinguistic studies. In traditional frameworks, "rational" speakers were thought to avoid redundancy, yet human-human communication research shows that both younger and older speakers include redundant information (e.g., color adjectives) in descriptions to facilitate listeners' visual search. However, this "cooperative" use of redundant expressions hinges on beliefs about listeners' perception (e.g., "pop-out" nature of human color processing). We explored the incidence and nature of younger and older speakers' redundant descriptions for a robot partner in different visual environments. Whereas both age groups produced redundant descriptions, there were important age differences for when these descriptions occurred and for the properties encoded in them.

Younger and Older Speakers' Use of Linguistic Redundancy with a Social Robot

From the ambient auditory environment, infants identify which communicative signals are linked to cognition. By 3 to 4 months of age, they have already begun to establish this link: listening to their native language and to non-human primate vocalizations supports infants’ core cognitive capacities, including object categorization. This study aims to shed light on the specific acoustic properties in these vocalizations which enable their links to cognition. We constructed a series of supervised machine-learning models to classify those vocalizations that support cognition from those that do not, based on classes of acoustic features derived from a collection of human language and non-human vocalization samples. The models highlight a potential role for spectral envelope and rhythmic features from both human languages and non-human vocalizations. Results implicate a potential role of underlying perceptual mechanisms relevant to spectral envelope and rhythmic features in infants’ establishment of the uniquely human language-cognition link.

Which acoustic features support the language-cognition link in infancy: A machine-learning approach

Individuals increasingly participate in online platforms where they copy, share and form they opinions. Social interactions in these platforms are mediated by digital institutions, which dictate algorithms that in turn affect how users form and evolve their opinions. In this work, we examine the conditions under which convergence on shared opinions can be obtained in a social network where connected agents repeatedly update their normalised cardinal preferences (i.e. value systems) under the influence of a non-constant reflexive signal (i.e. institution) that aggregates populations' information using a proportional representation rule. We analyse the impact of institutions that aggregate (i) expressed opinions (i.e. opinion-aggregation institutions), and (ii) cardinal preferences (i.e. value-aggregation institutions). We find that, in certain regions of the parameter space, moderate institutional influence can lead to moderate consensus and strong institutional influence can lead to polarisation. In our randomised network, local coordination alone in the total absence of institutions does not lead to convergence on shared opinions, but very low levels of institutional influence are sufficient to generate a feedback loop that favours global conventions. We also show that opinion-aggregation may act as a catalyst for value change and convergence. When applied to digital institutions, we show that the best mechanism to avoid extremism is to increase the initial diversity of the value systems in the population.

Synchronising the emergence of institutions and value systems: a model of opinion dynamics mediated by proportional representation

We present a computational cognitive model of referential choice that models and explains the choice between a wide variety of referring forms using a small set of features important to situated contexts. By combining explainable machine learning techniques, data collected in situated contexts, and recent computational models of cognitive status, we produce an accurate and explainable model of referential choice that provides an intuitive pragmatic account of this process in humans, and an intuitive method for computationally enabling this capability in robots and other autonomous agents.

Givenness Hierarchy Theoretic Referential Choice in Situated Contexts

We present a computational model of interactions between a Speaker and a Hearer in a signalling game. The partly cooperative/partly competitive interaction is intended to reproduce essential aspects of human communication, with the of approximating human language use and analysing it by means of simulation studies.
This was accomplished by implementing a language that accommodates for compositional signals, allowing agents to express infinitely many meanings with a finite set of signals. Personal attributes integral to human decision making such as sympathy and trust were implemented as adjustable parameters, providing the opportunity to create and study individuals with different 'personalities'. Over the course of several rounds, agents learn their optimal strategies using the Moran algorithm. The model was able to substantiate widely confirmed notions about human communication such as correlations between truthfulness and trust, demonstrating the possibility of correlation between aspects of linguistic cognition and social aspects of language use.

Modelling Human Communication as a Rejection Game

In an effort towards predicting mental workload while driving, previous research found interactions between working memory load and visuospatial demands, which complicates the accurate prediction of momentary mental workload. To investigate this interaction, the cognitive concepts working memory load and visuospatial attention were integrated into a cognitive driving model using the cognitive architecture ACT-R. The model was developed to safely drive on a multi-lane highway with ongoing traffic while performing a secondary n-back task using speed signs. To manipulate visuospatial demands, the model must drive through a construction site with reduced lane-width in certain blocks of the experiment. Furthermore, it is able to handle complex driving situations such as overtaking traffic while adjusting the speed according to the n-back task. The behavioral results show a negative effect on driving performance with increasing task difficulty of the secondary task. Additionally, the model indicates an interaction at a common, task-unspecific level.

Utilizing ACT-R to investigate interactions between working memory and visuospatial attention while driving

Although Artificial Intelligence (AI) is expected to outperform humans in many domains of decision-making, the process by which AI arrives at its superior decisions is often hidden and too complex for humans to fully grasp. As a result, humans may find it difficult to learn from AI, and accordingly, our knowledge about whether and how humans learn from AI is also limited. In this paper, we aim to expand our understanding by examining human decision-making in the board game Go. Our analysis of 1.3 million move decisions made by professional Go players suggests that people learned to make decisions like AI after they observe reasoning processes of AI, rather than mere actions of AI. Follow-up analyses compared the decision quality of two groups of players: those who had access to AI programs and those who did not. In line with the initial results, decision quality significantly improved for the players with AI access after they gained access to reasoning processes of AI, but not for the players without AI access. Our results demonstrate that humans can learn from AI even in a complex domain where the computation process of AI is also complicated.

Human Learning from Artificial Intelligence: Evidence from Human Go Players’ Decisions after AlphaGo

Digitally-enabled means for judgment aggregation have renewed interest in "wisdom of the crowd'' effects and kick-started collective intelligence design as an emerging field in the cognitive and computational sciences. A keenly debated question here is whether social influence helps or hinders collective accuracy on estimation tasks, with recently introduced network theories offering a reconciliation of seemingly contradictory past results. Yet, despite a growing body of literature linking social network structure and the accuracy of collective beliefs, strategies for exploiting network structure to harness crowd wisdom are under-explored. In this paper, we introduce a potential new tool for collective intelligence design informed by such network theories: rewiring algorithms. We provide a proof of concept through agent-based modelling and simulation, showing that rewiring algorithms that dynamically manipulate the structure of communicating social networks can increase the accuracy of collective estimations in the absence of knowledge of the ground truth.

Rewiring the Wisdom of the Crowd

Semantic relatedness decisions – decisions whether two concepts are semantically related or not – depend on cognitive processes of semantic memory and decision making. However, behavioral findings are mostly interpreted in the light of memory retrieval as spreading activation but neglects decision components. We propose that sequential sampling models (SSM) of decision making as a novel computational account of choice and response time data. In a simulation study, we generate data from basic SSM versions. We show if and how these models can account for two established behavioral benchmarks: The inverted-U shape of response times and the relatedness effect. Further, one of the SSM, the Leaky Competing Accumulator model makes a novel prediction: The relatedness reverses for weakly related word pairs. Reanalyzing a publicly available data set, we found credible evidence that this prediction holds empirically. Our results provide strong support for SSM as a viable computational account of semantic relatedness decisions.

A sequential sampling account of semantic relatedness decisions

The simple gesture of pointing can greatly augment one’s ability to comprehend states of the world based on observations. It triggers additional inferences relevant to one’s task at hand.We model an agent’s update to its belief of the world based on individual
observations using a partially observable Markov decision process (POMDP), a mainstream artificial intelligence (AI) model of how to act rationally according to beliefs formed through observation. On top of that, we model pointing as a communicative act between agents who have a mutual understanding that the pointed observation must be relevant and interpretable. Our model measures “relevance” by defining a Smithian Value of Information (SVI) as the utility improvement of the POMDP agent before and after receiving the pointing. We model that agents calculate SVI by using the cognitive
theory of Smithian helping as a principle of coordinating separate beliefs for action prediction and action evaluation. We then import SVI into rational speech act (RSA) as the utility function of an utterance. These lead us to a pragmatic model of pointing allowing for contextually flexible interpretations. We demonstrate the power of our Smithian pointing model by extending the Wumpus world, a classic AI task where a hunter hunts a monster with only partial observability of the world. We add another agent as a guide who can only help by marking an observation already perceived by the hunter with a pointing or not, without providing new observations or offering any instrumental help. Our results show that this severely limited and overloaded communication nevertheless significantly improves the hunters’ performance. The advantage of pointing is indeed due to a computation of relevance based on Smithian helping, as it disappears completely when the task is too difficult or too easy for the guide to help.

Downloads

Next from CogSci 2021

Younger and Older Speakers' Use of Linguistic Redundancy with a Social Robot

Similar lecture

Fast and Flexible: Human program induction in abstract reasoning tasks