Portugal

In this paper we present R-VAE, a system designed for the exploration of latent spaces of musical rhythms. Unlike most previous work in rhythm modeling, R-VAE can be trained with small datasets, enabling rapid customization and exploration by individual users. R-VAE employs a data representation that encodes simple and compound meter rhythms. To the best of our knowledge, this is the first time that a network architecture has been used to encode rhythms with these characteristics, which are common in some modern popular music genres.

ICCC&#39;20

Creating Latent Spaces for Modern Music Genre Rhythms Using Minimal Training Data

music generation

latent space

personalization

machine learning

stages 

COLING, the International Conference on Computational Linguistics, is one of the premier conferences for natural language processing and computational linguistics. Often grouped within the field of artificial intelligence, but actually pre-dating the development of artificial intelligence, advances in computational linguistics and natural language processing are now some of the major drivers behind the use of artificial intelligence for commercial and social applications – for example, on-line search, machine translation and with voice-assisted conversational devices.

First established in 1965, the biennial COLING conference is held in diverse parts of the globe and attracts participants from both top-ranked research centers and emerging countries. Today, the most important developments in our field are taking place not only in universities and academic research institutes, but also in industrial research departments and in technological startups. COLING conferences provide opportunities for all these communities to showcase their exciting developments.

COLING 2020

COLING, the International Conference on Computational Linguistics, is one of the premier conferences for natural language processing and computational linguistics. Often grouped within the field of artificial intelligence, but actually pre-dating the development of artificial intelligence, advances in computational linguistics and natural language processing are now some of the major drivers behind the use of artificial intelligence for commercial and social applications – for example, on-line search, machine translation and with voice-assisted conversational devices.

technical paper

Computational creativity is the art, science, philosophy and engineering of computational systems which, by taking on particular responsibilities, exhibit behaviours that unbiased observers would deem to be creative. As a field of research, this area is thriving, with progress in formalising what it means for software to be creative, along with many exciting and valuable applications of creative software in the sciences, the arts, literature, gaming and elsewhere.

The ICCC conference series organized by the Association for Computational Creativity since 2010 is the only scientific conference that focuses on computational creativity alone and also covers all aspects of it.


ICCC'20

The ICCC conference series organized by the Association for Computational Creativity since 2010 is the only scientific conference that focuses on computational creativity alone and also covers all aspects of it.
Computational creativity is the art, science, philosophy and engineering of computational systems which, by taking on particular responsibilities, exhibit behaviors that unbiased observers would deem to be creative. As a field of research, this area is thriving, with progress in formalizing what it means for software to be creative, along with many exciting and valuable applications of creative software in the sciences, the arts, literature, gaming and elsewhere.



The task of translating text to images holds some valid creative potential and has been the subject of study in Computational Creativity. In this paper, we present preliminary work focused on emoji translation. The work-in-progress system is based on techniques of information retrieval. We compare the performance of our system with three deep learning approaches using a text-to-emoji task. The preliminary results suggest some advantages of using a knowledge-base as opposed to a purely data-driven approach. This paper aims to situate the research, underline its relevance and attract valuable feedback for future developments.

An Approach for Text-to-Emoji Translation

A growing number of visual artists use neural networks in their practice. While these networks show promise as an art form, the lack of interpretability limits control to high level decisions based on observations. As an alternative, this research investigates the hand-crafting of network weights coupled with explanatory visualizations as a form of creative control over the internal and lower level processes. Two experimental tools were developed: one for parametrically generating first layer kernels and the second for editing multiple layers. These tools attempt to transform the hand-crafting of features into ``crafting'' in a richer sense by bringing network weights and visual materials into a tight feedback loop. The first author extensively engaged with these tools and these case studies serve to examine the affordances of internal interaction for art-making. The findings suggest that direct manipulation can be used intentionally and can yield insights into network representations, but that hand-crafting networks of greater sophistication would likely require a hybrid approach integrating data-driven methods.

Hand-Crafting Neural Networks for Art-Making

In human-human collaboration, gesture, verbal communications, and emotional responses are among the communication strategies that shape the interactions between collaborators to negotiate the contributions. Emotional feedback allows human collaborators to passively communicate their stance about the experience and convey their perception of the process without distracting the flow of the task. In human-human co-creative collaboration, participants inter-act and contribute to the task based on their perception of the collaboration over time.  In human-AI co-creativity, perceiving the cognitive and emotional state of the user in order to determine the dynamics of collaboration and decide what the agent should contribute to the artifact are two primary challenges of building effective co-creative Artificial Intelligence systems. This paper addresses these challenges with the design of a co-creative agent that can sense the user’s emotional state through facial expression.

ARNY: A Co-Creative System Design Based On Emotional Feedback

We present a general computational approach that enables a machine to generate a dance for any input music. We encode intuitive, flexible heuristics for what a 'good' dance is: the structure of the dance should align with the structure of the music. This flexibility allows the agent to discover creative dances. Human studies show that participants find our dances to be more creative and inspiring compared to meaningful baselines. We also evaluate how perception of creativity changes based on different presentations of the dance. Our code is available at github.com/purvaten/feel-the-music.

Feel The Music: Automatically Generating A Dance For An Input Song

Cross-Domain   creative   systems   can   learn   how   to creatively  map  and/or  transform  semantic  concepts from remote domains to their own. This project sets out to explore how a computational system relies on what it has learned from analyzing images to produce creative work.  Specifically, the system presented here extracts semantic content and themes from an input image and uses this knowledge to creatively generate emotionally-appropriate musical artifacts to accompany the image. An independent experiment was conducted to provide a performance assessment and direct future work.

Emotive Music Composition from Visual Sources of Inspiration

"Flow 1 is the first part of two pieces of music, Flow 1 and Flow 2 
While Flow 1 is more classically inspired, Flow 2 is more experimental.

This piece is influenced by the way in which humans, technology and intelligent systems work together to bring music into being. It works both as personal experiment in compositional style and as a way to bring together tradition and technology to explore the interplay between the composer, the tools they use and they ways in which we can explore this. In all works of this kind there are challenges that the composer has to deal with, using new technologies to invoke feelings is a difficult and thoughtful process. It is important to both be in the ‘flow’ of the music creation process and to understand the ways in which working technology can bend your practices to the ‘workflow’ of the system. 

"

Flow 1

Flow 2 is the second piece in the series of works and is a more experimental piece which is the counterpoint to Flow 1. These pieces are influenced by the way in which humans, technology and intelligent systems work together to bring music into being. It works both as personal experiment in compositional style and as a way to bring together tradition and technology to explore the interplay between the composer, the tools they use and they ways in which we can explore this. In all works of this kind there are challenges that the composer has to deal with, using new technologies to invoke feelings is a difficult and thoughtful process. It is important to both be in the ‘flow’ of the music creation process and to understand the ways in which working technology can bend your practices to the ‘workflow’ of the system.

Flow 2

Generative models for singing voice have been mostly concerned with the task of “singing voice synthesis,” i.e., to produce singing voice waveforms given musical scores and text lyrics. In this work, we explore a novel yet challenging alternative: singing voice generation without pre-assigned scores and lyrics, in both training and inference time. In particular, we outline three such generation schemes, and propose a pipeline to tackle these new tasks. Moreover, we implement such models using generative adversarial networks and evaluate them both objectively and subjectively.

Score and Lyrics-Free Singing Voice Generation

We consider the challenge of designing an artificial agent capable of interacting with humans in collaborative dialogue to produce creative, engaging narratives. Collaborative dialogue is distinct from chit-chat in that it is knowledge building, each utterance provides just enough information to add specificity and reduce ambiguity without limiting the conversation. We use concepts from information theory to define a narrative arc function which models dialogue progression. We demonstrate that this function can be used to modulate a generative conversation model and make it produce more interesting dialogues, compared to baseline outputs. We focus on two modes of modulation: reveal and conceal. Empirically, we show how the narrative arc function can model existing dialogues and shape conversation models towards either mode. We conclude with quantitative evidence suggesting that these modulated models provide interesting and engaging dialogue partners for improvisational theatre performers.

Shaping the Narrative Arc: Information-Theoretic Collaborative Dialogue

The Creative Systems Framework (CSF) formalises creativity as search through a space of concepts.  As a formal account of Margaret Boden’s descriptive hierarchy of creativity, it is at the basis of multiple studies dealing with diverse aspects ofComputational Creativity (CC) systems. However, the CSF at present neither formalises action nor action selection during search, limiting its use in analysing creative processes.  We extend the CSF by explicitly modelling these missing components in the search space traversal function.  We furthermore integrate the distinction between a concept and an artefact, and provide stopping criteria for creative search.  Our extension,  the  Creative  Action  Selection  Framework  (CASF),  is informed by previous studies in CC and draws on concepts from Markov Decision Processes (MDPs). It allows us to de-scribe a creative system as an agent selecting actions based on the value, validity and novelty of concepts and artefacts. TheCASF brings more analytical depth for creative systems that can be modelled as utilising an action selection procedure.

Downloads

Next from ICCC'20

An Approach for Text-to-Emoji Translation

Similar lecture

Word Embedding Binarization with Semantic Information Preservation

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from ICCC'20

An Approach for Text-to-Emoji Translation

Similar lecture

Word Embedding Binarization with Semantic Information Preservation

Downloads