Google Summer of Code 2015 | Gibber: Creative Coding for JavaScript

Introduction

Hello developers,

Gibber is a live coding system for audiovisual programming using JavaScript. It is used by programmers and visual artists to create music and visuals in realtime performance, on stage or in the studio (see toplap.org for more info about live coding performance). It has been successfully used to teach JavaScript programming to middle school, high school and university students. Here’s a brief introduction to Gibber:

For a much more detailed overview, please see the follwing talk from jsConf.Asia last year:

Below are some ideas for extending Gibber that have come both from our interests and the interests of the Gibber community. We are very open to other ideas and proposals; please get in touch with us to share them before submitting, as we may have thoughts on how you can refine your proposals to be more successful.

1. Port the ixi lang to Gibber

Although we’ve designed Gibber to be a JavaScript environment that is easy to use for musical performance, a number of higher level domain specific languages (DSLs) and mini-languages exist that specifically target the creation of musical pattern. We believe the constrained syntax of such languages provides a valuable scaffolding for beginning programmers in addition to serving as useful and visually appealing performance environments in their own right.

We propose porting the ixi lang mini-language to Gibber, and in the process define a set of best practices for authoring other DSLs and mini-languages on top of Gibber. This will provide the student with insights into how the Gibber environment operates, excellent practice in JavaScript, and a focused project, where the ixi lang mini language is ported to one of the most experimental live coding systems of today.

The ixi lang is a live coding language that has been used all over the world in musical performance and higher education, providing a front-end for defining musical patterns controlling the SuperCollider (http://supercollider.github.io) audio programming environment. Here, we would create a web front-end for ixi lang performances that integrates with Gibber; ideally the individual components would be abstracted so that they could be used with other audio libraries as well, such as Tone.js or WAAX.

Expected Results

The student will port ixi lang to the Gibber system. The student will create simple documentation about how to create a mini language on top of Gibber. Time permitting, the student will design their own DSL for performance and/or audiovisual composition, using the lessons learned from the ixi lang port as a template for what is possible.

This project is an exciting venture into language design for the bespoke purpose of musical performance and composition. The student will be supported by the creators of the two systems – Gibber and ixi lang – during the specified project period.  

Knowledge Prequisite

JavaScript programming experience is required for this project. In addition, some basic knowledge of music theory would be helpful; an interest in creative coding is essiential.

Mentors

Charlie Roberts | charlie@charlie-roberts.com | http://www.charlie-roberts.com
Thor Magnusson | http://www.ixi-software.net/content/about.html | t.magnusson@sussex.ac.uk

2. Create a music theory library for digital audio programming that supports non-Western musical traditions

Although a number of music theory libraries for JavaScript exist (see teoria and music.js) none support non-Western musical theory via freely customizable tunings and modes. This is essential to contemporary electronic music practice and is sorely missed in the JavaScript digital audio ecosystem. For this project a student will design such a library, enabling music programmers to use harmony and tunings from a variety of musical traditions. The student will assist in integrating the library into Gibber after its completion; however, the intention is for the library to be usable in conjunction with any JavaScript digital audio programming system.

Expected Results

The student will create a JavaScript library for music theory that fully supports both Western and non-Western musical traditions. The student will create simple documentation for their library and assist in integrating it into Gibber as a showcase for the library’s capabilities.

This project will give students the opportunity to design and implement a useful JavaScript library for musicians, and also to learn more about musical traditions from around the world. Mentors for the project include Dr. Matthew Wright, a computational ethnomusicologist, and Dr. Thor Magnusson, who has authored multiple live musical performance systems that heavily rely on non-Western modes and tunings. Dr. Charlie Roberts will assist with JavaScript programming and Gibber integration.

Knowledge Prequisite

JavaScript programming experience is required for this project. Knowledge of Western music theory is also required; an interest in non-Western musical traditions is preferred. Students with existing knowledge of non-Western musical traditions are highly encouraged to apply.

Mentors

3. A physical modeling synthesis library for JavaScript

Although the audio nodes included in the Web Audio API are the best vehicle for most synthesis algorithms due to their efficiency and low-latency performance, it is difficult to realize some forms of synthesis with them as they process samples in blocks; this precludes short feedback loops, which are an important tool in digital signal processing. Short feedback loops are particularly important to algorithms for physical modeling synthesis, which rely on waveguides and other feedback techniques to create physically-informed models of acoustic sound.

The ScriptProcessor node in the Web Audio API (soon to be replaced by a greatly improved Audio Worker node) enables us to define signal processing routines in JavaScript that handle a single sample at a time, enabling the feedback needed for physical modeling synthesis and other types of synthesis that rely on feedback, such as chaotic oscillators. Since most physical models will never be realized using the audio nodes included by the Web Audio API without significant architectural changes, a library that handles this type of synthesis and can still plug into other libraries would be a valuable contribution.

Expected Results

The student will create a JavaScript library for physical modeling synthesis. The student will create simple documentation for their library and assist in integrating it into Gibber as a showcase for the library’s capabilities. Ideally this library would support constructing arbitrary instruments out of a variety of exciters, dampers, and resonators.

Knowledge Prequisite

JavaScript programming experience is desired for this project, although students versed in another C-like language will strong DSP and physics knowledge will be considered. Knowledge of physics and digital signal processing is required.

Mentors

Charlie Roberts | charlie@charlie-roberts.com | http://www.charlie-roberts.com
Matthew Wright | matt@create.ucsb.edu