Chirophone

project for Master of Arts (MA) in Visual Arts, AVA, HKBU by Aby. Emmet Shaw, 2023-2024.

About the project

1. Background

1.1 Musical instruments made from clay

Since earliest times the simplest of materials - clay or mud - has been used to make tools for producing music. Many of them originated as domestic pots, but their use changed with the discovery of their sound-producing properties. All instrument families are represented: drums, winds, strings, and even unique hybrids such as Peruvian water whistles, and for each type of instrument special construction techniques specific to clay were developed. Pre-Columbian ceramic wind instruments - ocarinas, whistles, flutes and whistling jars - were often made in human or animal likeness and elaborately decorated with ceramic pigments, emphasizing the importance of music in ancient societies, while percussion instruments from the Middle East show off surfaces of Islamic abstract design. For example a jaguar ocarina-rattle hybrid was found at the ancient Maya site of Pacbitun, Belize (fig - ). Ceramicists today such as Brian Ransom and Susan Rawcliffe (fig - ) are continuing and expanding these traditions of ceramic instrument making while at the same time embodying a very contemporary formal aesthetics.

My CHIROPHONE project operates within these past and present practices of ceramic music making, but it explores rich new directions by introducing technologies to sonify the shape and surface features of ceramics.

1.2 Converting sound into material forms, and material forms into sound.

The idea of getting a formal expression for musical vibrations occurred to Chladni, a physicist of the last century. He fastened a plate of glass by its center, and then, having scattered some sand over the surface, threw it into sonorous vibrations by means of a violin bow. (fig- ) The opera-singer Margaret Watts Hughes invented the Eidophone in 1904. She would sing into a pipe, the end of which had been covered with an elastic membrane. On this membrane some substance (sand, paste, or colored glycerin) was spread; and when the membrane was set into vibration by the musical note sounding beneath it, the substance above gathered itself together into wonderful forms.(fig- )

Leon Theremin, inventor in 1919 of the early electronic music instrument named after him, also invented the ‘Illumovox’, where ‘a beam of light was projected through a rotating disc containing a strip of gelatin tinted with a colour spectrum’. In this way the composer could link the performance of the Theremin with the projection of corresponding colours.

In the late 1870s Thomas Edison invented the phonograph, a machine that could record and reproduce sound by means of the vibration of a stylus, or needle, following a groove on a rotating wax coated disc. The recording diaphragm (microphone) was connected to a stylus that cut a groove into a rotating cylinder covered in wax, and stored a replica of sound waves as a series of undulations in grooves inscribed in the wax. When the cylinder is played back, another stylus responds to the undulations, and its motions are then reconverted into sound.

The CHIROPHONE has a strong affinity with the analogue technologies of the phonograph as well as the gramophone with its turntable where the stylus tracks a groove, its pickup that converts the mechanical movements of the stylus into electrical impulses, its amplifier that intensified these electrical impulses, and its loudspeaker that converts the amplified signals back into sound. Because my CHIROPHONE uses an optical rather than mechanical reader, it is also closely related to the digital technologies of the laser disc where input signals are recorded as a sequence of coded holes and during playback the signals are read out with a laser. Variations in the amount of light reflected from the disc are sensed by a photodetector, and electronic circuitry translates the light signals into audio signals. The CHIROGRAPH detects light variations as well as displacement to sonify its ceramic objects.

1.3 The conjunction of sound and image

The CHIROPHONE is also closely related to the trajectory of the history of practices that explore the inter-relationships between sound and image.

The idea of a link between music and fine art can be traced to ancient Greek philosopher Plato, who was the first to apply musical terms ‘pitch’ and ‘harmony’ to paintings. The English hermeticist Robert Fludd (1574-1637) linked the Ptolemaic universe to musical intervals.(fig. - ). Schubert compared E minor with ‘a young lady dressed in white, with a pink and red bow on the chest.’ Goethe declared that architecture was "frozen music". Pictures at an Exhibition, composed by Modest Mussorgsky in 1874, depicts his tour of an exhibition where each of the ten numbers of the suite serve as a musical illustration of an individual painting. Around 1913, Wassily Kandinsky began working on paintings that came to be considered the first totally abstract works in modern art, “…drawing coloured lines and blobs on the canvas … and making them sing just as powerfully as I knew how." For Kandinsky, music and color were inextricably tied to one another, and he associated each note with an exact hue, and shades that resonated with each other to produce visual 'chords'. Scriabin, a composer of Kandinsky's generation, not only declared that he heard keys and chords as colours, but composed orchestral works to include a "colour keyboard" illuminating the concert hall with a flood of colours. To support his colour theories, Kandinsky appealed to the phenomena of synaesthesia, and recently a group of neuroscientists has managed to prove that a synaesthete exhibits brain activity in areas responsible for visual activity when they listened to sounds. (ref. -)

By the late 19th. and early 20th. century, the blurring of the edges between music and the other arts had become a widespread obsession and in the early 20’s film allowed painting to break out of its static frame and enter a new world of motion, thereby bringing visual art closer to music. For example, Hans Richter explored the rhythmic motion of abstract shapes, seeing abstract ‘evolutions and revolutions’ as being ‘analogous to the sounds of music’ (fig. -)(ref. -), and Laszlo Maholy-Nagy recognized that Viking Eggeling’s films ‘leaned upon musical frames of reference such as the division of time, regulation of tempi and over-all structure.’(fig. -)(ref. -). Another 20th Century development that embodies this convergence between image and sound are the graphical and ‘picturesque’ music scores created by musicians such John Cage (ref. -) Brian Eno (ref. -) and Cornelius Cardew (ref. -). Cardew’s Treatise (1963-1967) consists of a 193 pages of graphic notations that employed ambiguous numbers, shapes and symbols that the composer intended to be freely interpreted by the performer. (fig.-)

2. The Chirophone

In its simplest configuration, my proposed Chirophone is a creative research project that enables the surface features and overall shape of a rotating ceramic vessel to be digitally measured so as to create a temporal data stream that can be translated and transformed into musical expressions. The Chirophone is further described in the section Choice of Media/Art Form(s).

2.1 Description

The attached diagram describes the core components of this apparatus (fig. -)

motorized turntable on which the ceramic vessel is placed
user activated speed control of the motorized turntable
motorized linear translator that enables two sensors to be moved vertically up and down in front of the rotating ceramic vessel
laser displacement sensor that measures the surface texture of the ceramic vessel with micro-millimetre resolution
photoelectric full-spectrum sensor that measures colour and grey-scale of the ceramic vessel surface
electronics and application software that converts the data streams from these sensors into a format that can be used for musical interpretations and expressions
computer, amplifer and speaker
a further feature that will be tested in this research project is the placement of a transducer in contact with the ceramic vessel, so that the vessel itself will resonate as a speaker for the musical output.

2.2 Technology.

The Chirophone’s multimodal technology is largely based on an innovative configuration of industry standard components (hardware and software), that conjoins the fields of mechatronics, opto-electronics and digital signal processing. A multidisciplinary approach will inform my exploration and implementation of technology for the Chirophone, and currently a close conversation with the Hong Kong media artist Joseph Chan – who specializes in automated machinic sculptures – is providing the preliminary structural designs for the Chirophone’s rotating platform, linear-translator and optical sensors (fig. -). As a result of initial discussions with Baptist University’s Professor of Computer Science Jiming Liu, there will be opportunities to cooperate with faculty and students in both computer science and engineering & materials science with respect to the implementation of the Chirophone’s optical sensing) apparatus for displacement and image measurement, as well as the coding of the software architecture that will receive the raw data, and then transform and output it into language that is suitable for sonic processing.

Intended Outcomes

The Chirophone

The making of a fully functional and artistically/musically compelling ‘machine’ – the Chirophone. This includes the software methodologies and applications that read, interpret and enable sonification of the data streams generated by the Chirophone’s sensors.

The manufacture of approximately 200 ceramic vessels (a number of which will be wood-fired) to enable a deep going exploration of the potentials of various shapes, textures and decorations for sonification via the Chirophone.

Art and technology

By means of the Chirophone, contribute to the burgeoning popularity of ceramic art that can be critically interpreted as a response to the increasingly digital media forms that are infusing the world. At the same time this contribution also embraces the great potential of the digital realm to enliven the ceramic arts (such as is already being evidenced by 3D printing).

Context

Locating one aspect of this research project within the context of Hong Kong ceramic collections by experimenting with the sonifications of ceramic vessels from the HKU Museum and Art Gallery.

Locating of another aspect of this project in the local cultural context by inviting BU faculty and students to test their own ceramic creations in the Chirophone.

Exhibition

An exhibition of the final Chirophone with a selection of these ceramic objects created during my MA that visitors can select and place on its interactive rotating platform. A suitable audio system would be installed in the gallery to enable the real time sonification compositions for each those vessels to be experienced by the visitors. In this way the project will provide an alternative to the more usual spectator/consumer/ connoisseur relationship to craft objects, and instead give these ceramic objects a new multimodal and interactive experiential identity.

While the primary research topic of this MA is the Chirophone as a singular apparatus that sonifies specific ceramic vessels, a number of Chirophones will be virtually combined to simulate the result of a multi-instrument (i.e. multi-ceramic vessel) performance such as a duet, quartet or even an orchestra.

Publication and sale

A publishable journal paper describing the objectives and process of my MA research and the making and testing of the Chirophone. This would include a classification of ceramic shapes, textures and decoration with respect to their characteristics as data streams that can constitute meaningful musical expressions

A video and photographic record of milestones during my MA research, that will accompany the academic record.

A ‘collectable’ (possibly NFT) artifact that would be an animated model of specific ceramic vessels created during this MA project, together with the sound files that were generated by the Chirophone from that vessel.

An evaluation of the possibility of a market implementation of the Chirophone as a saleable artwork whereby the buyer would be purchasing a specific ceramic vessel, Chirophone platform, Bluetooth sound system and/or audio transducer attached to the ceramic.

Objectives

To assimilate and inform my research project by Semester 1’s teaching on Research Methodology for the Visual Arts.

To connect to traditions of clay musical instrument making that go back to ancient times, by creating a new modality of ceramic music production that brings contemporary technologies (as described above) into play. In this way the hitherto silent features of ceramic vessels - shape, texture, decoration - can now constitute what are in effect musical scores.

To take Thomas Edison’s invention of the phonograph as a paradigmatic first instance of sound being recorded and replayed via a physical medium – in this case via inscriptions on a wax coated cylinder. His invention provides a model for the Chirophone in it’s capability to also read the inscriptions on a ceramic vessel utilizing the contemporary technologies.

To undertake a research project in the domain of ceramics that will be driven by interdisciplinarity engagement with music, engineering, electronics and computation, both in terms of the media forms themselves and in terms of the people with their particular skill sets engaged in its production. In this way it will constitute itself as an exemplary art+tech creative undertaking. It also tests a model for an interdisciplinary studio practice with a sustainable future.

To engage with the traditions of experimentation and beliefs in the inter-relationship between sound and image, especially those contemporary painters, film makers and composers who have developed creative ways of giving musical expression to images, and visual expression to music. Interestingly, the Chrophone also introduces the 3rd dimension of form into this exchange. A further aspect in this context are the contemporary practices of purely graphical and pictorial music scores (ref.) and this will influence our experimentation with the surface decoration of ceramic vessels as these themselves become musical scores for the Chirophone.

To explore the ways in which the functionity of the Chirophone will influence design thinking with respect to the ceramics that will be created in this context. This is an outcome of accumulated knowledge with regard to the sonic implications of various shapes, textures and decorations. Part of this process will involve testing paradigmatic geometric forms that one encounters in nature (crystals, spirals, spheres, polygons, etc.) for their sonic signatures.

While the history of clay has primarily produced functional ‘crafts’, many contemporary ceramic artists are exploring clay’s indefatigable creative potential in the context of fine art. The Chirophone intends to expand this potential in new directions through its digital conjunction of ceramics and sound that will bring about innovation in both ceramics and music making.

To use the Chirophone concept and implementation as a platform to explore new ways in which abstract data streams can be interpreted and converted into profound musical experiences. Beginning with Alvin Lucier’s sonification of his alpha brain waves in 1965 (ref. -) numerous artists have been creatively sonifying various data sets (e.g. astronomy, weather, finance), and in the sphere of New Media Art it was Lev Manovich who coined the term ‘database art’.(ref.-)

Schedule

Semester 1 – research, prototyping, prototyping ceramics, identify colleagues and consolidate partners, data analysis,

Discuss Chirophone project with Ray Sylvester Mok and MA ceramics peers to get their input/advice
Assimilating and implementing the guidelies provided by the taught module ‘Critically Engaged’

Concerning the audio aspects of the Chrophone, discussions with colleagues in the Academy of Music/AVA including Eugene, Giovanni, Dimeo, Roberto, and consolidation of creative partnerships
Research into potential musical types, genres and aesthetics that can inform the sonic designs and outcomes of the Chirophone in semester 2.
With respect to the technical aspects of the Chirophone, discussions with colleagues in the BU Dept. of Physics (opto-electronics) and with the engineer/artist Joseph Chan (mechatronics) to establish the design specifications and engineering of the Chirophone protype
Purchase of components for the Chirophone prototype incl. optical sensors, linear translator, motorized platform, computing architecture, relevant software such as MAX/MSP.
Make first set of 100 pots based on estimation and predictions of their performance characteristics in the Chirophone
Identifying pots in Hong Kong collections (HKU and private) that are relevant to the Chirophone and organizing access for semester 2.
Focused research into styles and forms of ceramic production, both historical and contemporary, and with a focus on Asia and Hong Kong, that will inform my ceramics making for this project.

Semester 2 – practice/implementation, exploring creative potential,

Manufacture of the Chirophone protype

Testing the Chrophone prototype and making revisions as required to achieve the final configuration
Make second set of 100 pots that are progressively designed in response to the Chirophone test results
Assimilating the understandings provided by the taught modules ‘Visual Arts Theory & Crticism’, and ‘Art & the Public’ to further inform my approach to the ceramics and musical shaping of the Chirophone.

On-going Research into potential musical types, genres and aesthetics to further inform the sonic development of the Chirophone

Invite BU faculty and students to test/embody their own ceramic creations in the Chirophone
Exhibition of the final Chirophone with a selection of these ceramic objects created during my MA that visitors can select and place on its interactive rotating platform.
A number of Chirophones will be virtually combined to simulate the result of a multi-instrument (i.e. multi-ceramic vessel) performance such as a duet, quartet or even an orchestra
Final report/journal paper (Leonardo) describing the objectives and process of my MA research and the making and testing of the Chirophone.
On-going video and photographic record of milestones during my MA research, that will accompany the academic record.
Reflect on the emerging potential of this project to expand into a second year MPhil undertaking.

Choice of Media

This research project occupies a creative space at the interface of ceramics, technology and music.

1. Ceramics

In the context of this research project, five aspects of ceramics will come into play:

Characteristics of the clay from which the vessel is made
Shape of the vessel
Method of construction
Surface treatment of the vessel
Decoration of the vessel

1.1 Characteristics of the clay from which the vessel is made.

The traditional three variants are earthenware (mixture of clay and temper/grog), stoneware (mixture of clay, powdered stone and iron) and porcelain (mixture of kaolin clay and powdered stone). All of these parameters can be varied to create custom clay bodies that are tailored to specific applications and creative outcomes.

With respect to the Chirophone, I will be experimenting with various custom clay bodies that in the first place lend themselves to be thrown on the potter’s wheel, that secondly have rich textural outcomes via the incorporation of various tempers (e.g. sand, feldspar, grog, etc.), and that thirdly respond favorably to surface treatment.

An example recipe which I expect to be productive would be:

15
5

medium grain white clay

fine grain clay

local powdered granite

temper

felspar granules

1.2 Shape of the vessel.

While the Chirophone achieves much of its musical expression as a consequence of the reading of the texture, surface treatment and decoration of the vessel, the overall shape of the vessel is also going to be a fundamental parameter and modulator, and as a consequence it is my intention to explore a large range of shapes (fig. -)

I will research and implement two procedures that will convert the overall shape of the vessel into musical expressions.

Global understanding of shape - this will be achieved by a pre-scan of the entire vessel so that a complete model of the vessel is used to define certain global musical parameters in advance of the real time musical output generated by its individual features.
Progressive understanding of shape - as the vessel rotates in front of the vertically moving sensor beam, the shape of the vessel will progressively reveal itself as each momentary reading references previous readings. This evolving understanding of the vessel’s shape will thereby modulate the sound.

1.3 Method of construction of the vessel.

In choosing a construction method, most of my research will be done by throwing on a wheel as this has both a conceptual and functional congruence with the nature of the Chirophone, itself being a rotational music-making apparatus.

At the same time, given the comprehensive resources available at Baptist University, I will also explore other construction techniques such as slab building, slip casting and 3D printing, as I foresee these will generate their own idiosyncratic and interesting musical signatures.

1.4 Surface treatment of the vessel

The real time musical performance generated by the Chirophone is very much determined by the surface treatment of the vessel. As a consequence a significant research component of this project is the experimentation of a large range of surface treatments so as to increase the richness of sonic spectrum.

These surface treatments fall into three groups:

Clay – including displacement, penetration, incising, applique, smoothing, scraping, burnishing and neriage
Slip – including painting, slip decorating, slip trailing and marbelization
Glaze – includes ash, lead, tin, enameling, salt, feldspar and alkali

1.5 Decoration of the vessel

While Chirophone utilizes a depth displacement sensor to measure the slightest variation in form and texture, it also utilizes another sensor to record grey scale and colour, which is especially tuned to read decorations on the surface of the vessel.

In the history of ceramics relating to their specific localities and cultures, numerous iconographies and motifs have become paradigmatic, such as Delftware, Jasperware, Ming Dynasty porcelain, Majolika, Karatsuware, etc. Referencing these traditions, as well as experimenting with my own aesthetic preferences, will provide another rich data source for the Chirophone’s musical expressions.

In the last century there has been a resurgence of interest in pottery made in wood-fired kilns because of the unique decorative aesthetics resulting from this technique, and it is my intention to also explore wood-firing at Baptist University’s kion built by Tetsuya Ishiyama. Sonically meaning surface effects that can be achieved in a wood-fired kiln include orange peel, crazing, chun, crystal growth, carbon trapping, etc.

1.6 Historical ceramics and the Hong Kong context

While a core focus of this project is the creation of completely new ceramic vessels suited to my musical experiments, I also intend to explore the opportunties provided by sonifying historical pottery that is extant in Hong Kong collections. For example, Mr. Florian Knothe, director of the HKU Museum and Art Gallery has already agreed to give me access to characteristic ceramic items that are in HKU’s collection. I will also pursue the possibility of obtaining examples of Egyptian faience chalice, Song dynasty Lonquan celadon jar, Song dynasty Jun ware bowl, Islamic lustre ware, Iznik tile, Medici porcelain bottle, K’Ang Hsi famille verte porcelain plate, Böttger porcelain Boucher soft-paste and Wedgewood jasper ware. When it comes to sonifying such historical ceramic vessels, I believe it will be also meaningful to reference the musical traditions that are concurrent with them, and I intend to explore this contextual synthesis with my music collaborators.

2. Music

As described elsewhere in this document, the Chirophone extracts digital data from rotating ceramic objects (their shape, texture and decoration) and converts this into real time musical expressions.

This sonification can be either entirely synthetically generated, or modulated pre-recording samples, or be a combination of both. Fundamental frequencies? DSP?

The Chirophone’s sonification process involves the real time articulation of:

sound (overtone, timbre, pitch, amplitude, duration)
melody.
harmony.
rhythm.
texture.
form (jazz, techno, pop, classical, contemporary, ethnic, natural, abstract, etc.)
expression (dynamics, tempo, articulation)

The Chirophone is a fundamentally multidisciplinary project in that its potential musical expressions will be brought about via by my co-operations with faculty at Baptist University’s Academy of Music. At this stage Eugene Birman, Alonso Trillo, Dimeo… and Santini… have all expressed interest in participating in this project. These music professionals each have their own music-making identities and aesthetics that I want the Chirophone to bring to the fore. In this sense the Chirophone can be understood as both a unique artistic creation and a generically new musical instrument that will enable a range of musical expressions to emerge via each artist’s personal approach to the interpretation of the data being generated by the Chirophone’s sensory apparatus.