University of Sussex
peerj-cs-205.pdf (832.42 kB)

Sample-level sound synthesis with recurrent neural networks and conceptors

Download (832.42 kB)
journal contribution
posted on 2023-06-09, 18:05 authored by Chris KieferChris Kiefer
Conceptors are a recent development in the field of reservoir computing; they can be used to influence the dynamics of recurrent neural networks (RNNs), enabling generation of arbitrary patterns based on training data. Conceptors allow interpolation and extrapolation between patterns, and also provide a system of boolean logic for combining patterns together. Generation and manipulation of arbitrary patterns using conceptors has significant potential as a sound synthesis method for applications in computer music but has yet to be explored. Conceptors are untested with the generation of multi-timbre audio patterns, and little testing has been done on scalability to longer patterns required for audio. A novel method of sound synthesis based on conceptors is introduced. Conceptular Synthesis is based on granular synthesis; sets of conceptors are trained to recall varying patterns from a single RNN, then a runtime mechanism switches between them, generating short patterns which are recombined into a longer sound. The quality of sound resynthesis using this technique is experimentally evaluated. Conceptor models are shown to resynthesise audio with a comparable quality to a close equivalent technique using echo state networks with stored patterns and output feedback. Conceptor models are also shown to excel in their malleability and potential for creative sound manipulation, in comparison to echo state network models which tend to fail when the same manipulations are applied. Examples are given demonstrating creative sonic possibilities, by exploiting conceptor pattern morphing, boolean conceptor logic and manipulation of RNN dynamics. Limitations of conceptor models are revealed with regards to reproduction quality, and pragmatic limitations are also shown, where rises in computation and memory requirements preclude the use of these models for training with longer sound samples. The techniques presented here represent an initial exploration of the sound synthesis potential of conceptors, demonstrating possible creative applications in sound design; future possibilities and research questions are outlined.


MIMIC: Musically Intelligent Machines Interacting Creatively; G2434; AHRC-ARTS & HUMANITIES RESEARCH COUNCIL; AH/R002657/1


Publication status

  • Published

File Version

  • Published version


PeerJ Computer Science







Article number


Department affiliated with

  • Music Publications

Research groups affiliated with

  • Centre for Research in Creative and Performing Arts Publications


Dataset for this research can be found at related URL.

Full text available

  • Yes

Peer reviewed?

  • Yes

Legacy Posted Date


First Open Access (FOA) Date


First Compliant Deposit (FCD) Date


Usage metrics

    University of Sussex (Publications)