Distant sonification: a critical computational method for sonic analysis of digital data

Computation enables new ways of engaging with cultural criticism in cognate disciplines like the humanities and social sciences. This thesis contributes to the field of digital humanities by introducing and developing sonification as a method of analysis of digital data while drawing on these to critically reflect on digital humanities’ computational techniques. It is argued that approaches such as distant reading, networks, and topic modelling support new forms of reading but they typically rely on visual or textual representation. Previous digital humanities work has tended not to address the methodological potential of sonic practice for computational analysis, so this thesis presents a novel method that I call “distant sonification” to both engage with and extend digital humanities techniques. The introduction contextualises the concept of creating and reading patterns as a signal that can be sampled and represented sonically. The research questions are introduced before concluding with a chapter overview. Chapter two presents a literature review that draws from Sound Studies, Sonification, Digital Humanities, and Digital Media to situate the thesis within the wider literature and theory. Chapter three outlines the research methods and ethics used in the thesis. Here I describe the “Polemic Tweet” data set. I explore the limitations of previous digital methods and discuss the combination of digital methods to develop and situate my approach. Chapter four outlines a brief history of the sonic as a method. I explore the history of sound both through its use as an analytical method and how it has been conceptualised. I examine the argument for the use of, and relationship with, technology in the history. I then examine how questions related to notions of signal and noise are developed within paradigms of thinking with technology. Technology’s role within the development of signal and noise is developed with questions about who develops them and the codes through which they are understood or created. Chapter five presents a theory of sonification that draws from a broadly medium-specific approach where I posit that sonification is developed from three objects: data, sound, and temporal objects. I present this novel method of sonification through analysis of a series of sonically constructed digital objects. I critically engage with sonification as data, sound, and temporal objects to understand them through a broadly medium-specific approach. These develop existing theories of sonification. The second part operationalises the theory by introducing the software tools developed and introduces the two case studies and how they use the method. Chapter six applies sonification as a form of distant reading to look for patterns using the concept of the network. The chapter is divided into three sections: firstly, creating a computational reading of the data as a network; secondly, an emergent reading of the and thirdly, a critical reading of an algorithmic criticism that builds on networks. The first reading 4 presents a reading of how devices support the creation of culture and enable the reading that raises questions about the re-use of method and form. In the second part, the Polemic Tweet data is used to explore the focus of the digital method – such as annotations – as a form of exploration of constellations through network structures. The third part of the chapter focusses on algorithmic readings to re-present community through de-forming the modularity score. This is a repurposing of algorithmic criticism to explore computational methods using sonification to examine the lost aspects of the reading. Chapter seven applies these methods to explore topic modelling to generate themes and concepts from the underlying data. As with the previous chapter, it is divided into a section of thematic sections and the critical reading of a related algorithmic reading. The first reading looks at the algorithmic readings of the data to develop thematic readings. The reading here is automated and becomes a way of using the sound medium to represent the algorithmic reading and how the readings might (de)form each other. The second section focusses on the conversion of signal into noise within a model of the reading and the methods by which noise is measured inside the topic model. In contrast to the exploration of networks as testing patterns to apply across the data, this study looks at the ones that emerge from the data. A deconstruction approach is taken to the idea of coherence and to examine the assumptions within the reading. Chapter eight concludes this thesis and argues for further work. Drawing from computational distant reading as engaging with patterns, distant sonification is seen as the representing the signal from a dataset and to read the signal as an object. This computational method is read as both a method to create a critical object and a computational object that is itself open to criticism through medium-specific approaches. “Distant listening” is discussed as an avenue for future research.