In an Organic exclusive we talk to Dr. J. A. Hockman about his work in the field of computational rhythm analysis, Breakscience and his studies in breakbeat research.
First of all I think for those who don't already know of you or your work, it would be good to properly introduce yourself and the methods and objectives of Breakscience...
My name is Jason Hockman, I'm a music researcher originally from NY now residing in Montreal, Canada. I've recently finished a PhD in Music Technology, and the subject of my research is the development of hardcore, jungle, and drum & bass. I've been involved in drum & bass for some time as Jason oS and more recently as DAAT with Joe Mnemonic. Back in 2004, I had a few releases, but I felt that I wasn't technically able to achieve the desired synthesis/sound effects that I wanted to apply to the tracks I was making, so I enrolled in Music Technology courses at New York University. In my second year, I took a course with Juan Bello in Music Information Retrieval (MIR), and got hooked on the ideas of onset detection and beat tracking —finding the exact location of note events and beat (tapping) points in the audio stream. At the time, I wasn't sure what the application of these areas were to drum & bass, but I felt that both were cutting edge disciplines, and that in the future there were bound to be ways in which the MIR techniques could be applied to analysis or creation of drum & bass. When I finished at NYU, Juan suggested that I spend a few months working with Matthew Davies, who was just finishing up a PhD at Queen Mary University of London. Matt's contributions to the area of beat tracking are monumental, and his trackers are used behind the scenes of various applications. Over the course of a couple months, he and I pieced together the first of our collab projects, entitled Automated Rhythmic Transformations of Musical Audio (ARTMA), in 2007. Matt and I have since worked together on several other projects, including the some of the work presented in the dissertation.
I returned to North America shortly afterwards, and began a PhD at McGill University in Montreal at the Distributed Digital Music Archives & Libraries Lab (DDMAL) under the supervision of Ichiro Fujinaga. For the first few years, I was waffling a bit between onset detection and beat tracking algorithms, but nothing was really sticking. I also wasn't really producing much music either. Then, Jason Chatzilias (0=0) stopped by and we had a discussion about the lack of concrete historical data from the perspective of producers, and a clear perspective on the tools/samples. This combined with my work in the lab regarding archival and preservation made it clear to me that this project was something that I needed to do, and not just for a thesis... But rather as a life's mission. Technology has a way of steam rolling over artists that utilize the tools that generate the art. As tech advances (and I don't just mean synths and samplers, but media technologies as well) it becomes hard to envision production techniques from technology that is 5 or 10 years old. I wanted to chronicle the method of production and the meaning behind the madness that created the music, which would allow people down the line (say in 100 years or so) to be able to understand what this music meant to the people that made it, and how they went about creating it.
Its a broad topic (really one thats too big for any one work), so I chose as a main focus the incorporation of breakbeats in these genres. The approach was sort of multifaceted I suppose, in that I wanted to get an idea of how musicians had become interested in breakbeats, how they decided to use them, the tools they use(d), and how the technology affected the output. To do this I combined information from interviews with over 20 hardcore, jungle and drum & bass (HJDB) producers, with a ton of user manuals and product reviews, as well as video/magazine/book interviews, and other research materials. These information sources served as the knowledge base for the three main components of the project: a written history of the genres, description of the technology and techniques used in track creation, and computational analysis methods for breakbeats in HJDB. I originally wanted to call the project Breakbeat Science (in homage to the record store), but it seemed more appropriate to leave that one alone, and Breakscience worked just fine. As a side note, I really tried to keep myself out of this as much as possible,not meaning directly, but my experiences and how I came to know the music as I felt that it was important to understanding the views of the interviewees. Drafts were sent to the interviewees throughout the process too, and corrections were made as requested. Even though the dissertation document is completed now, I view the historical component as a work in progress, that will undergo several changes, becoming more detailed as more interviews are conducted.
What does the computational analysis do? What is it and how far reaching is it?
The computational analysis is comprised of 3 methods: Breakbeat classification, downbeat detection, and breakbeat modification analysis. The breakbeat classification method is useful for researchers to figure out which breakbeat is being used; downbeat detection finds the first beat in each measure, and the modification analysis incorporates the other two methods in an estimation of how the breakbeat has been resequenced. Of these the breakbeat classification method is in an early stage, and currently can only work with a limited number of breakbeats. In order to test it, I requested favourite x tracks on facebook, where x was a particular breakbeat (e.g., Amen, Apache, Funky Mule). Given ~100 tracks, the system was able to figure out the breakbeat used 87% of the time by searching for kick drums and comparing them to pitched and distorted versions of kicks. Of course hats and snares still need to be considered, but its not a bad start. This type of information is hopefully useful for researchers to identify trends in the timeline, such as the popularity of the Funky Drummer between 1990 and 1996. A nice application would be a Shazam-type application that could provide return the likely breakbeat, but in my opinion that's years away; possibly something someone else or a team of people can develop.
The downbeat detection algorithm is based on the idea that producers who use breakbeats *typically* use drums in a manner characteristic of the original breakbeat. I say typically, because there are of course a number of tracks that would break this statement. At the core of the downbeat detection method is a machine learning algorithm that is trained on consecutive 8th-note segments of 30 commonly-used breakbeats. The algorithm associates incoming 8th note segments with positions in the training data, providing a likely downbeat signal. This signal is then used along with information from beat tracking and onset detection algorithms to return downbeat points, that are useful in the scope of this project for alignment with the original breakbeats to determine how a breakbeat has been resequenced in a track, or outside the project in such applications as auto-mixing, or auto-effects (think bbcut).
The breakbeat modification analysis uses the other two methods to compare measures of the original breakbeat to the modified breakbeat in the track.
Both during the creation of the work and looking back now, what are your favourite bits?
Hands down, the interviews. I can't express how interesting and exciting these were to do. I'd gotten a taste of what the interviews were like through reading magazines and such, but the kind of in-depth information that the producers and label owners provided, I'd never seen before. Because the interviews were open-ended to some extent, they could take any shape (after a few basic questions), and the result was as many varied discussions as there were interviews. Each was an experience in and of itself, so its hard to pick favourites.
Across the interviews you did with artists and industry insiders, did any patterns emerge? What similarities did you find? And also were there any unexpected anomalies?
Patterns definitely emerged. An appreciation of and an interaction with hip hop, which led to an interest in searching out and selecting breaks. Techniques for mangling breakbeats using PCs and samplers that developed out of a desire for innovation with a backdrop of competition. There really is a sense of a ride that all were on. Throughout the timeline, the interaction between the producers, their instruments, and a receptive audience created multiple critical junctions at which subgenres either emerged or lost momentum (e.g., the division between happy hardcore and jungle or the split between jungle and drum & bass). That part isn't really new per se, but it's a wonderful thing to see it through the voices of people that made it happen.
Technology is now and always has been a driving force for advancement within the music. What were the key changes that you observed? What were the results?
Really its due to the development of samplers and PCs capable of storing/playback of large amounts of audio data. The earliest samplers used by interviewed hardcore musicians were models such as the Akai S612. These samplers had limited memory and minimal functionality which restricted the amount of breakbeat manipulation.
In the following years, they became capable of containing more memory, had internal effects and the ability to modulate sound-shaping parameters. Samplers like the S950 and S1000 were huge leaps ahead as they allowed additional effects such as timestretching etc, while the Ensoniqs had the unique modulation parameters associated with them. Then the affordable E-mu's came along, with the sigma delta encoding, which some have suggested allowed better resolution for bass frequencies. A completely separate sampling/sequencing technology, tracking software —or trackers— was also vastly improved and the invention of the DAW changed the musical landscape as well. At each of these stages, its likely be possible to point to the emergence/decline of camps or subgenres that had had their own styles and sonic character.
From what we understand, the Breakscience project is ongoing and casts it's net far further than what has already been achieved? What do you have planned for the future?
As there are so many people involved in the genres, there are just as many stories to tell. Therefore, I don't see the project as being something that could possibly be completed in the course of a few years. I'd really like to keep going in a few different directions: (1) Interviews: continue doing these as these are the stories that define the culture and serve to generate data for the output. As mentioned, I'd like to keep developing the historical section, to make it more complete and detailed, and really, this is the best way of doing so. (2) Improvement of algorithms: continued development of the breakbeat classifier and the addition of other types, such as snares/hats/hand percussion, and the inclusion of rhythm models that depict likely usage (think Funky Mule). (3) Vinyl archiving: much of the back catalog only exists on vinyl, and should be recorded using high-quality archival gear. This will ensure that people will listen to the highest-quality recordings of this music years in the future, regardless of changes in the vinyl player industry.
'An Ethnographic and Technological Study of Breakbeats in Hardcore, Jungle, and Drum & Bass' is available in full HERE
Photo by J. Thibodeau