Sound and vision blog

19 September 2018

Seeing sound: What is a spectrogram?

Greg Green, Audio Project Cataloguer for Unlocking our Sound Heritage writes:

In this digital age, most of us are familiar with audio waveforms, the ‘wavy’ images that represent the dynamic course of a particular sound recording. Waveforms are in fact a type of graph, with time on the X axis and amplitude (or loudness) on the Y.

Waveform exampleFigure 1: a waveform represents a sound recording by showing amplitude over time

Waveforms are very useful for conveying basic information about a recording e.g. where the loud bits are, where the quiet bits are, and how dynamic the recording is. If you were listening to an interview, a waveform can clearly show you where someone is speaking. Unfortunately, waveforms cannot tell us much about the pitch, frequency, or harmonic content of a recording. For that we can use a different visual representation of sound… say hello to the spectrogram!

How to read a spectrogram

Spectrograms keep time on the X axis but place frequency on the Y axis. Amplitude is also represented as a sort of heat map or scale of colour saturation. Spectrograms were originally produced as black and white diagrams on paper by a device called a sound spectrograph, whereas nowadays they are created by software and can be any range of colours imaginable!

Wave form v spectragramFigure 2: waveform and spectrogram of the same recording. An oscillating low frequency buzz dominates the waveform, only the spectrogram reveals where the bird is calling

Spectrograms map out sound in a similar way to a musical score, only mapping frequency rather than musical notes. Seeing frequency energy distributed over time in this way allows us to clearly distinguish each of the sound elements in a recording, and their harmonic structure. This is especially useful in acoustic studies when analysing sounds such as bird song and musical instruments. So not only do these graphs look really cool, but they can tell us a lot about the sound without even listening.

Spectrogram example (whooper swan)Figure 3: a spectrogram showing the harmonically rich calls of whooper swans

Whooper Swan calls recorded by John Corbett (BL shelfmark WS1734 C5)

In the above example, we can see the calls of a Whooper Swan represented in a spectrogram. The fundamental frequency of the calls is at about 750Hz, which is the frequency with the most energy (usually the lowest frequency of a sound), and gives the sound its perceived pitch. Above that are the harmonics - additional, quieter frequencies that give the sound its ‘colour’ and make up a sort of sonic signature – a Whooper Swan singing a perfect ‘G’ note will have a very different harmonic structure to a piano playing that same note. This information could be used to analyse bird songs and calls in different locations, or to understand the vocabulary of a species.

Creative uses for spectrograms

Clearly spectrograms can tell us a lot about the acoustic elements of a sound, but they are not just used for scientific studies. Audio editing is most often performed with waveforms as it's easier to make cuts or process a selected time range. When editing software uses spectrograms however, it opens up a whole new realm of possibilities! With this spectral editing, we are able to look into the microscopic details of a sound and apply processes to very specific time and frequency ranges. For example, an obtrusive footstep, or car alarm could be identified and removed from a recording, just like ‘photoshopping’ sound!

Spectral repair exampleFigure 4: a recording of a robin singing was ruined by a dog barking and some low end noise – a spectrogram reveals the unwanted noises and allows the recordist to remove them.

Spectral repair BEFORE

Spectral repair AFTER

Musicians can also use spectral editing to compose and generate sounds that could not be made any other way. Patterns and shapes can be ‘drawn’ into spectrograms and played back as frequency content. In some cases, detailed graphic images can be hidden within spectrograms. Aphex Twin used this technique to hide an image of a face within the second track of his ‘Windowlicker’ EP (1999).

Aphex Face

You can find some more examples of images hidden in the spectral content of popular songs here:

So now you know what spectrograms are, how to read them, and some of their many scientific, creative, and bizarre uses. Keep an eye out for our #SpectrogramSunday @BLSoundHeritage tweets, starting this weekend!

  UOSH_Footer with HLF logo