In 1804 Ludwig Van Beethoven began penning his 5th Symphony in C Minor by scrawling a series of dots, flags, and soft curls across evenly spaced lines on a page of parchment. This combination of letters, numbers, symbols, and text—read left to right—was translated into the airborne vibration of music in 1808 for the symphony halls of Vienna, and later for all Europe and ultimately the world. The treble clefs and bass clefs writ large that evoked the emotions of generations were originally just marks on a page.

In strong contrast, when Roger and Katy Payne first listened to Frank Watlington’s 1960’s recordings of the voices of humpback whales, they were not yet called songs. Songs were reserved for humans and birds and implied structure, repeatability, and—to a certain degree—beauty. Roger, a bioacoustician, and Katy, a musician, were uniquely qualified to listen to and identify the presence of vocal patterning. It was by ear alone that the artist and the biologist recognized that the organized vocalizations of humpback whales fit the criteria for song: structured, repeated, and lovely.

So, how do bio-acousticians, who study the sounds of ecosystems, translate the symphonies of the natural world onto paper? Symphonies composed by animals that have, in their own right, tempo, rhythm, rhyme and repetition, that cannot be neatly captured by quarter notes and adagio notation? We use the scientific version of sheet music, the spectrogram.

Spectrograms are a visual representation of sound, with time across the horizontal axis and frequency – which we perceive as pitch – across the vertical axis. (See the top portion of our figure above for our initial example.) In this way, a spectrogram is read much like music. Unlike western music, however, the natural world is not confined to a major or minor heptatonic (seven-tone) scale, nor do naturally occurring sounds follow discrete units of exact time, like half-notes or quarter notes. Scientists had to develop a method of visualization that included a continuous representation of both pitch and time that could cover any naturally existing value. As a result, the original spectrograms, which were hand tracings of computer output, came out looking somewhat like hieroglyphics.

Sound is also perceived in three dimensions: time, frequency and loudness (or energy content). In traditional musical notation, this is indicated by small letters directing the musician to play pianissimo or mezzo forte. Naturally, animals in nature are not playing instruments, but rather are composing sounds of their own accord. For animals, loudness is produced as its own important ecological variable. To account for this, in addition to time and pitch, loudness is embedded in the spectrogram in the form of color. The brighter the color (or in the case of a black and white spectrogram, the darker the shade of black) the louder. In this way, the specific emphasis of each unique element of the call is made obvious on the page. Ultimately, this visual interpretation is essential for humans who—unlike whales and dolphins—rely on vision to interpret our world.

Let us then further unpack the above figure, which first appeared in the Payne and McVay article Song of the Humpback in Science Magazine in 1971. The article—and this corresponding figure—documented whale song for the first time in scientific history. Further, this work has been credited with sparking the Save the Whales movement that brought humpback whales back from the brink of extinction and into the minds of the public.

What this image conveys to us is two-fold: it shows us that song is both hierarchically structured and also indicates the acoustic properties of each unique song element. Each of the elements can be found by reading the figure. Humpback whale song is made up of ‘sub-units’ which are combined to form a single ‘unit,’ similar to the way that letters or phonics are combined in human language to form a word. Units are then combined in a specific stereotyped order that may include unit repetition to form a ‘phrase.’ Phrases are then combined in a specific stereotyped order that may contain repetition to form a ‘theme.’ Themes are then combined to make a song, and songs are often sung multiple times within a single ‘singing session.’

Beyond structure, the figure shows us the sounds of the voice of the whale song. The first unit is a series of ascending trills which would be rapidly pulsed, followed by two short tones akin to a low whistle. With a little practice, acousticians can see these symbols and sing along. (I admit, I am doing this now writing this piece.) Thus, in one figure we see the sounds of the natural world coupled with the structure of the natural world: nuance and organization that demonstrates that the ocean is filled with more than either silence or chaos.

The use of spectrograms in bioacoustics is widespread. We visualize the mating calls of polar seals, and the squeaking sounds of ice. One can look at a year of sound in a panel and identify choruses of calling fish amidst periodic storms. The figure above, however, did more than just capture the sound. It also captured something that at one time was believed to be uniquely human: creativity. Whale songs change, evolving within a population before being shared with others in a rapid phenomenon of cultural exchange that spans ocean basins. Songs are created each year, and eventually morph into something different, something new.

The discovery of structure, repetition, and a shared cultural experience occurring among whales changed our human perception of the ocean. In the 1970’s humpback whale populations had dwindled into the hundreds, and whaling was both legal and largely unregulated. Industrial whaling was the single largest removal of biomass of any animal group in human history. Whales were perceived and treated as a cash crop, fueling industries ranging from cosmetics to residential heating. The discovery of whale song gave rise to the perception of whales’ intelligence, sociality, and creativity; and made plain the atrocities of industrial whaling that nearly ran this and other intelligent species extinct.

In whales, humans saw something of themselves. Much like Beethoven’s symphony, the voices of humpback whales spread quickly around the world, played on record players, tape players, and even alongside church bells and symphonies. Today, there are between 120,000 and 150,000 humpback whales in our global oceans. They are here, in part, because of this figure. ♦