Data sonification with Sonic Pi + Planet (Labs)

Image for post
Image for post
Sweet logo by Tanya Harrison

Oh, man! Last week was Planet Hack 2020, and it brought together several of my favorite things:

  • Significant advancement in creating generative music with Sonic Pi! I love making music, and I want to produce license-free background tracks for my videos. I got a bunch of help from Sam Aaron, the creator of Sonic Pi... #goals, honestly.
  • Getting to mind-meld with my bestie, Moheeb, whom I haven’t been able to hang out with for ages because of you-know-what… plus, catch up a bit with Tanya Harrison (who created that SWEET logo + gave Planet-related help) and Melissa Lamoreux.
  • My third piece of space music created in tandem with Planet — I love making art that builds on a body of work; it feels more deeply personal, tied in with my history.
  • Bonus: getting to play with some of my favorite chord progressions, and just listen to them forever 💙💙💙

OK, so! Planet provides images of the earth, taken by space robots, aiming to image the entire earth once per day. Moheeb built a thing that pulls in cloud-cover values, for photographs of a defined area of interest, over a defined timespan. He normalized those cloud-cover values to a 0–15 scale.

I wrote a Sonic Pi, uh, patch? script?? that takes that stream of data (or anything normalized to 0–15), and turns it into musical notes in an arpeggio across 5 octaves. The melody goes through a 4-chord progression, leading to something that sounds truly musical!

Moheeb chose Egypt as the area of interest (AOI), and from the cloud cover in those photos, I generated this clip. Not a lot of variation, but it’s kind of nice :)

Then, he generated a track from San Francisco, and that sounded much lovelier!

(My) philosophy of data sonification

  1. Accurately represent the data, and
  2. Make it really sound like music. (And not just noise music ;) )

For me, the whole point of data sonification is to get a sense for the information. On the one hand, it has to be listenable, but if you sacrifice too much information, there’s no point in this at all. But, our auditory system is an incredible computational engine!! And we can use that to process data intuitively, far beyond our capacity to understand lists of numbers.

In my Sonic patch, the arpeggiation and chord progressions handle item (2), while (1) is a little fuzzier, but in this kind of project there will always be tradeoffs — “music” as I think of it has structure, and you can’t impose that structure on raw data without losing a bit of granularity. However, in my opinion, we’ve traded away a relatively small amount of information; the same raw value may produce a note that’s a half-step different at different times, but you get a pretty good sense of it, and that’s what I really wanted.

Sonic Pi

I’ve used Sonic Pi once before, to make this Satie cover:

…But this new project needed to grow and change more easily. I’m no great shakes with software, but despite barely having written a line of Ruby before, I found Sonic Pi very friendly. My partner Ted also helped me make the code leaner.

Here are a bunch of my experiments from developing our project; none of them rely on outside data, so you can just grab them and play!

Planet music collaborations

Image for post
Image for post
My side panel, with soundwaves from a recording of the song.

My second musical Planet homage happened at last year’s Planet Hack. I didn’t end up doing much overlap-hacking with the rest of the group, but wrote and performed a cover of “All You Need Is Love” with lyrics that described our project. (Inspired by seeing the phrase “All You Need Is Dove” at the Planet office.) Unfortunately, we have only one short video clip, which I’ll hopefully upload somewhere soon…

So, this year’s creation ties into what’s swiftly becoming a tradition! I don’t put much stock in tradition for tradition’s sake, but I do like to create my own, with friends.

Chord progressions

So, this patch allows you to switch between a few of my favorites! They’re inspired by:

  • Chumbawamba’s soundtrack for Revengers Tragedy, which you should really watch if you like Eddie Izzard, Christopher Eccleston, Alex Cox films, or awesome things.
  • A nostalgic-feeling progression that can make anyone homesick, which features in “Take Me Home, Country Roads”, as well as “Wagon Wheel” by Old Crow Medicine Show.
  • Its conceptual opposite, the piercing lament upon Gandalf’s fall in Moria — a gorgeous progression that you might know from “Zombie” by the Cranberries, or Eminem/Rihanna’s “Love the Way You Lie”. If the previous progression were 1–2–3–4, then this one would be written 3–4–1–2, swapping the pairs of chords, and they’re both very powerful.
  • A slight modification on that progression, which Philip Glass seems to like a lot — the final chord should be a B7, but I didn’t know how to put that in without making the code much more annoyingly complex.
  • The “purty” one is just some chords I threw in there while protoyping. They sound nice c:

These favorite progressions bring up decades of memories, and I love being able to just let them churn in the background. I can do that with this fourth script.

Next steps

Later, I’d like to pull in other types of data, and maybe add further tracks for drum hits (when certain features are recognized) or similar. Stay tuned… :)

Written by

DIY robots, music, EEG, wearables, languages. FIRST alumna. Hardware Nerd @hacksterio. She/her

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store