Data sonification with Sonic Pi + Planet (Labs)
Oh, man! Last week was Planet Hack 2020, and it brought together several of my favorite things:
- Significant advancement in creating generative music with Sonic Pi! I love making music, and I want to produce license-free background tracks for my videos. I got a bunch of help from Sam Aaron, the creator of Sonic Pi... #goals, honestly.
- Getting to mind-meld with my bestie, Moheeb, whom I haven’t been able to hang out with for ages because of you-know-what… plus, catch up a bit with Tanya Harrison (who created that SWEET logo + gave Planet-related help) and Melissa Lamoreux.
- My third piece of space music created in tandem with Planet — I love making art that builds on a body of work; it feels more deeply personal, tied in with my history.
- Bonus: getting to play with some of my favorite chord progressions, and just listen to them forever 💙💙💙
OK, so! Planet provides images of the earth, taken by space robots, aiming to image the entire earth once per day. Moheeb built a thing that pulls in cloud-cover values, for photographs of a defined area of interest, over a defined timespan. He normalized those cloud-cover values to a 0–15 scale.
I wrote a Sonic Pi, uh, patch? script?? that takes that stream of data (or anything normalized to 0–15), and turns it into musical notes in an arpeggio across 5 octaves. The melody goes through a 4-chord progression, leading to something that sounds truly musical!
Moheeb chose Egypt as the area of interest (AOI), and from the cloud cover in those photos, I generated this clip. Not a lot of variation, but it’s kind of nice :)
Then, he generated a track from San Francisco, and that sounded much lovelier!
(My) philosophy of data sonification
I value two things in this kind of project, and it’s hard to do both:
- Accurately represent the data, and
- Make it really sound like music. (And not just noise music ;) )
For me, the whole point of data sonification is to get a sense for the information. On the one hand, it has to be listenable, but if you sacrifice too much information, there’s no point in this at all. But, our auditory system is an incredible computational engine!! And we can use that to process data intuitively, far beyond our capacity to understand lists of numbers.
In my Sonic patch, the arpeggiation and chord progressions handle item (2), while (1) is a little fuzzier, but in this kind of project there will always be tradeoffs — “music” as I think of it has structure, and you can’t impose that structure on raw data without losing a bit of granularity. However, in my opinion, we’ve traded away a relatively small amount of information; the same raw value may produce a note that’s a half-step different at different times, but you get a pretty good sense of it, and that’s what I really wanted.
Sonic Pi
It’s a framework that’s really designed for live-coding, where someone (a code jockey? CJ?) codes sound and/or visuals in real-time for an audience. I’ve seen a few performances, and it’s AWESOME, but my aims are more for generative music that I can twiddle now and then as it runs in the background. Sonic Pi was created by Sam Aaron, and I was lucky enough to get direct input from him on this project!
I’ve used Sonic Pi once before, to make this Satie cover:
…But this new project needed to grow and change more easily. I’m no great shakes with software, but despite barely having written a line of Ruby before, I found Sonic Pi very friendly. My partner Ted also helped me make the code leaner.
Here are a bunch of my experiments from developing our project; none of them rely on outside data, so you can just grab them and play!
Planet music collaborations
My first collab with Planet came while I was an Autodesk/Instructables artist in residence at Pier 9. Forrest Stearns and Emi Watanabe asked us to propose designs to laser-etch into the side panels on Doves (the Planet satellites). I wrote “Spinning Up” — a sort of reverse lullaby, a waking-up song narrated by the Doves — and invented an “iris notation” for writing down the melody and chords. “My” Dove, 2b-7, orbited the Earth for about a year before burning up on re-entry. It still brings up Big Feelings. 💙
My second musical Planet homage happened at last year’s Planet Hack. I didn’t end up doing much overlap-hacking with the rest of the group, but wrote and performed a cover of “All You Need Is Love” with lyrics that described our project. (Inspired by seeing the phrase “All You Need Is Dove” at the Planet office.) Unfortunately, we have only one short video clip, which I’ll hopefully upload somewhere soon…
So, this year’s creation ties into what’s swiftly becoming a tradition! I don’t put much stock in tradition for tradition’s sake, but I do like to create my own, with friends.
Chord progressions
This is a big one for me! When I find a chord progression that I love, it is an endless delight, and I could eat it up for hours and hours on end. (Really, I’ve made Spotify playlists and mashup tracks for some of them…) Chord progressions have the power to evoke strong emotions, in a way that hits me deeply.
So, this patch allows you to switch between a few of my favorites! They’re inspired by:
- Chumbawamba’s soundtrack for Revengers Tragedy, which you should really watch if you like Eddie Izzard, Christopher Eccleston, Alex Cox films, or awesome things.
- A nostalgic-feeling progression that can make anyone homesick, which features in “Take Me Home, Country Roads”, as well as “Wagon Wheel” by Old Crow Medicine Show.
- Its conceptual opposite, the piercing lament upon Gandalf’s fall in Moria — a gorgeous progression that you might know from “Zombie” by the Cranberries, or Eminem/Rihanna’s “Love the Way You Lie”. If the previous progression were 1–2–3–4, then this one would be written 3–4–1–2, swapping the pairs of chords, and they’re both very powerful.
- A slight modification on that progression, which Philip Glass seems to like a lot — the final chord should be a B7, but I didn’t know how to put that in without making the code much more annoyingly complex.
- The “purty” one is just some chords I threw in there while protoyping. They sound nice c:
These favorite progressions bring up decades of memories, and I love being able to just let them churn in the background. I can do that with this fourth script.
Next steps
For now, I’m getting this onto an actual Raspberry Pi so I can add a hardware interface! Buttons, knobs, switches… looking forward to that.
Later, I’d like to pull in other types of data, and maybe add further tracks for drum hits (when certain features are recognized) or similar. Stay tuned… :)
