This blog post originally appeared on the blog for my sonic branding agency, Electric Raindrop, though I figured the tech portion of this was relevant for this blog and wanted to share here as well.
Last week, I had the pleasure of speaking with an agency in regards to my “Taste the Music” series of works. I can’t say too much about what they reached out to me about, but it very much encouraged me to keep developing this idea and researching, as well as sharing more of my insights for those who are also interested in this niche world.
While I was at Berklee, I discovered a Ted talk by Berklee professor Ben Houge discussing his work doing food operas and wanted to do that myself. Unfortunately I was unable to get into his class, but I did manage to do tons of research and come to understand just how much sound can influence taste, as well as the entire dining experience.
Part I: Understanding what this format even is
What Ben did was he essentially created an interactive and immersive sound installation not just to accompany food, but also where different elements of the dining experience are control parameters in the music. First off, in media scoring we typically have two main categories of music: linear and nonlinear. Linear music is like a film score. It plays left to right the same way every time. Non-linear music is a whole category of audio where different interactive music systems create and/or recompose the music on the fly. It can involve looping, generative music techniques, and much more. The non-linear musical mindset is a strong starting point in an experience such as this, because you never know quite how long diners are going to enjoy each dish, what they will order or whatever other factors you chose as control parameters here.
So with that in mind, a great method for understanding the musical techniques utilized here would be to take a look at video game music (which is also heavily influenced by 20th century minimalists in particular if you’d like to dig deeper) In game audio, we have no idea how a player might play a game, how long it might take them to beat a level, and/or if they are going to even play the game the way it is intended or if they will go on and try to “break” the game. If we were to create a linear piece of music, the player may not finish the level before the music ends, or it may build and get too intense before the player even leaves the starting area. Interactive music techniques can allow the music to continue to play, and even adapt and change over time, in a manner perfectly suited to each gamers particular playing style.
Similarly, in a dining experience we need to be able to account for the various lengths of time it may take a person to eat a meal. If someone is on their own, they may eat faster than when with a table full of loved ones and conversing. Besides that, a major difference here compared to game music is in a video game a single player is often in control of the music system, in the food opera there can be many people whos individual choices can impact the music (depending on how we structure it) so we need to account for that - with the added challenge of this being much more analog and hands on than a pre-programmed system that serves the player the proper music without an additional person controlling things manually.
Part II: So what can we really control (or use as a control?)
My initial struggle with bringing these ideas to life were the massive amounts of tech involved behind it. Ben’s concept of the food opera involves a speaker at every seat and a custom coded musical engine inside of max msp where different food orders affect different musical elements uniquely suited for each diners table. The entire soundscape of the restaurant modulates to different keys together over time, but each diner’s individual meal contributes to this ever evolving soundscape of the place. Because the music is impacted by the diners choices and diners eat at different speeds, this creates an asynchronous experience where one diner might be adding “dessert music” to the mix while another is just ordering their first drink for the night. This presents two issues for me 1) the sheer cost of the set up alone and 2) the programming know-how to create such a music system
A huge solution to dealing with these two issues is to utilize the stereo set up I have, and figure out a format that can be applied in a more synchronous manner saving me the cost of new equipment as well as being able to utilize a lot of my abilities from my video game career (mainly using FMOD & WWise) to create these generative structures. The only “linear” experience I can think of related to food was a guided tasting of alcoholic beverages such as a wine tasting or a beer tasting. This is perfect now because we can blast the stereo music in the room (normal for a brewery) and it can be synchronous - saving a lot of time on the programming here.
So now we have a format we can adhere to within a reasonable scope, utilizing an adjacent skillset I possessed (video game music techniques) and an idea of how the event would go in general. What’s next?
Part III: The connection between sound and taste
In the early days of cinema, even in “silent” film, live music often accompanied the medium. But it was not until we could have synchronized soundtracks where we truly had film scores crafted to intentionally suit the movie and highlight certain moments, themes and more - heightening the cinematic experience. Restaurants and bars have almost always used music in the dining experience, though it was usually a background element or forethought to just enhance the overall experience or aesthetic. For example, think of a fancy steak house or high end restaurant. A small live jazz band or a pianist would be very appropriate here, but a grimey punk rock band likely would not be. A dive bar may have loud and fast music and a club may have an electronic DJ. Any of these choices were intentional aesthetic choices made to create various levels of experiences at these locations.
If we were to take a look at a theme restaurant like rainforest cafe or hard rock cafe - they go even more intentional with the aspects of their sonic branding language. Rain Forest cafe projects sounds of animals, thunderstorms, and more throughout the dining hall - even incorporating light shows and animatronics to create a more “immersive” experience. Hard Rock Cafe does something similar - they blast classic rock tracks throughout the restaurant and surround you with tons of rock and roll memorabilia. These places are more about the “Experience” than the food. Almost like an amusement park but for dining. The food itself at both of these locations is nothing super impressive, but these examples highlight how “eating” is much more than just about flavor.
Some research examples
There has been a lot of research into how music and sound can influence taste and the dining experience. Dr. Charles Spence has found for example that by utilizing “crunchy” audio, they were able to manipulate the perceived crunchiness of potato chips by up to 15% by playing “crunchy sounds” into consumers ears while they were eating. We eat with our ears, and when they replaced the crunchy sounds with more muffled sounds, the chips were perceived to be more stale. The human ear is so sensitive to sound, that we are even able to accurately hear the difference between hot and cold water being poured. Hot water is less viscous than cold, and that subtle difference is enough to affect the sound by a great enough degree for us to differentiate. In his experiments, Dr. Spence has been able to even use music to make toffee appear to be up to 10% more bitter than without the music. Sound clearly plays an important role in the perception of taste, and we can take advantage of this research to both enhance consumer experience and make an artistic statement.
Examples you can try at home
Going back to our film scoring examples, we can often mute a film and play various pieces of music over it to see how different elements of the music affect the perception of the movie. Our brain is constantly looking for connections between various medium, and when we are told there is one we often “find it”. This phenomenon is perhaps most famous in the “Dark Side of Oz” experience where someone “synchronizes” playback of the wizard of oz with Pink Floyd’s Dark side of the moon. When doing so, you’ll find all sorts of connections that might even lead you to believe that Pink Floyd intentionally composed the album as a film score to the movie. You can do this with any album and any movie, but your mileage may vary.
We have to understand the various mediums we work in, their conventions, & more in order to truly take advantage of composing music for these experiences intentionally. So we need to look at what the dining experience is, what “taste” and “flavor” actually are, and what different flavors might be expected in different dishes or drinks and how we can emphasize them with music. It is not unsimilar to understanding what a “hit point” or a “montage” is in film scoring. By understanding the medium we can write stronger music for it.
Anyway, to experiment with these ideas and to see how you might be impacted by sound consider going to your favorite fast food spot soon and grabbing a burger. For example, I went to wendys and got a baconator. I cut that baconator into 8 slices and played a different piece of music with each one, striving for diversity as much as possible. What I noticed, was different pieces of music did not just bring different elements of the dish to my attention but actually impacted how I ate the food itself. This made too much sense - a faster more active song made me eat faster. A more ambient and meditative song made me more aware of all the textures and flavors in my mouth. Whether or not I could perceive things such as “a 5% difference in taste” really didn't matter as much to me anymore (personally). It became quite clear that the different sounds around me impacted the experience regardless of if it was just about taste or flavor or something much more in the experience. Supporting this idea (and in a practical application), some restaurants are using different tempos of music to influence table turnover rate - faster upbeat music during peak hours to encourage faster eating and drinking rates, but then slower tempos when it is slower to encourage more drinking and lingering.
Another fun variation to try is to listen to the following audio with headphones on while eating chips. Even if you can't eat chips right now, please close your eyes and pretend to eat them. Do not listen ahead of time and try to chew in time with the rhythm:
Most people freeze up at the change halfway through. Your brain is listening to all the sounds of your food for feedback and impacting your decisions based on that. When the sound changes, it causes you to reinterpret what you’re eating subconsciously and make new decisions based on that.
Part IV: The Impact of Multi-sensory experiences, intentionality, & more on remembrance and perception
In addition to impacting taste, it has been shown that using custom audio for edibles has improved perceived value, improved memory of the experience, as well as deepening the relationship between consumer and product. In a 2016 experiment, consumers were asked to try beers with no label, a label, and a soundtrack. A significant increase occurred in the consumers evaluation of the beverages when they were presented as a sound-tasting experience. This study references a 1993 study by Areni and Kim where consumers were willing to spend significantly more on wine when classical music was playing rather than top 40 music. Furthermore, studies have been done exploring how various genres of music can make a beer taste sweeter or more bitter.
Outside of the studies, we can use music and sound to paint or curate a drinking experience and increase immersion. At a sonic tasting I did with Slate Point Meadery, we filled the room with a crackling bonfire and the sound of crickets while they enjoyed a cinnamon cyser (a hybrid between an apple cider and a mead) to invoke that feeling of being outside on a late autumnal night in the woods inside the walls of the taproom in a city. Ben Hogue has experimented with playing source recordings of a farm in a farm-to-table dining experience, as well as stories of the farmers talking about the ingredients/food. The goal was to keep the conscious idea of where the food comes from in the diners minds’ throughout the dining experience. In my sonic tasting series, I apply a similar idea and try to engage the audience with history of the beer style and facts of the beers development itself in between courses, and from my anecdotal experience customers tend to remember this much better and increase their enjoyment of the beverage than when a bartender tells them this stuff in a typical experience.
Anyway, you may already intentionally curate these experiences in your own life. After a long week you may pour yourself a glass of wine and play some classical music or jazz. If you’re a gamer, you may drink mead while playing a game like God of War. Perhaps you had a good bourbon while listening to some delta blues. There are a lot of cross-modal sensory experiences you may or may not already be creating for yourself, and even if you can not quantify the influence it has on your taste in cold hard numbers, it is hard to deny that this multi-sensorial experience is common. it provides a deeper enjoyment of the beverage or meal than just the product alone.
Part V: An example of how I implemented this ultimately
To see how this all culminates together, we can take a look at the usual format of my sonic tasting series. This is just one of the many possible applications of scoring for the dining hall, but it is one that I have found a lot of success with.
Typically, I run a sonic tasting as a private ticketed event at a brewery. We have a curated flight of four beverages, with the bar open and available for people to buy more drinks as they please. Each beverage is usually paired with both a food item (usually from an additional vendor we bring in for the event) and its own custom soundtrack for each pairing. So for example, at a previous one I did at Two Ladders in West Nyack, NY: we had 4 beers and each beer had a chicken wing pairing from a local food truck as well as a soundtrack for each pair. After a lot of trial and error, we have found that including the food element here really increases the value of the event, as well as added an additional element in the experience to create a deeper connection with the consumers. We also utilized the TV’s in the bar to put visuals up (like ambient beaches, bonfires etc) throughout the night to further curate the multi-sensorial experience.
The music was played back in custom coded Max MSP playback engines, drawing on techniques from composers such as Brian Eno, Terry Riley, & more. These minimalist ideas create an ever evolving soundscape that changed over time without repeating itself, but from minimal compositional material. These are typically more on the ambient side to promote a more conscious and lingering drinking experience. Each “course” typically lasts between 15-25 minutes, depending on what is being served and at what point in the night it is (earlier beverages tend to go faster than later - both due to ABV as well as the way we typically structure the order of the drinks by style).
In the earliest iterations of this, I included a QR code with a custom built website for people to read the beer notes/history/musical information but I found almost nobody liked using the QR codes and instead preferred for me to talk about all of this audibly before each beer. Speaking of talking, the first couple times I ran this I had to make it very clear it is not a concert. The music is meant to be talked over and just an element of the night, not the focal point. Many customers were initially afraid to talk over these ambient layers and it definitely created a bit of an awkward vibe until they relaxed and understood the night a little more.
I have had a lot of success running sonic tastings, but I am looking to go further in the food opera direction, and work more closely with high end (even michelin star) restaurants where dining is more of an experience and an art-form rather than just a meal. I feel like this is truly where the sound element of the meal will shine.
Part VI: The future and beyond
There are a lot of fun potential applications of these ideas that can be explored in the future. One thing I would like to do is build up a catalog of readily available generative soundscapes via RNBO on my website that go for various widely available commercially available food and beverages to people all over the world. In addition, it could be really cool to partner and/or collaborate with a brewery to build this into their brand itself. Attach a QR code on their cans that takes them to the website. If you buy a can to go from a bottle shop, the super craft beer nerds can get linked to the brewery’s website and experience a sonic tasting at home at their own will. This can create a sort of XR experience, and possibly even engage consumers further than typical social media means on it’s own. I think there is a real opportunity for a brewery to exploit this angle both as a legitimate means of consumer enjoyment of their product (backed up by research!) as well as a huge marketing standpoint for being “the first brewery in the world that has an official custom score for drinking”.
Pilsner Urquell is one of the only alcoholic brands at the moment utilizing sonic branding, complete with a sonic logo and brand anthem. Amplify has put out an issue of their magazine focusing on sonic branding in the alcohol industry and how nearly no brand across any of the beverages (beer, champagne, hard seltzer etc) utilize custom music and instead rely on licensed tracks. This could be a real opportunity for a brand that attempts to get into this space.
I would really like to be more involved in the food world, and find a way to scale these events beyond just one-off ticketed experiences. That would rely on incorporating much more technology and finding a way to extract data based on customers orders, and automate the playback systems without the need for human input. I’m sure it is totally possible to connect with the POS systems API or have a method for the wait staff to input a few pieces of information into some software, but I imagine AI will be an excellent solution for maintaining a general soundscape and then manipulating control parameters based off of incoming as well as historical data.
Lastly, I fully believe in the potential of XR in the branding experience. Perhaps your product is very organic and natural. If it is a beer, you use photographs of beautiful natural landscapes on your labels you can have a QR code that plays soundscapes related to or from that environment. Or your labels suggest some sort of aesthetic element that can be paired with audio and music from a matching genre can be played, or perhaps the brewer/owner wants to personally record an audio discussing the beer that customers can scan and listen to in the store while making their selections. Maybe something can even be done similar to how grocery stores may play thunderstorm sounds when they wash the vegetables. The possibilities for using audio in food and beverage marketing is endless, and the impact it can have on the brand and the consumer enjoyment of the product is definitely significant enough that it should be considered far more extensively than it currently is.
This post has been brought to you by Electric Raindrop Audio Company: A full service sonic branding agency for brands looking to cut through the noise and be heard.
Please make sure to join our Screen Scoring weekly newsletter to stay up to date with all scoring relate news!
Comments