Audio Design and Branding for the Internet of Everything

Archive: originally published in the Audio Branding Yearbook 2013, while I was Audio Brand Lead at Cisco (Audio Identity Lead for the Collaboration Technology Group). Republished from my later manuscript; its examples and forecasts reflect that period.

1. Beep boop beep!

Beep! The microwave finished thawing the bread. Beep beep! The oven is at temperature. Boop! The pot is misaligned on the induction cooker. Beep beep beep! Woops, you left the refrigerator door open. Boop boop! The dishwasher is done.

In everyday life, this sine wave orchestra surrounds you with a backdrop of mechanical noise from fans and kitchen appliances, doorbells and mobile phones. Imagine how much more pleasant our soundscape would be if this cacophony was designed with intent, and composed like a symphony instead. As more and more devices in our lives are connected and fight for our attention, it is our job as sound advocates to design the soundscape of tomorrow.

Hopefully we can avoid a situation like this: Beep! Your calcium levels are low, eat some broccoli. Beep Beep Beep Beep! Relax, your blood pressure is high, stress less. Beep! Turn over, you’e snoring. Beep! Your light bulb is due for replacement. Beep! Your other light bulb is due for replacement.

1.1 The Internet of Things and The Internet of Everything

Here at Cisco we define the Internet of Everything (IoE) as the networked connection of people, process, data, and things. By comparison, the “Internet of Things” (IoT) refers simply to the networked connection of physical objects. We estimate that 99.4 percent of physical objects that may one day be part of the IoE are still unconnected (see Figure 1) and that the overall global value at stake will amount to a whopping $19 trillion over the next decade.

The value this increased connectedness creates as “everything” comes online is producing unprecedented opportunities for organizations, individuals, communities, and countries by “connecting the unconnected” either by people-to-people, machine-to-people, or machine-to-machine connections.

There is a danger that if the devices in the IoE are designed without guidance from passionate sound advocates, the level of noise will increase. The IoE is actually an opportunity to remove sounds from nodes when applicable and let a central control hub reproduce the sound if needed. The 2014 Consumer Electronics Show (CES) in Las Vegas really showed that the IoT was more than a buzzword; it is entering the mainstream in a significant way. “Wearables” emerged as a top trend at CES, where wearable devices with power-efficient sensors are gathering information about our lives, and (hopefully) using it for good. By using vibrating motors in a wristband or smart watch, audio notifications can become superfluous in many situations.

Of course, just because a device is a node in the IoT, does not imply that it will have a loudspeaker or communicate with a user directly. In many cases, it would make sense to use devices like smart phones as user communication hubs. Instead of a self-diagnosing light bulb notifying you directly that it is due for replacement, an app on the phone could convey this information in a more subtle way. Using personal devices as hubs in the IoE brings up the discussion on the use of platform specific sounds or branded sounds. Without going into detail, use of branded sound is valuable in many cases. Brands like Skype have a distinct presence on their mobile apps, and I think this works to the brands advantage. However, if the branded approach was more prevalent and applied by most applications, the user might suffer from sound fatigue.

For the devices that do communicate directly to the user via sound, the experience will improve greatly through a modest investment into crafting a slightly better loudspeaker. By enabling a device with higher fidelity sound reproduction, sound designers are given the freedom to create something more meaningful and gentle than arbitrary beeps and boops. Cash registers, health care machines and kitchen appliances are all examples where a better loudspeaker and well thought-out sound design can have a major positive impact on the soundscapes in which they are used.

1.2 Contextual awareness

Contextual awareness will be key when designing the soundscape of the future. Using the information gathered in the cloud, we can tailor the sound experience and optimize it according to the current situation and environment. Devices that have your calendar information and presence status can play back appropriate sounds with that information in mind.

Imagine you are attending a scheduled meeting or making a call- sounds can be changed to more discreet ones. The sound level can be appropriately and automatically adjusted to the amount of background noise. You may choose to have a different set of sounds at work than at home, and this can be automated with geographic information.

Anyone using multiple devices with Facebook Messenger or Google Hangouts can relate to the pain of having multiple notifications alerting you to the same event. The TV series “The Office” takes this situation to a hilariously over-the-top level with WUPHF.com; a website that sends a fax, text message, email, chat, page, tweet and phones you up when you get a communication. The pervasiveness of available communication solutions is something we all can appreciate, but context-awareness can be used to improve the experience and avoid being alerted by multiple instances simultaneously.

Here at Cisco we have a technology we call Intelligent Proximity, where your mobile device automatically pairs to the video conferencing endpoint when you walk into the room. This a good example of devices using contextual information to simplify the user experience.

1.3 UI Sound design

I must admit, the first thing I do when booting an operating system for the first time, is to disable all system sounds. As a sound designer, many will find this counterintuitive or even hypocritical. The reason is not necessarily caused by my like or dislike of the sound set. The fact is, the sounds rarely offer me any perceived value.

On a similar note, I believe most people turn off the in-game sounds on their mobile devices, as it is frowned upon to have them active in public spaces (as it should be!). Even if headphones are used, I think most would enjoy their own music in that setting. It would be interesting to see a quantified study on this.

We have probably all experienced elderly people's insistence on having the key press sounds on their mobile phones set to the maximum level, often to the distress of their surroundings. The auditive feedback adds value to the user in this case, as eyesight grows weaker and fingers less dexterous as we get older. Haptics can be an equally good or even better solution for the user here, and people close to the user would surely be thankful.

Most devices today have the option to turn sounds either on or off. There is an opportunity here to increase the user flexibility by introducing a sliding scale with levels of sonification, much like the security notification settings in Microsoft Windows. In addition to the usual on/off/vibrate, modes like “Night”, “Contextual aware” or “Urgent” can add user value.

Sounds are most useful in user interfaces (UI) where there is an underlying uncertainty, like when you send an email on an iPhone. You accept that the transmission might take some time, but you would like to move on to other tasks while waiting. There is some uncertainty about whether the message is actually being sent, since there might be network connectivity issues etc. The characteristic sound that is played remedies this uncertainty. Swoosh- you are now confident that the message was sent.

Using infrared remote controls also introduces some uncertainty. Am I pointing it in the right direction? Is the battery working? Did the system receive my input? Having cleverly designed UI sounds will often instill confidence and add value to the user experience in these cases. The less responsive the visual UI is, the more important an auditive confirmation becomes.

The art of UI sound is managing to evoke an emotion and tell a story in a very short time, sometimes only a few hundred milliseconds. A lot can be learned from sound designers for games. Many are experts at creating a holistic relationship between elements within a sound set. They are also finely attuned to the players’ sensitivity to sound fatigue. A misplaced or misconceived in-game sound can truly ruin an otherwise great gaming experience. The best game sound designers create an acoustic ecology that must fit with both the brand of the game and the developing studio.

An aspirational approach to UI sound design is for it to be succinct. That is to say that one should strive to use the least amount of layers and sound events in order to achieve the desired effects. Let us not use sounds for their own sake, although that can often be tempting for “audiophiles”. In order for a sonification to be justified, it has to add value that supersedes the possibility of polluting our sound environment. We should have courage to move away from some old conventions. Alarms nowadays do not necessarily need to spike our stress hormones like sirens and bells of the past. The last thing we need is an injection of cortisol each time a device encounters an error.

Again, as a rule of thumb, UI should be as succinct as possible. An example from our video conferencing systems: When a notification event arises and it is backed up by a visual cue, the need for having separate sounds for various notifications is unnecessary. A notification sound does not need to be played if I am sitting in front of a computer and it has a visual counterpart. It is when you step away and cannot see the visual notification that the sound is valuable. This could be controlled in a smarter way if the computer knew whether or not if you were sitting there.

1.4 Process

In today’s highly competitive marketplace, most technology (especially that which is directed at consumers) must have a short time-to-market strategy in order to be successful. Using external consultancies can be demanding, and time-consuming. As the pressures increase to release products rapidly, I predict the lack of execution time will lead to some weak and incoherent designs.

Internal audio branding and design teams like the ones Nokia and Samsung have can test more thoroughly and collaborate with R&D while the product is being developed. A company will probably not delay a product launch as a result of delays in the sound design deliverables. The time advantage may indeed be the time needed for strategic planning and careful orchestration. Having internal teams will secure consistency over time, and they often have the benefit of being collocated.

However, having internal representatives for the audio identity does not exclude the use of additional consultancies. The level of competence in a large branding agency with talent and experts in every subfield is undoubtedly higher than with most internal teams. The internal model is mostly applicable for companies of a certain size, as this function might not be warranted or prioritized in smaller companies. Simultaneously being both the customer and the client requires discipline and constant self-scrutiny.

As our world grows noisier, having the time to test sound sets in a realistic environment will be imperative. Creating a unique sound expression in an ecosystem of many different brands is challenging. Even more so when doing this in a way that does not pollute the soundscape further. The aesthetics of our work should always be kept to the highest standard. No matter how much documentation of user testing, statistics and clever strategy is executed, the whole experience will crumble if it does not sound good.

With mobile productivity tools, the divide between work and free time is eroding. People are demanding the same functionality in the tools they use at work as they do at home, and bringing consumerism into work life affects the way we design business-targeted products. As a sound designer for mostly business-related collaboration tools, I feel an extra responsibility to soften the office soundscape. The same level of choice of ringtones you experience as a consumer, cannot be expected in the workplace. In a world where open landscapes are deployed with little consideration to the proven loss of productivity in noisy environments, enabling the possibility to choose a ringtone in your preferred genre, is not advised. Free jazz, black metal or opera does not sit well in an open office soundscape. The choices should be limited in order to protect the users from themselves.

A technological advance we as audio branders can utilize in the future is automated deployment of sound sets from the cloud. The possibility of having all assets located in a single service, can speed up implementation and make iterations easier. As the brand evolves over time, legacy products can be a part of the updated story when products’ sounds are sourced from the cloud, instead of being hardcoded on a device. Having this possibility also allows for and facilitates an agile and continuous integration workflow for sound design.

Having APIs (application programming interface) that are future-proof with respect to branded assets, is also important. Many problems can occur when backward-compatibility is needed and APIs are not prepared for the needs beyond the present feature set of a device.

2. Tomorrow starts HERE

The Internet of Things is already here, but will grow rapidly into the Internet of Everything in the coming years. We are ambassadors for quality sound experiences and have a responsibility and an obligation to guide this development in the best possible direction. The work we do as sound designers in the coming years will heavily influence the soundscapes of tomorrow. We have many opportunities for creating holistic audio branding experiences across new touch points. By taking a succinct approach and utilizing the increased contextual information available, we can create stronger brands while adding value to peoples lives.

References

For more information on IoE, see http://internetofeverything.cisco.com (original reference).