Dr. Bruce Wiggins: We're going to talk about our applied electro-acoustic research focusing on what we've put together actually for the impact case study for the new REF and actually this builds on some of the work that was an impact case study for REF 2014 as well and that went very well. So what we're going to talk about today is the democracy of sound and Adam's going to talk a little bit about what that means in the next slide. Our impact case study is shaped into three main themes which all overlap and actually draw from each other in some way shape or form but have all elicited impact internationally in different companies and with different practitioners so we'll start to talk about that now.
Dr. Adam Hill: When Bruce and I started to put together our current impact case study for REF, we had to figure out a way to be able to make our research coherent, because we kind of work in two separate corners of the audio industry. We realized that we really were working towards the same goal with our research, which is broadly the democracy of sound, which means we're trying to deliver a consistently high quality and safe listening experience for everyone listening to sound and that may be at a massive music festival or in a huge sports arena or someone sitting in the room listening to headphones something streaming over the internet.
So it's really giving everyone the equal chance to hear something really amazing and that's something that our current sound delivery systems just can't do. So the research generally is focusing on first understanding the challenges of sound reinforcement and reproduction, what's going wrong, how can we fix it, and then really trying to optimize the systems in a practical way that can be used immediately by practitioners, to make a real difference for audiences. As Bruce said a lot of the research dates back to the last round of REF. Bruce has been researching this area for over 20 years now. I don't think I can claim to be going that long but, maybe 10-15 years, but it is well established. I think in our respective audio engineering fields, we are known as experts in very specific areas, so our impact has become well established and international in reach.
Bruce: You can see from this slide, this is just an example of some of the companies that have benefited from our work either directly or indirectly and we're going to talk about some of these today. Some companies we work with aren't included in the case study, or they've been historically used, and our work goes between festivals, using headphone type listening on YouTube, VLC media player, using software like Reaper, speaker companies like Function One, working with societies like the Audio Engineering Society or the Institute of Acoustics when it might be guest talks or tutorials and that kind of thing, or sometimes directly with bands, like The Prodigy, which again, we will bring up examples of that in a moment.
Adam: We have three impact themes that generally encapsulate our work. First is expanded and enhanced creative immersive audio technology. What does that mean? Essentially it's 3D audio. It's something beyond just listening to speakers in front of you, which has pretty much been the case for about what 50 to 60 years. So that's a lot to do with Bruce's work; taking audio to the next step and making it a bit more exciting. Second is enhanced live event listening experiences, so making going to gigs much better, and third is improving the understanding of health-related aspects of sound. I'll talk later about what this is but really it's about making sure audiences are safe at events.
Bruce: Before we actually talk about the research and the research is obviously a really important aspect of our work and something that drives us in terms of writing papers and giving presentation, but actually a really important aspect of any impact that's going to happen is the enablers of that impact. It's not just the research and actually, there's really important things that we both do and other people at the university also do to encourage the impact to actually happen, because it sometimes happens a bit by accident. But, you have to put things in place, in order to get the work and the knowledge out, there, so just this slide is really just some examples of that. For example, we've run a research symposium here at Derby called Sounds in Space since about 2011.
We run it every year, except this year where we didn't run it because of the COVID restrictions, because it happens in June and, that's where, quite a long time ago now, we as a research group at the time, were working in 3D audio and doing what we thought was lots of interesting things and we struggled actually to have meetings. We decided to have a big meeting really at the end of the year and then thought well actually if we demonstrate some of the things we're doing, and this was actually across arts and engineering at the time, we could actually turn this into a research symposium so we started this mini-conference really in 2011 and it's gone from strength and strength ever since.
The picture on that slide is actually the auditorium three at Markeaton Street that we set up for this particular event and we normally have between 20 and 30 loudspeakers set up in that room that enables really high-quality, three-dimensional audio to be reproduced at the conference live. That was a really novel thing at the time, was actually having good quality audio at an audio conference, it sounds odd but that that was a novelty at the time and, we got quite a name throughout the world, and we get researchers from all over the world coming in.
We've had the BBC come in, we've had Pottermore, so the Harry Potter, a 3D audio rendition actually at Derby, that we weren't allowed to tell anyone about. It's the only other place it happened in the world, other than London because then there were various legal things as to why we couldn't tell anyone about it at the time, but actually, that really helped get the word out there about some of the things we were doing and get other people in to talk to them about. We also produce software tools that implement some of the things we research.
So we talk about the algorithms of 3D audio or reproducing sound in large spaces, but then if you've just got a research paper, no-one can really use it, unless they're also a researcher. So creating software tools that allow students and other people to use that in a more useful way is really important and to go alongside that, also blog posts and tutorials. And all the other things that make all that stuff more accessible. Again research papers are great, you might have lots of equations in them and make them sound very academic, but that's not great for getting an impact on its own.
You need to allow other people to weigh into that technology so they can actually use it. Then there are the usual things like conference presentations which then hopefully leads to invited talks and being in a leadership position, whether it's leading a research group, or leading something more internationally, in terms of some of Adam's work in the AES and the noise control, but actually, there's a lot of work that goes alongside actually doing the research that enables impact and some of the impact that we're going to talk to you about today. So the first subject is actually some of the work I've been doing.
As Adam mentioned I've been looking at immersive audio now for over 20 years. I started and completed my PhD here at Derby as it happens. I started that in 1999, so I've been here a long time, and the goal really of this research is to enable and improve the possibility of transparent immersive audio both for they are here and the you are there situation so really what I'm trying to do is fool people into thinking there is sound coming from places that there is not sound coming from. The image behind me that you can see is our research surround lab at the University of Derby, a little bit nicer than it is normally for a photoshoot, but nonetheless, this is one of the labs we work on, and the idea is, so for people using headphones, you want to make it if they just got headphones on, an augmented reality site situation where you want to make it sound like there's something in the room that they're currently in but isn't really there.
So some of the work we're currently doing is trying to fool people into thinking that they can almost touch that particular audio object because we've got head tracking whether that's using a webcam or using some other gyroscopes and whatnot so that's the augmented reality although they are here and then the other way is the you are there. So, if you think about virtual reality, where you're putting a whole headset and you take out the vision element and you replace the vision with something else, you're trying to transport the listener into somewhere where they're currently not, and again, audio is actually a really good way of fooling people into thinking things that are happening that aren't really there.
It's well known actually in the computer gaming industry that really good audio will make the graphics look better. Audio never gets the praise, it's always oh the graphics were amazing - yeah - we just changed the audio, but there we go. So some of the research that went into this subject over the years has been novel 3D reverberation simulation techniques and software, so basically, I've created one of the first three-dimensional reverberation algorithms or bit of software that was available for people to use. There was a lot of debate at the time as to how best to do it and I got so fed up with people arguing about it, I just made one and then just let people use it. Actually, that was some of the things that enabled some of the early practitioners at YouTube when they were doing 360 videos, that I'll talk about in a moment, to actually create mixes for new formats.
In my PhD I documented some of the first published methods for deriving psychoacoustically optimized surround decoders for regular speaker arrays which, in layman's terms, is basically just trying to make the surround audio that comes out of speakers, no matter where they are, sound better. That's essentially what it's doing. The psychoacoustics is, we need to know something about the system we're trying to fool, which is the ear brain system. It's not just the signals received by the air, it's actually how we process it and there's a lot of psychology in there as well, in the psychoacoustics, and trying to work out how we can fool that mechanism.
There's some shortcuts we can take and some we can't. Then some of my later work that we're looking at now, and actually this is what the little animated gif is on the screen, is looking at what called high order approximation of head-related transfer functions and their use in head-tracked oralization so, if you know anything about virtual reality, then the headsets or actually just your phone, can track the orientation of the user and we can do things to alter the audio to make the audio track with the user and there are ways of doing it that are sort of computationally expensive and there are ways that are doing it that are much more efficient and it's actually looking at that trade-off: how can we get something really efficient to work on a mobile phone or something similar, but still give a good indication that there is an audio source in front of you that isn't really there, so the augmented reality type thing. So that's the research in a very short nutshell.
In terms of applications of the impact, well YouTube actually started the ball rolling really in 2016 when they enabled immersive audio onto their 360 videos. 360 videos is basically a video you can watch on YouYube and you can navigate to look at any direction. The videos happened first. They then enabled audio that happened to be based on the technique that I was actually carrying out researching to as well, so I already had tools that I'd let students use and other people around the world use and tutorials to go with it, that enabled early adopters of that technology, to actually produce videos for that format.
So one of the first and second videos ever produced for YouTube in that format were using some of the software that I created a few years earlier because there just wasn't anything out there at the time because YouTube released it without the companies having time to catch up I also reverse engineered what YouTube did because I knew what they were doing in their particular algorithm and managed to improve it, because actually initially it didn't sound that great. So the little picture on the top right there is an inverse filter that I created to basically fix the sound in YouTube 360 videos and a lot of early adopters used that to actually make their videos sound much better and again these are just things that we released on the web and the research circles got talking to people, finding the practitioners, getting into a dialogue of what they needed and looking at what the problems were, to try and get that impact going basically.
Then VLC Media Player - so once 360 videos were enabled, they're mostly designed for headphone usage actually, so for personal listening, and VLC Media Player allowed those videos to be played over loudspeakers. It's the only media player that will do it and they needed some numbers that would allow them to optimize this particular set up for a standard 5.1 or a 7.1 home theatre array. They came to me and said, look we need some numbers, we know you can make us these numbers, can you give us some, to which I obviously said yes as VLC has got over three billion downloads. I thought that probably would be quite a good impact. So since version three, they've got some of my numbers, and you can see on the bottom left there me having a credit in the source code for VLC that enables listeners to actually listen to those 360 videos over at loudspeakers, which wasn't possible and still isn't possible, using their any other bit of software for consumers actually. So that, in a nutshell, is some of the stuff I've been doing with immersive audio.
Adam: So the second theme of impact that bridges Bruce's and my research is live event listening experiences, which is sort of sad to talk about now, just because we're without live events, and probably will be for the next at least half year. The general goal from all the research that kind of pours into this, is not necessarily to achieve the same sound level, at every seat in the house, because that is physically very difficult to do, but at least achieve the same tonality, so it sounds the same, the musicality is the same, as well as achieving the same sonic image. So, if you want the sound to be sounding like coming from just off left it is, in every seat of the house, which again feeds a lot into Bruce's research.
The research, Bruce already mentioned his ambisonic system, and that is used at live events as well, I think I'll mention it at some point in the next slide, but my research tends to focus on maybe less the spatial audio aspect and more just if you want to call it, the brute force of live sound. You know the thing that's exciting about live sound is the impact. When the drummer hits one of their drums, you physically shake, you feel it. I tend to specialize in the low frequency or, the bass aspect of sound systems, so I've done quite a bit of work on subwoofer system analysis and optimization to the point where, over the Summers when festivals are running, I get quite a few emails from system designers saying, "here are the subwoofers we have available, tell us what to do". I send them a few numbers, tell them what to do, they implement it and everyone seems to be happy.
Then dynamic diffused signal processing, I'm not even going to begin to try to explain what it is. In a nutshell, it's if you have multiple speakers outputting the same audio, they interfere with each other and this is a signal processing approach, to avoiding that interference issue, making everything sound the same everywhere in the audience. So applications: I think we have some fairly exciting applications for this stream of research, I'll let Bruce talk a little bit on his exciting work at Glastonbury. For me, I've worked for over 10 years now at Pitchfork Music Festival in Chicago, which draws about 60 000 people annually.
I've gained their trust, the people who put on Pitchfork, to the point where the company I work for in Chicago, designs and operates the sound systems on their two main stages. You'll see a photo from the festival, from a few years back on that, the top photo. They trust me now enough, to the point where they let me do experiments during the festival, and they know I'm not going to completely destroy everything, or make it sound horrible. So the past maybe four or five years, I go there every Summer with my research equipment and I'm doing experiments, there and then, as the festival's happening and I think there are very few opportunities in the world to do that. I mean it's about building trust and that all feeds back into the research at Derby, while at the same time hopefully giving those tens of thousands of people, a great experience listening to live music.
One of the last festivals that I was involved with, was the Ashley for the Arts Festival that was in the state of Wisconsin, so just north of Chicago, last summer where I didn't set foot on that site. The chief engineer of the sound company that was providing for that festival just got in touch, said here are our system specs, this is what we have, make it work. I ran some simulations on my software, sent him over the information and he came back saying, I don't know what you did but the engineers were absolutely raving about this. He said it was the best sound they had with the low frequencies, they were really happy.
So there's a lot of examples of things like that. We also did some work with the Prodigy. My relationship with them goes back about five years. I'm known for getting the base quality at events sounding really good. I got to talking to John Burton, who is their long time front of house engineer, who consequently now works at the University of Derby. He retired from the road and he joined our academic team since he had done an MRes at York. He officially brought me on for the Notorious tour, which is the last tour that the Prodigy did, before tragically their lead singer lost his life. I helped John design the sound system with that and really achieve the best impact you can with the sound system, which was very successful.
Aside from that, I've worked with companies like Alto Aviation, they do private and corporate aircrafts, so not a lot of events, but I'm using the same technology, with a bit of machine learning, to make it sound great in every seat of these private aircrafts for the rich and famous. Then lastly, one of the most recent bits of impact was on diffuse signal processing research. I had a PhD student who worked on this and we managed to get the algorithm to the point where it was ready to put into place in a real venue. So it is actually up and running at the National Library of New Zealand, it's halfway around the world, making their auditorium for speech and small music events actually sound quite good, at least from the feedback that we've received. So you know kind of that's my bit for the live sound. Bruce, do you want to say anything about Glastonbury?
Bruce: I can do yeah. Function One is a company that runs stages at Glastonbury and actually used to do all the sound at an event called Glade Festival. We trialled our immersive sound over six very large speakers at Glastonbury and found that at Glade Festival initially, that the sound was the best quality on-site, as voted by the punters, it was the loudest on site but the quietest off-site and so actually since then, they've been using some of my software at the Glastonbury Festival every year from that point, which was about 2008 say, until last year, which was the last year that Glastonbury's run, because it didn't run this year and again that's a good example of how some of our research crosses over in terms of this case study.
Adam: Okay so we're nearing the end here I know, I'm conscious of time, so I won't spend too long in this. The third and final aspect of our research that we do is health aspects of sound exposure and this is something that I really, to be honest, didn't care that much about until maybe three or four years ago. I was quite happy to build these sound systems that were really super loud and thought it was great but I did eventually get pulled into this initiative with the World Health Organization, looking into making listening safe; it's their Making Listening Safe initiative and actually trying to protect audiences going to all sorts of different venues around the world, to make sure they can enjoy themselves, without absolutely blowing out their hearing.
So a lot of the research has gone into making sure audiences are safe and to give you an idea of what we're dealing with, the front row of an audience at a music festival, based on my measurements, is receiving at peak about 140 db of sound energy, which the only published work I could find on it, was from NASA, from the studies they did in the Apollo missions when they were strapping people to rockets and sending them into space. So that's about the same sound pressure level if you're strapped to an old Apollo rocket going into space, as you are in the front row of the music festival. So a few alarm bells went off let's say. So we've looked into that.
We've also looked into festivals happening near people's dwellings, where people live, and we're looking at ways to really best practice to minimize annoyance in the community so we can coexist having these big events and having people living there and everyone's happy, easier said than done. Then lastly, just through a contact I have in the industry, who's very involved with speech intelligibility, we've started to pick apart the methods for quantifying speech intelligibility, which is really important in public address systems. Think about if you're on the tube in London, if there was a fire in there, you need to know that you have to get out of there and there have been big disasters in the past where that hasn't been able to happen. Speech intelligibility is something that's required by law to be at a certain good enough level so that people can hear emergency announcements if there's a reason for them to get out of somewhere because there's a problem.
So with the AES, the Audio Engineering Society, I think I effectively ended up doing another PhD over the past few years, where it was a massive literature review and a bit of original research, just looking across the world, seeing how we're managing sound exposure on-site and noise pollution offsite, at big outdoor events and highlighting best practice. So this was done with about 14 other people around the world, so some academics, some members of industry and it ended up being about 150 page technical document and it's the first time to my knowledge that noise regulations around the world have been actually compiled into a single document, and it's been quite a good reference document for members of industry, so that was released in May of this year.
Then a couple of other again standards that have come out based on the research, relating specifically to speech intelligibility, have been one through the IEC and one through British Standards. The British Standards one was a code of practice for designing and specifying sound systems in public buildings venues. So in some of that research, we found some flaws in the way speech intelligibility is quantified, fairly major flaws actually, so that's made it into the standards basically saying, look here's a problem, if you encounter these situations this is what you need to do and similarly, with the IEC standard, we're one of only I think five papers referenced in there. I should say that's one of our undergrad students who was the lead author on that conference paper that's referenced at an international standard which I'm very proud of. He stayed on to do a master's and now a PhD with us and he's doing very well.
Then just lastly, it's this make Listening Safe Initiative with the WHO: I think that really has the chance for the most impact out of anything with this research. I was in Geneva this past February where it's round table discussion, all sorts of different stakeholders with live events and mobile listening devices, where we're really trying to create an international regulatory framework to make things safe. Any way you listen for entertainment, make it safe. That's hopefully coming out next year and that will be international in scope. So that's us - and thanks for listening.