A Composer In His Natural Habitat
A Composer In His Natural Habitat
Habitat #003 - Mean Beats - A Sound Effects and Interactive Music Adventure with Jon Clark
Experienced video game sound designer and educator Jon Clark joins the show to discuss interactive audio and music for video games and other media.
- Join the ongoing discussion about this episode on our Discord channel: https://discord.gg/8F7Yuc9Mne
- Tex Murphy Series on Steam: https://store.steampowered.com/sub/45894
- Tex Murphy The Pandora Directive (1996) Trailer - https://www.youtube.com/watch?v=xGDg_acw6Uc
- Pathstorm App: https://www.bigfishgames.com/games/1562/pathstorm/
- Amped (snowboarding game) game play: https://www.youtube.com/watch?v=LgW4BJRHNKE
- Amped 3 game play: https://www.youtube.com/watch?v=aP29qq9dQeI
- Amped 3 cut scenes: https://www.youtube.com/watch?v=3S1OpmHwO4M
- Utah Wind Symphony Performance of "Firefly" by Ryan George: https://www.youtube.com/watch?v=8ndYH5hmJnE
- GameSoundCon (Annual game sound conference): https://www.gamesoundcon.com/
- Game Audio Network Guild (G.A.N.G.): https://www.audiogang.org/
- Music Composition for Games and Interactive Media is MUSC 2510 in the SLCC catalog. Game Audio Design is MUSC 2570.
Welcome to a composer in his Natural Habitat, a podcast about creativity and music featuring Thomas C. Baggaley. Habitat number three, Mean Beats, a sound effects and interactive music adventure. Hello, everyone. I'm Thomas Baggaley. And with me today as today's guest cohost is my good friend and colleague, Jon Clark. Jon, why don't you tell us a little bit about yourself and your background?
Jon:Oh, Jeez. I am ... ancient. [Laughter] I started working in video games in 1989. Prior to that, I was a technical director for a couple of theaters, fairly large theaters in different parts of the country. So my background, my college degree, all that stuff is theatrically based, which, in a way, as I was reflecting on it this morning, kind of set me up to do video game sound. I actually started working in video games as a visual artist. There wasn't really much sound in games in 1989. Number one, we weren't doing console games. We were doing computer games. So when I was hired, the company was doing Commodore 64 games Commodore was a big market. They were doing pretty well. They hired me. And that Christmas, while I was packing up in West Texas to move to Salt Lake, the Commodore market collapsed. The games didn't sell. Everything got shipped back to the company. They had no money. It was really ugly. And it was really scary because when I arrived all ready to go to work, they said, "We don't know what we're going to do. Our market is gone." They made the decision to go ahead and do games for Intel based computers. Back then, the computers were running 8086 and 8088 processors. But, yeah, things were pretty primitive when I look back at it. And I was actually contacted by and hired by a friend from college, which I always tell my students get to know the people around you because they're your network. These are the people that you're going to work with down the road. They're going to be the ones that call you and ask for help. Yeah, it'd been quite a long time and just kind of out of the blue he called me and asked if I was interested in coming to work for him. So anyway, we started doing PC games, and we were working on a golf game called Links. We had a guy in the office who was creating this game engine that could recreate the terrain. It was a three dimensional modeling program. So we could kind of build the terrain of the course, and then the ball would react and roll down hills and, you know, it would bounce based on how the train was. It was pretty groundbreaking, literally groundbreaking material. And they needed some sound for it. So we wanted to put in some little sounds. Another engineer in the office came up with something called real sound, and his thing was to make the little PC speaker that would beep when you turned the computer on ... He figured out a way you could control that and modulate it so you could play sounds out of it. Little bits of sound. So we decided we wanted some stuff to play on that ... things like golf ball, hit the golf ball or ball rolls into the cup. So, because I'd done sound in the theater, I was the sound guy. It was an editor. There was no Protools. There was no Cubase. So, the programmer also ... who did all the terrain stuff, also wrote a sound editor. It was a six bit editor, and we could record the sound, and I could go in and with primitive controls, I could cut it. But I could see the waveforms, which I had never seen. I had cut tape all my life, where you rock the tape reels back and forth and you'd have to find the sound. And now I could see this wave form, and I could see the spaces between things and, boy, I made editing so easy, and I never want to go back after that. I never have really. Ultimately, because of my background in sound, for a long time I did graphics and sound. It was a small business. I think I was employee number twelve or 13 in this company. So everybody did anything they could to get the products out. And so it took a long time, and I did more and more sound as sound cards became more common. and we needed more sounds to fill in the gaps. And we hired more people. Visual artists were easier to hire than sound people, I guess. It's probably still the case. So I just kind of became the defacto sound guy. The company was called Access Software, and we were in business until 1999. I believe it was late 99 Microsoft Game Studios bought us. We were with them for about five years. Then, they sold our game unit off to Take Two Interactive, which is another big game-producing conglomerate. We had a pretty broad background in sports, so we worked on games like tennis. We did snowboarding, which was also one of my products ... a product called Amped. An adventure game on snowboards, I guess. That was interesting. Yeah. Anyway, that lasted until 2006. And for whatever reason, in April 2006, Take Two decided to close a number of their studios. I'm kind of out floating around and just happened to bump into a guy I hired as a voice actor. I told him I was out of work, and he said, "Gee, it's too bad you don't have a degree in theater." And I said, "I do." "I have an opening at Salt Lake Community College." I applied and got the job, so I've been there for 15 years. Anyway, probably too much information, but it's kind of the background.
Thomas:Well, you have a wealth of experience, and I'm so glad that you were willing to come on to the podcast and help out. Kind of a tradition where I have to explain the title that's happened already for the first two episodes, So let me explain the title of today's episode. It's "Mean Beats" and that comes from one of your first games that you worked on from the Tech Murphy series from 1989. And it was called Mean Streets and that's actually a game that I played. I remember playing back then, so there we go.
Jon:I could hum you the theme tune from it. Actually, it's funny when we were putting that game together, I said we need music because the character was kind of this film noir fetective kind of thing, and I said,"Let's do some music that's kind of a reminiscent ... I love Perry Mason. That triplet figure in the theme. And I said,"Let's do something like that." And I don't remember who actually composed the music for that, but that's where it came from. So it has that triplet thing kind of reminiscent of Perry Mason's music.
Thomas:Thank you, Jon. And I look forward to seeing what you have to offer here as we talk about interactive audio and opportunities that are there for composers and other creative people working in interactive audio. What are you working on? Alright, Jon. So what kinds of things are you working on lately?
Jon:I'm a teacher. At this point, that pretty much fills up my life. I don't ... I don't have any freelance game work in the pipeline right now. I've worked on a few games since 2006. Not ... not a great many. Right after I started teaching, I worked on an interesting game where you would drop a marble into a maze, but you couldn't see the maze, and you had to figure out where the marble was. I think the game is called Path Storm. I think you can still get it. And that was a fun project because the idea was that the sound is what would tell you how the marble bounced around inside this maze. So if you dropped it in at the top, it would go down, and if it bounce in one direction, you'd get one sound. If it bounced in another direction, you get another sound. If it spun around, you'd get another sound. So I had to come up with a sound scheme that explained what was happening pretty clearly. That was a fun game. I still like the game. I enjoyed playing it, which is unusual. I'm not a game player. Anyway...
Thomas:That is interesting, right? I mean, a lot of times when you work in this, you spend so much time testing and working on stuff that when you go home, you don't want to play games.
Jon:Yeah, I ... At Microsoft, when we were working on Project Gotham Racing. I was testing that. I was kind of between projects of my own. And so I became a a beta tester for Project Gotham Racing, and I would spend hours every day driving these race cars around cities. And you learn things to win the race, like doing a pit maneuver on somebody to get past them because you just can't get past them. And I'd get on the freeway to go home in the afternoon, and I had to really fight the urge to do a pit maneuver on somebody to get past them. [Laughter] Yeah. It's kind of overwhelming. Yeah. I'm not inclined to play games. For me, the game has always been creating content ... figuring out how all this stuff's going to fit together and solving the problems of making the game. So I never really got into actually playing the games. Plus, you've got to realize when I started doing it, it was early enough in the game world that I didn't have a computer. Most of the people I knew didn't have computers, so I just didn't really have access to that to get that addiction. I worked on a couple of casual games for some of the people that I'd worked with at Access. I can't even remember the names of the games now, but they were fun because they were adventure games, and we recorded all the audio and I created basically these radio plays of all of the scenes in the game. Then they added the animation over the top of that. So we got all the sound edited and built, and I was really happy with it. I think it plays really well as a radio play.
Thomas:Cool.
Jon:We also did a follow up to Tex Murphy as a radio play. I think it was probably the early two thousands just after Microsoft acquired the company, and we had left people hanging with the Tex Murphy franchise. I think we had three games, and at the end of the last game, the hero and the heroine get shot with darts and then we don't know what happens ...
Thomas:Oh no! The eternal cliffhanger.
Jon:Yeah. So it was kind of strange. So anyway, we did a radio play to kind of pick up where we had left people hanging and ... and took them through. And that was fun. Yeah, the casual games, probably seven or eight years ago. I've helped them out with a couple of other things, just small pieces of games in the interim. But I haven't really focused on that part of my career. I've kind of left that behind and been teaching full time.
Thomas:And of course, there's a lot of creativity in teaching too, especially at Salt Lake Community College where you and I both were hired at the same time to create a program that didn't exist. And so how do we go about doing this? How do we teach these students the skills and how do we set up a program and design the classes? And so there's a lot of creative activity that goes into that ... figuring out the best way to help them. Alright. Well, on my part, this past little while I've been actually contacting publishers. As the listeners know, I've been writing a textbook on music composition, and the last little while I've been contacting some publishers and some agents trying to figure out how to get in. It's brought up the interesting question at this point. A lot of the music that I've done in the past, I've actually self-published in one way or another. I've either been hired for a specific project or I've written the music, and I've actually just kind of put it out there for self publishing, but with the textbook and in general, it's one of those decisions that you kind of have to face. Do you self publish? That means you have to do all the promotion and all the legwork yourself. A lot of people worry about ... you know ... Do I want to control the copyright? So ... That's only one piece of the puzzle. One of the reasons that you go to a publisher, whether it's with a book or with music. If you go through a label and part of the reason you often sign over rights is because they're going to do a whole lot of work for you, hopefully. And they have connections, and they're much more efficient at getting things out to the public and getting people to notice your product than you can be individually. And so I've been exploring that, talking to talked to a lot of publishers. It's really interesting the experience that I've been having, because as you do that, if they respond to you quickly, it's usually bad news. Usually they're saying something like we love this idea that you have, but it really doesn't quite fit with what we do, and they're really nice about it. And you don't burn any bridges or getting mad. Just say, okay. Thank you. I appreciate your quick response, and then you move on to the next person. But if it's good news or has the potential to be good news, you send it off and then you wait. It's a waiting process because they'll actually ... "Okay, This is interesting, but we're not sure, so we'll talk about it." And they'll think about it. So I've got a number of agents and publishing companies that I've reached out to, and you just give a short little spiel. You don't send everything to them. You don't want to overload them. But a few of them, I haven't heard back from yet. And you wonder ... Okay, what does that mean? Does it mean that they're actually thinking about it, or they just haven't even read your email yet. You don't know. And so you just keep working.
Jon:My wife is a writer, and we live with that around here. She's always got a bunch of stuff out waiting for publishers to get back. Yea, it's a game.
Thomas:Yep, it certainly is. But you keep writing and keep working and you just keep projects going. I've also been working, of course, on my sabbatical project, which is to write a musical or opera something, a music drama of some kind of how I put it in the documentation and on that, it's also a waiting game because I want to write something for a story that people know already. That's what I've been trying to do. And so that means getting the rights to those stories. And I've had contact with a number of ... of agents for authors, because you go through their agents. You find out who their agent is and you go through them. And I've had some good interactions. Nothing turned out great yet. I did contact one very well known author who has a lot of books out, and I was interested in a specific book that's not actually been optioned, and they actually had some interest. But it turns out that it's part of a larger world. And in order to even do a project in this world, even though other parts of the world are the parts that have been optioned out to film companies, you still have to get permission from the film company, and their response was, well, if you write something that we really, really like ... which I might be able to do that, but you never know. Then we'll go to bat for you. And I was like, for this project, maybe I need something a little bit more secure than that, but it's something I definitely want to do in the future when I can take a little bit of a risk because it's a great story that I'd like to do, but that's all part of the game. You just keep working at it.
Jon:I should say I haven't completely given up on doing sound. I'm in my home studio in the backyard of my house, and I do still ... I record music. I record ... until COVID shut everything down, I was the engineer for a group called the Utah Wind Symphony, which is a semi professional wind band. An excellent wind band. If you do a web search for them on on the Internet You'll find a couple of pieces that have been posted. We'd just started talking about what are we going to ... We need to release an album. We need to put a bunch of stuff together. We have a large body of recorded work. We just haven't published it yet, and I hope to get that published and available soon, because it's really excellent. I love band music. I mean, orchestra is great. I have nothing against an orchestra, but the power of a large wind ensemble doing something like Holst's "Planets," or there are a lot of symphonic pieces that have been transcribed for wind band that are just incredible. I love that ... the ener ... the power coming off the stage. To me, it's just ... nothing better. So I've really enjoyed doing that. I've done that now f ... well, pretty much for the whole time that I've been teaching. I've also been doing these recordings for them and for another organization that I'm not associated with any more. But that's kind of where I'm at now. I really enjoy doing that. Probably when I retire from teaching, I'll throw my name out there and try and pick up work recording high schools and things. It's just the location recording is an interesting challenge and getting a good recording. Anyway, I'll give you a link if you want to link to one of their pieces, and ...
Thomas:You bet, I'll put that on the show notes so people can listen to it.
Jon:I'm pretty happy with the results we get from these, and it's a good band.
Thomas:Questions! Alright. For today's questions, we have a few questions actually. We're getting some good listener participation. I really appreciate you sending in your questions. If you have any questions for the show, please get onto the Discord channel where we have our discussions. The link's in the show notes for that as well. Or you can just send me an email at thomas@baggaley B-A-G-G-A-L-E-Y music ... baggaleymusic.com. And we'll take a look at those as the show goes on. So first of all, the first question we have is from Robert, and Robert asks, "What has interactive audio taught us about the importance of silence?" And I think ... he goes on to talk specifically about with regard to our health. I'm not an expert on that myself, but I did find a web page that has some pretty interesting information. I'll put the link to that in the show notes, but I think I can talk a little bit about ... and John, you certainly can, of course ... about the importance of using silence as part of the composer's arsenal ... as part of the sound field in games. I know that when you started out with games, you think about games like Super Mario Brothers, where it just kind of had constant looping music, and yet, that's kind of a nostalgic point for a lot of us. but now a lot of the games you have, sometimes you're going to get very ambient, very quiet places within the game. And I think a big part of that is that they discovered if you're sitting there playing for three or four hours. After a while, you kind of need a break on the audio side.
Jon:Yeah, we figured that out pretty early on when we started doing the Tex Murphy adventure games, we had these music themes. We actually had a theme for every location. So every place you went had a different character to it. Yeah, we found out that if you're in this location, it was a search and find objects and put them in your inventory and use them later kind of thing. And you might spend an hour or two in a location, and it just became so overwhelming and annoying. The music was just so annoying that we said, you know, let's play the music for so long and then let's give it a break for a while, and then we'll play the music again, and then we'll give it a break, and then we'll play the music and we'll give it a break. So yeah, there's always too much of a good thing, right?
Thomas:Right.
Jon:It's like eating a whole bag of cookies. It just sounds like a great idea at the time, and then after the fact, maybe not. [Laughter] That's an interesting question. What has it taught us about silence? That's probably the main thing is that ... you need the contrast. Certainly in a game, you're going to spend hours and hours. Our games we were always looking for ... with the adventure games ... about a 16 to 20 hours experience was the ideal. And so you just can't have music playing and big sound for 20 hours. It doesn't work. And the same with movies. I mean, if you go to a movie, they are very good at using that contrast between loud and soft and silent and bombastic. And ... I don't know that I have an answer for that question. Beyond that, that's something I've never really thought that much about, beyond the fact that you really need the contrast. You should never be afraid of using silence. If it's the thing that needs to be there, then it needs to be what's used.
Thomas:And I think it's important to think about silence as part of the sound design. Just because there isn't any sound doesn't mean that you aren't doing it on purpose.
Jon:And there is no silence, really. We always put something in there, even if it's just the sound of air conditioning or the sound of ... of machinery or computers in the background. Maybe quiet sounds. But I can't imagine using absolute silence, because that would take you out of the experience. You've got to have something that ties you to what's happening on the screen. It has to be something on the screen, and the same, again, with movies. It's always some sort of an ambient background, even in the quietest parts.
Thomas:Good. Let's move onto the next question, then. Cassandra asks "How much thought and intention actually goes into video game music?" I guess that depends from getting to game, but certainly from my perspective, there's a lot of opportunity, at least to be very intentional. You'd think that with games, you don't know when something is going to happen necessarily, but because of the tools that are available to you ... You have all these game variables. You can design the sound in such a way that it reacts to what's happening. And so there's a lot of opportunity for you to be very purposeful about what's going to happen, music-wise and audio wise in a game. It's not just, "Let's put some music on here to create a mood."
Jon:Yeah. As far as how much thought goes into that, that is a huge thing, I mean, you'll get a piece of music back from a composer. I'm not a composer. I'm a sound designer. I've worked as a sound lead where I was in charge of the overall soundscape of the game, but we, with rare exception, would contract the music out to a composer who would provide finished tracks to us, whether it was MIDI content or scored orchestral clips or whatever it was. That stuff would come back and it was scrutinized and considered and everybody sits around ... That's good. That doesn't work here ... and send it back and have them rework it. Everything in the game is very thoroughly thought out and thoroughly considered. The visuals are scrutinized frame by frame. If there's an animation sequence, they'll sit and they'll go through frame by frame. And if something's out of place, they'll go back and re-render the whole thing because you just never know who's going to notice those things that just don't fit. So the music is very carefully considered. It's part of the game design as far as what kind of music. Sometimes we use temp tracks like they do in movies where you find a piece of music you like, that fits what you're looking for, and you'll kind of place that over the visuals, or you'll send that to the composer and say, here's the kind of stuff we want here. Now, it doesn't mean the composer is going to just turn the melody upside down and give it back to you. It's like here's, here's the character. Now you got a figure out how to make it work, how to create something that has that kind of character, but it isn't that.
Thomas:So we've got one more question here. Amanda asks, "Does the kind of the game ... if it's like an RPG or scrolling ... change how difficult it is to do the audio and the music for a game?"
Jon:Probably. And I would say that only because in my mind, a scroller isn't going to have as many options, isn't going to have as many possible turning points or as many possible surprises for the player. I mean, if things are moving back and forth across the screen, you have a limited field of possibilities and it's all on the screen. It's either what's off to the left or the right side of the screen. You can always stay ahead of it. But in first person experience, you don't know which way the person is going to turn. You don't know which way they're going to go, you don't know which door they're going to go through or which hill they're going to snowboard down, or ... There's a lot of work that goes into any of those things. But I think the first person experience or something where you have a pretty wide open world requires more consideration and more design work.
Thomas:You can compare that, for instance, to some of the social games out there, like where they're matching three ... where they're in the same thing quite repetitively, and it's a very limited kind of activity. Then it's a little bit easier to know what the player is going to do, as opposed to an open world like you mentioned, where they're just exploring.
Jon:The marble game I mentioned before where you drop the marbles in has a sound set of probably 50 or 60 sounds in the whole game just for the gameplay. And then there were some additional sounds for interface stuff, which I don't think people really think about, but every time you click on something, you need a sound that tells you that the game knows you clicked. Right? You need a response back from the game. So there's a lot of sound that goes with the interface and interaction as well. But the marble game had a very small sound set, and yet I would finish sounds, and I'd send them off to the game designers, and they play the game with those sounds, and they'd come back and they say, "These work, this one doesn't work. This one needs to be changed." And so even that small game took me four months, I think, to finally get it all worked out with them. Not that I was working on it every hour of every day, but it took time.
Thomas:Yeah, there's a lot of thought and time that goes into it. Well, we've been talking about it all along, but let's go ahead and go on with ... the theme of the day. It's time for the theme of the day. So the theme of the day, of course, as we've been talking about is interactive audio. One of the things that I think we can talk about right here. Let's talk about what's different about interactive audio. How does it change the workflow of the project? Now, one of the things that I thought about, for instance, if I'm writing music for a film, a lot of times there's been a lot of work that's gone into the film before they ever get to me. It's like the last thing. Some directors like to send you a script or something like that. And it's nice. You read the script and get some ideas, and then you get to the final edit and you throw out all the ideas you had before because none of them work with video once you see it. But I know that's not quite the same workflow for video games. A lot of times you're working on audio the whole time that they're designing the game itself.
Jon:The design process ... It was actually pretty interesting for me to find out, because I worked for this small company with 15 or 20 people working on a project. So everybody is involved with everything. But all of the design work was going on kind of behind the scenes. They would just come to me and say, "We need this sound," or we need this to happen or they give me an animation and I would come up with the sound for it. But when Microsoft picked us up, of course, they had a number of game studios, and all of those studios had kind of developed these processes for designing games. And all these people kind of had a process that they were used to working with. It was all new to me. I think they assumed that we worked the same way, I guess, because they just threw stuff at us. [Laughter] Yeah. It was an interesting eye opening experience to kind of learn how the game design process worked. But ultimately, after one or two games of struggling with that, I finally was involved with the game design process to the point where I kind of understood what we were trying to do. And that is we would all get together in a room, all of the leads and designers and work out ..."Okay, here's high level stuff. What is the game and what's the overall thing? What are we going to do? What makes this game fun? What's the game player do, what's the game action?" And that kind of is going to inform everything about the game. It's going to inform the artwork. It's going to inform the music and the sound design. So you get that high level stuff done, and then you start just kind of working through. Create the levels, create the world. And as that stuff becomes a little bit focused, you start to get kind of a blurry vision of it. You start to create the content based on that vision. So the artists go off and they start creating prototype characters and backgrounds and worlds. And the sound designers go off and we start creating sound content, and we start finding temp music, things that that are evocative of that world and come back and we all present all that stuff. The decisions are made. Yes, that works. No, that doesn't work. Let's go this direction. Let's go that direction. And everything just kind of comes more and more and more into focus over the process of designing the game. And so by the time you get to production, where everybody is, "We know what everything is going to be." You've already prototyped and built a lot of this stuff, and you have all the models for the visuals. And you have an idea of exactly what kind of sound you need and what kind of music you're going to have composed. So, then the artists go off and they start creating. The programmers go off and they start building the geometry and the interactivity. And that's probably something I should point out is ... The part of the programmers or developers, as they're called in the game world is huge and all of us artistic types, the creative people, the artists and the sound people and the composers work very closely with the programmers because you're ... the programmers are the ones who are creating the elements in the game or creating the code that's going to make these things happen. So you give them a a piece of animation or you give them a sound and they will make it happen within the game. It's been 15 years since I've worked on a large scale game, so I have have to admit, there may be some differences now to this process. It may have been streamlined, and I'm sure every studio does it a little differently, but ...
Thomas:I know with some of the engines that they created now ... Unreal and what's the other one I can't remember ...
Jon:And that made a huge difference because that was always a big part of what was happening in our studios. We were constantly ... we had a group of programmers, especially with Xbox, who were creating the game engine, all of that stuff. How does this even work? And now you're right with Unreal and Unity and these commercialized engines where you kind of plug and play in a lot of ways. That's a time saver.
Thomas:Yeah, it really is.
Jon:They are amazing. I've just been looking at the most recent Unreal demos and it just boggles my mind to think of doing that. And you got to realize I have a background in art and the visual side of this thing, too. So I've spent a lot of my time building worlds and building artwork. So when I see something like Unreal or Unity, where I can take some amazing visuals and and make them into three dimensional objects, I want to go play with that, too.
Thomas:How long does it typically take to develop one of these games? How long is the process now? At least at least, was it 15 years ago, at least when you were working on it?
Jon:I don't imagine that's changed an awful lot. I think for a large scale game like an Xbox or PlayStation game from conception to release is probably still in the range of a year and a half to two years. I know we would ... our cycle was we would shoot for a general issue for a Christmas release, which meant the thing went out the door to manufacturing somewhere in late October, so it could be on the shelves in November for Christmas, and we'd get a chance to take a break after that or Christmas, and maybe just a little into the new year. And then we would start talking about the next project in January, and the design process itself could take up to a year of talking about it, whiteboarding ideas, prototyping ideas, creating game engine elements, creating artwork, prototypes, sound, kind of putting together these little bits and pieces to see if this is even going to work. At the end of that eight months to a year, then you would say, okay, this is what we're going to do. Go into production and you'd shoot for the next Christmas. From the development side and the creation side, it means you're pretty much finished in June or July, because now you've got to get into beta testing, which is where you're just looking for all the bugs and all the problems, and everybody plays the game and tries to do anything they can to break it. We had a thing in an Amped Snowboarding, the first version of Amped, that sneak through into manufacturing where you could jump off of a cliff in a certain place and the background went away and you fell forever and your character was flailing in the air, and it would just flail until you turned the machine off. You try to find all those bugs and get them out there. Because especially with some of those early games where everything was delivered on DVD, that's a hard copy. You don't do downloadable updates to that hard copy. What's on the disk is the game. So you got to get rid of all that. I think it's better now where you get up your games online.
Thomas:It's kind of a double edged store, isn't it, with the way that they do it online? Because now there's a tendency sometimes to release games that aren't as polished and we'll fix it because we can. And then also the other thing, you and I have talked to some people in the industry, and they say they don't have down time anymore because especially with all the updates that they do and new content for games that they're just ... they release the game and ... They're not done with that game.
Jon:Yeah, if you have a successful franchise, it's going to get used and rebuilt and re-released and fixed up and new content and all that stuff. And that's probably worth mentioning about the industry in general. It's intense. The expectation is that you will work more than 8 hours a day. I don't know anybody in that business that works eight hour days, and often you work six day weeks when you get down to the crunch. There have been complaints in some companies about the crunch becoming the norm where you're always working six days, 14 hours a day, which is just not healthy. So hopefully that's changing. I hope it will change. As we're recording this, The IATSE stage hands union has authorized strike against the Producers Guild because they're working crazy hours or creating media for people who are making billions of dollars. And you know, here are people who are still making less than $20 an hour in that industry. I feel pretty strongly about the fact that they really need to have some life in your work life balance. Our society, I think, leans heavily toward work, and not so much toward life, and I hope we can make changes to that in some way.
Thomas:I agree. So what are the different kinds of audio assets that a sound designer needs to develop for a game? You mentioned some, like some feedback kinds of things. If you're going to give some different kinds or classes of audio assets to go into a game, what would those be?
Jon:You're going to have the interface that I mentioned before. You'll have the gameplay. So let's say it's the first person ... we'll use snowboarding instead of a first person shooter. Everybody uses shooters and we'll use snowboarding. So I need ... For snowboards, I need the sound of the snowboard, and I need the snowboard to interact in some way. So as you cut into the snow, the sound needs to change a little bit. Different surfaces of snow are going to require different sounds. If you hit a tree, that's going to sound like one thing. If you hit a building, that's going to sound different. What does it sound like when you go off a jump or when you're flying through the air? Or if you land on a ... say you jib on a pipe or a piece of wood or a cable as ... a ski lift cable, which is one of the things we had in our game. The sounds ... interaction with whatever objects the game designers decide are going to be in the world. So if there's ... Whatever object you need to interact with, there's probably going to be a sound for that. If there's walking involved, you'll have to have footsteps on whatever surfaces are involved. So that becomes a very large group of sounds, and all of those things have to be able to be queued up and played within just a few milliseconds so that they appear to be happening in real time. The next class of sounds would be probably the cutscene. Cutscenes are there to move the action forward, progress the story, and those are going to be linear, which is kind of a pleasant relief from doing all the non-linear stuff. So you have all those ... Those who just take the clip and you cut the sound effects to it. You cut everything into it
Thomas:And the music as well can be synchronized.
Jon:Yeah. So that's all synchronized, and it's the same every time, so that you just create a soundtrack for that. And then, of course, the music. And in the music you're going to have the music for the cut scenes and you'll have interactive music which plays during gameplay. And all of that has to be figured out. And there are a number of different ways that you could probably talk about for changing the music during the course of gameplay to reinforce a level of intensity or the level of action or the location or the whatever. So that becomes a pretty large thing too. If I had to classify them, those are kind of the things. It would be the gameplay, the interaction with the interface and the storytelling elements.
Thomas:I know that you've mentioned before as we've talked about this, that there are often thousands of audio files that can be involved in a game, and they all have to be organized so that they can be accessed really quickly and you know where they are. And if you make a change, you know where to go, make the change. How do you stay organized with all this?
Jon:During the design process, hopefully, you come up with naming schemes for things. You come up with a file system that everyone agrees on. So scene numbering, line numbering, action numbering, character names. All kinds of things go into this. So by the time you get into production and you're just producing content you've already got lists that you're just filling. So we might have a spreadsheet with all of the voice recordings that you need.
Thomas:Right.
Jon:And then as you recorded it and edit it and save it, you just save it with that file name. Then the programmers can just pop it into the game. I worked on one product, a baseball game where we had a script. It was probably 200 pages of script for commentators. And before we recorded the commentators ... It was the Fox Sports commentators. Before we brought them into the studio, and spent all that money, I sat and read all of their lines and temped those in using the file names. And then the programmers plugged all that in. The testers could play it and make sure that that stuff worked. And then after that was all done, we sent a guy to New York and recorded all of this commentary from the scripts, and it came back to me. And then I just went through and replaced my stuff with the actual stuff. You come up with all this stuff during design and the lists just grow naturally. And as long as you just keep track of all this stuff and make ... and have good spreadsheets, you just start filling in the blanks and once all the blanks are full, you're done.
Thomas:I've seen your spreadsheets and each step of the process along the way and there's a blank for it and you can check off and say, "Okay, I've done this now for that audio file and so forth."
Jon:And that includes things like localization too. It's an interesting thing that I had never really thought of before I started doing games, and suddenly we've got to localize it. They want to do this in German, which meant but I had to go through and take out all of the dialogue, but leave all the sound effects in. And I hadn't intended to do that. So everything was in one piece. I had to go back to all my original projects and try and remove dialogue tracks. It was difficult. So ever since that experience, I always assume that there's going to be localization. I always maintain separation between dialogue and everything else. Whether I'm working on a game or working on a film ... I worked on a number of small films, and I always ... even though none of those films have been localized, I always maintain that option because it's standard practice in the industry and I don't ever want to have to go back through and try and remove dialogue and stuff again. That was a nightmare.
Thomas:[Laughter] So what's really fun in all this? Where do you find your creative uses flowing and having a lot of fun when you're doing all this stuff? In figuring out how things are going to interact ... how the sounds are going to interact with each other. Right? So if I got ... let's go back to my snowboarding game. You got the snowboard and it's running down the hill on groomed snow. It's going to sound one way straight down. If I turn to one side, the snowboard is going to cut into the snow and it's going to sound a little different. You're going to hear that edge cutting in. Well, I need some way to blend ... to transition from this sound to this sound. And what's that? How am I going to do that? And then back again? And then I go from groomed snow to powder snow, which sounds totally different. And there are hundreds of those changes and those transitions that are going to happen in a game. I really enjoy that challenge. And then the other thing I really enjoy is recording the sound, finding things that sound right. It's just like movies. The real thing doesn't always sound right for what you're doing. You're kind of creating something that's bigger than reality. I can't think of a game I've worked on where we didn't want things to be bigger than real. In an adventure game, everything should be bigger. Gunshots are bigger. The characters are bigger. Music's bigger. Yeah, with reality, you take the original sound and you ignore a bunch of stuff that's not important and then the important stuff you make bigger and less real so that it can actually be believable for the listener ... the player.
Jon:It's interesting. In real life, our brains are really good at filtering out the unnecessary parts and ... even sitting in a room and recording. For example, I'm listening to your voice, and I can ... Just listening I can tell that the room you're in isn't acoustically treated. It's got hard surfaces and the sound bounces around. If I'm sitting in the same room with you and listening to you talk, I don't notice that at all. But hearing you over the microphone, I notice it a lot. It's obvious. That's one of the things that you have to overcome when you're doing any kind of presentational material. It needs to sound better than real life, because real life, we ignore an awful lot of stuff. Recording stuff is just a lot of fun for me and not just going on a trip. I used to get to go record golf crowds and tournaments or tennis crowds at tournaments. And that's ... I can't say I don't like that. That was a lot of fun to be involved in that way. To stand behind the backstop at the game that's leading into the playoffs in 1992 or whenever it was. That was pretty cool standing next to the dugout of guys who in a month are going to be the World Series winners. I enjoyed that. [Laughter] And looking down, I was at a tennis tournament once in Florida, and they put me in the press box. They didn't have cameras there during the week. They came in for the weekend for the finals, so they put me in the camera box at the end of the tennis court with my recorders. And then I ran all my microphones out onto the court, and I'm sitting there looking straight down the court at Serena Williams serving. And she did hit me a couple of times. That's an intense experience to be that ... that close and see that perspective and experience that perspective on things, you know. Same with golf. Being right there at the ropes or be inside the ropes with some of the greatest athletes and performers on the planet. That's cool. It's a lot of fun. Beyond that. I enjoy going into an empty room with a bunch of junk and just making sounds. I was doing a demonstration, doing a video while we were doing kind of online teaching, and I'm in my garage saying, look, you can record sound effects anywhere, and I'm picking up different things. I picked up a drill motor and a battery driven drill, and I squeezed it."See, that sounds ... That's a good motor." I picked up something else. I think it's a little reciprocating saw, and it had this really cool little sound that I've never heard before."I've got to use that somewhere. I'm definitely going to keep this sound or I'm going to record it again and keep it somewhere in my library, because it was just an awesome sound." And those discoveries are probably as much fun for me as going off to a big tournament or something. Probably more fun because it's a whole lot less trouble.
Thomas:Now, a lot of our listeners, of course, because it's a big focus of this podcast is music ... A lot of our listeners are composers. You've had the opportunity to hire composers and work with them. How does the composer fit into the picture? When do they come in and that kind of stuff? How does that process work with composers? The composer is going to be hired as soon as the game design is finished. Once you know what the game is going to be and you've made a lot of those decisions, we would hire the composer to ... to start work. And because you can't really hire somebody until you know what it is they have to do, right? Even at Microsoft, there's not unlimited money. Once you know and can define the requirements of their project, then you can hire them and hand it over to them and they can start bringing you content. They'll start submitting stuff to you, which you then plug into a game or a prototype of the game in that section and make sure it works. And if it works and everybody likes it, then move on to the next piece. So there's probably a five or six months window where the composers are going to be working. And again, when I was doing this, doing a score with an orchestra was unusual. It wasn't unheard of, but it was unusual. I think it's a lot more common now to go in and record orchestral cues, which probably is going to push the timeline out a little more. But the game has to be designed before the composer can start creating. One of the things where I think the composers have benefited lately in video games has kind of been that there has been a lot of emphasis, I think, on the sound of the games, and it's not something that is taken for granted by a lot of the ... especially the big productions, but even some of the small social games where they can be playing on a little phone and you go listen to some of the music that they've done for some of these games. And it's just like "Wow. They really put some resources into this." Because they understand the importance of audio in the experience and in helping you to become immersed in the product.
Jon:I think with the advent of some of the big game console production where we started to add things like the surround sound tracks, it became more cinematic. And while most people probably still play in front of a small screen with headphones, there are people who play in a home theater with big surround sound systems, and it's a pretty exciting experience to do that. So yeah, I think there's a demand for that cinematic experience, and that includes cinematic music.
Thomas:Yeah. I think it's a big area of opportunity, certainly living in Utah. I tell this to my students all the time ... that it's an area of growth in the industry for composers and audio ... games in general. I think with Silicon Slopes happening, and there's a lot of opportunity, I think, for a young composer or a young audio designer, young audio engineer to get some work and get some experience in this field while it's still growing.
Jon:And again, I tell students that your chances of getting a job at a big studio right out of school are very, very slim. Now, it might happen. I won't say it won't happen, but it's really slim. The people that you're going to be working with most likely are going to be your peers and the people that are around you and the people that you network with. I know we have a a chapter of the G.A.N.G. Audio Network Guild here that meets occasionally, and there's networking opportunity there. There's game developers meetups where if I were really interested in getting back into doing sound design for games today, that's the first thing I would do is try and network. Go find people who need my skills, who need what I have to offer. And not just that, but I need what they're doing so I can ply my trade. So it's a two way street. I need them and they need me. Networking is really key in the business, I think.
Thomas:Yeah. What I tell students is put together a portfolio. Have something that shows your work that you can do. Get it up online. Go to these networking events. Get to know people. Networking can be scary for some of us that are more introverted. It can be a difficult kind of thing to do. But if you think of it as making friends in the industry, rather than networking, you're just making friends, and there's not an expectation there. You don't know when that connection, that friendship will lead to something. You mentioned, the friendships you made in college, it was 20 years later that some of those connections really made a difference for you.
Jon:I mean, it wasn't just a casual thing. It's a life changing career change. No, I went from working in a theater, doing live events, to working in an air conditioned office every day. And for six years of that, I did work 8 hours a day and had my weekends off. It was great. That was a wonderful thing that happened that kind of saved me, I think. So, don't burn bridges. That's the other thing I would recommend. Try not to burn bridges. Don't make enemies. Don't be a jerk.
Thomas:Because you never know. I mean, even if you're in a supervisory position, you never know when it's going to turn around and they'll be in a supervisory position in the future. It's just a good practice to just be ... be somebody that people like working with. In school, when you have group projects to work on, be the reliable person. Establish your reputation now, because you never know what that could lead to later on. Be professional from day one.
Jon:Develop a good work ethic and always give the best that you can give. Don't ever hold back.
Thomas:Excellent. So, Jon, how can our listeners find you and your work online? Do you have a presence or anything that they can find?
Jon:I'm top secret. I don't have a strong web presence. Like I said, I'll give you a link to one of the music ... a piece of music that I've recorded a couple of years ago, and maybe I'll give you links to some of the YouTube videos of gameplay from games that I worked on. I still think they're fun. Some of the ... The snowboarding. The third iteration of snowboarding was just bizarre, and I think in a good way. I really enjoyed it. It was like five different animation genres, all in one game, and each character ... There were, I think, five characters, and each character had a different kind of animation world that they lived in. One was anime, and one of them was a boy band thing, you know. So it's kind of crazy, but it was a lot of fun. So I'll give you some links to those. Sure, if you'd like.
Thomas:Excellent. And certainly they can find you at Salt Lake Community College, And if you happen to be able to take classes at Salt Lake Community College, especially, I think one of the really nice things about our program is that we do have the game audio class, the music for games classes. A lot of audio and music programs kind of ignore the interactive audio side of things. Maybe it's just a unit or something like that within a larger class, but I think it's ... it's a bigger field that there's a lot more to learn ...
Jon:To create content for that stuff is ... it takes a while to wrap your head around what you're actually doing. I'm surprised there are game programs, game production programs around the country, that as near as I can tell, do not have any sound design element involved. It's like, "We'll teach you how to program, and we'll teach you how to do the artwork on the animation," but I don't know where they get their sound. I don't know how that happens, but it can be and I think should be its own piece of that. It's a whole separate field, and game audio designers are not common.
Thomas:And part of it is for most projects you'll have one audio designer and 20 artists and 20 programmers and so maybe that's part of it. But certainly that audio component is essential to the success of any project in my mind.
Jon:I will throw a plug in here for GameSoundCon, which is coming up in November. It's online and it's very affordable and a great way to kind of get a better idea about how game audio is and where it fits in the world of games. And these are all top notch sound designers currently practicing game sound designers. So if anyone watching this is interested in game sound, I would definitely say, "Go find GameSoundCon and sign up." I think it's $75 for registration. Panel discussions, interactive, networking. All that stuff's available there.
Thomas:Excellent. Thank you, Jon. I appreciate your time. Thank you for providing a wonderful insight. I really appreciate your expertise and thanks for being a part of the podcast.
Jon:Alright. Enjoy your sabbatical.
Thomas:Thanks for listening. In our next episode, music educator Dr. Craig Ferrin will join the show as we discuss the economic value of creativity. If you have enjoyed today's discussion, please subscribe to the podcast. It really helps us, and you can be alerted about new episodes coming out every two weeks. We welcome your thoughts and comments on today's show on our Discord Channel, A Composer's Natural Habitat. A Composer In His Natural Hhabitat is made possible by the support and donations of listeners like you. If you would like to help support the show, please subscribe to our Patreon for access to exclusive episodes and other content. Musical Themes by Thomas C. Baggaley. Special thanks to Mario, Sonic, Link, Princess Zelda, Pacman, Ms Pacman, Kirby, Laura Croft, Ezio Auditore d'Firenze, Donkey Kong, and of course, Tex Murphy. Games from the Tex Murphy series, including Mean Streets, are currently available for purchase on Steam. Until next time, keep creating.