Sound Design(erfall)

Show off your mod creations or just a work in progress.
User avatar
Hoon
Posts: 11
Joined: Thu May 14, 2015 5:49 am

Sound Design(erfall)

Post by Hoon »

So, I figure it might be prudent to start a thread now to have a discussion about what can and should be done for sound design in Daggerfall. I've already started scouring the internet for resources, and I'm doing okay in that regard, but let's talk about actually getting the sounds into DFTFU and all of the extra stuff.

My goal, in short, is to completely replace all of the audio in Daggerfall and add new elements to bring Daggerfall up to modern standards of audio. So let's break it down:

Replacing vanilla sounds. Finding/designing/recording suitable sound effects is my bag (and a good 90% of the whole project), actually getting those sounds into the game is something I need a little guidance on. I unpacked the DAGGER.SND archive with Daggerfall Jukebox (awesome little utility, sir!) and it's a great reference, but gosh is it an unorganized mess in there. I did a little experiment to see what would happen if I replaced just one sound in DAGGER.SND, and the result was a complete scrambling of what makes what sound. I used the DaggerSound utility to make the new archive, and its readme says that Daggerfall doesn't like anything other than raw 11025 Hz/8-bit unsigned/mono Wav format. Which is untenable. How does DFTFU know what sound effects to play, and how can we get it to play different (better) ones?

While we're here, are there any recommended guidelines for the audio quality and formats Unity likes best? My current plan is to deliver files with 44.1kHz/16-bit quality; .wav format for single sounds and .ogg format for loops.

Adding more sounds. Replacing all of the vanilla sounds one for one will do a lot to put Daggerfall's audio into the 21st century, but there are still things modern games have that Daggerfall lacks, and things we'll have to add. The ideal would be a system that can get information on your location and play appropriate ambient loops and randomly fired sounds. For more footsteps, the engine would have to know what the player is stepping on and what they're wearing (and what the actor is, if actors other than the player are to have footstep sounds). Then, turning more objects into sound sources: howling wind coming from a dungeon entrance, blacksmith hammering, dogs that actually bark, etc. And randomization: so instead of the engine playing a single sound for something every time, it can randomly choose from a list of similar sounds (eg, multiple creature idle sounds, combat impacts, or footstep sounds).

And finally, I've got a few questions about how Unity handles audio, and audio effects. The reverb zones are exactly what I have in mind; how will we go about attaching them to dungeon blocks? Will it be possible to create zones for other audio effects? I've got other effects in mind, such as using the low-pass filter when the player character goes underwater, or applying other effects when the player character is under the effect of certain spells.

I don't have much to show off yet. I've only just started collecting resources from wherever I can; I am keeping an attributions list of every sound I download, which will be handy for honoring intellectual property laws among the wonderful people that provide their recordings for use. Right now I've designed a few random menu and level-up sounds; I'll try recording some journal pageflips/scribbles and a coin-grab sound today.

If any of you have questions, suggestions, or sound effects I can use, then that's why I made this thread. Shoot! And happy hearing!

edit: it might be more reasonable for this thread to be moved to another forum. I'll let someone with the power to do so make that decision. :P

User avatar
Interkarma
Posts: 7236
Joined: Sun Mar 22, 2015 1:51 am

Re: Sound Design(erfall)

Post by Interkarma »

Awesome! I'd love to hear better quality sound in DF. I'll answer your post as best I can in the time I have over lunch. Might have to come back and edit later.

As you discovered, there's no point packing sounds back into DAGGER.SND. The only reason to do that is so you can change audio in the original game. You can completely ignore DAGGER.SND for your purposes. Same goes for all the native binary files, they are just there as a baseline.
How does DFTFU know what sound effects to play, and how can we get it to play different (better) ones?
Here's a rundown of the current process for playing native Daggerfall audio. I'll link the code in case anyone is interested.

1. DAGGER.SND file is instanced by SoundReader.cs, part of the Content Readers family. It also has the power to spit out a Unity-compatible AudioClip from the raw data in DAGGER.SND.
2. I have categorized native sound indices into an enumerated type in SoundClips.cs. This lets the programmer reference native sound effects in a type-safe manner, such as SoundClips.EnemySpiderAttack.
3. Then DaggerfallAudioSource.cs peers with and extends Unity's standard AudioSource component. This provides a simple way to easily play any of Daggerfall's sounds in Unity (even inside the editor).
4. Finally, the individual GameObjects have a DaggerfallAudioSource component attached to play the appropriate sound. For example, torches loop SoundClips.Burning with a linear rolloff. This is no different to any other sound-emitting object in Unity, only there's a special tool at hand to play native Daggerfall sounds.

So that's how the Daggerfall-Unity bridge works. Now to play different/better sound clips is very easy in theory. You just need to play the correct clip the usual way through Unity's AudioSource component. There's nothing too special going on here. All I really need to do from a tools point of view is provide the means to "get out of the way" so you can make the most of Unity's powerful audio features.
While we're here, are there any recommended guidelines for the audio quality and formats Unity likes best? My current plan is to deliver files with 44.1kHz/16-bit quality; .wav format for single sounds and .ogg format for loops.
That should be fine. Check the Unity Manual on Audio Files for more information. Basically, you don't have to think too hard about formats as Unity will import them back into an internal format anyway. Unity lets you do almost everything in a drag and drop way, so it should be easy to play with this in a test scene.
Adding more sounds. Replacing all of the vanilla sounds one for one will do a lot to put Daggerfall's audio into the 21st century, but there are still things modern games have that Daggerfall lacks, and things we'll have to add. The ideal would be a system that can get information on your location and play appropriate ambient loops and randomly fired sounds. For more footsteps, the engine would have to know what the player is stepping on and what they're wearing (and what the actor is, if actors other than the player are to have footstep sounds). Then, turning more objects into sound sources: howling wind coming from a dungeon entrance, blacksmith hammering, dogs that actually bark, etc. And randomization: so instead of the engine playing a single sound for something every time, it can randomly choose from a list of similar sounds (eg, multiple creature idle sounds, combat impacts, or footstep sounds).
It would be trivial to play direct replacements for existing sounds. The complexity of adding new sounds is variable depending on the spec. For example, easy to play a random sample from a collection of sounds, harder to map footsteps to surfaces (as surface properties don't currently exist in asset and would need to be added by hand). Basically all doable, just a matter of designing an idea then have a programmer wire it up with code. Like I said, the important thing is that my code can "get out of the way" so you can all make the most of Unity's features.
And finally, I've got a few questions about how Unity handles audio, and audio effects. The reverb zones are exactly what I have in mind; how will we go about attaching them to dungeon blocks? Will it be possible to create zones for other audio effects? I've got other effects in mind, such as using the low-pass filter when the player character goes underwater, or applying other effects when the player character is under the effect of certain spells.
I recommend playing with some of these things in Unity itself. Start with Dungeon Basics in Tutorials to create a static scene, then drop in some cubes on the ground and wire them up for different sound effects. It's much easier to bring these concepts into the procedural world once there's a clear example of how you need it to work. You can do an awful lot in Unity without writing a single line of code. :)

The main thing to keep in mind is that Unity handles audio, and DFTFU just bridges Daggerfall into Unity. For the most part, the things you want to do aren't specific to DFTFU or Daggerfall. These things are common to any game developed in Unity, so all the usual resources can be used. One of the joys of having such a mature and powerful engine at your fingertips.

User avatar
Arl
Posts: 202
Joined: Sun Mar 22, 2015 10:57 am

Re: Sound Design(erfall)

Post by Arl »

I think this is very important, I would say even more necessary than visual enhancements, because Daggerfall's audio in general is mostly noisy and loud.

Maybe is just me, but I feel that all sounds plays at the same volume, so I tend to feel some source of mental fatigue after a while, there's no sound distance fading or anything 'cause is a really dated game of course.
My Deviantart page, I have some Daggerfall stuff in there.

User avatar
Hoon
Posts: 11
Joined: Thu May 14, 2015 5:49 am

Re: Sound Design(erfall)

Post by Hoon »

Thanks for the guidance, Interkarma! I'm messing with things now, and it's mostly just intuitive. I'm going through the Dungeon Basics tutorial, and I've noticed a few things: For some reason, I can't get player footsteps to work. And the for the campfire we start next to with the burning loop, the rolloff on the audio source isn't working; there's no volume change, and it just cuts off once I pass the end point of the fade. The same seems to be the case for the creatures, too; I can hear the rat and the imp at full volume as soon as I'm in proximity. I'm guessing some things are different now because Unity got an update?

I dropped in a reverb zone, though, and that was fun. How will I be able to add sound design information like reverb zones directly to the base objects (buildings, dungeon blocks, etc) so that Daggerfall's procedural generation can take care of building the world? Same goes for attaching new sounds to things like creatures, NPCs, and billboards, but I'm guessing that's a lot more intuitive.

Fun fact about Daggerfall's sound files: Since they reduced the sample rate of the sound effects to 11025Hz to save on memory, that means any frequencies above 5512.5Hz in the actual waveforms are just gone. You can't accurately recreate frequencies greater than half of a given sample rate. That's why standard CD audio quality is 44.1kHz; 22kHz is well beyond the range of human hearing, with wiggle room. Nyquist theorem, yo.

User avatar
Interkarma
Posts: 7236
Joined: Sun Mar 22, 2015 1:51 am

Re: Sound Design(erfall)

Post by Interkarma »

If you're running Unity 5, check you downloaded DFTFU Developer Preview 1.3.8. The last release version (1.2.41) only works with Unity 4.6. There were major changes to the audio system in Unity 5, so much of the 1.2 audio code simply won't work.

I'm going to work on promoting 1.3 to a release version soon, just for the Unity 5 compatibility. This won't have the text&translation features intended for 1.3, but at this stage I think that Unity 5 compatibility is more important. I'll bump text&translation features to 1.4.
Hoon wrote:I dropped in a reverb zone, though, and that was fun. How will I be able to add sound design information like reverb zones directly to the base objects (buildings, dungeon blocks, etc) so that Daggerfall's procedural generation can take care of building the world? Same goes for attaching new sounds to things like creatures, NPCs, and billboards, but I'm guessing that's a lot more intuitive.
I'm planning features to support this. Best I can say right now is it will be a blended solution that will improve over time. Keep an eye out for the event system coming in 1.3.12. With only a small amount of coding, you'll be able to extend any game object when it's instantiated. This will expand towards using prefabs for things like enemies (so easy to tweak in editor) and systems like Uncanny_Valley's WorldObjects mod. I will also continue to expose information about the player and the surfaces they are touching. It will take time, but the solutions will come. :)
Hoon wrote:Fun fact about Daggerfall's sound files: Since they reduced the sample rate of the sound effects to 11025Hz to save on memory, that means any frequencies above 5512.5Hz in the actual waveforms are just gone. You can't accurately recreate frequencies greater than half of a given sample rate. That's why standard CD audio quality is 44.1kHz; 22kHz is well beyond the range of human hearing, with wiggle room. Nyquist theorem, yo.
That's really interesting! Thank you. :)

User avatar
Hoon
Posts: 11
Joined: Thu May 14, 2015 5:49 am

Re: Sound Design(erfall)

Post by Hoon »

Oh dang, I was definitely not using the developer preview, sorry. I'll have to revisit all of that, then.

Anyway, here's some of the stuff I've been working on. It's not much, but I've worked on a few random things and I'd like to show them off.

They're all in this little zip file here, on me google drive!

A brief rundown of the files inside:

BookWrite and CoinGrab were sounds I actually recorded myself. I'm surprised at how well they turned out, but I still feel like there's too much self-noise from the mic and whatnot. Whatever little things I can record myself though, I'd like to.

The Loop - Ambient files are a few environmental loops. RainInterior.wav is intended for when you're inside a building (but not a dungeon) during the rain (does it sound realistic? not sure I got the filtering right). I'm fairly certain that the audio loops seamlessly, although if your audio player hiccups when it restarts it'll be hard to tell. Also notice that the ambient effects are really sparse - I want to use something like what vanilla Daggerfall does (or the Ambient Effects Player in the demo) to fill out the soundscape with more distinct effects like birds or rattling chains dynamically. That way it's a bit less predictable.

The Hearthfire loop is just for fireplaces; it's probably not big enough for campfires, but maybe it can be used for torches...

[I'm going with a minute-long duration for environmental loops, and 10 seconds for object-based loops like the fireplace. Would different durations be preferred?)

And then there are two possible menu sound effects. Since vanilla Daggerfall's menu is mostly stonework-themed, I'd like to design menu sounds that actually sound like stones; I'm not sure you would think that without me telling you, though.

What do you guys think? There isn't much there, but I feel a bit more comfortable having a few sounds on the table. Also keep in mind that this is very much a work in progress; I'll be surprised if any of these sounds make it to the final product in their current state.

User avatar
Interkarma
Posts: 7236
Joined: Sun Mar 22, 2015 1:51 am

Re: Sound Design(erfall)

Post by Interkarma »

These sound really great! I look forward to hearing more sounds this quality. If you create more sounds, I'd be happy to showcase them in a future mod demo. :)

I'll keep on working on the means for you to experiment with adding new audio to procedural scenes.

User avatar
LypyL
Posts: 512
Joined: Sun Mar 22, 2015 3:48 am

Re: Sound Design(erfall)

Post by LypyL »

It sounds like you're off to a good start.

ifkopifko
Posts: 195
Joined: Thu Apr 02, 2015 9:03 am

Re: Sound Design(erfall)

Post by ifkopifko »

Sounds good to me. :) In the rain loop, perhaps it would be better to decouple the thunder rumble from the actual rain sound. Thunder is a very important atmosphere component, I think it would be desirable, to use variety of high quality thunder sounds. If matched in a realistic way with lightning flashes, that could be awesome. Perhaps even the possibility to have different kinds of storms would be a nice touch... like a really windy one for example.

User avatar
Hoon
Posts: 11
Joined: Thu May 14, 2015 5:49 am

Re: Sound Design(erfall)

Post by Hoon »

ifkopifko wrote:Sounds good to me. :) In the rain loop, perhaps it would be better to decouple the thunder rumble from the actual rain sound. Thunder is a very important atmosphere component, I think it would be desirable, to use variety of high quality thunder sounds. If matched in a realistic way with lightning flashes, that could be awesome. Perhaps even the possibility to have different kinds of storms would be a nice touch... like a really windy one for example.
That's one thing I was concerned about with that particular loop. There absolutely will be random thunder effects played exactly how you describe on top of this loop; with that in mind, I honestly don't know if the rumbling inside the loop itself will be too distinct.

Help me out here, everybody: what are all of the different weather types in Daggerfall? To my memory, there's clear, rainy, snowy, and thick fog. Sooo, I'd need ambient loops for:

Clear: Four different loops for winter, forest, jungle, and desert environments; anything else?
Night: Different nighttime loops for the same environments above.
Rainy: Technically there's only one type of rain, but it's no thing to prepare two loops for light and heavy rain. Rain in buildings (but not castles or dungeons), too.
Snowy: Same, two loops for light and heavy snowfall.
Foggy: Spooky, oppressive fog.

And then other environment loops for interiors like dungeons, buildings, and castles (castle loop should transition into dungeon loop once you get deep enough in, where the music and lighting switches in vanilla Daggerfall). Is that everything?

Additionally, I can use something like the ambient effects player to randomly play sounds complementary to the environment. For clear weather, I'd differentiate between towns and wilderness (day and night). I'd also like to attach loops and randomly played sounds to more game objects so that some distinct sounds actually come from a source in the world. For nighttime, it would be things like cricket loops, or owls in the trees. For daytime (in desert regions), it could be cicada loops. You get the picture. The more distinct the sound, the more I want it to be a 3D clip instead of 2D "ambience"; that'll do a lot more for the atmosphere and, uh, dynamicness.

If you can get them in-game, Interkarma, I could prepare the rest of the environmental loops for the next showcase. I could probably have some of the randomly played ambience ready, too. I'm having a much easier time finding quality ambient recordings than things like combat impacts and footsteps :P. If that happened, though, I'd ask that my attributions doc I linked in my first post found it's way into the data; it's not sufficient for a full release, but for a preview or showcase I'd still like to include some kind of credits.

Post Reply