(MOD) Daggerfall Audio Books - proof of concept, help needed

[King of Worms]

So, thanks to Zoran and his dedication, there was some progress made in the exploration of sound options in DFU. Most of the paths he tried, lead to the dead end, due to the nature of the DFU and unity. He spend 10s of hours on it :/

Another attempt was at Audio Books. It KIND OF works, here is a DFMOD proof of concept.
My personal MEGA, so it stays online:
https://mega.nz/file/3zxSyCSK#L9byNxKYJ ... Fhu0rRDnUo

It has many issues, and thats why I post it here, so maybe someone will help or point us in the right direction.

As it is now:
- its incompatible with Better Ambience mod
- works properly only for one book "Arkay the God"
- every other book you open, will play the Arkay the God voice lines THATS THE MAIN ISSUE WE NEED TO SOLVE.

To spawn the book, type addbook BOK00005 than read it and bath in the glory of audio book. It adds so much to the game!!

The main issue is this, will quote Zoran

the problem is that I can't find the way to differentiate which book is opened in the book window cause the DFU variable that has this info is protected by DFU and inaccessible

uX8lUdY.png (392.75 KiB) Viewed 926 times

Test it, take a look, and if you can help, it would benefit the game greatly.

Thank you
KoW

[King of Worms]

The enthusiasm here is deafening lately

[Seferoth]

It had only been one day, and it's nearly Christmas, many might have more important things in their minds currently.

[King of Worms]

It was the same with my previous question about sound, which is 1 month old. Its just how it is.

[Daniel87]

It was the same with my previous question about sound, which is 1 month old. Its just how it is.

Is the mod using CoquiTTS to generate audio files of the book text strings?
I built my own AI assistant in python with which I communicate via TTS and STT models but haven't tried running the Text2Speech in Unity. Not sure if there is a handle to run CoquiTTS AI using python and C# in Unity.
Would love to know how he managed to do this.

[King of Worms]

It was the same with my previous question about sound, which is 1 month old. Its just how it is.

Is the mod using CoquiTTS to generate audio files of the book text strings?
I built my own AI assistant in python with which I communicate via TTS and STT models but haven't tried running the Text2Speech in Unity. Not sure if there is a handle to run CoquiTTS AI using python and C# in Unity.
Would love to know how he managed to do this.

I think he generated the audio file on some site which provides text to speech. U can open the mod maybe in unity or somehow and see. The audio generation was not the main focus of the mod. Main focus was, to make the sound play in DFU, which failed.

[Daniel87]

It was the same with my previous question about sound, which is 1 month old. Its just how it is.

Is the mod using CoquiTTS to generate audio files of the book text strings?
I built my own AI assistant in python with which I communicate via TTS and STT models but haven't tried running the Text2Speech in Unity. Not sure if there is a handle to run CoquiTTS AI using python and C# in Unity.
Would love to know how he managed to do this.

I think he generated the audio file on some site which provides text to speech. U can open the mod maybe in unity or somehow and see. The audio generation was not the main focus of the mod. Main focus was, to make the sound play in DFU, which failed.

Ah, I guess he used 11.ai to generate the text.
This would mean every book needs to be generated beforehand and stored (GB of data, 11.ai is also very expensive to use outside the free character limit).
A better approach would be to dedicate some GPU resources for CoquiTTS AI and generate the text on demand. (I used a sentence by sentence approach that breaks the string into single sentences, generates the first one and while playing that one back, it starts generating the next one, etc. - It's not super fast but much faster than generating the whole thing and waiting 5 minutes for the result).

But to be able to use CoquiTTS, we would need a handle to run Python programs and python code within Unity3D - If you manage to get that done, you would be better of being one of the first people to create an actual RPG with AI characters that react dynamically. It's a huge task and not sure if the result would be very rewarding in daggerfall, as you need a really good GPU to run AI locally.

[King of Worms]

Yes but the problem was, we were not able to get it in the game. The issue is described in the opening post. Until that is solved, all other work is IMHO on halt :X I will take a rest now from DFU, not gonna dive in anything demanding or serious, I did that for last few years

[King of Worms]

Zoran saw your post here and this is what he said to me (hes only on Nexus)

Daniel87 idea with audio books seems really cool. His method is nice because it would take far less space and be compatible with other sound mods. Using AI text to speech on fly is better then making the sound files but it would require far more coding to get it to work in DFU. I don't know how to program with Python so implementation is way above my head. Hope that somebody joins who has better coding skills and makes the audio books possible, kinda feel like a lot of players will skip on the great TES lore just cause they prefer playing itself or think that the books take a lot of time to read - voiced audio would make more folks interested in them and the lore. Keeping the fingers crossed that someone helps us with the audio books or makes its own TTS mod.