Page 1 of 1

Texture conversion and caching question

Posted: Fri Jun 25, 2021 12:04 am
by numidium3rd
I've been doing a little programming for the fine folks working on Redguard Unity and we had a question about Daggerfall Unity's texture loading process. From what I can tell, DFU converts textures vanilla textures and then caches them for a period of time in memory. Is there are a specific reason why we don't cache converted textures to disk as PNGs so that we can skip the conversion process? I'm assuming it's because there wouldn't be an appreciable difference in load times. Am I correct or is there another reason?

Re: Texture conversion and caching question

Posted: Sat Jun 26, 2021 12:59 am
by Kab the Bird Ranger
It could be cached to disk, but then we'd have to know when to invalidate the cache. Mods can override those textures, so the system would have to know when to use the cache files and when to reload the "right texture".

Also, between loading a TES2 texture from disk and loading a PNG from disk... Both requiring accessing the disk, and both require conversions before being sent to the GPU. I'm not sure the performance would be that different.

Re: Texture conversion and caching question

Posted: Sat Jun 26, 2021 5:08 pm
by numidium3rd
Thanks for the reply. I agree, there's probably very little performance difference between streaming in a native texture versus a PNG. I honestly should have thought this through better before posting lol.

Re: Texture conversion and caching question

Posted: Sun Jun 27, 2021 5:03 am
by Interkarma
Hey Numidium. :)

Our gamedata isn't identical, so my experience might be of little value here, but there were some good reasons for DFU to use runtime conversion over pre-converting and caching everything beforehand.

First up, it's pretty inexpensive either way. It's still fast to read in a small file and expand it memory and it saves the hassle of maintaining cache differences between builds. Your target formats will probably change a bit as you don't know what you don't know yet.

This key one is that Daggerfall stores more data with textures than just images. It also stores scaling and offsets for world flats and position data for paper doll textures. This doesn't translate well to PNG so the original files need to be read anyway (or this data or also pre-cached to XML or something). And some images (e.g. weapons) are held in an indexed format for palette swaps that also doesn't translate well to PNG.

Also keep in mind that DFU started as Daggerfall Tools for Unity, a gamedata importer based on my old exploring tools. Part of this process is just an evolution from one thing to another. Personally, I think there's a niceness to just running the game and everything loading straight from classic gamedata without user needing to build a cache or re-build when formats change. The hard part of getting people to use good-quality gamedata in the first place.

Hopefully that's of some help, considering our requirements are probably different between the two games. I reckon just do whatever you think is best and let it grow from there. Good luck!

Re: Texture conversion and caching question

Posted: Sun Jun 27, 2021 5:30 am
by numidium3rd
Thanks a bunch for the detailed post, Interkarma. I didn't know that Daggerfall textures had all that extra data in them. Redguard's texture files are just bitmaps with simple headers as far as we can tell (except for animated textures - I'm still working on reversing those). I agree that it's best to have the fewest steps possible between reading a vanilla file and instantiating a Unity texture. Fewer things can go wrong in translation as you mentioned.
And thanks for the well-wishes. You're an inspiration to Elder Scrolls game hackers everywhere :) .