Texture conversion and caching question

Discuss coding questions, pull requests, and implementation details.
Post Reply
User avatar
numidium3rd
Posts: 187
Joined: Sun Mar 25, 2018 12:34 am
Location: United States

Texture conversion and caching question

Post by numidium3rd »

I've been doing a little programming for the fine folks working on Redguard Unity and we had a question about Daggerfall Unity's texture loading process. From what I can tell, DFU converts textures vanilla textures and then caches them for a period of time in memory. Is there are a specific reason why we don't cache converted textures to disk as PNGs so that we can skip the conversion process? I'm assuming it's because there wouldn't be an appreciable difference in load times. Am I correct or is there another reason?

User avatar
Kab the Bird Ranger
Posts: 123
Joined: Tue Feb 23, 2021 12:30 am

Re: Texture conversion and caching question

Post by Kab the Bird Ranger »

It could be cached to disk, but then we'd have to know when to invalidate the cache. Mods can override those textures, so the system would have to know when to use the cache files and when to reload the "right texture".

Also, between loading a TES2 texture from disk and loading a PNG from disk... Both requiring accessing the disk, and both require conversions before being sent to the GPU. I'm not sure the performance would be that different.

User avatar
numidium3rd
Posts: 187
Joined: Sun Mar 25, 2018 12:34 am
Location: United States

Re: Texture conversion and caching question

Post by numidium3rd »

Thanks for the reply. I agree, there's probably very little performance difference between streaming in a native texture versus a PNG. I honestly should have thought this through better before posting lol.

User avatar
Interkarma
Posts: 7236
Joined: Sun Mar 22, 2015 1:51 am

Re: Texture conversion and caching question

Post by Interkarma »

Hey Numidium. :)

Our gamedata isn't identical, so my experience might be of little value here, but there were some good reasons for DFU to use runtime conversion over pre-converting and caching everything beforehand.

First up, it's pretty inexpensive either way. It's still fast to read in a small file and expand it memory and it saves the hassle of maintaining cache differences between builds. Your target formats will probably change a bit as you don't know what you don't know yet.

This key one is that Daggerfall stores more data with textures than just images. It also stores scaling and offsets for world flats and position data for paper doll textures. This doesn't translate well to PNG so the original files need to be read anyway (or this data or also pre-cached to XML or something). And some images (e.g. weapons) are held in an indexed format for palette swaps that also doesn't translate well to PNG.

Also keep in mind that DFU started as Daggerfall Tools for Unity, a gamedata importer based on my old exploring tools. Part of this process is just an evolution from one thing to another. Personally, I think there's a niceness to just running the game and everything loading straight from classic gamedata without user needing to build a cache or re-build when formats change. The hard part of getting people to use good-quality gamedata in the first place.

Hopefully that's of some help, considering our requirements are probably different between the two games. I reckon just do whatever you think is best and let it grow from there. Good luck!

User avatar
numidium3rd
Posts: 187
Joined: Sun Mar 25, 2018 12:34 am
Location: United States

Re: Texture conversion and caching question

Post by numidium3rd »

Thanks a bunch for the detailed post, Interkarma. I didn't know that Daggerfall textures had all that extra data in them. Redguard's texture files are just bitmaps with simple headers as far as we can tell (except for animated textures - I'm still working on reversing those). I agree that it's best to have the fewest steps possible between reading a vanilla file and instantiating a Unity texture. Fewer things can go wrong in translation as you mentioned.
And thanks for the well-wishes. You're an inspiration to Elder Scrolls game hackers everywhere :) .

Post Reply