Page 2 of 2

Re: Reload/Refresh Textures

Posted: Fri Jan 12, 2018 7:04 pm
by ifkopifko
Just a note: PNG files are compressed, it is lossless compression, but it is compressed. For 1024x2048 pixels, 24 bits per pixel, it is 6MB uncompressed. So all your upscaled textures could take up 10GB in 24bit bitmap format. Not sure if it would explain the behaviour you described though.

Re: Reload/Refresh Textures

Posted: Sat Jan 13, 2018 12:08 am
by Interkarma
King of Worms wrote:The problem with upscaled enemy NPCs in dungeons for ex. they just massacre the system memory. Dont know what is going on there. Every upscaled NPC set has like 10-30mb on HDD. Ingame, it goes to 10s of gigabites of memory... I THINK maybe if the dung has 100 enemies, it caches each enemy? So 20*30mb for 20 rats. 10*30mb for 10 skeletons. 20*30mb for 20 bats etc?


ifkopifko wrote:Just a note: PNG files are compressed, it is lossless compression, but it is compressed. For 1024x2048 pixels, 24 bits per pixel, it is 6MB uncompressed. So all your upscaled textures could take up 10GB in 24bit bitmap format. Not sure if it would explain the behaviour you described though.


ifkopifko pretty much nails it. The size of source image on disk doesn't correspond with how engine stores that image internally for GPU. Currently this is RGBA32 format, so a full 32-bits per pixel. With DXT5 compression, this will come down to 4:1 ratio, or around 8-bits per pixel on average with almost the same overall fidelity.

I'll see what I can do with improved cache management once modded textures are compressed. :)

Re: Reload/Refresh Textures

Posted: Sat Jan 13, 2018 10:33 am
by King of Worms
This sounds very promising, thanks Interkarma. Many games had "compress textures" option, and usually, I was not able to tell the difference between compressed and not. So lets hope it will work here as well. Fingers crossed. And thanks Pifko for the explanation as well!

PS: Its interresting to me, that the original 256 color textures benefit from the 32 bit depth, but I dont understand jack in this department anyway ;) maybe it adds up to smooth gradients when its rendered

Re: Reload/Refresh Textures

Posted: Wed Jan 17, 2018 9:25 pm
by TheLacus
Interkarma wrote:MaterialReader.CompressSkyTextures used to be a field that applied to all textures (I think it was just called "CompressTextures" at the time). The setting had a negative impact on small textures such as flats, so I just retained setting for sky images where compression was barely noticeable. The same should be true for any large texture.

I'd still rather not have texture compression for the classic textures, as it corrupts their aesthetic and has minimal impact on memory usage. Perhaps we can add a new MaterialReader field called "CompressModdedTextures" or similar, and load as compressed when field is true. It could even be enabled by default.

Sounds good. The only issue i found is a discrepancy in normal maps: they look different, somewhat lighter, with DXT5 (second picture) compared to ARGB32. I'm aware Unity uses alpha and green channel, but afaik is the same for both formats.

normals (1).png
normals (1).png (1.72 MiB) Viewed 70 times
normals (2).png
normals (2).png (1.48 MiB) Viewed 70 times


Or would it be better to make this a setting in the mod manager so it can be toggled on a per-mod basis?

Do you think we should call Compress on textures imported with models? Compression is enabled by default in editor so many mods will probably leave it as is.

Textures for vanilla meshes, npcs, etc. are all imported from StreamingAssets so only a global toggle is possible.

I have been thinking about importing materials from mods because this would fit great with load order (if mod0 has albedo + normal and mod1 only albedo, only the latter is imported, while dropping everything on disk can create unintended mixes) but also has cons (unity editor is required, atlases and arrays requires textures to be stripped, mods like Nystul reflections inject only specfic properties like metallicgloss on purpose) so i kept this discussion for more mature times.

Re: Reload/Refresh Textures

Posted: Wed Jan 17, 2018 9:56 pm
by Interkarma
TheLacus wrote:Sounds good. The only issue i found is a discrepancy in normal maps: they look different, somewhat lighter, with DXT5 (second picture) compared to ARGB32. I'm aware Unity uses alpha and green channel, but afaik is the same for both formats.

Interesting. Not sure what is causing that right now. Can any other Unity devs weigh in?


TheLacus wrote:Do you think we should call Compress on textures imported with models? Compression is enabled by default in editor so many mods will probably leave it as is.

Textures for vanilla meshes, npcs, etc. are all imported from StreamingAssets so only a global toggle is possible.

Global is fine for loose textures, and mod creators have control over this themselves as they're packaging with editor. If there are any problems with consistency, this can be revisited later.


TheLacus wrote:I have been thinking about importing materials from mods because this would fit great with load order (if mod0 has albedo + normal and mod1 only albedo, only the latter is imported, while dropping everything on disk can create unintended mixes) but also has cons (unity editor is required, atlases and arrays requires textures to be stripped, mods like Nystul reflections inject only specfic properties like metallicgloss on purpose) so i kept this discussion for more mature times.

Yeah, this is a tougher one. I think the loose textures on disk are great for testing. But anyone distributing a large-scale mod should take the time to create a .dfmod package for user convenience. But this does complicate the pipeline in other ways as you say. Agreed it's best to defer for now until the systems are a bit more mature.