Reload/Refresh Textures

Post here if you need help getting started with Daggerfall Unity or just want to clarify a potential bug. Questions about playing or modding classic Daggerfall should be posted to Community.
User avatar
King of Worms
Posts: 4747
Joined: Mon Oct 17, 2016 11:18 pm
Location: Scourg Barrow (CZ)
Contact:

Re: Reload/Refresh Textures

Post by King of Worms »

Fantastic, thanks a lot! Fingers crossed for a smooth ride :)

User avatar
Interkarma
Posts: 7234
Joined: Sun Mar 22, 2015 1:51 am

Re: Reload/Refresh Textures

Post by Interkarma »

OK, super atlas is disabled now. From next builds, texture archives 97, 205, 211, 212, 213, 301 will no longer be combined into one huge atlas. Rather, each archive will each become an atlas of its own like other flats archives.

Something else to consider: the super atlas was only used in exterior RMB blocks, it was never used in dungeon RDB blocks. This means it's possible even a single archive might experience overflow. I wouldn't be surprised if archive 211 experiences this as it has a lot of small items. Even in its own atlas, it's practically a super atlas.

But at least now you can control the problem on a per-archive basis, and have substantially more overall leeway per archive set.

Next step might be to allow users to disable texture atlasing where possible. There are a few spots where atlasing is enforced - notably exterior foliage and ground textures. But TheLacus and modders seem to have these quite under control already. I'm a bit worried about performance doing this, but a quick test shows Unity's dynamic batching has come a long way since I started building this code in Unity 4.x days.

One step at a time. :)

User avatar
King of Worms
Posts: 4747
Joined: Mon Oct 17, 2016 11:18 pm
Location: Scourg Barrow (CZ)
Contact:

Re: Reload/Refresh Textures

Post by King of Worms »

Great! I think further testing will tell us more. If the atlas for each set is still 4k Im sure we will make it.

Ad the exterior/interior blocks - I saw atlas overflow in both cases... which is strange. I deffo saw 097 statues overflow outside, and that hanging light in dungeons is IMO from super-atlas as well. BUT, thats just my assumption now. I will test it thoroughly and see whats up after next build.

I think we can keep the atlases for now, its not a huge problem to fit in those. If it improves the performance that is... Agreed on the one step at time attitude ofc.

QUESTION:

Lacus mentioned NPC flats have different handling. But I have not received any specs according to that. I have not witnessed any problems like atlas overflow on them. But it would be fine to know what we are working with here. Here I speak about the basic flats like archive 182 which has like 100 entries, arch 183, 184 etc...

Than, Id like to know, how enemy animated NPCs are handled? Monsters like Rat, Bat etc. Each enemy has cca 40 - 90 frames in a set. Has that set size some limits I should fit in? Same would apply to neutral animated NPCs like town people etc.

Thanks a lot! Good luck in the dev process :geek:

User avatar
TheLacus
Posts: 1305
Joined: Wed Sep 14, 2016 6:22 pm

Re: Reload/Refresh Textures

Post by TheLacus »

King of Worms wrote: Than, Id like to know, how enemy animated NPCs are handled? Monsters like Rat, Bat etc. Each enemy has cca 40 - 90 frames in a set. Has that set size some limits I should fit in? Same would apply to neutral animated NPCs like town people etc.
Textures for Interior billboards like static npcs (albedo and emission) are loaded individually so there are no size issues. The same is true for enemies except that they only support albedo and also have animations.

I used them to test an alternative workflow for imported textures while preserving the original behaviour, with performance-saving features like atlas, for vanilla. I think unsing texturearray (or even atlas) for animation frames (instead of the entire archive) may be a future improvement to maintain some performance and support high resolution at the same time.

Outdoor wandering npcs share an atlas with original images, it's only increased up to 4096 as usual.

User avatar
LorrMaster42
Posts: 65
Joined: Thu Dec 24, 2015 12:09 am

Re: Reload/Refresh Textures

Post by LorrMaster42 »

Does Unity have LOD enabled for these sprites? That would help save some memory, right? Just wondering if there is any other optimization to be done.

User avatar
King of Worms
Posts: 4747
Joined: Mon Oct 17, 2016 11:18 pm
Location: Scourg Barrow (CZ)
Contact:

Re: Reload/Refresh Textures

Post by King of Worms »

Thanks mr Lacus for explanation, much appreciated! ;)
I will keep the high res flats archived in case they can be used in future.

Ad LorrMaster42 - I have not noticed any sprite LOD scaling nor have I idea if such a approach could help free up some resources in this particular case. All I know is sprites are kept in atlases or atlas-clusters to keep the draw calls low. They can occupy quite a bit of memory (everything you have x2 :lol: - in case of my very high res flats ) so every optimalization would be welcomed...

User avatar
Interkarma
Posts: 7234
Joined: Sun Mar 22, 2015 1:51 am

Re: Reload/Refresh Textures

Post by Interkarma »

LorrMaster42 wrote:Does Unity have LOD enabled for these sprites? That would help save some memory, right? Just wondering if there is any other optimization to be done.
There is no LOD for billboards. They are a quad with only two tris and cannot get any simpler.

If you're thinking of textures in the same way, mipmaps are already a system that stores small through large versions of a texture to be displayed based on distance to camera. This doesn't save texture memory though, quite the opposite.

The best way to optimise texture memory is to simply make textures smaller. Commercial game artists are given memory budgets to work within based on design spec. In this case spec is for individual textures to be under 128x128 pixels. You can obviously push things a lot further, but "infinity and beyond!" Should never be the default starting point. :D

Part of the job of a responsible game artist is to find the correct balance between fidelity and conservation of resources. It can be as much of an engineering challenge as programming.

User avatar
LorrMaster42
Posts: 65
Joined: Thu Dec 24, 2015 12:09 am

Re: Reload/Refresh Textures

Post by LorrMaster42 »

Okay, I see now. The individual sprite isn't the problem, its the image's memory space. That adds up a lot with all the NPCs.

One this I was wondering is if you could take a 3D model and add in a shader to make it look like a 2D sprite. Reason being would be to allow for different clothing options on character models and such while still maintaining a similar art style. Would there also be anything to gain development-wise from doing this?

User avatar
King of Worms
Posts: 4747
Joined: Mon Oct 17, 2016 11:18 pm
Location: Scourg Barrow (CZ)
Contact:

Re: Reload/Refresh Textures

Post by King of Worms »

Im testing the new version, using the imported texture compression. I have not made any direct comparisons on the memory used with on/off settings yet. Game runs better Id say. The visual fidelity of compressed textures is ok, I can see the degradation but its nothing dramatic - no problems there.

But Ive noticed one thing,it happened in the previous version as well but now Ive got it documented better.

I GUESS it has something to do with the cache system. Cache seems to use every MB of the memory left, which might not be a problem. But in case of paging file its excessive. And at the end of the day, the absolute saturation of memory is contraproductive and causes performance drops.

Game uses more paging file than I have allowed on my PC.

Ive never seen this in any other game, even games which massacred my PC like "mass effect" andromeda, which used the full 12288mb of my paging file. But nothing used more than I allocated - ever.

Maybe something to look into when the right time comes? Thanks, and gratulations on a new version!
Swap file.jpg
Swap file.jpg (449.37 KiB) Viewed 3610 times

User avatar
Interkarma
Posts: 7234
Joined: Sun Mar 22, 2015 1:51 am

Re: Reload/Refresh Textures

Post by Interkarma »

I would be interested in an apples-with-apples comparison of memory usage with compression on vs. off if possible. I'm intrigued how much saving is present at larger scale.

I'm not familiar with the tool you're using for overlaid stats here. It doesn't look exactly granular, so can't comment on your actual page file utilisation caused by Daggerfall Unity. I'd recommend using perfmon and setting up custom counters or drilling into Process Explorer by Sysinternals to get more granular information.

Your page file size of 12GB is more or less appropriate for a system with 8GB RAM (general rule of thumb is 1.5x system RAM). Can you confirm this is an 8GB system?

Daggerfall Unity doesn't use your page file directly. Rather, Windows has its own memory handling system and will move data between RAM and page file as needed. This page swapping can cause serious performance issues if system memory is constantly over-utilised (say above 95% at all times).

I'm still thinking about ways to manage cache more effectively to minimise waste and keep load performance high. Expect to see more on this shortly after I change over to 0.5. :)

Post Reply