Page 3 of 5

Re: Reload/Refresh Textures

Posted: Sun Jan 21, 2018 3:46 pm
by King of Worms
Fantastic, thanks a lot! Fingers crossed for a smooth ride :)

Re: Reload/Refresh Textures

Posted: Sun Jan 21, 2018 10:08 pm
by Interkarma
OK, super atlas is disabled now. From next builds, texture archives 97, 205, 211, 212, 213, 301 will no longer be combined into one huge atlas. Rather, each archive will each become an atlas of its own like other flats archives.

Something else to consider: the super atlas was only used in exterior RMB blocks, it was never used in dungeon RDB blocks. This means it's possible even a single archive might experience overflow. I wouldn't be surprised if archive 211 experiences this as it has a lot of small items. Even in its own atlas, it's practically a super atlas.

But at least now you can control the problem on a per-archive basis, and have substantially more overall leeway per archive set.

Next step might be to allow users to disable texture atlasing where possible. There are a few spots where atlasing is enforced - notably exterior foliage and ground textures. But TheLacus and modders seem to have these quite under control already. I'm a bit worried about performance doing this, but a quick test shows Unity's dynamic batching has come a long way since I started building this code in Unity 4.x days.

One step at a time. :)

Re: Reload/Refresh Textures

Posted: Tue Jan 23, 2018 11:08 am
by King of Worms
Great! I think further testing will tell us more. If the atlas for each set is still 4k Im sure we will make it.

Ad the exterior/interior blocks - I saw atlas overflow in both cases... which is strange. I deffo saw 097 statues overflow outside, and that hanging light in dungeons is IMO from super-atlas as well. BUT, thats just my assumption now. I will test it thoroughly and see whats up after next build.

I think we can keep the atlases for now, its not a huge problem to fit in those. If it improves the performance that is... Agreed on the one step at time attitude ofc.

QUESTION:

Lacus mentioned NPC flats have different handling. But I have not received any specs according to that. I have not witnessed any problems like atlas overflow on them. But it would be fine to know what we are working with here. Here I speak about the basic flats like archive 182 which has like 100 entries, arch 183, 184 etc...

Than, Id like to know, how enemy animated NPCs are handled? Monsters like Rat, Bat etc. Each enemy has cca 40 - 90 frames in a set. Has that set size some limits I should fit in? Same would apply to neutral animated NPCs like town people etc.

Thanks a lot! Good luck in the dev process :geek:

Re: Reload/Refresh Textures

Posted: Tue Jan 23, 2018 3:44 pm
by TheLacus
King of Worms wrote: Than, Id like to know, how enemy animated NPCs are handled? Monsters like Rat, Bat etc. Each enemy has cca 40 - 90 frames in a set. Has that set size some limits I should fit in? Same would apply to neutral animated NPCs like town people etc.
Textures for Interior billboards like static npcs (albedo and emission) are loaded individually so there are no size issues. The same is true for enemies except that they only support albedo and also have animations.

I used them to test an alternative workflow for imported textures while preserving the original behaviour, with performance-saving features like atlas, for vanilla. I think unsing texturearray (or even atlas) for animation frames (instead of the entire archive) may be a future improvement to maintain some performance and support high resolution at the same time.

Outdoor wandering npcs share an atlas with original images, it's only increased up to 4096 as usual.

Re: Reload/Refresh Textures

Posted: Thu Jan 25, 2018 4:28 am
by LorrMaster42
Does Unity have LOD enabled for these sprites? That would help save some memory, right? Just wondering if there is any other optimization to be done.

Re: Reload/Refresh Textures

Posted: Thu Jan 25, 2018 2:37 pm
by King of Worms
Thanks mr Lacus for explanation, much appreciated! ;)
I will keep the high res flats archived in case they can be used in future.

Ad LorrMaster42 - I have not noticed any sprite LOD scaling nor have I idea if such a approach could help free up some resources in this particular case. All I know is sprites are kept in atlases or atlas-clusters to keep the draw calls low. They can occupy quite a bit of memory (everything you have x2 :lol: - in case of my very high res flats ) so every optimalization would be welcomed...

Re: Reload/Refresh Textures

Posted: Thu Jan 25, 2018 10:10 pm
by Interkarma
LorrMaster42 wrote:Does Unity have LOD enabled for these sprites? That would help save some memory, right? Just wondering if there is any other optimization to be done.
There is no LOD for billboards. They are a quad with only two tris and cannot get any simpler.

If you're thinking of textures in the same way, mipmaps are already a system that stores small through large versions of a texture to be displayed based on distance to camera. This doesn't save texture memory though, quite the opposite.

The best way to optimise texture memory is to simply make textures smaller. Commercial game artists are given memory budgets to work within based on design spec. In this case spec is for individual textures to be under 128x128 pixels. You can obviously push things a lot further, but "infinity and beyond!" Should never be the default starting point. :D

Part of the job of a responsible game artist is to find the correct balance between fidelity and conservation of resources. It can be as much of an engineering challenge as programming.

Re: Reload/Refresh Textures

Posted: Fri Jan 26, 2018 5:33 am
by LorrMaster42
Okay, I see now. The individual sprite isn't the problem, its the image's memory space. That adds up a lot with all the NPCs.

One this I was wondering is if you could take a 3D model and add in a shader to make it look like a 2D sprite. Reason being would be to allow for different clothing options on character models and such while still maintaining a similar art style. Would there also be anything to gain development-wise from doing this?

Re: Reload/Refresh Textures

Posted: Sun Jan 28, 2018 5:22 pm
by King of Worms
Im testing the new version, using the imported texture compression. I have not made any direct comparisons on the memory used with on/off settings yet. Game runs better Id say. The visual fidelity of compressed textures is ok, I can see the degradation but its nothing dramatic - no problems there.

But Ive noticed one thing,it happened in the previous version as well but now Ive got it documented better.

I GUESS it has something to do with the cache system. Cache seems to use every MB of the memory left, which might not be a problem. But in case of paging file its excessive. And at the end of the day, the absolute saturation of memory is contraproductive and causes performance drops.

Game uses more paging file than I have allowed on my PC.

Ive never seen this in any other game, even games which massacred my PC like "mass effect" andromeda, which used the full 12288mb of my paging file. But nothing used more than I allocated - ever.

Maybe something to look into when the right time comes? Thanks, and gratulations on a new version!
Swap file.jpg
Swap file.jpg (449.37 KiB) Viewed 3635 times

Re: Reload/Refresh Textures

Posted: Sun Jan 28, 2018 11:24 pm
by Interkarma
I would be interested in an apples-with-apples comparison of memory usage with compression on vs. off if possible. I'm intrigued how much saving is present at larger scale.

I'm not familiar with the tool you're using for overlaid stats here. It doesn't look exactly granular, so can't comment on your actual page file utilisation caused by Daggerfall Unity. I'd recommend using perfmon and setting up custom counters or drilling into Process Explorer by Sysinternals to get more granular information.

Your page file size of 12GB is more or less appropriate for a system with 8GB RAM (general rule of thumb is 1.5x system RAM). Can you confirm this is an 8GB system?

Daggerfall Unity doesn't use your page file directly. Rather, Windows has its own memory handling system and will move data between RAM and page file as needed. This page swapping can cause serious performance issues if system memory is constantly over-utilised (say above 95% at all times).

I'm still thinking about ways to manage cache more effectively to minimise waste and keep load performance high. Expect to see more on this shortly after I change over to 0.5. :)