Reload/Refresh Textures

Post here if you need help getting started with Daggerfall Unity or just want to clarify a potential bug. Questions about playing or modding classic Daggerfall should be posted to Community.
User avatar
King of Worms
Posts: 4751
Joined: Mon Oct 17, 2016 11:18 pm
Location: Scourg Barrow (CZ)
Contact:

Re: Reload/Refresh Textures

Post by King of Worms »

Hi! I will do the comparison today. Its not exactly trivial as my memory gets saturated with or without texture compression anyway ;)

Tool used is MSI Afterburner, its VERY reliable piece of SW. The numbers it shows were always 100% correct. No other game ever exceeded my page file settings.

Yes, 12gb is ideal for me and I confirm my system is indeed 8gb RAM (4gb VRAM)

The system memory is over utilized, can be, its constantly full, thus it fills the paging file as well. My streaming assets folder has 800mb tho.... I fail to see the reason for these 800mb with compression enabled to saturate 8gb RAM+12gb paging file +4 gb of VRAM memory.

I understand this is just a modding, and I should wait for systems to mature more, and thats what I will do.

Btw I found memory leak:

Start the game, click load, and at the loading screen the memory gets filled by cca 20mb/sec rate and it never ends. It basically saturated all my memory in 10 or 15 minutes.

User avatar
Interkarma
Posts: 7236
Joined: Sun Mar 22, 2015 1:51 am

Re: Reload/Refresh Textures

Post by Interkarma »

King of Worms wrote: Btw I found memory leak:

Start the game, click load, and at the loading screen the memory gets filled by cca 20mb/sec rate and it never ends. It basically saturated all my memory in 10 or 15 minutes.
Thank you, I can reproduce this by selecting up and down through my save games. Just sitting on any particular save doesn't cause this for me however, I need to actively select up and down through list. It's still a problem though, and very likely related. I've added this to my priority list for correction.

I might move this thread over to Help & Support as it's deviated a ways from the Work Requests charter for a few pages now. I haven't forgotten your original request however. This is in my medium priority list for early 0.5. :)

User avatar
Interkarma
Posts: 7236
Joined: Sun Mar 22, 2015 1:51 am

Re: Reload/Refresh Textures

Post by Interkarma »

I've just fixed a pair of memory leaks evident in the save/load window:
  1. TextLabel now destroys old label textures when updating text. This is why scrolling up and down through many items resulted in additional memory usage. This should be a universal fix to all text labels in game. Long scrolling lists were most affected by this bug.
  2. Save screenshot preview panel now destroys old texture when moving between saves.
Not sure if this solves your specific issue on this window, but it should. If you can, please report back when next build drops as it will contain this fix. Cheers. :)

User avatar
King of Worms
Posts: 4751
Joined: Mon Oct 17, 2016 11:18 pm
Location: Scourg Barrow (CZ)
Contact:

Re: Reload/Refresh Textures

Post by King of Worms »

That was fast, I will test it with a new build! In my case, memory leak is going on without moving the cursor. I use modded loading screen backgrounds tho, to confuse things some more :) But thats just a few textures for buttons etc...
New version will tell! And thanks for moving the thread, I agree.

Ive tested compressed/uncompressed
Test was done like this:

Start the game with texture compression ON
Load the game (in Daggerfall city)
Wait cca 5 seconds after load, write the numbers, close the game

Start game, switch to Compression OFF, end game

Start game
Load the same save
Wait 5 sec after load and write the numbers


TEST RESULTS - 1st Run:


COMPRESSION OFF
VRAM....1543 MB
RAM......4635 MB
SWAP....6282 MB
----------------------
TOTAL - 12460 MB

COMPRESSION ON
VRAM.....795 MB - (748 MB Saved)
RAM.....3904 MB - (731 MB Saved)
SWAP....4809 MB - (1473 MB Saved)
-------------------------------------------
TOTAL - 9508 MB - (2952 MB Saved)


So its deffo working. Seems like we saved approximately 50% Vram and 25% of total memory used.
I can imagine the results will vary by the place you are located at, and how many modded vs unmodded textures are there.
In this testing approach Ive tried to minimize the cache settings influence. I know that if I will start to actually play, travel a lot on the map, I will saturate whole memory cache system in both cases. BUT, the test proves the compression works and thats important for now!

I will do another test tomorrow and I will restart the PC in between the tests, so nothing stays in the swap file etc. And I will try some other save/location.

User avatar
Interkarma
Posts: 7236
Joined: Sun Mar 22, 2015 1:51 am

Re: Reload/Refresh Textures

Post by Interkarma »

Thank you for testing, especially for outlining your process. That's a great result for compression, now we need to make caching more efficient. There's also likely to be some other places where texture memory leaks need to be plugged.

It's just going to be a process of finding the problem areas and fixing them. Your spot on the load screen was excellent as it allowed me to find a universal memory leak with TextLabel, not just on the load screen itself.

I'm honestly not sure why you're experiencing a constant climb in the save/load UI. Even with modded textures, there shouldn't be anything happening - it's just idly waiting for input. This means problem is likely a bit deeper in the BaseScreenComponent when using replacement textures, something in the update loop. I'm not able to reproduce this one yet, but will look closer soon.

User avatar
King of Worms
Posts: 4751
Joined: Mon Oct 17, 2016 11:18 pm
Location: Scourg Barrow (CZ)
Contact:

Re: Reload/Refresh Textures

Post by King of Worms »

No problem testing was a pleasure. Just a quick notice. Ive put a "-" before the textures folder, so no modded texture is in the game, and the memory leak is going on like I described before. So it was not caused by modded textures - one variable less. Anyway, I will wait for a new release before I test/report this particular thing again. Im very glad my reports helped detect those leaks! The power of MSI Afterburner .)

User avatar
Interkarma
Posts: 7236
Joined: Sun Mar 22, 2015 1:51 am

Re: Reload/Refresh Textures

Post by Interkarma »

I just popped up some new builds containing fixes for loading screen leaks I was able to reproduce yesterday. Unfortunately, I haven't been able to reproduce your exact issue at this time.

When you have time, please retest and let me know if problem continues. We'll go from there. :)

User avatar
King of Worms
Posts: 4751
Joined: Mon Oct 17, 2016 11:18 pm
Location: Scourg Barrow (CZ)
Contact:

Re: Reload/Refresh Textures

Post by King of Worms »

Glad to report the leak is fixed at the loading screen for me in a new version. Great job! I will keep an eye on things like this in the future.

Im also very surprised that when I loaded the game with everything modded I have, even the upscaled monsters at 100% resolution, game is working great when I have compression enabled. At Privateers hold I hover around 2 gb VRAM, 6gb RAM and 8gb Swap, and that is great. Without the compression, my whole memory system got saturated at load, and that was even when I downscaled the monsters to 50%...

Also, the problem with super-atlas is gone, so everything is being rendered as intended.

I will do more tests and report. But we really getting somewhere ;)

User avatar
Interkarma
Posts: 7236
Joined: Sun Mar 22, 2015 1:51 am

Re: Reload/Refresh Textures

Post by Interkarma »

That's all really great to hear! Thank you for your feedback and patient testing. :)

Hopefully with some tweaks to the cache in future, we'll be able to bring down the remaining memory usage to a more reasonable level.

User avatar
King of Worms
Posts: 4751
Joined: Mon Oct 17, 2016 11:18 pm
Location: Scourg Barrow (CZ)
Contact:

Re: Reload/Refresh Textures

Post by King of Worms »

Was a pleasure! The improvements as it is now are great as well. My mem went from totally saturated to playable with reserves (which get saturated laters on travel, loads etc -> cache)

Made new test and added loading times.
Same test method as before.
We CANT see the whole picture because with compression to OFF my whole mem system is saturated on load in all departments - VRAM, RAM, SWAP - if I had more memory, it would be filled... that 3,6gb VRAM is my max (GTX970)

Its night and day now. That compression phase adds some seconds, but its worth it. Loading should be a bit faster but thats a thing of future optimalization Id say. I use SSD and FX 8300 CPU.
Compare.jpg
Compare.jpg (352.52 KiB) Viewed 3990 times

Post Reply