[MOD] Distant Terrain
- Nystul
- Posts: 1501
- Joined: Mon Mar 23, 2015 8:31 am
Re: [MOD] Distant Terrain
can you tell us the map location (nearest town e.g.) from where that screenshot was captured? thank you
- herman2000
- Posts: 18
- Joined: Sun May 10, 2020 2:08 pm
- Location: Dresden, Saxony, Germany
- Contact:
Re: [MOD] Distant Terrain
Hm. With Distant Terrain's improved Terrain setting, I'm getting some weird results in south-Dak'fron. Fast-Travelling to Lothten Orchard spawns me beneath the Terrain. Going from The Queen's Huntsman's Lodge and running South East for a while, I'll eventually see giant holes in the terrain. With the setting turned off, all is well. Quite bizarre.
Spoiler!
- pango
- Posts: 3359
- Joined: Wed Jul 18, 2018 6:14 pm
- Location: France
- Contact:
Re: [MOD] Distant Terrain
They're known issues going near the southern border indeed
viewtopic.php?f=27&t=10&start=440#p32197
viewtopic.php?f=27&t=10&start=410#p23689
viewtopic.php?f=4&t=37&start=350#p40893
viewtopic.php?f=27&t=10&start=440#p32197
viewtopic.php?f=27&t=10&start=410#p23689
viewtopic.php?f=4&t=37&start=350#p40893
Mastodon: @pango@fosstodon.org
When a measure becomes a target, it ceases to be a good measure.
-- Charles Goodhart
When a measure becomes a target, it ceases to be a good measure.
-- Charles Goodhart
- Nystul
- Posts: 1501
- Joined: Mon Mar 23, 2015 8:31 am
Re: [MOD] Distant Terrain
thanks for investigating - this is indeed an easy fix - thanks for your efforts!Hazelnut wrote: ↑Thu Oct 10, 2019 6:33 pm I took a look and it's not due to the jobs update. I did track down the error to these lines 606-608 of ImprovedTerrainSampler.cs:
The array out of bounds happens when the mapPixelY becomes 500, so when multiplied by 1000 it's 500,000 which is the length of the array. Subtracting 1 from the map width and height values (as shown in the modified code above) stops the OOB but it may be that the actual mapPixelY/X need 1 subtracting for the array. I will leave that to you Nystul as I have no idea what this code is actually doing.Code: Select all
int mapPixelX = Math.Max(0, Math.Min(mx + x - 2, WoodsFile.mapWidthValue -1)); int mapPixelY = Math.Max(0, Math.Min(my + y - 2, WoodsFile.mapHeightValue -1)); waterDistanceMap[x, y] = (float)Math.Sqrt(ImprovedWorldTerrain.MapDistanceSquaredFromWater[mapPixelY * WoodsFile.mapWidthValue + mapPixelX]);
update: there is a new version 2.6.1 with retro rendering mode support and map border fix in first post of this thread
- Nystul
- Posts: 1501
- Joined: Mon Mar 23, 2015 8:31 am
Re: [MOD] Distant Terrain
I found time to improve the heights generation in the terrain sampler so that neighboring map pixels with different climate (and thus different height noise parameters) are blended into each other making the "trench" workaround obsolete.
I will also try to get rid of the Resources.UnloadUnusedAssets() calls - performance is better without (less hickups) - we have to test and wait for feedback if some people experience crashes again but if we need it I plan to implement a mod setting to disable/enable it - but lets try to get along without it
I might have found the issue with the crash on exit as well
link to alpha build "Distant Terrain 2.7.0a" updated on first post of this thread - pls test and report back how it works for you
update: here is a comparison screenshot regarding the trenches (mysterious tree spawned):
- BadLuckBurt
- Posts: 948
- Joined: Sun Nov 05, 2017 8:30 pm
Re: [MOD] Distant Terrain
Looking good, I'm going to sift through the code to see how you did the trench fix. I'm close to finishing my own terrain sampler which has climate merging as well but the code could probably more elegant.
I'm also curious, do you use different types of noise for climates or is it just the scaling settings that are changed?
I ended up on this blog at some point: https://www.redblobgames.com/maps/terra ... #elevation while doing research and incorporated some of those functions into mine to give it more variation.
I'm also curious, do you use different types of noise for climates or is it just the scaling settings that are changed?
I ended up on this blog at some point: https://www.redblobgames.com/maps/terra ... #elevation while doing research and incorporated some of those functions into mine to give it more variation.
DFU on UESP: https://en.uesp.net/w/index.php?title=T ... fall_Unity
DFU Nexus Mods: https://www.nexusmods.com/daggerfallunity
My github repositories with mostly DFU related stuff: https://github.com/BadLuckBurt
.
DFU Nexus Mods: https://www.nexusmods.com/daggerfallunity
My github repositories with mostly DFU related stuff: https://github.com/BadLuckBurt
.
- Nystul
- Posts: 1501
- Joined: Mon Mar 23, 2015 8:31 am
Re: [MOD] Distant Terrain
I preprocess the small height map from daggerfall to make it more realistic (e.g. heights of map pixels depend on distance from water and climate)
baseheight from updated small height map (bicubic interpolation between map pixels) multiplied by a factor (=main height) + noise height from large heightmap (bicubic interpolation again) multiplied by a factor (extra height variation) + small extra noise height (small terrain spikes) multiplied by a factor depended on climate (bilinear interpolation between map pixels).
bilinear interpolation between map pixels for climate does work like this:
there is a height multiplicator for every climate
for every map pixel I compute:
baseheight from updated small height map (bicubic interpolation between map pixels) multiplied by a factor (=main height) + noise height from large heightmap (bicubic interpolation again) multiplied by a factor (extra height variation) + small extra noise height (small terrain spikes) multiplied by a factor depended on climate (bilinear interpolation between map pixels).
bilinear interpolation between map pixels for climate does work like this:
there is a height multiplicator for every climate
for every map pixel I compute:
- a "top left height multiplier" as average between neighbor to the top left, to the top, to the left and current map pixel
- a "top right height multiplier" as average between neighbor to the top, current map pixel, neighbor to the top right, and right
- a "bottom left height multiplier" as average between neighbor to the left, current map pixel, neighbor to the bottom left, and bottom
- a "bottom right height multiplier" as average between current map pixel, neighbor to the right, and bottom, and bottom right
yes, different types of noise for climates - that caused the problems in the first place when 2 neighboring map pixels had different climateBadLuckBurt wrote: ↑Sat May 30, 2020 7:11 pm I'm also curious, do you use different types of noise for climates or is it just the scaling settings that are changed?
cool, really interested in what your terrain sampler will look likeBadLuckBurt wrote: ↑Sat May 30, 2020 7:11 pm I ended up on this blog at some point: https://www.redblobgames.com/maps/terra ... #elevation while doing research and incorporated some of those functions into mine to give it more variation.
- pango
- Posts: 3359
- Joined: Wed Jul 18, 2018 6:14 pm
- Location: France
- Contact:
Re: [MOD] Distant Terrain
Glad to see this mod be worked on again!
So, I tested the alpha,
The good:
Also as I side note, you can try using DaggerfallGC.ThrottledUnloadUnusedAssets() instead of directly calling Resources.UnloadUnusedAssets() so that collection does not happen more than once every 3 minutes. That may be a bit too slow when used with high time compression tedious travel though, but one benefit vs. reimplementing the same logic in each mod is that the timer is shared between all throttled calls, so collections happen the least number of times. Maybe the API should be modified so one can specify the minimum delay between collections? eg. ThrottledUnloadUnusedAssets(int minDelay)
But I really hope forced garbage collection can be avoided altogether... (native arrays pooling?)
So, I tested the alpha,
The good:
- works fine under Linux
- with retro mode support too, yay!
- I checked the world borders, North, Each and South, everything seems fine
- I like the improvements on terrain generation itself
- I had a few cases of "missing towns" with traveling around on horse (without Tedious Travel, just plain moving around), but that's a problem that is known to happen already so it's hard to tell if that mod version really has an impact on that problem.
- I hate to be this guy, but the last problem to report is high memory usage when the mod is stressed, like when using Tedious Travel. As probably expected, it gets worse with high Terrain Distance; While tedious traveling with a Terrain Distance = 4, I've seen memory usage of almost 21GB with no texture mod (then the game just exited, no log message).
Also as I side note, you can try using DaggerfallGC.ThrottledUnloadUnusedAssets() instead of directly calling Resources.UnloadUnusedAssets() so that collection does not happen more than once every 3 minutes. That may be a bit too slow when used with high time compression tedious travel though, but one benefit vs. reimplementing the same logic in each mod is that the timer is shared between all throttled calls, so collections happen the least number of times. Maybe the API should be modified so one can specify the minimum delay between collections? eg. ThrottledUnloadUnusedAssets(int minDelay)
But I really hope forced garbage collection can be avoided altogether... (native arrays pooling?)
Mastodon: @pango@fosstodon.org
When a measure becomes a target, it ceases to be a good measure.
-- Charles Goodhart
When a measure becomes a target, it ceases to be a good measure.
-- Charles Goodhart
- Nystul
- Posts: 1501
- Joined: Mon Mar 23, 2015 8:31 am
Re: [MOD] Distant Terrain
version 2.7.0a does not have any of these - so I now know it was in there for a reason :/
I really need to find out what leaks all this memory. earlier tests showed that even without any mods dfunity leaks memory on fast travel/map pixel changes. But it is less and thus much later problems start to occur. And this might have been already fixed - have to give it another try
Need to learn that profiler feature of unity I guess and take a look.
I always thought GC should clean up all data which is not referenced anywhere anymore - but I think something is not working as intended at least in distant terrain mod. I think some of the terrains are not correctly deleted for some reason.
- pango
- Posts: 3359
- Joined: Wed Jul 18, 2018 6:14 pm
- Location: France
- Contact:
Re: [MOD] Distant Terrain
Well, that's what garbage collectors do eventually for all managed memory.
Unreleased managed memory can come from some dangling reference, the object not released could be in some container somewhere, that is itself considered a "live" object so it's not released. Since calling GC.Collect() (thru Resources.UnloadUnusedAssets()) seems to fix the issue, I guess it's not the case.
Unmanaged memory: Distant Terrain is handling native memory thru NativeArrays, and I suspect jobs themselves need more native memory internally. But they should be all fenced by managed objects that should released the native memory when their Dispose() method is called (which should be automatic) and all should be fine. Again, the fact that calling GC.Collect() keeps memory under control means there should be no bug with native memory handling.
Eventual collection: garbage collection happens concurrently or in parallel to the main program, and how fast it scans memory for objects to reclaim is adjusted using an estimate of how fast garbage is being produced. Since most of the memory used is native memory, the garbage collector can be made aware of allocations outside of managed memory using AddMemoryPressure() and RemoveMemoryPressure(). I wonder if NativeArray does that already, or if it needs to be done explicitly.
Some of this is best guesses, maybe we're hitting a bug in some external code for example...
Indeed the next step is using the profiler (Window > Analysis > Profiler). Doesn't work too well under Linux though, I can't resize its window to begin with
Mastodon: @pango@fosstodon.org
When a measure becomes a target, it ceases to be a good measure.
-- Charles Goodhart
When a measure becomes a target, it ceases to be a good measure.
-- Charles Goodhart