Switching DREAM to Linear build

Discuss Daggerfall Unity and Daggerfall Tools for Unity.
Post Reply
User avatar
King of Worms
Posts: 4215
Joined: Mon Oct 17, 2016 11:18 pm
Location: Scourg Barrow (CZ)
Contact:

Switching DREAM to Linear build

Post by King of Worms »

I moved to Unity 2019 new Unity build using Linear branch and DFU 0.13 linear build

At the transition it Imported all the DREAM assets, it took 2 hours of 100% 12 thread CPU usage.
Why if I build a 1st mod in this unity version, its doing the 2 hour import again?
This was always the case, but I was too lazy to write this report.

To top this, when I will build for Linux, it will be another 2 hour import.
And when I will build for OsX it will be another 2 hour import.
Only THAN all imports will be done and I will never have to import again, until I change Unity version.

Thats 8 hours of import and torture of CPU in total.

Is that really necessary?
Last edited by King of Worms on Tue Aug 17, 2021 7:45 pm, edited 3 times in total.

User avatar
Interkarma
Posts: 6527
Joined: Sun Mar 22, 2015 1:51 am

Re: Mod Builder - Importing all assets takes hours for each OS

Post by Interkarma »

I don't know, but I'll try to get the ball rolling. :)

If you've set up a fresh project for each upgrade and copied over assets, this will trigger a full asset cache rebuild. If you upgrade project in place (e.g. sync changes from git and open in new version), I think this will be faster overall as it doesn't need a full cache rebuild. You've only recently started using github to pull down changes, so you might not have had much benefit from the asset cache until after this round.

I'll admit to not knowing a lot about how Unity handles asset cache. DFU basically has its own asset pipeline loading from DOS gamedata. It's not something I've had to struggle with much. I only know other games need to manage this a lot more.

About to sign out for the evening here. Will loop back when I can if nobody can offer something more concrete.

User avatar
King of Worms
Posts: 4215
Joined: Mon Oct 17, 2016 11:18 pm
Location: Scourg Barrow (CZ)
Contact:

Re: Mod Builder - Importing all assets takes hours / Builder Crash

Post by King of Worms »

yeah so I clickled on the Git Desktop to switch to Master. Than back to Linear and all mod files are gone.

It deleted everything and created clean Linear branch without any changes Ive did...
Thanks god I kept the old install with all the files, because this click just deleted whole freakin dream without a question prompt or anything. Moving away from Git desktop, that thing is dangerous for newb - it really deleted all my work.

I will just place the linear branch over the 2019 install and hope it works after I switch to linear in the editor and setup all the textures again...

EDIT: ok so it works :idea:

What I see is there will be a lots of work needed :D It looks good at some places, not so good at others. That was expected. I can see this new Linear space is more accurate and displays things in more real manner, and that brings up the stuff which was not seen in gamma, good or bad...

Do you think the Improved light mod https://www.nexusmods.com/daggerfallunity/mods/144 will need a rebuild or it works as it is in 0.13? To me it seems it works as intended, but the light color looks very different in Linear and that changes whole img. Not sure if I should tweak the mod settings (light colors and strength) or wait for a rebuilt.

Anyway, took some promising screens at places where it worked ok-ish. And that indeed is improvement from the past ;)
DaggerfallUnity 2021-08-17 21-30-33.jpg
DaggerfallUnity 2021-08-17 21-30-33.jpg (381.17 KiB) Viewed 703 times
DaggerfallUnity 2021-08-17 21-36-38.jpg
DaggerfallUnity 2021-08-17 21-36-38.jpg (293.77 KiB) Viewed 703 times
DaggerfallUnity 2021-08-17 21-27-39.jpg
DaggerfallUnity 2021-08-17 21-27-39.jpg (414.92 KiB) Viewed 703 times

User avatar
Interkarma
Posts: 6527
Joined: Sun Mar 22, 2015 1:51 am

Re: Switching DREAM to Linear build

Post by Interkarma »

Dang these look good! :D It will all come together soon, especially once all the other graphical mods are tuned/rebuilt where needed also. I'll wrap up my side and get some public tests out next week. Have a small patch to 0.12 coming this week.

King of Worms wrote: Tue Aug 17, 2021 4:20 pm Thanks god I kept the old install with all the files, because this click just deleted whole freakin dream without a question prompt or anything. Moving away from Git desktop, that thing is dangerous for newb - it really deleted all my work.
I don't know what happened there. :( If your mod assets are in a folder below Assets/Game/Mods, this path has a .gitignore set and isn't tracked by github desktop at all. I have a bunch of stuff here that never gets touched and I teleport around different branches and PRs all the time. Even if the changes are tracked elsewhere in project, it gives you a warning when switching branches to either stash the change or bring them with you to new branch.

At the end of the day, it's your project and you need to manage it in a way you feel comfortable and in control of the process. I support you whatever that is. Just make sure you keep plenty of backups (3-2-1 is your friend!) it would be a sad day for DFU community if something ever happened to DREAM. :)

King of Worms wrote: Tue Aug 17, 2021 4:20 pm Do you think the Improved light mod https://www.nexusmods.com/daggerfallunity/mods/144 will need a rebuild or it works as it is in 0.13? To me it seems it works as intended, but the light color looks very different in Linear and that changes whole img. Not sure if I should tweak the mod settings (light colors and strength) or wait for a rebuilt.
Ideally every graphics .dfmod involving materials, lights, or shaders will do a tuning pass and rebuild for new version. Some might need it need less than others, but they'll probably want to touch things up based on how linear lights differently. Worst-case, a mod will look outright wrong until author rebuilds for 0.13 versions.


Anyway, I'm super happy to see you making great progress and in charge. I'm really looking forward to seeing this all come together with other updated mods. :D

User avatar
King of Worms
Posts: 4215
Joined: Mon Oct 17, 2016 11:18 pm
Location: Scourg Barrow (CZ)
Contact:

Re: Switching DREAM to Linear build

Post by King of Worms »

Hi :) Yup I will wait for others to catch up, dont want to tweak anything, if a light mod is gonna change...

Git Desktop deleted all my Dreams because they were located "...Assets\Game\Addons\My Mod" - have nothing nice to say about that SW and its gone from my PC .)

Anyway, thanks a lot for support, its up and running ;) ... Honestly only mod I wait for it the light mod... I have a feeling it works as expected tho, just the light color look different in linear... who knows.

User avatar
Hazelnut
Posts: 2867
Joined: Sat Aug 26, 2017 2:46 pm
Contact:

Re: Switching DREAM to Linear build

Post by Hazelnut »

King of Worms wrote: Wed Aug 18, 2021 12:07 pm Hi :) Yup I will wait for others to catch up, dont want to tweak anything, if a light mod is gonna change...

Git Desktop deleted all my Dreams because they were located "...Assets\Game\Addons\My Mod" - have nothing nice to say about that SW and its gone from my PC .)

Anyway, thanks a lot for support, its up and running ;) ... Honestly only mod I wait for it the light mod... I have a feeling it works as expected tho, just the light color look different in linear... who knows.
I usually link my mods repo clones into Assets/Game/Mods/modname as they're in a different github repo to DFU (obviously) but I think the issue is where you had your mod. Addons is part of the DFU repo and is not ignored by git like the Mods folder is. So you should not blame github desktop as it did exactly what it was told to do. Move where you put your mod to the correct place and it should not happen again.

Let me know if you want help with linking a separate local clone of dream into this location. This saves on copying files and allows you to manage the separate repos (DFU + Dream) separately nice and easy.
See my mod code for examples of how to change various aspects of DFU: https://github.com/ajrb/dfunity-mods

User avatar
King of Worms
Posts: 4215
Joined: Mon Oct 17, 2016 11:18 pm
Location: Scourg Barrow (CZ)
Contact:

Re: Switching DREAM to Linear build

Post by King of Worms »

Thanks for offer honestly but nah, I removed that prog, never needed it in 5 years here, so why start at the end of the journey. This brief meeting with Git Wrecked, I mean Git Desktop, was enough for me. Our paths have parted...

User avatar
King of Worms
Posts: 4215
Joined: Mon Oct 17, 2016 11:18 pm
Location: Scourg Barrow (CZ)
Contact:

Re: Mod Builder - Importing all assets takes hours for each OS

Post by King of Worms »

Interkarma, while you at it, having a look at underwater fog would be great.

I think it makes the water way too bright (no matter the color space, it was always like that) The underwater sections are in the dark dungeons, so that stark change from dark dungeon to bright blue underwater is something which should be IMO either tweaked for all users - or if you want to keep it - than opened for modding?

2nd question is about the Dungeons in Linear. I have a impression the Linear pushed the dungeon fog much further? So if Im in a long hall, the lights at distance are quite bright. But that can be caused by other variables as well I guess :X

User avatar
Interkarma
Posts: 6527
Joined: Sun Mar 22, 2015 1:51 am

Re: Switching DREAM to Linear build

Post by Interkarma »

King of Worms wrote: Thu Aug 19, 2021 3:52 pm Interkarma, while you at it, having a look at underwater fog would be great.
The underwater fog is intended to look very similar between classic and DFU. I don't have any plans to change much, but it could be definitely open to modders. On a personal level, I agree with what you're saying and would never implement underwater like this in my own game. :)

Edit: Had a quick look and the underwater fog is managed by a class called UnderwaterFog in PlayerEnterExit. This could be exposed to modders so they can change the colour. I need to fix some fog code anyway as it's incompatible with the new postprocessing stack, I can make underwater fog colour public at the same time.

King of Worms wrote: Thu Aug 19, 2021 3:52 pm 2nd question is about the Dungeons in Linear. I have a impression the Linear pushed the dungeon fog much further? So if Im in a long hall, the lights at distance are quite bright. But that can be caused by other variables as well I guess :X
I haven't changed dungeon fog, but I can definitely take a look. The effect you describe might be down to how linear falloff looks brighter towards the edges of light radius.

User avatar
King of Worms
Posts: 4215
Joined: Mon Oct 17, 2016 11:18 pm
Location: Scourg Barrow (CZ)
Contact:

Re: Switching DREAM to Linear build

Post by King of Worms »

Yup I understand you will keep the underwater fog vanilla, making the setting public will be great, thank you :)
Agree that the distant light change is most likely linked to different light behavior under linear... I will keep on tweaking the materials/light sources and see what can be done about the looks.

Post Reply