Switching DREAM to Linear build
- King of Worms
- Posts: 4753
- Joined: Mon Oct 17, 2016 11:18 pm
- Location: Scourg Barrow (CZ)
- Contact:
Switching DREAM to Linear build
I moved to Unity 2019 new Unity build using Linear branch and DFU 0.13 linear build
At the transition it Imported all the DREAM assets, it took 2 hours of 100% 12 thread CPU usage.
Why if I build a 1st mod in this unity version, its doing the 2 hour import again?
This was always the case, but I was too lazy to write this report.
To top this, when I will build for Linux, it will be another 2 hour import.
And when I will build for OsX it will be another 2 hour import.
Only THAN all imports will be done and I will never have to import again, until I change Unity version.
Thats 8 hours of import and torture of CPU in total.
Is that really necessary?
At the transition it Imported all the DREAM assets, it took 2 hours of 100% 12 thread CPU usage.
Why if I build a 1st mod in this unity version, its doing the 2 hour import again?
This was always the case, but I was too lazy to write this report.
To top this, when I will build for Linux, it will be another 2 hour import.
And when I will build for OsX it will be another 2 hour import.
Only THAN all imports will be done and I will never have to import again, until I change Unity version.
Thats 8 hours of import and torture of CPU in total.
Is that really necessary?
Last edited by King of Worms on Tue Aug 17, 2021 7:45 pm, edited 3 times in total.
DREAM Nexus https://www.nexusmods.com/users/73250038?tab=user+files
DREAM Forums viewtopic.php?f=27&t=1168
DISCORD: king_of_worms_cz
DREAM Forums viewtopic.php?f=27&t=1168
DISCORD: king_of_worms_cz
- Interkarma
- Posts: 7243
- Joined: Sun Mar 22, 2015 1:51 am
Re: Mod Builder - Importing all assets takes hours for each OS
I don't know, but I'll try to get the ball rolling.
If you've set up a fresh project for each upgrade and copied over assets, this will trigger a full asset cache rebuild. If you upgrade project in place (e.g. sync changes from git and open in new version), I think this will be faster overall as it doesn't need a full cache rebuild. You've only recently started using github to pull down changes, so you might not have had much benefit from the asset cache until after this round.
I'll admit to not knowing a lot about how Unity handles asset cache. DFU basically has its own asset pipeline loading from DOS gamedata. It's not something I've had to struggle with much. I only know other games need to manage this a lot more.
About to sign out for the evening here. Will loop back when I can if nobody can offer something more concrete.
If you've set up a fresh project for each upgrade and copied over assets, this will trigger a full asset cache rebuild. If you upgrade project in place (e.g. sync changes from git and open in new version), I think this will be faster overall as it doesn't need a full cache rebuild. You've only recently started using github to pull down changes, so you might not have had much benefit from the asset cache until after this round.
I'll admit to not knowing a lot about how Unity handles asset cache. DFU basically has its own asset pipeline loading from DOS gamedata. It's not something I've had to struggle with much. I only know other games need to manage this a lot more.
About to sign out for the evening here. Will loop back when I can if nobody can offer something more concrete.
- King of Worms
- Posts: 4753
- Joined: Mon Oct 17, 2016 11:18 pm
- Location: Scourg Barrow (CZ)
- Contact:
Re: Mod Builder - Importing all assets takes hours / Builder Crash
yeah so I clickled on the Git Desktop to switch to Master. Than back to Linear and all mod files are gone.
It deleted everything and created clean Linear branch without any changes Ive did...
Thanks god I kept the old install with all the files, because this click just deleted whole freakin dream without a question prompt or anything. Moving away from Git desktop, that thing is dangerous for newb - it really deleted all my work.
I will just place the linear branch over the 2019 install and hope it works after I switch to linear in the editor and setup all the textures again...
EDIT: ok so it works
What I see is there will be a lots of work needed It looks good at some places, not so good at others. That was expected. I can see this new Linear space is more accurate and displays things in more real manner, and that brings up the stuff which was not seen in gamma, good or bad...
Do you think the Improved light mod https://www.nexusmods.com/daggerfallunity/mods/144 will need a rebuild or it works as it is in 0.13? To me it seems it works as intended, but the light color looks very different in Linear and that changes whole img. Not sure if I should tweak the mod settings (light colors and strength) or wait for a rebuilt.
Anyway, took some promising screens at places where it worked ok-ish. And that indeed is improvement from the past
It deleted everything and created clean Linear branch without any changes Ive did...
Thanks god I kept the old install with all the files, because this click just deleted whole freakin dream without a question prompt or anything. Moving away from Git desktop, that thing is dangerous for newb - it really deleted all my work.
I will just place the linear branch over the 2019 install and hope it works after I switch to linear in the editor and setup all the textures again...
EDIT: ok so it works
What I see is there will be a lots of work needed It looks good at some places, not so good at others. That was expected. I can see this new Linear space is more accurate and displays things in more real manner, and that brings up the stuff which was not seen in gamma, good or bad...
Do you think the Improved light mod https://www.nexusmods.com/daggerfallunity/mods/144 will need a rebuild or it works as it is in 0.13? To me it seems it works as intended, but the light color looks very different in Linear and that changes whole img. Not sure if I should tweak the mod settings (light colors and strength) or wait for a rebuilt.
Anyway, took some promising screens at places where it worked ok-ish. And that indeed is improvement from the past
DREAM Nexus https://www.nexusmods.com/users/73250038?tab=user+files
DREAM Forums viewtopic.php?f=27&t=1168
DISCORD: king_of_worms_cz
DREAM Forums viewtopic.php?f=27&t=1168
DISCORD: king_of_worms_cz
- Interkarma
- Posts: 7243
- Joined: Sun Mar 22, 2015 1:51 am
Re: Switching DREAM to Linear build
Dang these look good! It will all come together soon, especially once all the other graphical mods are tuned/rebuilt where needed also. I'll wrap up my side and get some public tests out next week. Have a small patch to 0.12 coming this week.
At the end of the day, it's your project and you need to manage it in a way you feel comfortable and in control of the process. I support you whatever that is. Just make sure you keep plenty of backups (3-2-1 is your friend!) it would be a sad day for DFU community if something ever happened to DREAM.
Anyway, I'm super happy to see you making great progress and in charge. I'm really looking forward to seeing this all come together with other updated mods.
I don't know what happened there. If your mod assets are in a folder below Assets/Game/Mods, this path has a .gitignore set and isn't tracked by github desktop at all. I have a bunch of stuff here that never gets touched and I teleport around different branches and PRs all the time. Even if the changes are tracked elsewhere in project, it gives you a warning when switching branches to either stash the change or bring them with you to new branch.King of Worms wrote: ↑Tue Aug 17, 2021 4:20 pm Thanks god I kept the old install with all the files, because this click just deleted whole freakin dream without a question prompt or anything. Moving away from Git desktop, that thing is dangerous for newb - it really deleted all my work.
At the end of the day, it's your project and you need to manage it in a way you feel comfortable and in control of the process. I support you whatever that is. Just make sure you keep plenty of backups (3-2-1 is your friend!) it would be a sad day for DFU community if something ever happened to DREAM.
Ideally every graphics .dfmod involving materials, lights, or shaders will do a tuning pass and rebuild for new version. Some might need it need less than others, but they'll probably want to touch things up based on how linear lights differently. Worst-case, a mod will look outright wrong until author rebuilds for 0.13 versions.King of Worms wrote: ↑Tue Aug 17, 2021 4:20 pm Do you think the Improved light mod https://www.nexusmods.com/daggerfallunity/mods/144 will need a rebuild or it works as it is in 0.13? To me it seems it works as intended, but the light color looks very different in Linear and that changes whole img. Not sure if I should tweak the mod settings (light colors and strength) or wait for a rebuilt.
Anyway, I'm super happy to see you making great progress and in charge. I'm really looking forward to seeing this all come together with other updated mods.
- King of Worms
- Posts: 4753
- Joined: Mon Oct 17, 2016 11:18 pm
- Location: Scourg Barrow (CZ)
- Contact:
Re: Switching DREAM to Linear build
Hi Yup I will wait for others to catch up, dont want to tweak anything, if a light mod is gonna change...
Git Desktop deleted all my Dreams because they were located "...Assets\Game\Addons\My Mod" - have nothing nice to say about that SW and its gone from my PC .)
Anyway, thanks a lot for support, its up and running ... Honestly only mod I wait for it the light mod... I have a feeling it works as expected tho, just the light color look different in linear... who knows.
Git Desktop deleted all my Dreams because they were located "...Assets\Game\Addons\My Mod" - have nothing nice to say about that SW and its gone from my PC .)
Anyway, thanks a lot for support, its up and running ... Honestly only mod I wait for it the light mod... I have a feeling it works as expected tho, just the light color look different in linear... who knows.
DREAM Nexus https://www.nexusmods.com/users/73250038?tab=user+files
DREAM Forums viewtopic.php?f=27&t=1168
DISCORD: king_of_worms_cz
DREAM Forums viewtopic.php?f=27&t=1168
DISCORD: king_of_worms_cz
- Hazelnut
- Posts: 3016
- Joined: Sat Aug 26, 2017 2:46 pm
- Contact:
Re: Switching DREAM to Linear build
I usually link my mods repo clones into Assets/Game/Mods/modname as they're in a different github repo to DFU (obviously) but I think the issue is where you had your mod. Addons is part of the DFU repo and is not ignored by git like the Mods folder is. So you should not blame github desktop as it did exactly what it was told to do. Move where you put your mod to the correct place and it should not happen again.King of Worms wrote: ↑Wed Aug 18, 2021 12:07 pm Hi Yup I will wait for others to catch up, dont want to tweak anything, if a light mod is gonna change...
Git Desktop deleted all my Dreams because they were located "...Assets\Game\Addons\My Mod" - have nothing nice to say about that SW and its gone from my PC .)
Anyway, thanks a lot for support, its up and running ... Honestly only mod I wait for it the light mod... I have a feeling it works as expected tho, just the light color look different in linear... who knows.
Let me know if you want help with linking a separate local clone of dream into this location. This saves on copying files and allows you to manage the separate repos (DFU + Dream) separately nice and easy.
See my mod code for examples of how to change various aspects of DFU: https://github.com/ajrb/dfunity-mods
- King of Worms
- Posts: 4753
- Joined: Mon Oct 17, 2016 11:18 pm
- Location: Scourg Barrow (CZ)
- Contact:
Re: Switching DREAM to Linear build
Thanks for offer honestly but nah, I removed that prog, never needed it in 5 years here, so why start at the end of the journey. This brief meeting with Git Wrecked, I mean Git Desktop, was enough for me. Our paths have parted...
DREAM Nexus https://www.nexusmods.com/users/73250038?tab=user+files
DREAM Forums viewtopic.php?f=27&t=1168
DISCORD: king_of_worms_cz
DREAM Forums viewtopic.php?f=27&t=1168
DISCORD: king_of_worms_cz
- King of Worms
- Posts: 4753
- Joined: Mon Oct 17, 2016 11:18 pm
- Location: Scourg Barrow (CZ)
- Contact:
Re: Mod Builder - Importing all assets takes hours for each OS
Interkarma, while you at it, having a look at underwater fog would be great.
I think it makes the water way too bright (no matter the color space, it was always like that) The underwater sections are in the dark dungeons, so that stark change from dark dungeon to bright blue underwater is something which should be IMO either tweaked for all users - or if you want to keep it - than opened for modding?
2nd question is about the Dungeons in Linear. I have a impression the Linear pushed the dungeon fog much further? So if Im in a long hall, the lights at distance are quite bright. But that can be caused by other variables as well I guess :X
I think it makes the water way too bright (no matter the color space, it was always like that) The underwater sections are in the dark dungeons, so that stark change from dark dungeon to bright blue underwater is something which should be IMO either tweaked for all users - or if you want to keep it - than opened for modding?
2nd question is about the Dungeons in Linear. I have a impression the Linear pushed the dungeon fog much further? So if Im in a long hall, the lights at distance are quite bright. But that can be caused by other variables as well I guess :X
DREAM Nexus https://www.nexusmods.com/users/73250038?tab=user+files
DREAM Forums viewtopic.php?f=27&t=1168
DISCORD: king_of_worms_cz
DREAM Forums viewtopic.php?f=27&t=1168
DISCORD: king_of_worms_cz
- Interkarma
- Posts: 7243
- Joined: Sun Mar 22, 2015 1:51 am
Re: Switching DREAM to Linear build
The underwater fog is intended to look very similar between classic and DFU. I don't have any plans to change much, but it could be definitely open to modders. On a personal level, I agree with what you're saying and would never implement underwater like this in my own game.King of Worms wrote: ↑Thu Aug 19, 2021 3:52 pm Interkarma, while you at it, having a look at underwater fog would be great.
Edit: Had a quick look and the underwater fog is managed by a class called UnderwaterFog in PlayerEnterExit. This could be exposed to modders so they can change the colour. I need to fix some fog code anyway as it's incompatible with the new postprocessing stack, I can make underwater fog colour public at the same time.
I haven't changed dungeon fog, but I can definitely take a look. The effect you describe might be down to how linear falloff looks brighter towards the edges of light radius.King of Worms wrote: ↑Thu Aug 19, 2021 3:52 pm 2nd question is about the Dungeons in Linear. I have a impression the Linear pushed the dungeon fog much further? So if Im in a long hall, the lights at distance are quite bright. But that can be caused by other variables as well I guess :X
- King of Worms
- Posts: 4753
- Joined: Mon Oct 17, 2016 11:18 pm
- Location: Scourg Barrow (CZ)
- Contact:
Re: Switching DREAM to Linear build
Yup I understand you will keep the underwater fog vanilla, making the setting public will be great, thank you
Agree that the distant light change is most likely linked to different light behavior under linear... I will keep on tweaking the materials/light sources and see what can be done about the looks.
Agree that the distant light change is most likely linked to different light behavior under linear... I will keep on tweaking the materials/light sources and see what can be done about the looks.
DREAM Nexus https://www.nexusmods.com/users/73250038?tab=user+files
DREAM Forums viewtopic.php?f=27&t=1168
DISCORD: king_of_worms_cz
DREAM Forums viewtopic.php?f=27&t=1168
DISCORD: king_of_worms_cz