[Feature request] Posterization effect

Talk about the mods or features you'd like to see in Daggerfall Unity. Give mod creators some ideas!
User avatar
pango
Posts: 2597
Joined: Wed Jul 18, 2018 6:14 pm
Location: France
Contact:

Re: [Feature request] Posterization effect

Post by pango »

Thanks mate :D

I spent most of the day trying to optimize it further, but it's hard! Shaders really don't like branching (and do like unrolling), and the non-trivial k-d tree algorithm is all about decimating a search tree...

The best I managed so far is to increase k-d tree depth by one, going from 8 leaf functions to 16, and consequently to usually test 18 colors instead of 36. But ShaderCompiler takes ages, the Unity Editor complains about some timeout after 600s, the shader eventually compiles anyway and works perfectly. Not sure it's too safe though :shock:
All attempts to speed up compilation resulted in either no change (the optimizing compiler undoing all my modifications) or code that doesn't compile (if I try to handle my own stack, the compiler complains it can't unroll my main loop).
Unless somebody comes with a totally new approach I guess it'll stay as-is... in any case, I'm learning about shaders...
And now I understand why barely anybody tried such insane idea :D (only a few using the naive approach on shadertoy)
Attachments
DaggerfallRetroPalettization-byhand.shader.gz
(3.55 KiB) Downloaded 21 times
When a measure becomes a target, it ceases to be a good measure.
-- Charles Goodhart

Lokkrin Zhataros
Posts: 200
Joined: Thu Nov 21, 2019 9:27 pm

Re: [Feature request] Posterization effect

Post by Lokkrin Zhataros »

I’ve been tinkering with the new post processing settings and comparing it with classic DF in DOS. Got to say that it’s pretty cool, I really like it. I’ll be using it in my gameplay from now on. I’m surprised at how accurate it is compared to classic color rendering. Great job! :)

One question though. I’m curious what the most optimal settings are for trying to simulate classic DF the best when using palletization? When using palletization mode it seems that the colors often appear a fair bit darker or more saturated than in classic. It’s a bit hard on my eyes. It could just be how Unity renders the colors and handles lighting differently than the classic DF engine, but that’s just a guess.

Thanks

User avatar
pango
Posts: 2597
Joined: Wed Jul 18, 2018 6:14 pm
Location: France
Contact:

Re: [Feature request] Posterization effect

Post by pango »

Thanks!

Palettization outputs RGB values taken from classic palette (from ART_PAL.COL to be precise).
So it has to be some color rendering difference after that step (darker/more saturated color is probably related to a higher gamma value).
(it could also be that plain DFU output - before post-processing - is brighter/more saturated than classic Daggerfall output, so palettization still picks color from classic palette, but has to pick more saturated colors. I think that's less likely though, but you can use Shift-F11 to dynamically switch post-processing on/off and check).
When a measure becomes a target, it ceases to be a good measure.
-- Charles Goodhart

Lokkrin Zhataros
Posts: 200
Joined: Thu Nov 21, 2019 9:27 pm

Re: [Feature request] Posterization effect

Post by Lokkrin Zhataros »

Thanks Pango for the information!

I did a bit more experimenting. The screenshots below compares two rooms in Privateers Hold.

With palletization enabled in DFU, when the light is dark it seems to choose very saturated/contrasty colors than classic DF. Maybe you are right about the gamma values being higher? In classic DF you can tell that the light sources illuminates rooms much more and more evenly compared to DFU.

No idea if that makes any sense or helps. I’m no expert. :D
palletization_compare.jpg
palletization_compare.jpg (186.79 KiB) Viewed 603 times

User avatar
pango
Posts: 2597
Joined: Wed Jul 18, 2018 6:14 pm
Location: France
Contact:

Re: [Feature request] Posterization effect

Post by pango »

Are you using default DFU lighting? (All ambient lightings to 1.0, player item based lighting off) It seems DFU uses less ambient lighting in your examples
I don't have DFU at hand to experiment right now, but I should probably do that.

By the way there has been a related discussion before on reddit: https://www.reddit.com/r/daggerfallunit ... nights_of/ (on brightness alone)
When a measure becomes a target, it ceases to be a good measure.
-- Charles Goodhart

Lokkrin Zhataros
Posts: 200
Joined: Thu Nov 21, 2019 9:27 pm

Re: [Feature request] Posterization effect

Post by Lokkrin Zhataros »

I used all default lighting, no mods, ambient light 1.0, item based player torch is off in these examples. When using any settings that lower the light or the use of Improved Interior Lighting mod only makes the colors even more saturated, so I didn't use them.

Gathering from all this, it seems that DFU uses lighting somewhat differently than classic DF. In DFU the lighting is often darker in certain places with less ambient lighting, therefore the palletization may be forced to use darker/saturated tones. This is just what I gathered from your comments here and on reddit.

Here are some more examples I put together below.

1) Palletization picks darker/saturated colors than in classic.
2) An example of DFU lighting being brighter in some interiors than in classic.
3) Palace lighting and colors seem to be rendered mostly accurate and consistent in all examples.
4) At night, another example of palletization picking darker/saturated colors than in classic.

palletization_compare02.jpg
palletization_compare02.jpg (294.61 KiB) Viewed 551 times

User avatar
MasonFace
Posts: 512
Joined: Tue Nov 27, 2018 7:28 pm
Location: Tennessee, USA
Contact:

Re: [Feature request] Posterization effect

Post by MasonFace »

I wonder how linear vs gamma lighting will effect this.

User avatar
pango
Posts: 2597
Joined: Wed Jul 18, 2018 6:14 pm
Location: France
Contact:

Re: [Feature request] Posterization effect

Post by pango »

We'll see, palettization does nothing about color balance, it just rounds each pixel's color to the closest one found in a palette, so balance may have to be fixed elsewhere.

On that topic, in 0.10.24 I've added 3 very dark grey values to the classic palette, that had no value between (0,0,0) and (15,15,15). That should help a bit with all low light condition, and as you mentioned they're a lot more of those in DFU than in classic. For example it should help at dawn and dusk times.
It makes the dungeons slightly less claustrophobic, but I think it's globally a good move:
darkness before.jpg
darkness before.jpg (133.76 KiB) Viewed 444 times
darkness after.jpg
darkness after.jpg (134.15 KiB) Viewed 444 times
When a measure becomes a target, it ceases to be a good measure.
-- Charles Goodhart

Lokkrin Zhataros
Posts: 200
Joined: Thu Nov 21, 2019 9:27 pm

Re: [Feature request] Posterization effect

Post by Lokkrin Zhataros »

Adding the extra gray colors brings interesting results in low light. It's looks good! :) Yeah, it doesn't make the scene feel as dark and claustrophobic as it previously did, but this is a good in-between look that's nice.

The previous issue with the dark/saturated colors I don't think had anything to do with the blacks/grays though. It's still picking weird over-saturated colors in certain lighting. I understand that it may not be possible to fix the issue with the colors. For me it's not a big deal and I'll still keep using palletization.

Thanks Pango for the update and improvements! :)

Lokkrin Zhataros
Posts: 200
Joined: Thu Nov 21, 2019 9:27 pm

Re: [Feature request] Posterization effect

Post by Lokkrin Zhataros »

Whoops, ignore my last statement about colors. Whatever you adjusted this time actually improved the color rendering a fair bit. It's pretty close to classic daggerfall look. It looks awesome!! :D

Palletization is best used with 'item based player torch' enabled, with player torch light at 0.5 and a bit of Ambient occlusion with Post Processing mod.

Thanks again Pango for your work! :)

Post Reply