[MOD] Pure Vanilla Extract

[MasonFace]

Thanks my dude! I'll probably incorporate them into PVE!

Sure, Im still completely burdened with the bugs and issues around paperdoll and funny stuff like that.
I remember you wanted some other parts of DREAM, which were they?

Life has been really busy for me lately, so I haven't had much time to devote to DFU mod stuff. From what I recall, I think it was UI and videos. There may have been other things, but it can wait since I don't have time to really implement any of this into PVE for a while.

[MrFlibble]

And since the character flats are mostly single frames, it would be reasonably simple to process them as batched sprite sheets like you generally do.

I'll try to get something done but for now not sure about the timing yet.

In the meantime I realised that my previous method of extracting the sprites from background is somewhat faulty, especially noticeable with the Fatality/Rebout/Lady interpolation because it blurs more than the other models, resulting in the sprite edges getting contaminated with the background colour. The flats are very colourful so it's tough to find a background that would be completely neutral to the colours used in the actual images except black or white.

So I came up with what seems like a better method, however it's also much more time consuming.

1. The idea is to make one upscale against black background and another against pink background with the grayscale version of the sprites, to be used as a mask:

The third image is the original sprites against pink background, we'll also need it later (duh).

For ease of reference, let's call these source images S1, S2 and S3. Use mtPaint to convert each to indexed palette (exact match).

2. Then I apply xBRZ softening (scale up 4x w/xBRZ Scaler Test, then back down to original res w/ Bicubic in GIMP) to S1 and S2 and process them with ESRGAN with Fatality/Rebout/Lady model.

3. With the S2 upscale, apply Kuwahara Nagao blur at 2 pixel radius, convert to the palette of the source image:

4. In GIMP, select the pink background from the result you got in step 2, paste over the S1 upscale, export this image, open in mtPaint and again apply KN blur at 2 pixel radius. Then convert to the indexed palette of S3:

5. Open the result in GIMP, select the pink background and grow selection by 1 pixel. Copy the selected area, then paste over the S1 upscale. You can then re-select the pink background (without growing though so that the copied sprite edges remain) on the S1 upscale and delete it:


On a separate note, I knew before that the background colour or at least brightness affects the results (even in areas not close to the sprite edges), but here it's noticable that with a white background the results are sharper and darker:

[MrFlibble]

Here's another test (sorry for double-posting but my previous post is already large!):

This time I used a dark blue background because pink gives off too much ringing:

I also used Kuwahara Nagao blur at 1 pixel radius with protect details on, and it didn't make a huge difference compared to the 2 pixel radius version.

If you have better ideas on how to extract the upscaled sprites from the background feel free to share, because with scaling up against black produces some real smooth results:

UPD: Nah, I think the KN 2 pixel radius is still better:

[MrFlibble]

I wasn't entirely happy with the blurriness that xBRZ softening produces, and as it turns out, not without reason!

I tried upscaling sprites without softening and realised that a different alpha would produce better handling of edges for non-softened source images. In fact, an interpolation of rebout_interp and 4x_Lady0101_v3_blender at alpha 0.3 gives better results with raw images:



A few more results here:
https://imgur.com/a/PGCMHPg
I'm using the same procedure as described above to get masks, save for the different model and without step 2 (xBRZ softening).

[MrFlibble]

Meanwhile I spent some time to do a more systematic research of what the currently available models can do.

The models that produce good results without pre-processing of the source images are:

• 4x_Fatality_01_265000_G
• 4x_rebout_325k
• rebout_interp
• 4x_Lady0101_v3_340000
• 4x_Lady0101_v3_blender
• 4x_SpongeBob_235000_G

None of these produce acceptable sprite results if used on their own, but can get quite good if interpolated with one another.

Previously I relied mostly on guesswork when interpolating models (using alpha = 0.5 most of the time) but more extensive testing showed that at certain levels (in particular close to the said middle value of 0.5) the models may actually disrupt one another, negating the efficiency with which images are processed. For a quick example, here's the same sprite upscaled with Fatality (left), Rebout Interp (right) and an alpha 0.5 interpolation of the two (centre):

You can clearly see that the priest's staff is a clean diagonal line in all but the centre image where it has become jittery.

With this in mind I conducted further tests using alpha values in the range of 0.1 to 0.3, or, conversely, from 0.7 to 0.9. Note that the alpha values are completely reversible, meaning that an X/Y interpolation at 0.3 is the exact same thing as Y/X interpolation at 0.7. (For clarity in notation, when I say X/Y I mean that in net_interp.py, X is the model in the net_ESRGAN_path and Y in net_PSNR_path).

Here's the same priest sprite with Fatality/Rebout interpolated at 0.3:

Another catch is that each model has its own peculiarities:

• Fatality results are overly sharp and with noticeable ringing
• rebout_interp fuzzies edges and produces colour banding
• Lady is rather blurry

For NPC flats, I mentioned above that the Lady models seem to work good. I picked two sprites that have issues with different models to illustrate the results using alpha = 0.7.

Series I.
4x_Lady0101_v3_340000 interpolated with: 4x_Fatality_01_265000_G, rebout_interp, 4x_SpongeBob_235000_G:

4x_Lady0101_v3_blender interpolated with: 4x_Fatality_01_265000_G, rebout_interp, 4x_SpongeBob_235000_G:

Series II.
4x_Lady0101_v3_340000 interpolated with: 4x_Fatality_01_265000_G, rebout_interp, 4x_SpongeBob_235000_G:

4x_Lady0101_v3_blender interpolated with: 4x_Fatality_01_265000_G, rebout_interp, 4x_SpongeBob_235000_G:


I'd probably say that the 4x_Lady0101_v3_blender interpolations with rebout_interp and 4x_SpongeBob_235000_G are the closest to the original image.

[MrFlibble]

Please forgive multiple posting!

I think I kinda-sorta figured out how to counter the blurriness of Lady interpolated models' output. There's a filter in Image Analyzer (kudos to Jimmy for suggesting this programme!) called Warp sharpen which produces some rather neat results. Here are some side-to-side comparisons:
https://imgsli.com/NzQ2Ng

[King of Worms]

Very interresting read and images. I think the place where this might have good impact on the visuals are Commoners (townfolk roaming the cities) and Id say nature flats, besides those redrawn by Jman0war.

I did the town ppl, but it can be better. Only thing Im 100% content with are the higher res front facing images from each commoners set. They were done with Lady algo and faces were redrawn by Adrinus.

I think the static NPCs are already beyond the level AI can get us honestly. Unless some breakthru is made that is..

[apprihensivsoul]

Bumping, but out of curiousity, is it possible (and if it hasn't been done, what would I be required to do) to make a version of this that has climate-based dungeon textures, like Dream? Keeping the original aesthetic and PVE quality, but maybe just different colour sets or with additional vines or other forms of coverage. This is my favorite graphics mod bar none, but one thing Dream has stronger is that function, especially in a game like Daggerfall where there are so many dungeon dives.

[King of Worms]

Well if you convert absolutely all textures, than the climate setting will work for any mod. You just select it in the game options.

So for this to work the same way DREAM works, it just needs the completion.

At least if I understand your idea correctly, that is

[MasonFace]

. . . a version of this that has climate-based dungeon textures . . .

Yeah, I'm not quite sure what you mean, either. I thought I had all the dungeon textures upscaled, but I most likely don't have the PBR maps generated for them. Could you take a screenshot to demonstrate what you mean?

If its just the PBR maps that aren't generated, then I could use your help making them. It really isn't hard, it's more just time consuming and I haven't had time to mess with it much and got burnt out working on it. KoW knows what I mean, far too well.

Thanks!