AI Upscaled Textures

Show off your mod creations or just a work in progress.
Post Reply
User avatar
King of Worms
Posts: 4752
Joined: Mon Oct 17, 2016 11:18 pm
Location: Scourg Barrow (CZ)
Contact:

Re: AI Upscaled Textures

Post by King of Worms »

Thanks for upscale MasonFace, I think I can make the arrows work by combining the good parts from them and than just rotating the result. Anyway, if I ever need any more upscales, I will make sure to just batch everything together so its easier for you to do.
Thanks again :)

User avatar
MasonFace
Posts: 543
Joined: Tue Nov 27, 2018 7:28 pm
Location: Tennessee, USA
Contact:

Re: AI Upscaled Textures

Post by MasonFace »

I antialias the image, followed by upscaling to create the mask, at the end I apply a threshold effect of 127 to make the mask binary.
You said before that you aren't using a neural network to upscale the masking layer, so are you using interpolation to upscale it? If so, which interpolation model? I'm using GIMP, so I may not have the same interpolation models; all of them produce poor results for upscaling the anti-aliased masking layer. I've even tried using AA, then upscaling 150%, running AA again, upscaling 150%, running AA again, . . . until I reach the target resolution, but it just looks bad.
BTW, could someone put together an archive with all the images that need masks in the same folder? Currently I have them in subfolders, so I can't just create a single batch to run through them all.
Depending on what operating system you're on, you should be able to search the parent folder that holds all of the subfolders for "*.png". This fuzzy search should return all the images which you could cut and paste into one consolidated folder. The problem with this is that you'll probably have multiple images with the same name... I don't think I have all the original images consolidated with the proper name formatting at the moment. It would take a little while for me to compile. I might could write a batch script to find the archive number from the folder name, then prepend the file name of each image in each folder, then pull them all out into one master folder.

However, I have refined my masking method further and I have had consistently good results on a small batch of samples. I don't know yet if it will work well in every situation, but so far the results look promising. As such, I may not need much more help with the masking. I really appreciate your help up to this point! If it turns out that I need more support with the masks, I'll make sure I have all the images consolidated so it will be much less tedious for you to batch. :)

User avatar
Jukic
Posts: 68
Joined: Sun May 13, 2018 5:51 pm

Re: AI Upscaled Textures

Post by Jukic »

@MasonFace:

I am swamped with work and reallife obligations but I am reading and will return!, if You dont finish yourself that is ....! :geek:

phredreeke
Posts: 71
Joined: Mon Mar 11, 2019 9:13 am

Re: AI Upscaled Textures

Post by phredreeke »

MasonFace wrote: Tue Mar 12, 2019 2:48 pm You said before that you aren't using a neural network to upscale the masking layer, so are you using interpolation to upscale it? If so, which interpolation model? I'm using GIMP, so I may not have the same interpolation models; all of them produce poor results for upscaling the anti-aliased masking layer. I've even tried using AA, then upscaling 150%, running AA again, upscaling 150%, running AA again, . . . until I reach the target resolution, but it just looks bad.
I use SmartSize, which to my understanding uses Bicubic when upscaling and Bilinear when downscaling. I only upscale at integer sizes (actually that's a slight lie with the latest mask, as I omitted 2 pixels on each axis for the last upscale, which I then pad back to the intended size)

For antialiasing I actually create a separate layer (I use the color selection tool then use shape-based antialias in PSP, and fill a layer on the inside setting and another layer on the outside setting then take the difference between those two) and use as a mask to soften the underlaying image (the actual mask for the upscaled image).

User avatar
MrFlibble
Posts: 410
Joined: Sat Jan 27, 2018 10:43 am

Re: AI Upscaled Textures

Post by MrFlibble »

All this talk about anti-aliasing prompted me to try out a very simple trick.

Back when I started trying to get waifu2x to properly process sprite edges and came up with the xBRZ softening method, the first thing I tried was simple Gaussian blur applied to the original image (at a radius of 0.5-0.7 pixels). This more or less worked but the result was, well, blurry.

Now I thought, why not scale up the image with xBRZ to 4x, and then apply Gaussian blur? I did that, and then scaled the image down with Bicubic interpolation. I tried radius of 2 and 3 pixels (since Bicubic downscaling sharpens the image, by my estimate a radius of 2 pixels ends up to the effect of a 0.5 pixel blur, and 3 pixels results in 1.5 pixel blur radius), with the latter being too blurry but the former working rather well.

I tried this image and scaled it with ESRGAN*Manga109 at alpha = 0.5 and with pure Manga109. Each image was then converted to the original palette with mtPaint, and then I used GIMP's select by colour tool to select the blue background and remove it. Here are the results:

ESRGAN/Manga109 interpolation:
Image

Pure Manga109:
Image

Blend of both (using the G'MIC Blend [median] function)
Image

I think the blended version is a pretty good compromise between the two.

User avatar
King of Worms
Posts: 4752
Joined: Mon Oct 17, 2016 11:18 pm
Location: Scourg Barrow (CZ)
Contact:

Re: AI Upscaled Textures

Post by King of Worms »

Interresting, 1st img is very sharp, 2nd quite blurry, 3rd is ideal... I think those are nice results.

User avatar
MrFlibble
Posts: 410
Joined: Sat Jan 27, 2018 10:43 am

Re: AI Upscaled Textures

Post by MrFlibble »

It's still not perfect because either network messes up small details... Like the golden skull ornament on the King of Worms' entourage's garments, it's become completely unrecognisable, and the witch cover guard's facial features are all messed up (which is not the case with the original low-res sprite; this is how ESRGAN generalised the details).

Here's another try, with a group of monsters (again, it's an official preview image), blended like No. 3 above:
Image
It's mostly rather good I think.

User avatar
MrFlibble
Posts: 410
Joined: Sat Jan 27, 2018 10:43 am

Re: AI Upscaled Textures

Post by MrFlibble »

Here's the result of blending three layers: the first two shown above (ESRGAN/Manga109 and pure Manga109) and the same image simply scaled up with xBRZ:
Image

It's a bit smoother than the two-layer blend. For comparison here's only the xBRZ scaled image:
Image

I think it's a good compromise between any of the traditional non-neural scalers and the neural scaling because neither do the job as well as if the sprites were redrawn or generated from high-res 3D models manually.

User avatar
MrFlibble
Posts: 410
Joined: Sat Jan 27, 2018 10:43 am

Re: AI Upscaled Textures

Post by MrFlibble »

Here's another attempt, this time I used xBR so scale up the image and an interpolated ESRGAN/Manga109 model at alpha = 0.2:
Image
No blending, sharpening or other post-processing here. I think it's overall better than the results above.

User avatar
MasonFace
Posts: 543
Joined: Tue Nov 27, 2018 7:28 pm
Location: Tennessee, USA
Contact:

Re: AI Upscaled Textures

Post by MasonFace »

.

Wow, a lot of movement here in the last 12 hours!

@MrFlibble:

I agree that the latest results look the best. It preserves the details very well and doesn't butcher the faces nearly as much as the others. Definitely the way to go on the NPCs. Am I to understand that you used the same xBR + Gaussian Blur method as in the first post and only changed the alpha value for the ESRGAN interpolation?

Also, what is the purpose of dropping the color depth back down to the original DF palette? Is that just an artistic touch or does it have a functional purpose?

One last thing. How are you able to use GIMP's select color to restore the alpha around the subjects' outlines? When I upscale using AI with a solid colored background, the background ends up having so much noise in it that I can't select by color and get a good outline. If I increase the threshold of the selection, I end up either not picking up enough of the outline, or it begins cutting into the sprite somewhere I don't want it to. My method of masking involves posterizing down to 2 or 3 levels so all the subtle blue colors are combined into just one unified color, then ramping up the contrast to make the outline very distinct.

The only problem I'm having is that if the sprite has any pixels on them that are close to being blue, it will mask that, too. I was just wondering if converting to the original DF palette helps this. If so, do you know how to change the palette in GIMP? I've tried using "Palette Map," but it doesn't come out looking right at all.

@Phredreeke:

Having said that, my masking method is really showing its weaknesses. :?

I have all the sprites (there's a LOT of them!) consolidated into one folder with the proper naming format. Would you be so kind as to generate the masking layer using your refined technique? :P

Consolidated Sprites:
https://drive.google.com/open?id=1IqPIg ... oVf-DnU0qo



.

Post Reply