AI Upscaled Textures
- King of Worms
- Posts: 4752
- Joined: Mon Oct 17, 2016 11:18 pm
- Location: Scourg Barrow (CZ)
- Contact:
Re: AI Upscaled Textures
Thanks for upscale MasonFace, I think I can make the arrows work by combining the good parts from them and than just rotating the result. Anyway, if I ever need any more upscales, I will make sure to just batch everything together so its easier for you to do.
Thanks again
Thanks again
DREAM Nexus https://www.nexusmods.com/users/73250038?tab=user+files
DREAM Forums viewtopic.php?f=27&t=1168
DISCORD: king_of_worms_cz
DREAM Forums viewtopic.php?f=27&t=1168
DISCORD: king_of_worms_cz
- MasonFace
- Posts: 543
- Joined: Tue Nov 27, 2018 7:28 pm
- Location: Tennessee, USA
- Contact:
Re: AI Upscaled Textures
You said before that you aren't using a neural network to upscale the masking layer, so are you using interpolation to upscale it? If so, which interpolation model? I'm using GIMP, so I may not have the same interpolation models; all of them produce poor results for upscaling the anti-aliased masking layer. I've even tried using AA, then upscaling 150%, running AA again, upscaling 150%, running AA again, . . . until I reach the target resolution, but it just looks bad.I antialias the image, followed by upscaling to create the mask, at the end I apply a threshold effect of 127 to make the mask binary.
Depending on what operating system you're on, you should be able to search the parent folder that holds all of the subfolders for "*.png". This fuzzy search should return all the images which you could cut and paste into one consolidated folder. The problem with this is that you'll probably have multiple images with the same name... I don't think I have all the original images consolidated with the proper name formatting at the moment. It would take a little while for me to compile. I might could write a batch script to find the archive number from the folder name, then prepend the file name of each image in each folder, then pull them all out into one master folder.BTW, could someone put together an archive with all the images that need masks in the same folder? Currently I have them in subfolders, so I can't just create a single batch to run through them all.
However, I have refined my masking method further and I have had consistently good results on a small batch of samples. I don't know yet if it will work well in every situation, but so far the results look promising. As such, I may not need much more help with the masking. I really appreciate your help up to this point! If it turns out that I need more support with the masks, I'll make sure I have all the images consolidated so it will be much less tedious for you to batch.
- Jukic
- Posts: 68
- Joined: Sun May 13, 2018 5:51 pm
Re: AI Upscaled Textures
@MasonFace:
I am swamped with work and reallife obligations but I am reading and will return!, if You dont finish yourself that is ....!
I am swamped with work and reallife obligations but I am reading and will return!, if You dont finish yourself that is ....!
-
- Posts: 71
- Joined: Mon Mar 11, 2019 9:13 am
Re: AI Upscaled Textures
I use SmartSize, which to my understanding uses Bicubic when upscaling and Bilinear when downscaling. I only upscale at integer sizes (actually that's a slight lie with the latest mask, as I omitted 2 pixels on each axis for the last upscale, which I then pad back to the intended size)MasonFace wrote: ↑Tue Mar 12, 2019 2:48 pm You said before that you aren't using a neural network to upscale the masking layer, so are you using interpolation to upscale it? If so, which interpolation model? I'm using GIMP, so I may not have the same interpolation models; all of them produce poor results for upscaling the anti-aliased masking layer. I've even tried using AA, then upscaling 150%, running AA again, upscaling 150%, running AA again, . . . until I reach the target resolution, but it just looks bad.
For antialiasing I actually create a separate layer (I use the color selection tool then use shape-based antialias in PSP, and fill a layer on the inside setting and another layer on the outside setting then take the difference between those two) and use as a mask to soften the underlaying image (the actual mask for the upscaled image).
- MrFlibble
- Posts: 410
- Joined: Sat Jan 27, 2018 10:43 am
Re: AI Upscaled Textures
All this talk about anti-aliasing prompted me to try out a very simple trick.
Back when I started trying to get waifu2x to properly process sprite edges and came up with the xBRZ softening method, the first thing I tried was simple Gaussian blur applied to the original image (at a radius of 0.5-0.7 pixels). This more or less worked but the result was, well, blurry.
Now I thought, why not scale up the image with xBRZ to 4x, and then apply Gaussian blur? I did that, and then scaled the image down with Bicubic interpolation. I tried radius of 2 and 3 pixels (since Bicubic downscaling sharpens the image, by my estimate a radius of 2 pixels ends up to the effect of a 0.5 pixel blur, and 3 pixels results in 1.5 pixel blur radius), with the latter being too blurry but the former working rather well.
I tried this image and scaled it with ESRGAN*Manga109 at alpha = 0.5 and with pure Manga109. Each image was then converted to the original palette with mtPaint, and then I used GIMP's select by colour tool to select the blue background and remove it. Here are the results:
ESRGAN/Manga109 interpolation:
Pure Manga109:
Blend of both (using the G'MIC Blend [median] function)
I think the blended version is a pretty good compromise between the two.
Back when I started trying to get waifu2x to properly process sprite edges and came up with the xBRZ softening method, the first thing I tried was simple Gaussian blur applied to the original image (at a radius of 0.5-0.7 pixels). This more or less worked but the result was, well, blurry.
Now I thought, why not scale up the image with xBRZ to 4x, and then apply Gaussian blur? I did that, and then scaled the image down with Bicubic interpolation. I tried radius of 2 and 3 pixels (since Bicubic downscaling sharpens the image, by my estimate a radius of 2 pixels ends up to the effect of a 0.5 pixel blur, and 3 pixels results in 1.5 pixel blur radius), with the latter being too blurry but the former working rather well.
I tried this image and scaled it with ESRGAN*Manga109 at alpha = 0.5 and with pure Manga109. Each image was then converted to the original palette with mtPaint, and then I used GIMP's select by colour tool to select the blue background and remove it. Here are the results:
ESRGAN/Manga109 interpolation:
Pure Manga109:
Blend of both (using the G'MIC Blend [median] function)
I think the blended version is a pretty good compromise between the two.
- King of Worms
- Posts: 4752
- Joined: Mon Oct 17, 2016 11:18 pm
- Location: Scourg Barrow (CZ)
- Contact:
Re: AI Upscaled Textures
Interresting, 1st img is very sharp, 2nd quite blurry, 3rd is ideal... I think those are nice results.
DREAM Nexus https://www.nexusmods.com/users/73250038?tab=user+files
DREAM Forums viewtopic.php?f=27&t=1168
DISCORD: king_of_worms_cz
DREAM Forums viewtopic.php?f=27&t=1168
DISCORD: king_of_worms_cz
- MrFlibble
- Posts: 410
- Joined: Sat Jan 27, 2018 10:43 am
Re: AI Upscaled Textures
It's still not perfect because either network messes up small details... Like the golden skull ornament on the King of Worms' entourage's garments, it's become completely unrecognisable, and the witch cover guard's facial features are all messed up (which is not the case with the original low-res sprite; this is how ESRGAN generalised the details).
Here's another try, with a group of monsters (again, it's an official preview image), blended like No. 3 above:
It's mostly rather good I think.
Here's another try, with a group of monsters (again, it's an official preview image), blended like No. 3 above:
It's mostly rather good I think.
- MrFlibble
- Posts: 410
- Joined: Sat Jan 27, 2018 10:43 am
Re: AI Upscaled Textures
Here's the result of blending three layers: the first two shown above (ESRGAN/Manga109 and pure Manga109) and the same image simply scaled up with xBRZ:
It's a bit smoother than the two-layer blend. For comparison here's only the xBRZ scaled image:
I think it's a good compromise between any of the traditional non-neural scalers and the neural scaling because neither do the job as well as if the sprites were redrawn or generated from high-res 3D models manually.
It's a bit smoother than the two-layer blend. For comparison here's only the xBRZ scaled image:
I think it's a good compromise between any of the traditional non-neural scalers and the neural scaling because neither do the job as well as if the sprites were redrawn or generated from high-res 3D models manually.
- MrFlibble
- Posts: 410
- Joined: Sat Jan 27, 2018 10:43 am
Re: AI Upscaled Textures
Here's another attempt, this time I used xBR so scale up the image and an interpolated ESRGAN/Manga109 model at alpha = 0.2:
No blending, sharpening or other post-processing here. I think it's overall better than the results above.
No blending, sharpening or other post-processing here. I think it's overall better than the results above.
- MasonFace
- Posts: 543
- Joined: Tue Nov 27, 2018 7:28 pm
- Location: Tennessee, USA
- Contact:
Re: AI Upscaled Textures
.
Wow, a lot of movement here in the last 12 hours!
@MrFlibble:
I agree that the latest results look the best. It preserves the details very well and doesn't butcher the faces nearly as much as the others. Definitely the way to go on the NPCs. Am I to understand that you used the same xBR + Gaussian Blur method as in the first post and only changed the alpha value for the ESRGAN interpolation?
Also, what is the purpose of dropping the color depth back down to the original DF palette? Is that just an artistic touch or does it have a functional purpose?
One last thing. How are you able to use GIMP's select color to restore the alpha around the subjects' outlines? When I upscale using AI with a solid colored background, the background ends up having so much noise in it that I can't select by color and get a good outline. If I increase the threshold of the selection, I end up either not picking up enough of the outline, or it begins cutting into the sprite somewhere I don't want it to. My method of masking involves posterizing down to 2 or 3 levels so all the subtle blue colors are combined into just one unified color, then ramping up the contrast to make the outline very distinct.
The only problem I'm having is that if the sprite has any pixels on them that are close to being blue, it will mask that, too. I was just wondering if converting to the original DF palette helps this. If so, do you know how to change the palette in GIMP? I've tried using "Palette Map," but it doesn't come out looking right at all.
@Phredreeke:
Having said that, my masking method is really showing its weaknesses.
I have all the sprites (there's a LOT of them!) consolidated into one folder with the proper naming format. Would you be so kind as to generate the masking layer using your refined technique?
Consolidated Sprites:
https://drive.google.com/open?id=1IqPIg ... oVf-DnU0qo
.
Wow, a lot of movement here in the last 12 hours!
@MrFlibble:
I agree that the latest results look the best. It preserves the details very well and doesn't butcher the faces nearly as much as the others. Definitely the way to go on the NPCs. Am I to understand that you used the same xBR + Gaussian Blur method as in the first post and only changed the alpha value for the ESRGAN interpolation?
Also, what is the purpose of dropping the color depth back down to the original DF palette? Is that just an artistic touch or does it have a functional purpose?
One last thing. How are you able to use GIMP's select color to restore the alpha around the subjects' outlines? When I upscale using AI with a solid colored background, the background ends up having so much noise in it that I can't select by color and get a good outline. If I increase the threshold of the selection, I end up either not picking up enough of the outline, or it begins cutting into the sprite somewhere I don't want it to. My method of masking involves posterizing down to 2 or 3 levels so all the subtle blue colors are combined into just one unified color, then ramping up the contrast to make the outline very distinct.
The only problem I'm having is that if the sprite has any pixels on them that are close to being blue, it will mask that, too. I was just wondering if converting to the original DF palette helps this. If so, do you know how to change the palette in GIMP? I've tried using "Palette Map," but it doesn't come out looking right at all.
@Phredreeke:
Having said that, my masking method is really showing its weaknesses.
I have all the sprites (there's a LOT of them!) consolidated into one folder with the proper naming format. Would you be so kind as to generate the masking layer using your refined technique?
Consolidated Sprites:
https://drive.google.com/open?id=1IqPIg ... oVf-DnU0qo
.