AI Upscaled Textures

Show off your mod creations or just a work in progress.
Post Reply
User avatar
Midknightprince
Posts: 1180
Joined: Fri Aug 11, 2017 6:51 am
Location: San Antonio TX
Contact:

Re: AI Upscaled Textures

Post by Midknightprince » Tue Jan 22, 2019 1:18 am

Hey this is sweet !
Check out my YouTube Channel!

User avatar
MasonFace
Posts: 454
Joined: Tue Nov 27, 2018 7:28 pm
Location: Tennessee, USA
Contact:

Re: AI Upscaled Textures

Post by MasonFace » Tue Jan 22, 2019 3:16 am

@Jukic:

I've re-uploaded all three sets of Orc sprites, cleaning up the naming convention. The previous links are now dead.

"262_SFTGAN.rar" is the one that you will need as it has the corrected dimensions matching the others now.

KoW's Orc Downscaled from 8x to 4x (xBRZ4x):
https://drive.google.com/open?id=1GQTG- ... Y07CoGAhfj

SFTGAN Upscaled Orc:
https://drive.google.com/open?id=1fEY6Y ... xi3BQh2l10

ESRGAN Upscaled Orc:
https://drive.google.com/open?id=1Qrt9q ... uDDZgsvNqX

Turns out the problem is in the SFTGAN script itself. For some reason, it cuts off a 4 pixel row and column from the right and bottom of the image. Not sure why... I used ImageMagick to add these rows back in but with just the background color.

User avatar
MrFlibble
Posts: 167
Joined: Sat Jan 27, 2018 10:43 am

Re: AI Upscaled Textures

Post by MrFlibble » Tue Jan 22, 2019 10:24 am

MasonFace wrote:
Mon Jan 21, 2019 7:55 pm
@MrFlibble:
Interesting results! I'd like to see how this method performs in other situations such as textures. So far, it looks like a good technique to put in our upscaling toolbox.
Yes, I was thinking the same, here's a sample:
https://imgur.com/a/2NkxqHn

I'm not sure if I like the resulting quality of the 4x upscales as they get this distinct "painted" look, but if you downscale to 2x and convert to the original palette it actually looks quite nice and retro-ish:
Image Image Image
MasonFace wrote:
Mon Jan 21, 2019 7:55 pm
BTW, what percentage are you interpolating between those two models?
alpha = 0.5
MasonFace wrote:
Tue Jan 22, 2019 3:16 am
Turns out the problem is in the SFTGAN script itself. For some reason, it cuts off a 4 pixel row and column from the right and bottom of the image. Not sure why... I used ImageMagick to add these rows back in but with just the background color.
BTW, I tried scaling stuff up with SFTGAN but for some reason it would only mess up the input image without increasing its size, and I didn't investigate further. I wonder what I did wrong though.

I ran a quick test with some orc sprites in the interpolated model (all images below converted to the original palette without dithering). As with textures, scaled 'em down to 2x to check if anything would look better (full album):
Image
original

Image
scaled down to 2x

Image
4x

As you can see the original sprite has the pixels arranged so that the orc's skin looks scaly, which is almost completely lost in the upscales.

I also tried adding noise to the result (G'MIC's Noise [additive], Uniform noise at amplitude 5):
Image

Doesn't seem much of an improvement though.

User avatar
King of Worms
Posts: 3159
Joined: Mon Oct 17, 2016 11:18 pm
Location: Scourg Barrow (CZ)
Contact:

Re: AI Upscaled Textures

Post by King of Worms » Tue Jan 22, 2019 12:26 pm

Hi Masonface,

Ive cleaned up some more tabs. It would be great if you could upscale it for me please ;)

Heres the link: https://mega.nz/#!ynwDEAxA!j3_ilhpRiPZa ... -zQeD1FOfE
Thank you!

User avatar
MasonFace
Posts: 454
Joined: Tue Nov 27, 2018 7:28 pm
Location: Tennessee, USA
Contact:

Re: AI Upscaled Textures

Post by MasonFace » Tue Jan 22, 2019 2:10 pm

@MrFlibble:
I really like the 2x upscaled cabinet textures. I never could get those particular textures to upscale well, but your method produces good results on it.
BTW, I tried scaling stuff up with SFTGAN but for some reason it would only mess up the input image without increasing its size, and I didn't investigate further. I wonder what I did wrong though.
I had the hardest time getting SFTGAN to work, but I think it's worth it just to have some extra tools at your disposal. For certain stuff it works much better than ESRGAN, but it is by no means a magic bullet.

Even though it is made by the same team, the workflow is completely different. For one thing, your image cannot have an alpha layer, so that has to be removed. Then you have to upscale it up 4x manually. I've found that using your xBR softening technique works well here, just don't downscale it back to the original size - keep it at 4x and then run SFTGAN. Basically, it enhances an image but doesn't scale it - I'm not sure why it's so different from ESRGAN in that regard.

For some textures, I got good results by upscaling with ESRGAN, then feeding the result into SFTGAN. This works well for wood and stone type textures.


@King of Worms:
I'll try to get those upscaled tonight when I get home.

User avatar
MrFlibble
Posts: 167
Joined: Sat Jan 27, 2018 10:43 am

Re: AI Upscaled Textures

Post by MrFlibble » Tue Jan 22, 2019 6:04 pm

King of Worms wrote:
Tue Jan 22, 2019 12:26 pm
Ive cleaned up some more tabs. It would be great if you could upscale it for me please ;)

Heres the link: https://mega.nz/#!ynwDEAxA!j3_ilhpRiPZa ... -zQeD1FOfE
Thank you!
I decided to give this a try with the RandomArt/Manga interpolated model, there's the result:
https://imgur.com/a/JMj5Ld7

Sorry I had to split one image into two halves, the file was too large and Imgur would auto-JPEG it otherwise.

The scrolling arrows didn't come out very straight.

UDP: I tested a few other interpolated models, the regular Manga 109/ESRGAN and Manga 109/PSNR might be better candidates:
Image
ESGRAN/Manga109 interpolation (alpha = 0.5)

Image
Manga109/PSNR interpolation (alpha = 0.5)

And here's Manga109/PSNR without any prior softening of the image:
Image

BTW, I kept getting an "incorrect sRGB profile" warnings from your source images both in xBRZ Scaler Tool and from ESRGAN too.
Last edited by MrFlibble on Tue Jan 22, 2019 6:19 pm, edited 1 time in total.

User avatar
King of Worms
Posts: 3159
Joined: Mon Oct 17, 2016 11:18 pm
Location: Scourg Barrow (CZ)
Contact:

Re: AI Upscaled Textures

Post by King of Worms » Tue Jan 22, 2019 6:13 pm

Thats quite a nice result for the tabs! Yes the arrows came out a bit curved. The spellmaker icons are hit & miss, some look nice, others are changed too much - that was expected tho, I did not expect a miracle on them .)

I will wait for Masonface results, for the sake of art consistency, lets see his kind of magic...

Thanks MrFibble!

User avatar
Hazelnut
Posts: 2189
Joined: Sat Aug 26, 2017 2:46 pm
Contact:

Re: AI Upscaled Textures

Post by Hazelnut » Tue Jan 22, 2019 6:19 pm

MrFlibble wrote:
Tue Jan 22, 2019 6:04 pm
I decided to give this a try with the RandomArt/Manga interpolated model, there's the result:
https://imgur.com/a/JMj5Ld7
Some of them are ok, but I'm not really liking the 'partially melted' overall look personally.
See my mod code for examples of how to change various aspects of DFU: https://github.com/ajrb/dfunity-mods

User avatar
MasonFace
Posts: 454
Joined: Tue Nov 27, 2018 7:28 pm
Location: Tennessee, USA
Contact:

Re: AI Upscaled Textures

Post by MasonFace » Tue Jan 22, 2019 9:29 pm

Yeah I think this is an example of when SFTGAN does a bit better, although I don't expect it will do any better on the icons. The icons may need to be upscaled separately with straight XBR, or it may be better to have VMBlast author some new ones.

@MrFlibble:
I don't think the sRGB color profile affected anything when I upscaled the others; it just complained about it but worked anyway.
Have you had a chance to try SFTGAN again with my advice? I'm really interested in seeing what all kinds of crazy new techniques you will dream up with another tool at your disposal.

User avatar
MasonFace
Posts: 454
Joined: Tue Nov 27, 2018 7:28 pm
Location: Tennessee, USA
Contact:

Re: AI Upscaled Textures

Post by MasonFace » Wed Jan 23, 2019 5:13 am

@King of Worms:

Here are the upscales: https://drive.google.com/open?id=1KJ02- ... WMbBmGZ3qT

As expected, the icons didn't turn out too well.

Post Reply