AI Upscaled Textures

Show off your mod creations or just a work in progress.
Post Reply
User avatar
MasonFace
Posts: 543
Joined: Tue Nov 27, 2018 7:28 pm
Location: Tennessee, USA
Contact:

Re: AI Upscaled Textures

Post by MasonFace »

That's really impressive, but I think it only works for videos, not still images.

Although it could probably be used for the intro video. My original upscale of that could use some improvement. Unfortunately, I doubt I'll have time to fool with it though...

User avatar
DigitalMonk
Posts: 111
Joined: Sun Nov 10, 2019 8:01 pm

Re: AI Upscaled Textures

Post by DigitalMonk »

I'm pretty sure you're right about "only videos". Especially with the discussion of the wavelet flows that originally needed motion vectors, but now can work with just video (which implies that the motion vectors are being calculated from the video). If you can accurately track how the camera is panning over a scene, you can extract a huge amount of additional information from image sequences. Older super-resolution used to work purely by this temporal alignment/stacking process. If you feed all of that information into a super resolution learning system, you get these amazing results. But it doesn't look like anything new on the single image front, unfortunately.

Still, damned impressive. Thanks for the link!

User avatar
Ralzar
Posts: 2211
Joined: Mon Oct 07, 2019 4:11 pm
Location: Norway

Re: AI Upscaled Textures

Post by Ralzar »

MasonFace wrote: Tue Sep 08, 2020 12:58 pm That's really impressive, but I think it only works for videos, not still images.
Make a film of all the sprites, run it through that program, then pick out frames and convert them into new sprites :D

User avatar
MasonFace
Posts: 543
Joined: Tue Nov 27, 2018 7:28 pm
Location: Tennessee, USA
Contact:

Re: AI Upscaled Textures

Post by MasonFace »

Ralzar wrote: Tue Sep 08, 2020 1:56 pm Make a film of all the sprites, run it through that program, then pick out frames and convert them into new sprites :D
That would be spooky as hell if that actually worked. :lol:

User avatar
MrFlibble
Posts: 410
Joined: Sat Jan 27, 2018 10:43 am

Re: AI Upscaled Textures

Post by MrFlibble »

I just tried a recent model called Rebout Blend (available here), which is specifically made for character sprites, and it gives quite decent results with small flats -- notably better than previous models:
Image
It does a very good job with medium flats too (open image in separate window to enlarge):
Image

User avatar
MasonFace
Posts: 543
Joined: Tue Nov 27, 2018 7:28 pm
Location: Tennessee, USA
Contact:

Re: AI Upscaled Textures

Post by MasonFace »

Oh wow! That really is the best results by far on the small sprites.

User avatar
King of Worms
Posts: 4747
Joined: Mon Oct 17, 2016 11:18 pm
Location: Scourg Barrow (CZ)
Contact:

Re: AI Upscaled Textures

Post by King of Worms »

Nice job on the artifacts upscales MasonFace, thanks a lot :)
Also, this new ReboutBlend algo indeed looks really good!

User avatar
MrFlibble
Posts: 410
Joined: Sat Jan 27, 2018 10:43 am

Re: AI Upscaled Textures

Post by MrFlibble »

I just tried a different model by LyonHrt called Lollypop that gets some good result on textures, and interesting results on sprites, and then decided to interpolate it with Rebout Blend (alpha = 0.3), and got some pretty nice results with the sprites (definitely better than from using either Lollypop or Rebout Blend alone):
Image
Image
Image
Image
Image
Image
Here's the net_interp code:

Code: Select all

net_PSNR_path = './models/lollypop.pth'
net_ESRGAN_path = './models/reboutblend.pth'

User avatar
MrFlibble
Posts: 410
Joined: Sat Jan 27, 2018 10:43 am

Re: AI Upscaled Textures

Post by MrFlibble »

I ran some further tests with Lollypop and the interpolated model on larger pre-rendered sprites, I think Lollypop alone (images on the left side) does good job but the other is fine too.
Image Image
Image Image
Image Image
Image Image
Image Image
Image Image

User avatar
MrFlibble
Posts: 410
Joined: Sat Jan 27, 2018 10:43 am

Re: AI Upscaled Textures

Post by MrFlibble »

I've been playing more with ESRGAN models lately and found one called Fatality MKII (available here) which does this to villager close-up faces:
Image Image

Post Reply