AI Upscaled Textures

Show off your mod creations or just a work in progress.
User avatar
MasonFace
Posts: 289
Joined: Tue Nov 27, 2018 7:28 pm
Contact:

Re: AI Upscaled Textures

Post by MasonFace » Sat Mar 30, 2019 4:35 am

Just a quick test. The results are... interesting?
93-0Test.jpg
93-0Test.jpg (87.81 KiB) Viewed 708 times
In all honesty, it was neither quick nor easy to get these results and they look fugly. This AI is probably best for modifying small parts of photos of real people. Not so much for fully upscaling tiny resolution hand drawn portraits.

Still plenty neat.

It would have a lot of more potential if I could use GIMP to create the masks and sketches instead of using their overly simplistic demo drawing tools.

MrFlibble
Posts: 111
Joined: Sat Jan 27, 2018 10:43 am

Re: AI Upscaled Textures

Post by MrFlibble » Sat Mar 30, 2019 7:47 pm

MasonFace wrote:
Sat Mar 30, 2019 4:35 am
In all honesty, it was neither quick nor easy to get these results and they look fugly. This AI is probably best for modifying small parts of photos of real people. Not so much for fully upscaling tiny resolution hand drawn portraits.

Still plenty neat.
Thanks for trying this out!

To be honest, when I read the project description in your previous post I did not have any high expectations. In fact, I thought that the results would be far more horrible than what you got :) It's a very impressive project but it's just not suited to what we're doing.

Speaking of which, I realised why scaling gets "better" with prior xBR/xBRZ softening: it's not because of anti-aliasing per se, it's because these scalers actually produce smoother outlines, and this first translates into the low-res image and then into the ESRGAN upscale. Here's my attempt to explain this phenomenon. I'm also assuming that a) upscaling results will not get fundamentally better than the high-res image that is used to produce low-res input (e.g. if xBRZ fails to properly smooth a line at a certain angle, ESRGAN will not be able to do much better when scaling the image preprocessed with xBRZ) and b) if we want to improve results, it's probably better to manipulate the (xBRZ-)upscaled image, rather than the low-res image (but I'm not 100% sure on this).

MrFlibble
Posts: 111
Joined: Sat Jan 27, 2018 10:43 am

Re: AI Upscaled Textures

Post by MrFlibble » Sun Mar 31, 2019 4:57 pm

My thoughts about how the xBR(Z) upscale affected the neural upscale result prompted me to test pre-processing with several passes of xBR. That is, upscale image once with xBR(Z), then scale down to the original resolution (this time I used Sinc instead of Bicubic because it produces less ringing as it seems), then back up with xBR(Z) etc.

It seems that two passes actually do a good job at smoothing straight lines at odd angles (more on this below). What's more I also decided to try this: first scale up with xBR and then with xBRZ. This actually helps to fix the jagged edges that xBRZ sometimes produces where there shouldn't be any. For a quick example, here's a comparison between an image (part of a Red Alert screenshot) scaled up with xBR only (left) and with two passes of xBR and then xBRZ (right)
Image Image

Since in other instances xBR produces somewhat better results that xBRZ, I thought why not combine the two methods. The only drawback is that the image gets somewhat blurry, but not to the point of losing a lot of detail. Comparison (one xBRZ pass to the left, combined xBR/xBRZ two-pass to the right; everything with ESRGAN/Manga interpolation at 0.2):
Image Image

Image Image
This also seems to do a decent job with low-res hand-drawn NPCs:
Image
In all instances the steps were as follows:
  • original image -> xBR upscale 4x
  • xBR upscale -> GIMP Gaussian blur 1 pixel radius -> scale down to original size w/ Sinc
  • GIMP output -> xBRZ upscale -> same GIMP steps as above to get original size w/ Sinc

User avatar
MasonFace
Posts: 289
Joined: Tue Nov 27, 2018 7:28 pm
Contact:

Re: AI Upscaled Textures

Post by MasonFace » Mon Apr 01, 2019 1:13 pm

4pR1lf0015.gif
4pR1lf0015.gif (189.57 KiB) Viewed 578 times

After quite a bit of tinkering, I think I've got the best results I can squeeze out of ESRGAN on the spider MOB. Particularly interesting is that the spider came out a completely different species than the low resolution input. Rightfully so, I suspected that the model was trained on many images of South American tarantulas exclusively. I haven't tried to composite the results with Gigapixel and SFTGAN yet; hopefully that will bring the results closer to vanilla. Last several MOBs have worked out this way and I expect this one will, too.

For what its worth, I think the results are very interesting. Other folks may disagree, and that's perfectly fine. Of course there are still some imperfections, but that just comes with the territory of using AI. Like all other AI algorithms, some artifacts will be present. Some touch ups may be required.
Last edited by MasonFace on Mon Apr 01, 2019 5:08 pm, edited 1 time in total.

User avatar
King of Worms
Posts: 2053
Joined: Mon Oct 17, 2016 11:18 pm
Location: Scourg Barrow (CZ)
Contact:

Re: AI Upscaled Textures

Post by King of Worms » Mon Apr 01, 2019 1:50 pm

Thats some Shodan kind of AI results, very interresting. Yes its different from the original but its quite awesome. Id like to see the full model in the game.

ifkopifko
Posts: 169
Joined: Thu Apr 02, 2015 9:03 am

Re: AI Upscaled Textures

Post by ifkopifko » Mon Apr 01, 2019 4:04 pm

Zoiks! I suspect the "Remove Spiders" MOD will become quite popular... and rightfully so...

User avatar
Ferital
Posts: 126
Joined: Thu Apr 05, 2018 8:01 am

Re: AI Upscaled Textures

Post by Ferital » Mon Apr 01, 2019 4:27 pm

Nice April fools :)

User avatar
MasonFace
Posts: 289
Joined: Tue Nov 27, 2018 7:28 pm
Contact:

Re: AI Upscaled Textures

Post by MasonFace » Mon Apr 01, 2019 5:05 pm

Ah, you got me! :lol:

Those are pictures of a toy I found and just cut out the background.

I was hoping someone would notice the filename "4pR1lf0015" was "April fools" and that the first letter of every sentence spells "APRIL FOOLS." (I've gone back and embellished the first letters so its more obvious).

Anyway, I hope you guys have a good laugh from it. I wasn't sure how universal the April fools tradition is around the world, and it's particularly challenging when some folks are a half day ahead. It's probably April 2nd right now for KoW and ifkpifko, so I won't give them a hard time if they truly fell for it. It was all just in good fun.

Anyhow, in other news I'm about 80% done upscaling and compositing the MOBs and should be releasing the Pure Vanilla Extract mod sometime this week, with the source files to follow shortly after. I will create a new thread specifically for the PVE mod and keep this thread open for continual exploration of AI upscaling methods.

@MrFlibble:

Interesting results. It looks like the xBR to xBRZ preprocessing yields improved outlines, but the sharpness suffers pretty badly. Have you tried this method with a lower blur value between the xBR and xBRZ passes?

User avatar
King of Worms
Posts: 2053
Joined: Mon Oct 17, 2016 11:18 pm
Location: Scourg Barrow (CZ)
Contact:

Re: AI Upscaled Textures

Post by King of Worms » Mon Apr 01, 2019 5:23 pm

I hate you :lol:

User avatar
Jay_H
Posts: 2526
Joined: Tue Aug 25, 2015 1:54 am

Re: AI Upscaled Textures

Post by Jay_H » Mon Apr 01, 2019 9:06 pm

Woooo, well done with the joke :lol:
Come join the Unofficial Daggerfall Unity Discord.
See the Daggerfall Unity Wiki on the UESP.
Progress on fixing classic quests here.

Post Reply