AI Upscaled Textures

Show off your mod creations or just a work in progress.
User avatar
MasonFace
Posts: 253
Joined: Tue Nov 27, 2018 7:28 pm
Contact:

Re: AI Upscaled Textures

Post by MasonFace » Wed Feb 27, 2019 5:46 pm

A large part of the problem I've been having with the sprite edges I thought was due to the bilinear filtering smoothing the edges in a way I didn't expect. I thought it had something to do with the suggestion that Interkarma made to you in the thread I referenced above, but I don't think that's the issue after all. Interestingly enough, the edge padding (or dilation) appears to be working now without any required work on the artists's behalf - the RGB values at the alpha edge are extended outwards so the bilinear filter has surrounding pixels to sample without contaminating the results. Not sure if this "fix" was a Unity update or if Interkarma updated his billboard shader but it seems to work fine without any effort on the artist's part.


See attached picture below with the alpha cutoff set to 0 so that all RGB values are displayed (no transparency).
PaddedSprite.PNG
PaddedSprite.PNG (231.66 KiB) Viewed 531 times


And with the alpha cutoff set to the default of "0.5"
SpriteCutoff.PNG
SpriteCutoff.PNG (180.05 KiB) Viewed 531 times


A lot of my problem was actually that I had feathered edges. Once I set the alpha to have a hard falloff, it works pretty good.

So my remaining problem is this:

Using your xBR upscales as a base, I downscale them from 8x vanilla resolution to 4x vanilla resolution, I do get smooth silhouettes but the xBR interprets some diagonal lines as odd melted stair steps which look weird... It does provide a good starting point though, and cleaning those issues is fairly easy to do manually, just time consuming and not possible to batch. For the most part, the results look pretty good, but up close the edges still look a little murky without a lot of careful editing to get clean the silhouette.

For example, look at the top part of the shield. It looks pretty close to round, but the bottom I didn't clean up as well and it still looks wavy.

I had to edit the battle axe pretty heavily to make the shaft straight and the blade curved properly.

User avatar
King of Worms
Posts: 1509
Joined: Mon Oct 17, 2016 11:18 pm
Location: Scourg Barrow
Contact:

Re: AI Upscaled Textures

Post by King of Worms » Fri Mar 01, 2019 5:43 pm

Hi, Im glad the main problem is dealt with, no matter how it happened :)

The XBR algo is quite limited in in certain situations, like the very low res lines which are in lets say 45 degrees direction, or the round shapes upscale from the low res. Exactly what you found out.

I take it as a artistic direction, and when these things get animated, it looks proper to me. Ofc, given enough efforts, these can be manually corrected, like you did.

And I did that as well, if you look at the skeletons shield when he is directly approaching the player (or directly attacks a player) its corrected. I just did not correct it in all those other degrees, like the 45 degree you posted. I made this decission because there were just too much frames to do, so I focused only on those, which are usually visible. (also, when I did this, the fights between enemies were not present, so basically, those frames were not possible to see to a player)

All mobs have manually cleaned the "direct to player" frames tho... there were countless artefacts and imperfections to deal with, and I did that .)

Plus other touches I gladly forgot, like the orc shamans staff changing color from yellow to red in certain animations was corrected etc etc etc... I dont really remember all...

I did what I was able to do in a given time, and I focused on things which are visible most of the time - lets call it bang for a buck approach... just its a bang for a invested time and efforts..

The skeleton looks great btw.. and in case you grow tired of the manual touches, it will look good even without the corrections when in movement. Its up to You to decide, how much time and effort you can invest in this. Id say, go on, until you cant stand it anymore :)

PS: I still think the "direct to player" frames are a priority. So a "walk towards a player" and attacks, plus mob being hit. Plus I usually did those 5 frames where the mob is rotating. They are at the end of each mob frame set, just after the hit frames.

PSS: Id say, batch process all the frames, see how it looks ingame, and than decide on the other tweaks maybe?

Good luck!! Im VERY happy to see the mobs being taken to another level by you. Its the best possible outcome! Thank you

User avatar
MasonFace
Posts: 253
Joined: Tue Nov 27, 2018 7:28 pm
Contact:

Re: AI Upscaled Textures

Post by MasonFace » Fri Mar 01, 2019 7:51 pm

The skeleton looks great btw.. and in case you grow tired of the manual touches, it will look good even without the corrections when in movement. Its up to You to decide, how much time and effort you can invest in this. Id say, go on, until you cant stand it anymore :)
Thanks for the encouragement! I figured I was being a bit overly self-critical of the results and it probably didn't look as bad as I thought. I'll try to post some screenshots tonight of what the raw output looks like without the silhouette getting manually cleaned up just for comparison.
I did what I was able to do in a given time, and I focused on things which are visible most of the time - lets call it bang for a buck approach... just its a bang for a invested time and efforts..
Very smart! I lost sight of this. I call it the "Pareto Approach" or the "80/20 rule," which states that you can expend 20% of the effort to achieve 80% of the results by focusing on the ones that matter most. Great suggestion! Of course I will release the results so others can clean them up if they have time and want to contribute.
PSS: Id say, batch process all the frames, see how it looks ingame, and than decide on the other tweaks maybe?
Unfortunately, I don't have any of this batched yet... I have a process that should theoretically be batch-able, but I haven't committed the time to writing a GIMP macro to test it (nor have I ever written a GIMP macro/script/routine/filter/whatever before, so it may take some time). Jukic was working on making a batch process to handle this, but I still owe him a larger test set. I'm still working on it Jukic, I swear! :lol:

User avatar
King of Worms
Posts: 1509
Joined: Mon Oct 17, 2016 11:18 pm
Location: Scourg Barrow
Contact:

Re: AI Upscaled Textures

Post by King of Worms » Fri Mar 01, 2019 8:11 pm

Looking forward more screens :)

MrFlibble
Posts: 107
Joined: Sat Jan 27, 2018 10:43 am

Re: AI Upscaled Textures

Post by MrFlibble » Sat Mar 02, 2019 9:41 am

In the meantime, Phredreeke of the Duke4.net forums released a Duke Nukem 3D Sprite Upscale pack, created using ESRGAN.

Phredreeke is using Paint Shop Pro scripts to get proper aliasing on the edges, and I think the results he got are way better than any other solution people have come up so far for this problem (I heard hidfan who created the Doom Neural Upscale edited alpha masks for sprites manually!):
Image
I already asked Phredreeke to share his scripts, that could give the Daggerfall sprite project a good boost.

User avatar
MasonFace
Posts: 253
Joined: Tue Nov 27, 2018 7:28 pm
Contact:

Re: AI Upscaled Textures

Post by MasonFace » Sat Mar 02, 2019 4:31 pm

Those are great results! I'm hoping he will share the scripts. That would be a HUGE boost towards this project!

User avatar
Kamer
Posts: 254
Joined: Mon Mar 05, 2018 9:26 pm

Re: AI Upscaled Textures

Post by Kamer » Wed Mar 06, 2019 11:16 pm

MrFlibble wrote:
Sat Mar 02, 2019 9:41 am
In the meantime, Phredreeke of the Duke4.net forums released a Duke Nukem 3D Sprite Upscale pack, created using ESRGAN.

Phredreeke is using Paint Shop Pro scripts to get proper aliasing on the edges, and I think the results he got are way better than any other solution people have come up so far for this problem (I heard hidfan who created the Doom Neural Upscale edited alpha masks for sprites manually!):
Image
I already asked Phredreeke to share his scripts, that could give the Daggerfall sprite project a good boost.

If we can get those scripts then I won't really need to work on the 3d models anymore. I wouldn't see the point in it if we can get the df prites looking this good.

User avatar
MasonFace
Posts: 253
Joined: Tue Nov 27, 2018 7:28 pm
Contact:

Re: AI Upscaled Textures

Post by MasonFace » Thu Mar 07, 2019 3:53 am

So I've refined my method to get good edges on the sprites without manual touch ups. The trick is to use the ESRGAN upscale with a high contrasting color (like purple) as the background, then alter to the color contrast of that image WAY up so that subject and background are as different as possible and the silhouette of the subject stands out perfectly, then you have a good high resolution masking layer to restore the alpha with.

The upscaling method below is a composite of 10% ESRGAN, 45% SFTGAN, and 45% Gigapixel AI results. The results of this composition is then fed into Topaz AI Sharpen tool, then one last pass with GIMP's unsharp mask. I'm quite pleased with the results.
RatUpscale.png
RatUpscale.png (291.87 KiB) Viewed 210 times
The next step will be for me to write some scripts for GIMP to automate this as much as possible and make progress much faster.
If we can get those scripts then I won't really need to work on the 3d models anymore. I wouldn't see the point in it if we can get the df prites looking this good.
Nonsense! Eventually 3D models could be implemented for the MOBs and NPCs so your 3D models will still have plenty of utility.

User avatar
Kamer
Posts: 254
Joined: Mon Mar 05, 2018 9:26 pm

Re: AI Upscaled Textures

Post by Kamer » Thu Mar 07, 2019 4:53 am

True but if we can get hi-rez upscaled versions of the original sprites, then making models and pre-rendering them would be a waste of time cause this looks like a much better option. If anything the goal of the project will change. Probably to modernize the sprites by giving them extra details but keeping to the artstyle as close as possible. That rat looks amazing btw.
Last edited by Kamer on Thu Mar 07, 2019 7:27 am, edited 2 times in total.

User avatar
NyanCommissar
Posts: 6
Joined: Wed Sep 12, 2018 7:31 am

Re: AI Upscaled Textures

Post by NyanCommissar » Thu Mar 07, 2019 5:17 am

MasonFace wrote:
Thu Mar 07, 2019 3:53 am
The upscaling method below is a composite of 10% ESRGAN, 45% SFTGAN, and 45% Gigapixel AI results. The results of this composition is then fed into Topaz AI Sharpen tool, then one last pass with GIMP's unsharp mask. I'm quite pleased with the results.
Well, it appears you got the best pattern for upscaling NPC sprites. But how about the humanoid ones?

Post Reply