Page 23 of 75

Re: AI Upscaled Textures

Posted: Wed Feb 27, 2019 5:46 pm
by MasonFace
A large part of the problem I've been having with the sprite edges I thought was due to the bilinear filtering smoothing the edges in a way I didn't expect. I thought it had something to do with the suggestion that Interkarma made to you in the thread I referenced above, but I don't think that's the issue after all. Interestingly enough, the edge padding (or dilation) appears to be working now without any required work on the artists's behalf - the RGB values at the alpha edge are extended outwards so the bilinear filter has surrounding pixels to sample without contaminating the results. Not sure if this "fix" was a Unity update or if Interkarma updated his billboard shader but it seems to work fine without any effort on the artist's part.


See attached picture below with the alpha cutoff set to 0 so that all RGB values are displayed (no transparency).
PaddedSprite.PNG
PaddedSprite.PNG (231.66 KiB) Viewed 2520 times


And with the alpha cutoff set to the default of "0.5"
SpriteCutoff.PNG
SpriteCutoff.PNG (180.05 KiB) Viewed 2520 times


A lot of my problem was actually that I had feathered edges. Once I set the alpha to have a hard falloff, it works pretty good.

So my remaining problem is this:

Using your xBR upscales as a base, I downscale them from 8x vanilla resolution to 4x vanilla resolution, I do get smooth silhouettes but the xBR interprets some diagonal lines as odd melted stair steps which look weird... It does provide a good starting point though, and cleaning those issues is fairly easy to do manually, just time consuming and not possible to batch. For the most part, the results look pretty good, but up close the edges still look a little murky without a lot of careful editing to get clean the silhouette.

For example, look at the top part of the shield. It looks pretty close to round, but the bottom I didn't clean up as well and it still looks wavy.

I had to edit the battle axe pretty heavily to make the shaft straight and the blade curved properly.

Re: AI Upscaled Textures

Posted: Fri Mar 01, 2019 5:43 pm
by King of Worms
Hi, Im glad the main problem is dealt with, no matter how it happened :)

The XBR algo is quite limited in in certain situations, like the very low res lines which are in lets say 45 degrees direction, or the round shapes upscale from the low res. Exactly what you found out.

I take it as a artistic direction, and when these things get animated, it looks proper to me. Ofc, given enough efforts, these can be manually corrected, like you did.

And I did that as well, if you look at the skeletons shield when he is directly approaching the player (or directly attacks a player) its corrected. I just did not correct it in all those other degrees, like the 45 degree you posted. I made this decission because there were just too much frames to do, so I focused only on those, which are usually visible. (also, when I did this, the fights between enemies were not present, so basically, those frames were not possible to see to a player)

All mobs have manually cleaned the "direct to player" frames tho... there were countless artefacts and imperfections to deal with, and I did that .)

Plus other touches I gladly forgot, like the orc shamans staff changing color from yellow to red in certain animations was corrected etc etc etc... I dont really remember all...

I did what I was able to do in a given time, and I focused on things which are visible most of the time - lets call it bang for a buck approach... just its a bang for a invested time and efforts..

The skeleton looks great btw.. and in case you grow tired of the manual touches, it will look good even without the corrections when in movement. Its up to You to decide, how much time and effort you can invest in this. Id say, go on, until you cant stand it anymore :)

PS: I still think the "direct to player" frames are a priority. So a "walk towards a player" and attacks, plus mob being hit. Plus I usually did those 5 frames where the mob is rotating. They are at the end of each mob frame set, just after the hit frames.

PSS: Id say, batch process all the frames, see how it looks ingame, and than decide on the other tweaks maybe?

Good luck!! Im VERY happy to see the mobs being taken to another level by you. Its the best possible outcome! Thank you

Re: AI Upscaled Textures

Posted: Fri Mar 01, 2019 7:51 pm
by MasonFace
The skeleton looks great btw.. and in case you grow tired of the manual touches, it will look good even without the corrections when in movement. Its up to You to decide, how much time and effort you can invest in this. Id say, go on, until you cant stand it anymore :)
Thanks for the encouragement! I figured I was being a bit overly self-critical of the results and it probably didn't look as bad as I thought. I'll try to post some screenshots tonight of what the raw output looks like without the silhouette getting manually cleaned up just for comparison.
I did what I was able to do in a given time, and I focused on things which are visible most of the time - lets call it bang for a buck approach... just its a bang for a invested time and efforts..
Very smart! I lost sight of this. I call it the "Pareto Approach" or the "80/20 rule," which states that you can expend 20% of the effort to achieve 80% of the results by focusing on the ones that matter most. Great suggestion! Of course I will release the results so others can clean them up if they have time and want to contribute.
PSS: Id say, batch process all the frames, see how it looks ingame, and than decide on the other tweaks maybe?
Unfortunately, I don't have any of this batched yet... I have a process that should theoretically be batch-able, but I haven't committed the time to writing a GIMP macro to test it (nor have I ever written a GIMP macro/script/routine/filter/whatever before, so it may take some time). Jukic was working on making a batch process to handle this, but I still owe him a larger test set. I'm still working on it Jukic, I swear! :lol:

Re: AI Upscaled Textures

Posted: Fri Mar 01, 2019 8:11 pm
by King of Worms
Looking forward more screens :)

Re: AI Upscaled Textures

Posted: Sat Mar 02, 2019 9:41 am
by MrFlibble
In the meantime, Phredreeke of the Duke4.net forums released a Duke Nukem 3D Sprite Upscale pack, created using ESRGAN.

Phredreeke is using Paint Shop Pro scripts to get proper aliasing on the edges, and I think the results he got are way better than any other solution people have come up so far for this problem (I heard hidfan who created the Doom Neural Upscale edited alpha masks for sprites manually!):
Image
I already asked Phredreeke to share his scripts, that could give the Daggerfall sprite project a good boost.

Re: AI Upscaled Textures

Posted: Sat Mar 02, 2019 4:31 pm
by MasonFace
Those are great results! I'm hoping he will share the scripts. That would be a HUGE boost towards this project!

Re: AI Upscaled Textures

Posted: Wed Mar 06, 2019 11:16 pm
by Kamer
MrFlibble wrote: Sat Mar 02, 2019 9:41 am In the meantime, Phredreeke of the Duke4.net forums released a Duke Nukem 3D Sprite Upscale pack, created using ESRGAN.

Phredreeke is using Paint Shop Pro scripts to get proper aliasing on the edges, and I think the results he got are way better than any other solution people have come up so far for this problem (I heard hidfan who created the Doom Neural Upscale edited alpha masks for sprites manually!):
Image
I already asked Phredreeke to share his scripts, that could give the Daggerfall sprite project a good boost.

If we can get those scripts then I won't really need to work on the 3d models anymore. I wouldn't see the point in it if we can get the df prites looking this good.

Re: AI Upscaled Textures

Posted: Thu Mar 07, 2019 3:53 am
by MasonFace
So I've refined my method to get good edges on the sprites without manual touch ups. The trick is to use the ESRGAN upscale with a high contrasting color (like purple) as the background, then alter to the color contrast of that image WAY up so that subject and background are as different as possible and the silhouette of the subject stands out perfectly, then you have a good high resolution masking layer to restore the alpha with.

The upscaling method below is a composite of 10% ESRGAN, 45% SFTGAN, and 45% Gigapixel AI results. The results of this composition is then fed into Topaz AI Sharpen tool, then one last pass with GIMP's unsharp mask. I'm quite pleased with the results.
RatUpscale.png
RatUpscale.png (291.87 KiB) Viewed 2199 times
The next step will be for me to write some scripts for GIMP to automate this as much as possible and make progress much faster.
If we can get those scripts then I won't really need to work on the 3d models anymore. I wouldn't see the point in it if we can get the df prites looking this good.
Nonsense! Eventually 3D models could be implemented for the MOBs and NPCs so your 3D models will still have plenty of utility.

Re: AI Upscaled Textures

Posted: Thu Mar 07, 2019 4:53 am
by Kamer
True but if we can get hi-rez upscaled versions of the original sprites, then making models and pre-rendering them would be a waste of time cause this looks like a much better option. If anything the goal of the project will change. Probably to modernize the sprites by giving them extra details but keeping to the artstyle as close as possible. That rat looks amazing btw.

Re: AI Upscaled Textures

Posted: Thu Mar 07, 2019 5:17 am
by NyanCommissar
MasonFace wrote: Thu Mar 07, 2019 3:53 amThe upscaling method below is a composite of 10% ESRGAN, 45% SFTGAN, and 45% Gigapixel AI results. The results of this composition is then fed into Topaz AI Sharpen tool, then one last pass with GIMP's unsharp mask. I'm quite pleased with the results.
Well, it appears you got the best pattern for upscaling NPC sprites. But how about the humanoid ones?