AI Upscaled Textures

Show off your mod creations or just a work in progress.
User avatar
King of Worms
Posts: 2354
Joined: Mon Oct 17, 2016 11:18 pm
Location: Scourg Barrow (CZ)
Contact:

Re: AI Upscaled Textures

Post by King of Worms » Fri Jan 11, 2019 8:20 pm

I solved that cutout issue by reducing the resolution using default Photoshop bicubic algo. My sprites had huge resolution, so I reduced it to 50% I guess... and that created that smooth alpha around the edges so all the white artefacts ingame vanished and sprites fit in perfectly.

I also like the pixelated look, but if you use bi/trilinear filtering ingame, its just a mess..
I was thinking about being able to use point filter on sprites and bi/tri filters on textures for example, like its done in Doom emulators etc..

That would be interresting

Untill than, I use sprites which are cca 1080p in height in case of NPCs. That way, they look sharp even with bi/tri filtering ON...


Thinking about it.... I have all the MOBs upscaled with XBR4x times two and than reduced to 50%, than some manual touches to clean up artefacts, plus I corrected skeleton animation phases, used better version of a bear etc.... all with correct full naming... so... maybe you can use that as a base if it helps you?

Looks like this
Spoiler!
261_0-1.png
261_0-1.png (295.48 KiB) Viewed 737 times
262_5-1.png
262_5-1.png (342.53 KiB) Viewed 737 times
270_5-4.png
270_5-4.png (355.8 KiB) Viewed 737 times

Narf the Mouse
Posts: 758
Joined: Mon Nov 30, 2015 6:32 pm

Re: AI Upscaled Textures

Post by Narf the Mouse » Fri Jan 11, 2019 11:51 pm

MasonFace wrote:
Fri Jan 11, 2019 10:17 am
Well, I got one MOB done. I decided to work on an enemy (Fighter Mage) that shows up pretty early in Privateer's Hold to make it easy to test in-game.
That one's an archer. :)

Ref: I have Privateer's Hold memorized, to the point I bug-reported on the "wrong" monsters showing up in DFU. lol
Previous experience tells me it's very easy to misunderstand the tone, intent, or meaning of what I've posted. If you have questions, ask.

MrFlibble
Posts: 130
Joined: Sat Jan 27, 2018 10:43 am

Re: AI Upscaled Textures

Post by MrFlibble » Sat Jan 12, 2019 10:59 am

MasonFace wrote:
Wed Jan 09, 2019 8:11 pm
Also, I certainly don't want to discourage you from trying to train new models.
Training models is beyond my capabilities ATM, 'cause I'm stuck with CPU mode even though I have an NVIDIA card but my current distro is officially unsupported by CUDA. I have asked at the distro dev forums, maybe they will suggest something.

The same PC has a Windows 8 OEM version, which is again unsupported by CUDA :(

On another note, I discovered that if you use Sinc interpolation instead of pixelise + nearest neighbour when applying xBR softening, you get sharper results with all the advantages of smoother lines.

UPD: There's also a difference depending on whether you scale the image up using xBR and xBRZ. It affects some of the shapes in the image but also the xBRZ version is even sharper. Here's a quick comparison:

Pixelise + nearest neighbour (xBR)
Image

Sinc (xBR)
Image

Sinc (xBRZ)
Image

User avatar
King of Worms
Posts: 2354
Joined: Mon Oct 17, 2016 11:18 pm
Location: Scourg Barrow (CZ)
Contact:

Re: AI Upscaled Textures

Post by King of Worms » Sat Jan 12, 2019 11:26 am

Nice, the sharper results will come handy, because to get those images ingame in a way they dont destroy everyones PCs RAM/VRAM, we need to compress them, and that will remove a bit of that sharpness indeed. So the 2nd/3rd method seems to be better.

MrFlibble
Posts: 130
Joined: Sat Jan 27, 2018 10:43 am

Re: AI Upscaled Textures

Post by MrFlibble » Sat Jan 12, 2019 3:23 pm

King of Worms wrote:
Sat Jan 12, 2019 11:26 am
because to get those images ingame in a way they dont destroy everyones PCs RAM/VRAM, we need to compress them, and that will remove a bit of that sharpness indeed
Actually I believe that the best way to process the sprites is to convert them back to the original palette, which will also reduce the file size. Surely not use lossless compression like JPEG!

In the meantime, someone trained another model (actually four models) on random pieces of art:
https://www.reddit.com/r/GameUpscale/co ... i_trained/

UPD: Did a quick run, the only remotely tolerable for sprites is the JPGFriendly variant, but it's far worse looking than the Manga model, with noticeable ringing and JPEG artifacts.

User avatar
King of Worms
Posts: 2354
Joined: Mon Oct 17, 2016 11:18 pm
Location: Scourg Barrow (CZ)
Contact:

Re: AI Upscaled Textures

Post by King of Worms » Sat Jan 12, 2019 4:52 pm

Unfortunatelly, this is not how it works, because Unity converts the 256 color original palete to 32bit or something. I converted my whole project to palette and it helps nada with vram/ram

I use unity compression, theres quite a lot of options... its absolutely crucial your images size can be divided by 4.... height and width

I use PNG and compress with unity. If done right, it looks crisp.

Converting to Dagger pallete saves ONLY disk space.

Hope it helps (if only I knew this one year back :) )

I can help with this process when needed laters down the road and we create a DFMOD packages...

Use PNG and keep the size so it can be divided by 4 and u will be golden

User avatar
King of Worms
Posts: 2354
Joined: Mon Oct 17, 2016 11:18 pm
Location: Scourg Barrow (CZ)
Contact:

Re: AI Upscaled Textures

Post by King of Worms » Sat Jan 12, 2019 7:22 pm

btw guys, any magic trick to upscale things like this nicely?
0-0.PNG
0-0.PNG (17.91 KiB) Viewed 628 times

User avatar
Jukic
Posts: 68
Joined: Sun May 13, 2018 5:51 pm

Re: AI Upscaled Textures

Post by Jukic » Sat Jan 12, 2019 7:27 pm

@mason: no prob, I wish You guys all the best! I am also "full-in" anyways :) Regarding sprites: dark background - no prob. just dont let it be complete black but some specific color, whatever suites You best.

User avatar
MasonFace
Posts: 346
Joined: Tue Nov 27, 2018 7:28 pm
Contact:

Re: AI Upscaled Textures

Post by MasonFace » Sun Jan 13, 2019 6:25 am

----------------------------

@Narf:
That one's an archer. :)
I was just going by what "Daggerfall Imaging 2" had populated in its description field. I guess it makes since that he's an archer and not a fighter mage since he shoots a bow but doesn't have any spell casting animations... lol

----------------------------

@King of Worms:
I think it should be 2x the resolution IMO tho... I will try it. Thank you
Sorry, I first thought you meant 2x the vanilla DF resolution, not twice the AI upscaled resolution (which would be 8x the vanilla DF resolution).
Until then, I use sprites which are cca 1080p in height in case of NPCs.
Here I go exposing my ignorance again, but what does "CCA" mean?
Thinking about it.... I have all the MOBs upscaled with XBR4x times two and then reduced to 50%, than some manual touches to clean up artefacts, plus I corrected skeleton animation phases, used better version of a bear etc.... all with correct full naming... so... maybe you can use that as a base if it helps you?
I think I can use these actually. I'll just pull them from the DREAM mod. See my response to Jukic below.
btw guys, any magic trick to upscale things like this nicely?
The AI messes up text so I clone brushed out the text at the top before upscaling it. I tried several things, but what ended up working best is using MrFlibble's classic xBR softening technique, then running it through SFTGAN. Then I ran it through the unsharp mask filter to accent a few subtle details.

Image

----------------------------

@Jukic:
Jukic wrote:
Sat Jan 12, 2019 7:27 pm
@mason: no prob, I wish You guys all the best! I am also "full-in" anyways :) Regarding sprites: dark background - no prob. just dont let it be complete black but some specific color, whatever suites You best.
Here are all the orc frames with a funky color background. They're a little rough around the edges. Beware: the AI adds a little bit of noise, even to the background for some reason which may complicate your workflow. It's a shame that the alpha has to be removed in order for the AI to work... :?

ESRGAN:
https://drive.google.com/file/d/1khYVar ... sp=sharing

SFTGAN:
https://drive.google.com/file/d/1cKnz3o ... sp=sharing

KoW's DREAM mod Orc downscaled to match ESRGAN and SFTGAN 4x scale
https://drive.google.com/file/d/1cKUBbO ... sp=sharing

Below I did a more complex procedure where I composited the results of all three. I think this best combines the strengths of the different upscaling methods (at least in this case). It uses KoW's upscales for the edge of the sprite, then combines some of the details from both ESRGAN and SFTGAN.
Image

Basically, I used KoW's orc from the DREAM mod (scaled back down from 8 times to 4x the vanilla DF resolution) as the bottom layer and used it for the edges. I selected its outline, grew the selection by 2 pixels, feathered by 3, then deleted that selected portion from the ESRGAN and SFTGAN layers. Basically, I wanted the outline of KoW's orc with the inside texture of the AI upscaled stuff. I then blended the ESRGAN and SFTGAN by setting the ESRGAN to the top layer and setting its opacity to 50%. SFTGAN was the middle layer with 100% opacity. I have no idea how feasible all that is to do with batching. Feel free to experiment with your own methods.

----------------------------

@MrFlibble:
Training models is beyond my capabilities ATM, 'cause I'm stuck with CPU mode even though I have an NVIDIA card but my current distro is officially unsupported by CUDA.
That sucks! :(
I think training is also outside my capabilities. From what I understand, it would take my GTX 1060 over a week to train a model even with CUDA... although I could be wrong about that.
On another note, I discovered that if you use Sinc interpolation instead of pixelise + nearest neighbour when applying xBR softening, you get sharper results with all the advantages of smoother lines.
So when you use the Sinc interpolation, you don't apply the pixelize filter at all? I tried that but I'm not quite getting the same results. One caveat is that I'm now running Gimp 2.10 which replaced the Sinc interpolation with two other algorithms, NoHalo and LoHalo, which are supposed to be basically the same but "better". Not sure how.

User avatar
VMblast
Posts: 502
Joined: Wed Mar 29, 2017 12:22 pm
Contact:

Re: AI Upscaled Textures

Post by VMblast » Sun Jan 13, 2019 8:06 am

Ive added the text. Now its properly rescaled :)


Image



And the same one but with added detail texture - it gives it more spunk. :P
Spoiler!
Image

Post Reply