AI Upscaled Textures

[MasonFace]

1) Obviously some assets were created at a low resolution from scratch.

Is that obvious? I can't say for sure they were originally drawn at higher resolution then downscaled. Likewise, I can't say for sure that you're wrong.

2) Now onto the pre-rendered sprites which are the most promising venue with neural upscaling. Even if you scale them up with the most simple waifu2x method you'll notice one thing: the models are rather underdetailed. The hands have no separate fingers, no toes if barefoot, the faces are very basic if visible at all etc.

True, but you also stated that this was due to time limitations not technical limitations, but I do concede there is a bit of a catch-22 in that adding more detail sometimes reveals a lack of detail.

Regardless, I think we're getting a bit into the weeds here. Why don't we make the project mission statement something a little less rigid:

What to upscale?
Whatever interests you! I'd prioritize graphics that haven't already been enhanced by others (looking for gaps in DREAM for example) and also prioritize graphics that can be batched. Try to get the most bang for your buck when it comes to your time and effort. For example, I'd like to upscale a lot of the textures, not because they haven't already been redone by DREAM, but because it interests me to enhance them closer to the vanilla style and I can bang them out fairly quickly.

Upscale to what extent?
Until it looks good to your eye! As long as it looks better than the original graphic and you're proud of the result, then it's good enough. In this sense, it may just be best to decentralize the decision and let the individual contributor be the judge of what looks best. If someone doesn't like it, they can roll up their sleeves and try to do better themselves.

Bottom line: Let's just have fun and apply our skills as best we can. This is a hobby project after all, so spend your time and effort wisely. And don't worry- the results couldn't possibly be any more disjointed than the originals.


On the topic of batching, I've got my GIMP script working to where it will correctly blend the ESRGAN, SFTGAN, and Gigapixel results to predetermined proportions using GIMP 2.10's new "Merge" layer mode. For example, the Giant results looked best with 60% Gigapixel, 30% ESRGAN, and 10% SFTGAN merged together. Then it applies Phredreeke's mask layer to restore the alpha and saves it in GIMP format and PNG. While it does all these steps autonomously, I have to manually iterate to the next image after each upscale... so not really batching.

It turns out, in order to batch it properly, I have to rewrite my script from Python into Scheme language so I can take advantage of filename globbing. Shouldn't be hard, I just have to carve out the time to do it.

[phredreeke]

In PSP I can choose a number of files in a folder, and they will all process using a script I choose and save to another folder I've set. Can't GIMP do something similar?

[King of Worms]

I feel the MOBs results shown in this thread are superior to what I have in DREAM. And im quite interrested in the AI textures. I THINK the NPCs in my mod are quite ok in most of the cases. I somehow doubt the AI upscaling and combination of methods can give better results than hours of hand work done on them. I would gladly be prooven wrong tho. For example the dancer NPC posted here was awesome and better than mine (the body) But tha was also because I created mine from a lower res spurce material (somehow I missed the better resolution sprite in the game files - my bad)

But ultimately. Do what makes you enjoy the process. I quite resonate with the mission statement above Have fun guys. You already have some great results! Just push long enough and you cant go wrong. More mobs pls

[MasonFace]

In PSP I can choose a number of files in a folder, and they will all process using a script I choose and save to another folder I've set. Can't GIMP do something similar?

GIMP has a plugin called BIMP that can batch process multiple images following registered scripts/filters. I tried it and it only spit out the masking layers as output. Now that you mention it, I think I may have uncovered the solution to getting BIMP working while researching how to batch with Scheme. I think I might give it another try before I go through the pain of re-coding the whole thing in Scheme. Thanks for bringing it up!

I THINK the NPCs in my mod are quite ok in most of the cases. I somehow doubt the AI upscaling and combination of methods can give better results than hours of hand work done on them. I would gladly be proven wrong tho.

I'm afraid you may be right. The vanilla NPC sprites are SO low resolution that the AI really struggles to extract novel details from it. I'd like to try my hand at it eventually, but I don't think I can top MrFlibble's results.

And hey! Aren't you supposed to be on holiday?! Get outta here! Go have fun or something!

[King of Worms]

Lets pretend this conversation never happened laters!!

[jayhova]

It seems like this thread poses the unanswerable question as to what would be more genuinely Daggerfall. All anyone can do is give an opinion. How do we derive the best, most genuine, results from what we have now? The truth is there is no perfect answer. We can guess what the theoretical art would have looked like from the low resolution renderings of it. Do originals exist in a drawer at Bethesda somewhere? Are they in Lynda Carter's closet? Did they ever exist? I think it likely that Mark did peleminay work, probably on paper before it was reduced down to what we have now.

If high resolution copies don't exist what is the next best solution. Even if Mark redid the art, It would change because he could show things he could not have shown before. Does this make the recreation less genuine? That seems to be mostly a matter of opinion. Maybe one day we can have a kickstarter campaign and give mark $10,000 to redo all the art and remake the dragon. To my mind that would be the best solution, to just let Mark do it his way however he pleases.

[MasonFace]

I think I may have uncovered the solution to getting BIMP working while researching how to batch with Scheme. I think I might give it another try before I go through the pain of re-coding the whole thing in Scheme. Thanks for bringing it up!

I fiddled with it for about 2 hours last night and I still can't get BIMP to batch the results. I think I'm just going to have to bite the bullet and rewrite the code in Scheme. Really isn't a very long script so it shouldn't take but a few hours to do with any luck.

To my mind that would be the best solution, to just let Mark do it his way however he pleases.

I see he has a LinkedIn account. Has anyone tried to reach out to him?

[MrFlibble]

Hey guys,

First off, I really like the new mission statement. After all, the neural network tech we're using is highly experimental even in the field of its primary application (i.e., upscaling low-res photos) so the more we experiment the better.

Secondly, I have now concluded that my previous method of xBRZ softening is lacking and produces unnecessary blur. It is better to scale the image up with xBR, apply Gaussian blur at 1.5 pixel radius (this is done only to compensate for the sharpening effect produced by bicubic interpolation when downsampling) and then resize to the original resolution using bicubic interpolation.

Here's a small comparison, both images upscaled using the ESRGAN/Manga109 interpolation at 0.2. As you can see the xBRZ softened image is just very noticeably blurry with no improvement of the edges or whatever.

Third, Phredreeke game me a very interesting tip to add noise to an image before upscaling. This seems to result in interesting effects. Here's the same bunch of dudes as above (also processed with ESRGAN/Manga109 0.2. interpolation) but with HSV noise (GIMP, default settings) added to the image before processing it with xBR (the image also had GIMP Antialiase before adding HSV):

It seems to result in smoother sprite edges in some places.

[phredreeke]

I'd like to clarify that my preprocessing script does not apply a fixed blur effect across the whole image. Here is a mask used internally in the script (halted it midway to show it)

zombieaamask.png (6.6 KiB) Viewed 2905 times

Black areas are left as is, white parts being fully blurred, with greys somewhere in between. In this frame the strongest blur is applied to the stairsteps in the falling axe,

[MasonFace]

@MrFlibble:

I gave your new method a try and can confirm that the results were better than the previous xBRZ softening method.

I applied the steps as below with the following caveats for GIMP 2.10:

• Scaled up with xBR to 4x the original size

• Applied gaussian blur with value of 0.50 (1.5 was WAY too strong with this version of GIMP, not sure why)

• Scaled the image down 25% using cubic interpolation (GIMP 2.10 renamed it from "Bicubic" to just "Cubic")

• Applied anti-alias. This didn't appear to do anything, but it may have just been too subtle to notice

• Added HSV noise. This seemed to do the most good. Makes sense considering the images the models were trained on.

I tried to make a script in GIMP to do all these steps at once, but I couldn't get it to work. I must be losing my Python mojo!

Also, I didn't have any luck getting my composition script to batch in GIMP over the weekend. I have one last thing to try before I give up and just run it individually on each frame. A single MOB with 80 frames takes me about 15 minutes doing it this way. I'd rather be able to just pass in the directory, the % ESRGAN, SFTGAN, and Gigapixel, then let file globbing take care of the whole set of frames for that MOB while I get some tea.

So far, I've finished most of the MOBs that are in privateer's hold. I'm only lacking the male and female thieves and the bear. So far they are all looking very good in game.

@Phredreeke:

Speaking of the bear, I'm having a very hard time getting the outlines to work right on the bear (archive# 457 I think?). I'll upload a picture when I get home so you can see what's going on, but the outline is both off center and too large. It looks like you may have padded the sprite with more pixels than the others I've used so far, but I'm not sure if that has anything to do with it. I have only tried the first two frames and they both had the problem, but I haven't check the rest yet.

Other than that, the outlines have been perfect about 98% of the time. I think the skeletal warrior (archive# 270) had one frame (270_15-0 I think) that was off by alot, but it wasn't hard to fix by hand.

Also, very interesting about how your script works. It's almost like the inverse of GIMP 2.10's "selective blur" which keeps the edges sharp while blurring the insides. Your script looks like it detects the edges, then blurs strongest there while keeping the inside contents sharp. Very cool.