AI Upscaled Textures

Show off your mod creations or just a work in progress.
Post Reply
User avatar
MasonFace
Posts: 355
Joined: Tue Nov 27, 2018 7:28 pm
Contact:

Re: AI Upscaled Textures

Post by MasonFace » Sun Dec 23, 2018 7:45 pm

I was tinkering around with ESRGAN and SFTGAN this morning and found a workflow that produces consistently decent results.

My new method involves appying a 1 pixel guassian blur to the vanilla texture then feeding it into ESRGAN, then applying a sharpen filter to the result, then upscaling that with SFTGAN. The results look a little better to my eye:

Image

Image

Then I got a bit curious what the finals results might look like in-game with normal, occlusion, and emission maps applied, so I manually created some to slap on there:

Image

I still don't consider the results stunning, but it gets acceptable results without much work. I could probably churn out some good textures in little time.

DassiD
Posts: 2
Joined: Sun Dec 23, 2018 7:56 pm

Re: AI Upscaled Textures

Post by DassiD » Sun Dec 23, 2018 8:04 pm

I suggest using ESRGAN mixed PSNR, that sould be even better than sharpening + SFTGAN. That way you can very nicely control the amount of noise, and do different levels for different types of texures. From the git page:

"You can interpolate the RRDB_ESRGAN and RRDB_PSNR models with alpha in [0, 1].

Run python net_interp.py 0.8, where 0.8 is the interpolation parameter and you can change it to any value in [0,1].
Run python test.py models/interp_08.pth, where models/interp_08.pth is the model path."

Image

Hope this can be of help!

User avatar
King of Worms
Posts: 2526
Joined: Mon Oct 17, 2016 11:18 pm
Location: Scourg Barrow (CZ)
Contact:

Re: AI Upscaled Textures

Post by King of Worms » Sun Dec 23, 2018 8:19 pm

Looks promissing! Id like to see some enemy sprites, or npcs, or tree sprites...

User avatar
MasonFace
Posts: 355
Joined: Tue Nov 27, 2018 7:28 pm
Contact:

Re: AI Upscaled Textures

Post by MasonFace » Sun Dec 23, 2018 8:30 pm

I suggest using ESRGAN mixed PSNR, that sould be even better than sharpening + SFTGAN. That way you can very nicely control the amount of noise, and do different levels for different types of texures.
The ESRGAN seems to put out a LOT of noise on these very small textures and I've been paying around with the interpolation to see how it reacts. Right now I'm using interpolation value of alpha = 0.1 to get a smoother output before feeding its result into the SFTGAN and it seems to work okay. The first images I posted to this thread using ESRGAN was using alpha = 0.8 and you can see how it added a lot of strange details into the texture. It was interesting, but not all that great. Blurring the vanilla texture by 1 pixel seemed to help a little in preventing ESRGAN from injecting more noise.

I think what would help this out the most would be for us to train the AI on a large set of HD textures so that it is trained for this specific task (I think the included models were trained on very general pictures). Does anyone happen to have a large set of HD textures (maybe 500 or more?) that we could run the training algorithm on? It would also help if said person has a Geforce 1080TI or two...
Looks promissing! Id like to see some enemy sprites, or npcs, or tree sprites...
I will try this method on some NPCs and vegitation when I get home this evening, but I suspect it won't work very well... but I hope I'm wrong!

DassiD
Posts: 2
Joined: Sun Dec 23, 2018 7:56 pm

Re: AI Upscaled Textures

Post by DassiD » Sun Dec 23, 2018 9:16 pm

"I think what would help this out the most would be for us to train the AI on a large set of HD textures so that it is trained for this specific task (I think the included models were trained on very general pictures). Does anyone happen to have a large set of HD textures (maybe 500 or more?) that we could run the training algorithm on? It would also help if said person has a Geforce 1080TI or two..."

In the Morrowind community, we're trying to source textures from several models, and train some of our own too.
It's in the starting phase, but I'll make sure to share the results of any successful training.

https://docs.google.com/document/d/1mrm ... sp=sharing

User avatar
MasonFace
Posts: 355
Joined: Tue Nov 27, 2018 7:28 pm
Contact:

Re: AI Upscaled Textures

Post by MasonFace » Mon Dec 24, 2018 6:01 pm

In the Morrowind community, we're trying to source textures from several models, and train some of our own too.
It's in the starting phase, but I'll make sure to share the results of any successful training.
I've been wanting to reach out to the Morrowind AI Upscaling community, but wanted to learn more before I did; I didn't want to sound too ignorant before asking for help. Thank you for taking the initiative and reaching out to us! :)

Also, I think your mission statement sounds right-on-the-money. I think it's an excellent approach. Maybe we can form a group within the DFU community to curate the results into one collection that best fit the original style while also fitting among themselves.

Please keep in touch!

User avatar
MasonFace
Posts: 355
Joined: Tue Nov 27, 2018 7:28 pm
Contact:

Re: AI Upscaled Textures

Post by MasonFace » Mon Dec 24, 2018 7:29 pm

Working on Dungeon Entrances:
Spoiler!

Image

Image

User avatar
King of Worms
Posts: 2526
Joined: Mon Oct 17, 2016 11:18 pm
Location: Scourg Barrow (CZ)
Contact:

Re: AI Upscaled Textures

Post by King of Worms » Mon Dec 24, 2018 9:37 pm

Well, that is REALLY good!

Narf the Mouse
Posts: 837
Joined: Mon Nov 30, 2015 6:32 pm

Re: AI Upscaled Textures

Post by Narf the Mouse » Mon Dec 24, 2018 11:52 pm

Hmm... Train it on manually-upscaled textures, then feed in the original textures? Theoretically, we could get greater than 100% efficiency out of this...
Previous experience tells me it's very easy to misunderstand the tone, intent, or meaning of what I've posted. If you have questions, ask.

User avatar
VMblast
Posts: 504
Joined: Wed Mar 29, 2017 12:22 pm
Contact:

Re: AI Upscaled Textures

Post by VMblast » Tue Dec 25, 2018 12:17 pm

These are good! 👍

One thing tho, textures need to be at least 512x1024 to work properly ingame. What I mean is that for the monitor HD resolution of 1920x1080 is best visually supported by (minimal) vertical texture size of 1024.

Post Reply