AI Upscaled Textures
- King of Worms
- Posts: 4753
- Joined: Mon Oct 17, 2016 11:18 pm
- Location: Scourg Barrow (CZ)
- Contact:
Re: AI Upscaled Textures
I would add that the textures need to be able to be divided by 4 on both dimensions. Its best to stick to standards like VMBlast mentioned
128
256
512
1024
2048
It saves a lot of time afterwards...
128
256
512
1024
2048
It saves a lot of time afterwards...
DREAM Nexus https://www.nexusmods.com/users/73250038?tab=user+files
DREAM Forums viewtopic.php?f=27&t=1168
DISCORD: king_of_worms_cz
DREAM Forums viewtopic.php?f=27&t=1168
DISCORD: king_of_worms_cz
- MasonFace
- Posts: 543
- Joined: Tue Nov 27, 2018 7:28 pm
- Location: Tennessee, USA
- Contact:
Re: AI Upscaled Textures
One thing tho, textures need to be at least 512x1024 to work properly ingame. What I mean is that for the monitor HD resolution of 1920x1080 is best visually supported by (minimal) vertical texture size of 1024.
Thanks for reminding me! My upscales were hitting weird resolutions and I couldn't figure out why they weren't looking quite right in-game. They look better now. BTW, any idea when custom streaming world texture injection will be supported?I would add that the textures need to be able to be divided by 4 on both dimensions. Its best to stick to standards like VMBlast mentioned
I've been poking around with this and it the results are very hit-or-miss (mostly miss) when it comes to sprites. The best I've gotten so far is the horse, and it needs a bit of manual cleaning up around the silhouette and it would look pretty good.Id like to see some enemy sprites, or npcs, or tree sprites...
I first used blur -> ESRGAN and the results were pretty good.
Then I used just SFTGAN and got much better detail, but a little worse silhouette.
The second best (and it's a far second) was the dog. The problem seems to be that there just isn't nearly enough pixels to start off with (a measly 56x40!) for the AI to infer any additional detail from...
It doesn't look any better than just a regular cubic interpolated scaling...
I tried a few trees and bushes, but they all came out either just terribly blurry or highly contaminated with artifacts.
I may try a few more experiments on the sprites to see if I can get consistently decent results. Otherwise, I'll continue to focus on textures.
Yup, that's the end-game! Would you be able to help? I need a compilation of textures close to or above our target resolution (1024x1024 would probably be okay), preferably of hand-painted textures of a variety of materials (wood, stone, brick, marble, metal, etc.). I will need a bare minimum of 100, but we will get increasingly better results with more training data. I will continue to read up on how to actually perform the training... from what I understand, that is the actual computationally expensive part of the AI.Hmm... Train it on manually-upscaled textures, then feed in the original textures?
-
- Posts: 177
- Joined: Sun Mar 22, 2015 9:52 am
Re: AI Upscaled Textures
Wow, how does SFTGAN know the horse's eye should actually be an eye, with round pupil and gleam and all, from only six pixels? I use neural networks at my job but I never stop being amazed.
- King of Worms
- Posts: 4753
- Joined: Mon Oct 17, 2016 11:18 pm
- Location: Scourg Barrow (CZ)
- Contact:
Re: AI Upscaled Textures
SFTGAN horse is quite ok.
Well... I have a idea... I can give you some upscaled versions of the original art so you can train the AI on it.
Im sure my NPCs and Sprites are the best learning material for it. Its VERY vanilla and high quality compared to source materials.
Than I have upscaled textures...
Im not so sure with them in terms of AI learning. Some of them are vanila upscaled with XBR4x... some of them are redrawn.
Tell me where to start.
NPC Example
Well... I have a idea... I can give you some upscaled versions of the original art so you can train the AI on it.
Im sure my NPCs and Sprites are the best learning material for it. Its VERY vanilla and high quality compared to source materials.
Than I have upscaled textures...
Im not so sure with them in terms of AI learning. Some of them are vanila upscaled with XBR4x... some of them are redrawn.
Tell me where to start.
NPC Example
Spoiler!
DREAM Nexus https://www.nexusmods.com/users/73250038?tab=user+files
DREAM Forums viewtopic.php?f=27&t=1168
DISCORD: king_of_worms_cz
DREAM Forums viewtopic.php?f=27&t=1168
DISCORD: king_of_worms_cz
- MasonFace
- Posts: 543
- Joined: Tue Nov 27, 2018 7:28 pm
- Location: Tennessee, USA
- Contact:
Re: AI Upscaled Textures
.
TL;DR - Let's start with textures. I'll need at least 100 unique hand-painted textures.
I think we would do best to train separate models: one for textures, another for NPCs/animals, and maybe another for vegetation/foliage. And despite what we've seen so far, I actually think ESRGAN would win out over SFTGAN if we train the models specific to our tasks, but I'd like to train both ESRGAN and SFTGAN with the same training data and see how they compare. I think the best place to start would be textures - seems it would be the least challenging to find enough training data to get useful results (100-500 unique samples I'm thinking).
First, we need to define what outcome we are shooting for; I personally prefer the "hand-painted" look for Daggerfall textures over the more realistic ones. We can import collections of hand-painted textures that all look approximately the same style, including those you and VMBlast have created. This set would be our "ground truth" samples and will be the target that the AI is striving to achieve. The training inputs will simply be the ground truth samples downscaled to vanilla DF resolution. It would be helpful if the hand-painted ground truth samples contain various materials (such as metal, marble, wood, stone, grass, etc.) and various features (pillars, beams, bolts/nail heads, window frames, cracks, etc.). Over the course of the training process, the GAN will "learn" how to fill in all the missing information during the upscale.
Again, the hand-painted ground truth samples don't have to be just previously upscaled DF textures. If you guys have any prior art or know of a royalty-free source of these types of textures, they will probably work just fine.
In the meantime, I'm going to mess around with some other upscaling techniques such as Waifu2x and XBR4x like you mentioned and see if I can somehow combine them with these GANs to enhance the output a little.
TL;DR - Let's start with textures. I'll need at least 100 unique hand-painted textures.
I think we would do best to train separate models: one for textures, another for NPCs/animals, and maybe another for vegetation/foliage. And despite what we've seen so far, I actually think ESRGAN would win out over SFTGAN if we train the models specific to our tasks, but I'd like to train both ESRGAN and SFTGAN with the same training data and see how they compare. I think the best place to start would be textures - seems it would be the least challenging to find enough training data to get useful results (100-500 unique samples I'm thinking).
First, we need to define what outcome we are shooting for; I personally prefer the "hand-painted" look for Daggerfall textures over the more realistic ones. We can import collections of hand-painted textures that all look approximately the same style, including those you and VMBlast have created. This set would be our "ground truth" samples and will be the target that the AI is striving to achieve. The training inputs will simply be the ground truth samples downscaled to vanilla DF resolution. It would be helpful if the hand-painted ground truth samples contain various materials (such as metal, marble, wood, stone, grass, etc.) and various features (pillars, beams, bolts/nail heads, window frames, cracks, etc.). Over the course of the training process, the GAN will "learn" how to fill in all the missing information during the upscale.
Again, the hand-painted ground truth samples don't have to be just previously upscaled DF textures. If you guys have any prior art or know of a royalty-free source of these types of textures, they will probably work just fine.
In the meantime, I'm going to mess around with some other upscaling techniques such as Waifu2x and XBR4x like you mentioned and see if I can somehow combine them with these GANs to enhance the output a little.
- VMblast
- Posts: 519
- Joined: Wed Mar 29, 2017 12:22 pm
- Contact:
Re: AI Upscaled Textures
What I meant when I mentioned HD resolution with upscaling to 1024, should be taken with a grain of salt. The resolution of 1024 vertical was aimed at textures which are the biggest in the ingame world, in the case of examples, these would be walls of the tavern interior or exterior. Anything (vanilla textures) smaller than that percentage vise, should be upscaled accordingly.MasonFace wrote: ↑Wed Dec 26, 2018 4:32 am The second best (and it's a far second) was the dog. The problem seems to be that there just isn't nearly enough pixels to start off with (a measly 56x40!) for the AI to infer any additional detail from...
It doesn't look any better than just a regular cubic interpolated scaling...
I tried a few trees and bushes, but they all came out either just terribly blurry or highly contaminated with artifacts.
Vanilla: 64x128 should be up to min. ---> 512x1024
If the dog texture was 64x64, percentage wise, upscale should be 512x512 (since it covers smaler portion of the actual HD screen, when compared to the walls textures).
PS -All the upscaling are looking good so far. A bit bury sometimes, but you are on the right track. Shame that no one havent made actual simple app for easier use.
-
- Posts: 833
- Joined: Mon Nov 30, 2015 6:32 pm
Re: AI Upscaled Textures
Unfortunately outside my area of expertise; I've just read some summary articles and watched a few cool Youtube videos. That opinion is about the farthest I can venture right now.MasonFace wrote: ↑Wed Dec 26, 2018 4:32 amYup, that's the end-game! Would you be able to help? I need a compilation of textures close to or above our target resolution (1024x1024 would probably be okay), preferably of hand-painted textures of a variety of materials (wood, stone, brick, marble, metal, etc.). I will need a bare minimum of 100, but we will get increasingly better results with more training data. I will continue to read up on how to actually perform the training... from what I understand, that is the actual computationally expensive part of the AI.Hmm... Train it on manually-upscaled textures, then feed in the original textures?
...Oh, and I also once trained a 5-neuron? NAND gate from a tutorial about 5? 6? years ago. Heh. Anyway, I'd unfortunately have to spend quite a bit more time than I have training myself up on the techniques and technologies.
But if you've got a training set you want run, and all I have to do is push a few buttons, I have a Ryzen 2700X.
Previous experience tells me it's very easy to misunderstand the tone, intent, or meaning of what I've posted. If you have questions, ask.
- Freak2121
- Posts: 60
- Joined: Fri Sep 08, 2017 6:14 am
Re: AI Upscaled Textures
This stuff is cool! I've always wondered how well AI upscaling would work for old games like Daggerfall, nice to see it works quite well. I imagine if it gets trained on actual sprites and textures rather than real photos, the results would be incredible.
Anyway, I was curious so I signed up for a 30 day trial with Topaz AI's upscaler, and the results are quite nice. Here's a bit of a dump of some textures and sprites I tested it on, it seems to do really well with flora and really bad with faces. Mostly Daggerfall, though a few are from other games.
Warning, a whole ton of images in the spoiler.
Honestly these would all be really good with a bit of manual retouching.
Anyway, I was curious so I signed up for a 30 day trial with Topaz AI's upscaler, and the results are quite nice. Here's a bit of a dump of some textures and sprites I tested it on, it seems to do really well with flora and really bad with faces. Mostly Daggerfall, though a few are from other games.
Warning, a whole ton of images in the spoiler.
Spoiler!
- Arctus
- Posts: 60
- Joined: Mon Mar 23, 2015 12:28 am
Re: AI Upscaled Textures
Damn those look great for procedural upscaling. So the entirety of DF textures and sprites could be run through that?
- Interkarma
- Posts: 7247
- Joined: Sun Mar 22, 2015 1:51 am
Re: AI Upscaled Textures
Wow, these look incredible for an AI upscaler. Some of those faces are a bit unsettling, but nothing a manual touch-up can't fix later. Excellent work!