AI Upscaled Textures
- MrFlibble
- Posts: 411
- Joined: Sat Jan 27, 2018 10:43 am
Re: AI Upscaled Textures
So I read through that ResetEra thread, and the folks there not only used pre-trained models for ESRGAN and SFTGAN, bu also created their own based on a selection of Manga images, which appears to produce better results.
However wouldn't it be more logical to train a model on authentic 8-bit graphics? For certain games, there exist pairs of low-res and high-res images. For example, the Macintosh port of Wolfenstein 3-D has all textures in double the resolution as the PC DOS version. Command & Conquer: Red Alert has a number of static screens in both 320x200 and 640x400 resolutions, including a selection of FMV "stills" (but actually 3D renders) that are displayed during the credits sequence. Air Power and some other Rowan Software sims also contain 320x200 and 640x400 versions of certain in-game screens which can be used (some available in the demo version).
However wouldn't it be more logical to train a model on authentic 8-bit graphics? For certain games, there exist pairs of low-res and high-res images. For example, the Macintosh port of Wolfenstein 3-D has all textures in double the resolution as the PC DOS version. Command & Conquer: Red Alert has a number of static screens in both 320x200 and 640x400 resolutions, including a selection of FMV "stills" (but actually 3D renders) that are displayed during the credits sequence. Air Power and some other Rowan Software sims also contain 320x200 and 640x400 versions of certain in-game screens which can be used (some available in the demo version).
- Freak2121
- Posts: 60
- Joined: Fri Sep 08, 2017 6:14 am
Re: AI Upscaled Textures
Well, I did all the object textures. Here is a download link, just plop the textures folder into DaggerfallUnity_Data/StreamingAssets
I didn't do sprites because if I keep them transparent, their outlines come out all blocky and gross. If I fill in the background with a dark colour, the sprites come out very nicely but are no longer transparent, and getting them cleanly transparent again is A LOT of work. There's gotta be some way around that.
A few comparisons
There's some issues with tiling, and some textures have just come out wrong for one reason or another.
I didn't do sprites because if I keep them transparent, their outlines come out all blocky and gross. If I fill in the background with a dark colour, the sprites come out very nicely but are no longer transparent, and getting them cleanly transparent again is A LOT of work. There's gotta be some way around that.
A few comparisons
There's some issues with tiling, and some textures have just come out wrong for one reason or another.
- MasonFace
- Posts: 543
- Joined: Tue Nov 27, 2018 7:28 pm
- Location: Tennessee, USA
- Contact:
Re: AI Upscaled Textures
Very nice! It's very much classic Daggerfall textures, but more crisp! I love it.Well, I did all the object textures. Here is a download link, just plop the textures folder into DaggerfallUnity_Data/StreamingAssets
If you get time, please do upscale the sprites. Someone else may be able to clean up the edges. Mr. Flibble may have an inventive method.I didn't do sprites because if I keep them transparent, their outlines come out all blocky and gross. If I fill in the background with a dark colour, the sprites come out very nicely but are no longer transparent, and getting them cleanly transparent again is A LOT of work. There's gotta be some way around that.
- Jukic
- Posts: 68
- Joined: Sun May 13, 2018 5:51 pm
Re: AI Upscaled Textures
@Freak2121:
Wow, this trully is vanilla Daggerfall reimagined! Nice work. Please do the sprites and I will also play with them to incorporate transparency in a batch manner. Just, can You fill the background of sprites some specific dark color (dark purple or whatever, not black!)
Otherwise just do all Daggerfall textures/sprites if You are able to! This is superb!
(also we will think of some tiling solution...)
Wow, this trully is vanilla Daggerfall reimagined! Nice work. Please do the sprites and I will also play with them to incorporate transparency in a batch manner. Just, can You fill the background of sprites some specific dark color (dark purple or whatever, not black!)
Otherwise just do all Daggerfall textures/sprites if You are able to! This is superb!
(also we will think of some tiling solution...)
- MasonFace
- Posts: 543
- Joined: Tue Nov 27, 2018 7:28 pm
- Location: Tennessee, USA
- Contact:
Re: AI Upscaled Textures
Yes it sure would in our case. I've been reading up on how to train these models and I think I have a good idea now on how to do it, we just need a collection of data to train the model on. The problem is getting enough training samples (I really think between 500-1000 is going to be the bare minimum for a GAN).So I read through that ResetEra thread, and the folks there not only used pre-trained models for ESRGAN and SFTGAN, bu also created their own based on a selection of Manga images, which appears to produce better results.
However wouldn't it be more logical to train a model on authentic 8-bit graphics?
It would be helpful to have an online repository that we can all contribute data into so it isn't just one person trying to compile 1,000 or more samples needed to train the AI by himself, so I created a Google Drive folder here: https://drive.google.com/drive/folders/ ... sp=sharing
(I haven't tested to make sure others can upload to it. Let me know if you have problems).
I've started putting in some hand-painted textures to get us started, but they don't have be hand-painted. It is probably more important that we get a large number of samples with a variety of features and materials.
We only need to focus on getting samples that roughly match our desired outcome- we can simply downscale these samples to make the training inputs. The training inputs would work best if we downscale to close to the vanilla resolution and color depth.
Also, I think the AI will operate much better if we separate them into different categories: One model for textures, one for sprites (might even need to separate this again into sub-categories?). This means unique training data sets for each.
I've updated the first post to make the current progress more visible.
- MrFlibble
- Posts: 411
- Joined: Sat Jan 27, 2018 10:43 am
Re: AI Upscaled Textures
Did you do that with Topaz AI? (I kind of missed your previous post on that while skimming the thread for the first time) The results look really amazing, the photohallucination method seems to provide uncannily good results.
I had a very simple idea which I used when I figured out how to trick waifu2x into properly handling sprite edges. Use the default black background (so that any dark pixels that were not cleaned up from the original sprites' edges will blend with the background), then convert the result back to the original 8-bit palette (I use mtPaint for very good results in this, you just need to set colour space to RGB from the default sRGB). Then open the image in any editor of your choice and use the simple bucket fill to replace the black background with some high contrast colour, like bright green. Just be sure not to have bucket fill set to smooth edges/smart fill or whatever it's called. You will then probably have a dark outline around each sprite but this can be eliminated by using the select by colour tool in GIMP. Just select all the bright green areas with it (effectively the background), then increase selection by one pixel, remove the selected area. It's crude but it should work.
Another method is to process each image twice, on black and white background, then blend the results with G'MIC (Layers > Blend [median]), and the background will become grey. However I have not tested this with Daggerfall sprites and it may not work very well because there are the aforementioned dark pixels that were not cleaned up when the sprites were created (this only applies to pre-rendered sprites).
Also Freak2121, have you tried "softening" the sprites as I described in my post above, before processing them with Topaz AI? I wonder if this might yield any improvement to the edges and/or the overall look of the images.
At least for textures, I think it is quite possible to generate data sets by using free textures from sources like OpenGameArt. They come in relatively large resolutions, and can be scaled down to any desired size. Here's a 128x128 and 256x256 source and target pair I made from this ground texture:
I scaled down the source image to both resolutions using Sinc3 interpolation in GIMP, then converted the 128x128 one to 256 colours using mtPaint (PNN quantification, no dithering, RGB colour space). After this I used the resulting palette to create an indexed version of the larger version.
The downside is that this one might be too noisy because it is based on a photograph. It is my understanding that back in the 90s photo-based textures (such as those found in Doom for example) were all touched up manually to get a proper look in-game. Maybe there are hand-painted textures available as well too.
- pango
- Posts: 3359
- Joined: Wed Jul 18, 2018 6:14 pm
- Location: France
- Contact:
Re: AI Upscaled Textures
It's probably a good idea to use one of Daggerfall palettes to convert to 256 colors...
I'm not sure which one is best though
I'm not sure which one is best though
Mastodon: @pango@fosstodon.org
When a measure becomes a target, it ceases to be a good measure.
-- Charles Goodhart
When a measure becomes a target, it ceases to be a good measure.
-- Charles Goodhart
-
- Posts: 833
- Joined: Mon Nov 30, 2015 6:32 pm
Re: AI Upscaled Textures
Indeed!
Previous experience tells me it's very easy to misunderstand the tone, intent, or meaning of what I've posted. If you have questions, ask.
- MasonFace
- Posts: 543
- Joined: Tue Nov 27, 2018 7:28 pm
- Location: Tennessee, USA
- Contact:
Re: AI Upscaled Textures
Narf the Mouse wrote: ↑Sat Dec 29, 2018 7:38 pm Hazelnut wrote: ↑Fri Dec 28, 2018 12:39 pm
MasonFace wrote: ↑Fri Dec 28, 2018 12:29 pm
Also, my first kid is due to be born in a few days, so I actually may not have a chance to do much of anything aside from changing diapers and cuddling for quite a while.
Best wishes for a smooth delivery!
Indeed!
Thank you Narf for the kind wishes! And congrats Jukic!
@Freak2121:
I updated the first post of this thread to include your download link so more people can start enjoying your work!
- King of Worms
- Posts: 4753
- Joined: Mon Oct 17, 2016 11:18 pm
- Location: Scourg Barrow (CZ)
- Contact:
Re: AI Upscaled Textures
Here are my upscaled NPCs for now.... https://mega.nz/#!y6RjzYKI!Miw4j3p9vmbt ... 5RCdtiR0uc
Not sure how well it will work tho... but maybe worth a try. Problem is, I used the NPCs from Daggerfall archive with higher resolution as a source. Not those which are used ingame. And I changed their heads for the heads from their portraits...
so.. who knows. Depends on usage...
I also have sprites... ask if u want those as well.. My textures are unusable, not the same source as vanilla. OR if they are vanilla based, its just a XBR4x upscale usually.... Than I have all the mobs upscalled with XBR4x blablabla
Not sure how well it will work tho... but maybe worth a try. Problem is, I used the NPCs from Daggerfall archive with higher resolution as a source. Not those which are used ingame. And I changed their heads for the heads from their portraits...
so.. who knows. Depends on usage...
I also have sprites... ask if u want those as well.. My textures are unusable, not the same source as vanilla. OR if they are vanilla based, its just a XBR4x upscale usually.... Than I have all the mobs upscalled with XBR4x blablabla
DREAM Nexus https://www.nexusmods.com/users/73250038?tab=user+files
DREAM Forums viewtopic.php?f=27&t=1168
DISCORD: king_of_worms_cz
DREAM Forums viewtopic.php?f=27&t=1168
DISCORD: king_of_worms_cz