AI Upscaled Textures

Show off your mod creations or just a work in progress.
Post Reply
User avatar
MrFlibble
Posts: 410
Joined: Sat Jan 27, 2018 10:43 am

Re: AI Upscaled Textures

Post by MrFlibble »

So I read through that ResetEra thread, and the folks there not only used pre-trained models for ESRGAN and SFTGAN, bu also created their own based on a selection of Manga images, which appears to produce better results.

However wouldn't it be more logical to train a model on authentic 8-bit graphics? For certain games, there exist pairs of low-res and high-res images. For example, the Macintosh port of Wolfenstein 3-D has all textures in double the resolution as the PC DOS version. Command & Conquer: Red Alert has a number of static screens in both 320x200 and 640x400 resolutions, including a selection of FMV "stills" (but actually 3D renders) that are displayed during the credits sequence. Air Power and some other Rowan Software sims also contain 320x200 and 640x400 versions of certain in-game screens which can be used (some available in the demo version).

User avatar
Freak2121
Posts: 60
Joined: Fri Sep 08, 2017 6:14 am

Re: AI Upscaled Textures

Post by Freak2121 »

Well, I did all the object textures. Here is a download link, just plop the textures folder into DaggerfallUnity_Data/StreamingAssets

I didn't do sprites because if I keep them transparent, their outlines come out all blocky and gross. If I fill in the background with a dark colour, the sprites come out very nicely but are no longer transparent, and getting them cleanly transparent again is A LOT of work. There's gotta be some way around that.

A few comparisons

Image
Image

Image
Image

Image
Image

There's some issues with tiling, and some textures have just come out wrong for one reason or another.

User avatar
MasonFace
Posts: 543
Joined: Tue Nov 27, 2018 7:28 pm
Location: Tennessee, USA
Contact:

Re: AI Upscaled Textures

Post by MasonFace »

Well, I did all the object textures. Here is a download link, just plop the textures folder into DaggerfallUnity_Data/StreamingAssets
Very nice! It's very much classic Daggerfall textures, but more crisp! I love it.
I didn't do sprites because if I keep them transparent, their outlines come out all blocky and gross. If I fill in the background with a dark colour, the sprites come out very nicely but are no longer transparent, and getting them cleanly transparent again is A LOT of work. There's gotta be some way around that.
If you get time, please do upscale the sprites. Someone else may be able to clean up the edges. Mr. Flibble may have an inventive method.

User avatar
Jukic
Posts: 68
Joined: Sun May 13, 2018 5:51 pm

Re: AI Upscaled Textures

Post by Jukic »

@Freak2121:
Wow, this trully is vanilla Daggerfall reimagined! Nice work. Please do the sprites and I will also play with them to incorporate transparency in a batch manner. Just, can You fill the background of sprites some specific dark color (dark purple or whatever, not black!)

Otherwise just do all Daggerfall textures/sprites if You are able to! This is superb!
(also we will think of some tiling solution...)

User avatar
MasonFace
Posts: 543
Joined: Tue Nov 27, 2018 7:28 pm
Location: Tennessee, USA
Contact:

Re: AI Upscaled Textures

Post by MasonFace »

So I read through that ResetEra thread, and the folks there not only used pre-trained models for ESRGAN and SFTGAN, bu also created their own based on a selection of Manga images, which appears to produce better results.

However wouldn't it be more logical to train a model on authentic 8-bit graphics?
Yes it sure would in our case. I've been reading up on how to train these models and I think I have a good idea now on how to do it, we just need a collection of data to train the model on. The problem is getting enough training samples (I really think between 500-1000 is going to be the bare minimum for a GAN).

It would be helpful to have an online repository that we can all contribute data into so it isn't just one person trying to compile 1,000 or more samples needed to train the AI by himself, so I created a Google Drive folder here: https://drive.google.com/drive/folders/ ... sp=sharing
(I haven't tested to make sure others can upload to it. Let me know if you have problems).

I've started putting in some hand-painted textures to get us started, but they don't have be hand-painted. It is probably more important that we get a large number of samples with a variety of features and materials.

We only need to focus on getting samples that roughly match our desired outcome- we can simply downscale these samples to make the training inputs. The training inputs would work best if we downscale to close to the vanilla resolution and color depth.

Also, I think the AI will operate much better if we separate them into different categories: One model for textures, one for sprites (might even need to separate this again into sub-categories?). This means unique training data sets for each.


I've updated the first post to make the current progress more visible.

User avatar
MrFlibble
Posts: 410
Joined: Sat Jan 27, 2018 10:43 am

Re: AI Upscaled Textures

Post by MrFlibble »

Freak2121 wrote: Sat Dec 29, 2018 12:31 pm Well, I did all the object textures.
Did you do that with Topaz AI? (I kind of missed your previous post on that while skimming the thread for the first time) The results look really amazing, the photohallucination method seems to provide uncannily good results.
MasonFace wrote: Sat Dec 29, 2018 2:57 pm If you get time, please do upscale the sprites. Someone else may be able to clean up the edges. Mr. Flibble may have an inventive method.
I had a very simple idea which I used when I figured out how to trick waifu2x into properly handling sprite edges. Use the default black background (so that any dark pixels that were not cleaned up from the original sprites' edges will blend with the background), then convert the result back to the original 8-bit palette (I use mtPaint for very good results in this, you just need to set colour space to RGB from the default sRGB). Then open the image in any editor of your choice and use the simple bucket fill to replace the black background with some high contrast colour, like bright green. Just be sure not to have bucket fill set to smooth edges/smart fill or whatever it's called. You will then probably have a dark outline around each sprite but this can be eliminated by using the select by colour tool in GIMP. Just select all the bright green areas with it (effectively the background), then increase selection by one pixel, remove the selected area. It's crude but it should work.

Another method is to process each image twice, on black and white background, then blend the results with G'MIC (Layers > Blend [median]), and the background will become grey. However I have not tested this with Daggerfall sprites and it may not work very well because there are the aforementioned dark pixels that were not cleaned up when the sprites were created (this only applies to pre-rendered sprites).

Also Freak2121, have you tried "softening" the sprites as I described in my post above, before processing them with Topaz AI? I wonder if this might yield any improvement to the edges and/or the overall look of the images.
MasonFace wrote: Sat Dec 29, 2018 4:32 pm We only need to focus on getting samples that roughly match our desired outcome- we can simply downscale these samples to make the training inputs. The training inputs would work best if we downscale to close to the vanilla resolution and color depth.
At least for textures, I think it is quite possible to generate data sets by using free textures from sources like OpenGameArt. They come in relatively large resolutions, and can be scaled down to any desired size. Here's a 128x128 and 256x256 source and target pair I made from this ground texture:
Image Image

I scaled down the source image to both resolutions using Sinc3 interpolation in GIMP, then converted the 128x128 one to 256 colours using mtPaint (PNN quantification, no dithering, RGB colour space). After this I used the resulting palette to create an indexed version of the larger version.

The downside is that this one might be too noisy because it is based on a photograph. It is my understanding that back in the 90s photo-based textures (such as those found in Doom for example) were all touched up manually to get a proper look in-game. Maybe there are hand-painted textures available as well too.

User avatar
pango
Posts: 3347
Joined: Wed Jul 18, 2018 6:14 pm
Location: France
Contact:

Re: AI Upscaled Textures

Post by pango »

It's probably a good idea to use one of Daggerfall palettes to convert to 256 colors...
I'm not sure which one is best though
Mastodon: @pango@fosstodon.org
When a measure becomes a target, it ceases to be a good measure.
-- Charles Goodhart

Narf the Mouse
Posts: 833
Joined: Mon Nov 30, 2015 6:32 pm

Re: AI Upscaled Textures

Post by Narf the Mouse »

Hazelnut wrote: Fri Dec 28, 2018 6:39 pm
MasonFace wrote: Fri Dec 28, 2018 6:29 pm Also, my first kid is due to be born in a few days, so I actually may not have a chance to do much of anything aside from changing diapers and cuddling for quite a while. :lol:
Best wishes for a smooth delivery!
Indeed!
Previous experience tells me it's very easy to misunderstand the tone, intent, or meaning of what I've posted. If you have questions, ask.

User avatar
MasonFace
Posts: 543
Joined: Tue Nov 27, 2018 7:28 pm
Location: Tennessee, USA
Contact:

Re: AI Upscaled Textures

Post by MasonFace »

Narf the Mouse wrote: Sat Dec 29, 2018 7:38 pm Hazelnut wrote: ↑Fri Dec 28, 2018 12:39 pm
MasonFace wrote: ↑Fri Dec 28, 2018 12:29 pm
Also, my first kid is due to be born in a few days, so I actually may not have a chance to do much of anything aside from changing diapers and cuddling for quite a while.
Best wishes for a smooth delivery!
Indeed!
Jukic wrote: Fri Dec 28, 2018 7:48 pm Re: AI Upscaled TexturesQuote Jukic
by Jukic » Fri Dec 28, 2018 1:48 pm

@MasonFace: All the best! (I am also a fresh father )
Thank you Narf for the kind wishes! And congrats Jukic!

@Freak2121:

I updated the first post of this thread to include your download link so more people can start enjoying your work!

User avatar
King of Worms
Posts: 4751
Joined: Mon Oct 17, 2016 11:18 pm
Location: Scourg Barrow (CZ)
Contact:

Re: AI Upscaled Textures

Post by King of Worms »

Here are my upscaled NPCs for now.... https://mega.nz/#!y6RjzYKI!Miw4j3p9vmbt ... 5RCdtiR0uc

Not sure how well it will work tho... but maybe worth a try. Problem is, I used the NPCs from Daggerfall archive with higher resolution as a source. Not those which are used ingame. And I changed their heads for the heads from their portraits...

so.. who knows. Depends on usage...

I also have sprites... ask if u want those as well.. My textures are unusable, not the same source as vanilla. OR if they are vanilla based, its just a XBR4x upscale usually.... Than I have all the mobs upscalled with XBR4x blablabla
001.jpg
001.jpg (385.01 KiB) Viewed 4791 times

Post Reply