Daggerfall Workshop Forums

I decided to give the previous faces another shot - this time with an interpolation of Spongebob Reloaded Swag (as used on the magazine in the upscale on last page) and Lollypop models. Afterwards I used imagemagick to remap the result to the palette of the original image.

Had a look thru them and they are nice, I know how difficult are these faces for AI so nice job.

I will personally not use them, because my goal for my mod is to have them all manually redrawn in HD (and that goal is lets say 80% ready)

Still, really impressive results indeed.

I wonder what results we could get on MOBs with these new algos, but I dont even dare to go there anymore I still think what we have is really good, especially the front facing frames which had a Topaz AI sharpening pass and lots of manual input after the AI upscale.

Thought I'd share these. The results of an experimental model that been shared with me.

This looks awesome!

I think with a few touches of a artist, these are really good. I wonder how hard would be to remove the black background (some algos are hard to automatically cut-out)

I miss last 23 faces. Can I please send em to you for this treatment?
Even if I need to cut em out manually afterwards, its doable.

In case you could do it, I will add a RAR with these faces
Thanks a lot! (& even if just some of those come out good, its still few new faces done, I really need to push this forward somehow)

Here goes:

sure, here are the upscales

Thanks a lot sir, will go thru them when I have some more free time, really appreciated

I'm awaiting on the release of a newer model from Nvidia that should be open sourced in the next couple of weeks for turning these results into 3D renders--but thought I'd share in the meantime. Humans only unfortunately--will need to work through the process of style transfer for non human NPCs (retraining).

I used pixel2style2pixel to generate the photorealistic human faces from an older download of the NPC upscaled sprites portraits. The one on the right is a render of the transferred style--I'm pretty sure there is a way to transfer an art style too, so if someone wanted to apply a non lore-breaking "flavor" of art to these, it could be done to make the originals look whatever style you want.

hopefully by the time the 3Dganiverse model is out I can have the remaining fine tuning for faces done and apply another StyleGan model to the NPC flats, fully automate the process of converting them to 3D models.

Edit: the point of this is to demonstrate for faces, that there is a way to generate a template human photo of a face resembling the original to then apply an additional art style that fits the original game art using something like StyleGAN2. Many of the networks take photos of humans as input--this is a way to generate those photos for use in a larger pipeline. The option to face swap onto the full body sprites after applying a StyleGAN2 layer for an artistic style is easy to do--I'll try to do an example over the weekend, time permitting.

Oh dang, those are surprisingly decent results.
Now I'm wondering what these would look like with the additional style step.

Yeah I am thinking the approach of super resolution needs to happen *in conjunction* with style transfer to attain the best results. I may be wrong but it's fun to play with in my free time anyway lol.

For the non humans I am considering extracting encodings of styles from images of Modern TES games, applying that style to the remaining non human NPCs. Then applying an extracted high res DF style (or a combination of styles) to all.

Additionally:
I'm thinking it might be fun to to achieve a good balance between the style of the game with the high resolution of form from actual photos + modern TES extractions. If the modern TES extractions may be foggy on licensing for using them here, I can automatically generate variations of a single extremely high resolution 3D model that I can put together myself (for anyone to use for commercial or non under MIT license), then encode those (custom dataset, would add a couple hours to generate over just using TES extractions from web scraped images).

Style and "Form" here (there's another word for "form" but I can't recall it off the top of my head) refers to 2 things, style being the art style and "Form" being sort of like the type of physical object that you are transforming the original sprite into (or applying a style to).

My understanding of StyleGAN2 is very high level at the moment, I'm just reading about models, playing with and retraining them where I can find usable code, and plugging them together using Python/OpenCV/PIL. I've trained YOLOv3 and YOLOv4 to detect body parts and typically use that for automatically cropping things prior to applying something this--made a repo tutorial for training YOLO here (works for V4 too):

https://github.com/WyattAutomation/Trai ... enImagesV4

Really impressive results.

Honestly, just simply overlaying the GAN faces over the originals looks pretty dang good.
But I'm excited to see what the GAN can do once it's trained with style transfer to match Daggerfall.