Using AI upscalling and filtering techniques on a pixel art and why I think it's not there yet.

Discuss modding questions and implementation details.
User avatar
carademono
Posts: 210
Joined: Sat Jul 11, 2020 3:20 pm

Re: Using AI upscalling and filtering techniques on a pixel art and why I think it's not there yet.

Post by carademono »

Ooh, that one's not a smoothing issue -- that one's a mistake. I accidentally including the blacksmith from a low-res archive rather than a high-res archive. That blacksmith frame should have looked like this:

Image

The trade-offs are pretty clear in the image. There's definitely a loss of detail, as you point out. But in exchange there's better depth and realism. I think that trade-off is particularly worthwhile with a game like Daggerfall that puts 2d sprites in a 3d world -- that's what distinguishes it from other pixel art games where detail is more important than depth.

Anyway, thank you for catching that error! It will be fixed in the next update.

sayber1
Posts: 46
Joined: Thu Sep 16, 2021 3:27 pm

Re: Using AI upscalling and filtering techniques on a pixel art and why I think it's not there yet.

Post by sayber1 »

carademono wrote: Sun Jan 23, 2022 7:27 pm The trade-offs are pretty clear in the image. There's definitely a loss of detail, as you point out. But in exchange there's better depth and realism. I think that trade-off is particularly worthwhile with a game like Daggerfall that puts 2d sprites in a 3d world -- that's what distinguishes it from other pixel art games where detail is more important than depth.
Now, that seems to be the point I missed. We just have a clearly different ideas about the way graphics for DF should function in general. As I think of it as a Pixel Art game and as such prioritize sharpness and clear detail, as well as not seeing the issue of 16bit color palette. With this approach a lot of your decisions start to make a lot more sense to me.

Again, sorry for being an asshole in my first post.

User avatar
King of Worms
Posts: 4752
Joined: Mon Oct 17, 2016 11:18 pm
Location: Scourg Barrow (CZ)
Contact:

Re: Using AI upscalling and filtering techniques on a pixel art and why I think it's not there yet.

Post by King of Worms »

Hi guys, your work is not there, but I dont have time to do it better. But I can tell u how

sayber1
Posts: 46
Joined: Thu Sep 16, 2021 3:27 pm

Re: Using AI upscalling and filtering techniques on a pixel art and why I think it's not there yet.

Post by sayber1 »

King of Worms wrote: Mon Jan 24, 2022 9:30 am Hi guys, your work is not there, but I dont have time to do it better. But I can tell u how
So basically, no discussion or criticism allowed.

And no I'm not not doing this myself because I do not have time for this, it's because I'm not interested in doing this specific thing at all. As I said in my last post, it was made clear we have a very different idea about the direction of graphics for this game.

I'm doing other projects that I think should be done, that comfort to my vision. But as I am invested in the modding scene of this game, I would still want to provide feedback and opinion to the projects that I may not be using personally. And in this case, a project that I believe has a potential to become a new standart in the Retro Style of modding the DFU, just like your DREAM has become a de facto standart for HD Style modding of this game.

User avatar
Ninelan
Posts: 187
Joined: Wed Jun 30, 2021 10:30 am
Location: Under your bed
Contact:

Re: Using AI upscalling and filtering techniques on a pixel art and why I think it's not there yet.

Post by Ninelan »

The same lack of substance or contribution should also apply not just for mods but comments too :mrgreen:
People constantly here did these 'well I wish, I want' in regards to 3D models all the time, so I see no reason why people push back on such a discourse when it comes to 2D art. :? And I've heard my own fair share of 'you should rather work on X than Y' and 'i'd rather your work be this instead' here.

An artist here, one well versed in pixel AND other forms of art.
And it is true, the upscale art usually looks like overly smooth pudding most of the time, and personally to me, I prefer to keep it vanilla in such cases.
However! one also has to consider that there are people out there that want to do such kinds of mods but have no technical knowledge on how to do it by hand and do it well on top of it.
As a decent share of folk here is into coding, not art, or neither of both, they scurry off to algorithms and upscaling programs as it's a lot more accessible for them than struggle through the thousands upon thousands of images in an egregious effort of labour. Keep in mind that programs are wonderful to do the bulk of the heavy lifting, even if you're an artist, with the artist touch coming last where needed.

So this is something to understand that such mods have existed, do exist and will continue to exist and nevertheless are useful to make more sophisticated end results.
Luckily some mods, including the one that sparked this conversation is accepting of multi-people contributions so an artist can offer their own contributions and improvements! (like my stars(some of which are delicately drawn pixel stars) and moon contributions for vanilla enhanced)


But back on the artistry of things of the topic! Because talking art stuff and art quality stuff is rare to come by!
There are two images linked, but I think the most glaring issue is that the enhanced image doesn't look like an improvement, it looks like the original image, (as the resolution is the same), but slightly blurred out. It loses its crispness, its sharpness.
I think the better argument would be, is the effort of painstakingly going through hundreds upon hundreds of image archives, painstakingly editing them for the end result not even being that noticeable... is the end result proportionate to the effort put in? ...should this effort even be proportionate?
Introducing more colours to pixel art doesn't necessarily improve pixel art, for the onlookers, believe it or not, a limited palette can be quite appealing to look at. (Cue the popularity of pixel indie games...and Minecraft LOL!)
However, if the range of colours is expanded, wouldn't the logical conclusion be to move onto larger images? So there is more space for all those colours without having to resort to blurryness?
The textures of the houses themselves per se, were IMO almost flawless. On the other hand, the ground pixels being 3 times larger than the wall pixels is a bit... eugh. But that's beside the point. (Daggerfall is simply riddled with mixels)
The blurryness also makes the pixels less noticeable, which then leads to why not just go for a full, complete drawn rendition of said sprites?
Can somebody really appreciate something being pixel art, if you can't see the individual pixels aside from highlights, where is the 'cut off' border between acceptably 'pixelated' and acceptably not pixelated? What's then up with the brain itself interpreting the pixels as tangible 'smooth' images in your mind? Does that not account for anything?
What is then the difference between sprites with pixelated edged but insides so smooth you can't discern pixels, to 3D models with completely smooth outlines yet pixelated textures?

And with the blacksmith, there isn't actually more depth or realism to it, it's just the original but slightly blurred.
It actually loses depth as it killed the bright highlights that make certain parts of the art pop out.
BadLuckBurt wrote: Sat Jan 22, 2022 11:15 am So, the King of Fighter hi-res pixel art does not count? https://kofaniv.snk-corp.co.jp/english/ ... /index.php
Size doesn't matter in pixel art, it's the deliberate manipulation of an image on a pixel level. Least I pull out my 19000px across pixel art to prove the opposite in an extreme scale :D . (the usage of rotoscoping 3D models as a template to build pixel art from is irrelevant)
You can support me and in turn my art on my Patreon.
https://www.patreon.com/fireflyness

sayber1
Posts: 46
Joined: Thu Sep 16, 2021 3:27 pm

Re: Using AI upscalling and filtering techniques on a pixel art and why I think it's not there yet.

Post by sayber1 »

Ninelan wrote: Mon Jan 24, 2022 3:50 pm However! one also has to consider that there are people out there that want to do such kinds of mods but have no technical knowledge on how to do it by hand and do it well on top of it.
As a decent share of folk here is into coding, not art, or neither of both, they scurry off to algorithms and upscaling programs as it's a lot more accessible for them than struggle through the thousands upon thousands of images in an egregious effort of labour. Keep in mind that programs are wonderful to do the bulk of the heavy lifting, even if you're an artist, with the artist touch coming last where needed.
That's the reason I wanted to bring this up. As a lot of the time modders do not have the specialized technical knowledge, they can sometimes make some big mistakes working on something without realizing it. I studied graphical design in college, and I would often see mods having some serious issues when it comes for example to putting highly detailed and contrast images as backgrounds for menus or using highly elaborate and stylized text which undermines the readability. Those mistakes wouldn't be tolerated even in my college which would sometimes set a really low bar for its students. Sometimes there are genuine reasons for doing things a certain, perhaps incorrect way. A lot of times there are not. If the latter is the case I think it should be pointed out for people to avoid such mistakes in the future. What I mean is the difference between "This is the only optimal way of doing this for me, even if it's not the best" and "This is probably the way to do it".

If there is a reason or a geniune vision behind such decidion I can ignore this. Even if I think it is wrong or doesn't work well or look good. But often times it's impossible to tell this just from the observation alone.

As for the pixel art itself, I do believe a lot of it's beauty and charm comes from its' ability to show details from small and deliberate suggestions(Don't know if I made that sound right, english is not my native language). Its' ability to correctly show you things that aren't really there.
This is what I got from my own observations as well as listening to other artists specialized in that field.
As I said in my first post of this thread I do think that AI filtering right now does not really improve pixel art. Its' effects are subtle and are usually hit or miss. I personally think it's more distracting that anything else.
I do not think I properly touched on this in my previous posts, as continuing this would make me sound even worse and I was already too unintetionaly rude when starting this discussion, but I do consider blurring a pixel art to go against what I learned and appreciated about it. If there is somekind of vision behind this, I'm okay with this, but I would still prefer it not go that route. A general blurryness and artificial look are more distracting for me than an odd miscolored pixel.

Then again there is probably an argument to be made, considering the oldschool games were made for often times blurry CRT screens and often took that into account when designing their visuals, like the famous example of Sonics' waterfalls. As I come from a generation for which pixel art was already a feature of retro gaming and matured to be deliberate form of art and not just a result of pure technical limitations, I do tend to appreciate it as its' own thing with its' own rules applied to it.

User avatar
Ninelan
Posts: 187
Joined: Wed Jun 30, 2021 10:30 am
Location: Under your bed
Contact:

Re: Using AI upscalling and filtering techniques on a pixel art and why I think it's not there yet.

Post by Ninelan »

sayber1 wrote: Mon Jan 24, 2022 4:59 pm As for the pixel art itself, I do believe a lot of it's beauty and charm comes from its' ability to show details from small and deliberate suggestions(Don't know if I made that sound right, english is not my native language). Its' ability to correctly show you things that aren't really there.
This is what I got from my own observations as well as listening to other artists specialized in that field.
As I said in my first post of this thread I do think that AI filtering right now does not really improve pixel art. Its' effects are subtle and are usually hit or miss. I personally think it's more distracting that anything else.
I do not think I properly touched on this in my previous posts, as continuing this would make me sound even worse and I was already too unintetionaly rude when starting this discussion, but I do consider blurring a pixel art to go against what I learned and appreciated about it. If there is somekind of vision behind this, I'm okay with this, but I would still prefer it not go that route. A general blurryness and artificial look are more distracting for me than an odd miscolored pixel.

Then again there is probably an argument to be made, considering the oldschool games were made for often times blurry CRT screens and often took that into account when designing their visuals, like the famous example of Sonics' waterfalls. As I come from a generation for which pixel art was already a feature of retro gaming and matured to be deliberate form of art and not just a result of pure technical limitations, I do tend to appreciate it as its' own thing with its' own rules applied to it.
Indeed, I believe this is what a shortcoming was, you can't keep the pixel charm of the originals by getting rid of everything that gives off that pixel charm.
When it comes to upscaling it does have its use. But yeah, you could probably voice yourself better in that thread with some more collected thoughts. But nevertheless, I do like that such a thread popped up regardless of it.

Incidentally, pixel art is one of the easiest artforms to learn, provided you get taught the proper way. Like not using soft brushes and ramping up the contrast when shading. I've done my fair share of short teaching sessions when i helped people fix up their pixel art for mods :D .

I assume when it comes to CRT screens we can chalk it up to 'ancient greek and roman sculptures were painted, yet we ogle at the marvels that is that pristine and sublime white marble'.
You can support me and in turn my art on my Patreon.
https://www.patreon.com/fireflyness

User avatar
carademono
Posts: 210
Joined: Sat Jul 11, 2020 3:20 pm

Re: Using AI upscalling and filtering techniques on a pixel art and why I think it's not there yet.

Post by carademono »

Both of you are mostly discussing pixel art in the abstract but neither of you are grappling with the specific problem posed by Daggerfall's actual pixel art, which is an utterly incoherent mess of completely different scales, poorly pixelized 3d models, and multiple styles of hand-drawn flats. How do you harmonize that into something that resembles a single coherent aesthetic with the pixel-by-pixel philosophies you're discussing above? Either you have to completely recreate all of Daggerfall's assets by hand or you have to accept some costs imposed by AI techniques.

sayber1
Posts: 46
Joined: Thu Sep 16, 2021 3:27 pm

Re: Using AI upscalling and filtering techniques on a pixel art and why I think it's not there yet.

Post by sayber1 »

carademono wrote: Mon Jan 24, 2022 11:29 pm Both of you are mostly discussing pixel art in the abstract but neither of you are grappling with the specific problem posed by Daggerfall's actual pixel art, which is an utterly incoherent mess of completely different scales, poorly pixelized 3d models, and multiple styles of hand-drawn flats. How do you harmonize that into something that resembles a single coherent aesthetic with the pixel-by-pixel philosophies you're discussing above? Either you have to completely recreate all of Daggerfall's assets by hand or you have to accept some costs imposed by AI techniques.
I do not think it's as bad as you say. Enviroment textures are generally nice and more or less look consistent. The only big problem I can think of would be the size difference between terrain textures and building texures, as Ninelan said in his post, but I do not think it's that striking , and is relatively easy to ignore.
That leaves us with NPC sprites. There are 3 main styles of NPC sprites used in game. Hand painted hirez sprites(334, 346, 357, 182, 183 and some Daedra Lords), low res handpainted sprites(184, 180, guilds), some of which are, from my knowledge are repurposed Arena sprites, and lastly, sprites made from 3D models(Enemies, some guild members and villagers). As 184 are used a lot in taverns and other enviroments where they are placed alongside hirez sprites they definitly should be reworked to look more consistently alongside them. Guild sprites are almost universally low res, so there is not a big dissonance among then, as there aren't usually hirez sprites next to them that would make them look out of place.

I do think it's definetly possible to rework lowres HandDrawns, the only issue being the need to animate a lot of guild ones.
Then again something like the .178(Necromacers), as far as I remember, are never actually seen together with other, more hirez sprites, so I feel their rework would be from the not needed to the last thing to do.

The real big problem are 3D based sprites. One thing I find very jarring about them are their colors. Especially their skin color. This yellowish, beige hue is not generally used on Hirez Sprites with one exception I can think of. I'm not sure how to tackle this problem yet, I did try to tackle some parts of this problem by attemting to create a sort of universal color correction and palletizing them, but I'm not happy the results yet. But(!) there is a texture leftover in 195(10-0) that is 3d sprite using warmer color palette for it's skin.
Then there is still an issue of resolution, I'm not sure what to do about this yet, or if anything should even be done about this. Overall I believe simply adjusting palette for them can do a lot of thnigs to improve the consistency of their look next to handdrawn sprites. Then there are examples of 3d sprites using hand drawn parts in the original, so this may be a solution. Weirdly that was not as big of an issue in vanilla, as those NPCs would show up next to HandDrawns much less than in DFU thanks to some mods.

Other than that I don't think there are inconsistencies on a level to warrant full scale graphical rework. Maybe small touch ups.

Also retro rendering mode. What I found when remaking 184 lowres sprites, is that difference is almost unnoticeable at times when playing with 320x200 full palettization mode.

User avatar
Ninelan
Posts: 187
Joined: Wed Jun 30, 2021 10:30 am
Location: Under your bed
Contact:

Re: Using AI upscalling and filtering techniques on a pixel art and why I think it's not there yet.

Post by Ninelan »

carademono wrote: Mon Jan 24, 2022 11:29 pm Both of you are mostly discussing pixel art in the abstract but neither of you are grappling with the specific problem posed by Daggerfall's actual pixel art, which is an utterly incoherent mess of completely different scales, poorly pixelized 3d models, and multiple styles of hand-drawn flats. How do you harmonize that into something that resembles a single coherent aesthetic with the pixel-by-pixel philosophies you're discussing above? Either you have to completely recreate all of Daggerfall's assets by hand or you have to accept some costs imposed by AI techniques.
No no, this is precisely relevant. Art theory explains why such a project fails to achieve what it sets up to. You have to look at this as technically not like a layman but like how a master chef looks at lamb sauce.

No amount of filtering will ever achieve style cohesion, at best it muddies what charm Daggerfall's handcrafted sprites had. The vast majority of sprites !do match! from the walls on the buildings to the trees outside to the people indoors.
Even the differently scaled people have stylistic cohesion between each other in the way they are drawn and shaded.

The only stylistic dissonance is between the 3D model renders and the hand-drawn ones.
And Daggerfall's 3D models can only really be 'fixed' by rotoscoping them in a similar style to static sprites.

Most of the 2D sprites are 'ideal' for what Daggerfall is.
The only way to actually get them in line is to redraw them to be inline.
You can support me and in turn my art on my Patreon.
https://www.patreon.com/fireflyness

Post Reply