Using AI upscalling and filtering techniques on a pixel art and why I think it's not there yet.

Discuss modding questions and implementation details.
User avatar
carademono
Posts: 210
Joined: Sat Jul 11, 2020 3:20 pm

Re: Using AI upscalling and filtering techniques on a pixel art and why I think it's not there yet.

Post by carademono »

Maybe I haven't taken enough art theory classes but I simply do not understand the argument that this:

Image

represents a more coherent aesthetic than this:

Image

sayber1
Posts: 46
Joined: Thu Sep 16, 2021 3:27 pm

Re: Using AI upscalling and filtering techniques on a pixel art and why I think it's not there yet.

Post by sayber1 »

carademono wrote: Tue Jan 25, 2022 6:10 pm Maybe I haven't taken enough art theory classes but I simply do not understand the argument that this:
Spoiler!
Image
represents a more coherent aesthetic than this:
Spoiler!
Image
Well, for me personally, it feels kinda similar to squinting your eyes for things to look more coherent. They are still different in style just with their finer details lost.
Does the cost justify the results? Well, it's for people to decide. As I said, if that is your geniune vision you should continue to work on it in ways you see fit.

As for cohesion, it's all the matter of context. You usually do not see sprites of all different styles lined up in your game.
2022_01_25_22_15_11.jpg
2022_01_25_22_15_11.jpg (382.88 KiB) Viewed 1098 times
2022_01_25_19_56_28.jpg
2022_01_25_19_56_28.jpg (338.6 KiB) Viewed 1105 times
If we are not talking about 3D sprites, which are the most problematic, as this discussion has shown, you would not really notice if sprites in one location are somewhat different from the sprites from another location.

But you will notice smudged details or upscaling artifacts even on a single sprite without any context.

User avatar
carademono
Posts: 210
Joined: Sat Jul 11, 2020 3:20 pm

Re: Using AI upscalling and filtering techniques on a pixel art and why I think it's not there yet.

Post by carademono »

sayber1 wrote: Tue Jan 25, 2022 7:02 pm If we are not talking about 3D sprites, which are the most problematic, as this discussion has shown, you would not really notice if sprites in one location are somewhat different from the sprites from another location.
Call it an overdeveloped sense of object permanence, but I do in fact notice when people suddenly look totally different after I walk through a door. Nor do I think "just turn on retro mode" is actually a solution. These are simply ways of saying you don't see a problem that needs fixing at all.
sayber1 wrote: Tue Jan 25, 2022 7:02 pm But you will notice smudged details or upscaling artifacts even on a single sprite without any context.
I completely agree. Retouching and restoring details (or adding new ones on upscaled sprites) is a big part of the next stage of the project. One way that I've done this is by layering the vanilla sprite on top of the AI enhanced layer and selectively blending parts of the sprite where detail matters most. So for example, this would be my final version of the blacksmith discussed earlier:

Image

But there are a huge number of frames to do, so as I keep pointing out, I could use some help.

sayber1
Posts: 46
Joined: Thu Sep 16, 2021 3:27 pm

Re: Using AI upscalling and filtering techniques on a pixel art and why I think it's not there yet.

Post by sayber1 »

That is the thing, I would not consider a blacksmith sprite to be in need of any improvement, as well as ane other higher res sprites. The only sprites that could, I think, be improved, are lower resolution hand drawn ones( to up their resolution, so they would not stand out next to other sprites) and of course the 3d sprites.
As Ninelan said, the only proper way to fix those, would be manually rotoscoping them with new artwork is style of handdrawns. The smoothing and AI upscaling would not do anything to properly blend them with others. As it is clear from your previous post, the dancer, even after all the filtering still looks completely different from the other sprites.
Pesonally, I still do not give up hope of there being a less painful way of doing it(Color correction, partial handdrawn additions, perhaps rerendering them entirely with models more closely in line with other DF art and going on from there).

While I would like to give such project a go, I need to focus on project, I've already started, and perhaps expanding it to guilds and daedra. If there would be artists willing to take on such project, it would be amazing. But as of right now there isn't such movement, not that I'm aware of.

Meanwhile, I would still consider doing those the smoothing way a misguided attemt, that ultimately would not solve the problem, and not without sacrificing the quality of other art in process. Would a blending 3d sprites be worth of the worsened other ones?

But as I said, this is a position of a person, who thinks of Pixel Art as of a separate art form. And such believing that bringing that art to consistensy, should be guided by principles that developed during the course of its' establishment.
But if looking from the position, there a pixel art is merely a limitation of it's time, and it's own value can be discarded for the sake of other goals, your take may be an optimal way. I do not know. I can only voice my opinion.

User avatar
carademono
Posts: 210
Joined: Sat Jul 11, 2020 3:20 pm

Re: Using AI upscalling and filtering techniques on a pixel art and why I think it's not there yet.

Post by carademono »

sayber1 wrote: Tue Jan 25, 2022 9:35 pm That is the thing, I would not consider a blacksmith sprite to be in need of any improvement, as well as ane other higher res sprites.
I also don't think the vanilla blacksmith sprite (or any of the higher res sprites) needed any improvement. They look great.
sayber1 wrote: Tue Jan 25, 2022 9:35 pm The only sprites that could, I think, be improved, are lower resolution hand drawn ones( to up their resolution, so they would not stand out next to other sprites) and of course the 3d sprites.
You have just described 98% of all the sprites in the game. THAT'S the point of my project.
sayber1 wrote: Tue Jan 25, 2022 9:35 pm As Ninelan said, the only proper way to fix those, would be manually rotoscoping them with new artwork is style of handdrawns.
Do you have any idea how many frames there are in this game? Not to mention they are at all different resolutions? This conversation is incredibly frustrating. "There's no problem at all with the vanilla sprites, except for 98% of them. And if you want to fix them, the only 'proper way' is to build a rocket ship to the moon in your unpaid spare time." Man, I'm not trying to do it the "proper way," I'm just trying to make 98% of the sprites in the game not look awful, out of place, and immersion-breaking.

sayber1
Posts: 46
Joined: Thu Sep 16, 2021 3:27 pm

Re: Using AI upscalling and filtering techniques on a pixel art and why I think it's not there yet.

Post by sayber1 »

carademono wrote: Tue Jan 25, 2022 10:00 pm Do you have any idea how many frames there are in this game? Not to mention they are at all different resolutions? This conversation is incredibly frustrating. "There's no problem at all with the vanilla sprites, except for 98% of them. And if you want to fix them, the only 'proper way' is to build a rocket ship to the moon in your unpaid spare time." Man, I'm not trying to do it the "proper way," I'm just trying to make 98% of the sprites in the game not look awful, out of place, and immersion-breaking.
I said the proper way, not the way it should realistically be done. The problem at hand is to find a good compromise for that. I do not consider AI filtering of the entire game such compromise. There are 551 texture folders in texture files of this game alone. Roughly an 80 of them are 3D sprites. You are suggesting filtering and smoothing all the graphics of this game for the sake of those 15%.

I do not consider sacrificing 85% of the game visuals for the sake of those 15%. Hell, I do not even find them distracting to the point of being irredimable.

I feel like this is not going anywhere. You think this is the only way it could be done. You do it.
I think there might be better ways to do this, I don't know. I only want to keep this statement, so people would continue this search and not abandon it because of the already existing alternative.

User avatar
King of Worms
Posts: 4752
Joined: Mon Oct 17, 2016 11:18 pm
Location: Scourg Barrow (CZ)
Contact:

Re: Using AI upscalling and filtering techniques on a pixel art and why I think it's not there yet.

Post by King of Worms »

Unless u show the results of your work, its all just a talk.. unpopular opinion but true. This theory talking is all nice and dandy, but unless there are new sprites happening to support your claims, its just a loss of time. Dont bash me for that.

User avatar
Ninelan
Posts: 187
Joined: Wed Jun 30, 2021 10:30 am
Location: Under your bed
Contact:

Re: Using AI upscalling and filtering techniques on a pixel art and why I think it's not there yet.

Post by Ninelan »

Now now, there's no need to get heated over some pixels =P
There are worse things that can happen than a critique of methods.
King of Worms wrote: Tue Jan 25, 2022 10:36 pm Unless u show the results of your work, its all just a talk.. unpopular opinion but true. This theory talking is all nice and dandy, but unless there are new sprites happening to support your claims, its just a loss of time. Dont bash me for that.
They have done sprite work if you look...
You can support me and in turn my art on my Patreon.
https://www.patreon.com/fireflyness

User avatar
carademono
Posts: 210
Joined: Sat Jul 11, 2020 3:20 pm

Re: Using AI upscalling and filtering techniques on a pixel art and why I think it's not there yet.

Post by carademono »

Ninelan wrote: Tue Jan 25, 2022 10:46 pm Now now, there's no need to get heated over some pixels =P
There are worse things that can happen than a critique of methods.
Look, I'm desperate for good feedback on this project aimed at helping me make it better. But "the entire premise of your project is bad and I have no further suggestions" is not that. God knows I've had heated debates with KoW over upscaling techniques (sorry KoW!) but always with the purpose of making DREAM's sprites look better within the framework of KoW's aesthetic vision.

sayber1
Posts: 46
Joined: Thu Sep 16, 2021 3:27 pm

Re: Using AI upscalling and filtering techniques on a pixel art and why I think it's not there yet.

Post by sayber1 »

King of Worms wrote: Tue Jan 25, 2022 10:36 pm Unless u show the results of your work, its all just a talk.. unpopular opinion but true. This theory talking is all nice and dandy, but unless there are new sprites happening to support your claims, its just a loss of time. Dont bash me for that.
I understand that. But if I would try to commit to this project, I would have to abandon the other half way. I know my limits, I'm finally trying to deliever at least something of value to this community, I don't want it to down in unfocused attempts. I've already started that project without enough preparation, and have to pay consequences of having to rethink my style and redo a lot of stuff.

I do not want to stop people from doing what they do. This discussion has started as kinda a way for me to figure out, are the problems discussed a compromise or a genuine unconscious mistakes. Then it got to discussing the actual values of pixel art.
I've stressed this out numerous times, people should do what they think is right. This thread is not to stop them from it.
This thread is about theory talking.

EDIT:
If this was about my work in general, there are some previews of it
184_30-0.png
184_30-0.png (3.01 KiB) Viewed 1034 times
184_22-0.png
184_22-0.png (4.45 KiB) Viewed 1034 times

Post Reply