Best way to upscale pixel art exactly 200%

Questions and postings pertaining to the usage of ImageMagick regardless of the interface. This includes the command-line utilities, as well as the C and C++ APIs. Usage questions are like "How do I use ImageMagick to create drop shadows?".
NicolasRobidoux
Posts: 1944
Joined: 2010-08-28T11:16:00-07:00
Authentication code: 8675308
Location: Montreal, Canada

Re: Best way to upscale pixel art exactly 200%

Post by NicolasRobidoux »

Once you have installed a very recent version of IM, you can try this:

Code: Select all

convert 3numbersscreenshot_1.jpg -alpha on -virtual-pixel transparent -filter Triangle \
distort Affine '0,0 0,0 %w,0 640,0 0,%h 0,960' -alpha off -crop 640x960+0+0 4numbers_Triangle.png
(The above is for a 320x480 picture enlarged to 640x960.)

Then, just replace "Triangle" by other filters (like Cubic, Lanczos, LanczosSharp, Robidoux=default).

Because your enlargement factor is only 2, having a smooth filter kernel is not that important. In other, a blurred Triangle may give OK results. I would also try

Code: Select all

convert 3numbersscreenshot_1.jpg -alpha on -virtual-pixel transparent -filter Triangle \
-define filter:blur=1.090909090909 \
+distort Affine '0,0 0,0 %w,0 640,0 0,%h 0,960' -alpha off -crop 640x960+0+0 4numbers_Triangle_blur.png
which I think gives an OK result. (This blur value has to do with "smoothing diagonals".) If you were enlarging by much more than 2, I would not not recommend Triangle, but you are not.
NicolasRobidoux
Posts: 1944
Joined: 2010-08-28T11:16:00-07:00
Authentication code: 8675308
Location: Montreal, Canada

Re: Best way to upscale pixel art exactly 200%

Post by NicolasRobidoux »

Oops! The special "minimize diagonal aliasing" blur for Triangle used in Clamped EWA is actually 4/3, not 12/11. In my opinion, with this value, things look really good.

Hmmm! When I have a minute, I'll compute its value for Hermite. At this magnification, Triangle and Hermite distort look OK!

(I'm not 100% sure, however, that the above code aligns things just right. Maybe we need to crop +2+2?)
Last edited by NicolasRobidoux on 2010-11-16T14:50:31-07:00, edited 1 time in total.
NicolasRobidoux
Posts: 1944
Joined: 2010-08-28T11:16:00-07:00
Authentication code: 8675308
Location: Montreal, Canada

Re: Best way to upscale pixel art exactly 200%

Post by NicolasRobidoux »

Another suggestion:

I think you could get something decent by blending the result of distort Cubic (B-Spline smoothing used radially) and distort Triangle (bilinear used radially).

All Cubic -> very smooth but a bit "faint." (It's an approximate Gaussian blur.)

All Triangle -> snappy but blocky.

(Still trying to get a handle on what the new distort does well and badly.)
NicolasRobidoux
Posts: 1944
Joined: 2010-08-28T11:16:00-07:00
Authentication code: 8675308
Location: Montreal, Canada

Re: Best way to upscale pixel art exactly 200%

Post by NicolasRobidoux »

Well, well: The world never ceases to amaze:

The Robidoux distort filter does seem to do a pretty good job.

-----

Note: When you enlarge by an exact factor of 2 with the image size convention which is the current default (actually the only one) in IM, you are better thinking of the filter as a convolution which four incarnations (corresponding to the four midpoints of the four squares which subdivide a square in two). I don't want to go there, but you could actually fine tune this convolution so it gives exactly what you want for your types of images. On my end, it forces me to make sure that I use distort with perfect alignment, and I am a bit dodgy w.r.t. to how. (If the alignment is off, the filter does not boil down to a convolution with four "avatars" anymore.)

What you are doing is not really a general "resize." It's very specific, and so properties which normally matter (e.g. filter kernel smoothness) are completely irrelevant.

In addition, because you are only sampling the kernel at very few distances (because of the exact factor of 2) if you get the alignment right (it is automatically with -resize, it takes a bit more work with +distort), methods that suck in general have a fighting chance. It appears that some of them may actually be winners.
Last edited by NicolasRobidoux on 2010-11-17T10:49:09-07:00, edited 1 time in total.
NicolasRobidoux
Posts: 1944
Joined: 2010-08-28T11:16:00-07:00
Authentication code: 8675308
Location: Montreal, Canada

Re: Best way to upscale pixel art exactly 200%

Post by NicolasRobidoux »

(Apologies for using this post to get my thoughts straight about a few things, and most likely being way too technical.)

Because you are enlarging by exactly 2 with the default image size convention of IM ("match corners") which is the best for integer enlargements (except possibly if you care about what happens near the boundary), what you really want (and what you get anyway if the alignment is just right, which it automaticaly is with -resize; distort is in a state of flux as far as resizing goes so...) is a so-called vertex-split subdivision method.

http://www.cs.nyu.edu/~dzorin/sig00course/

That I know, they are not implemented in IM. Not that they necessarily should be: They are a pain to program for general resampling or resizing without using recursion, and they only work better than standard filtering methods in fairly specific cases (namely, enlargement by a small power of 2). Yours is likely to be one of those specific cases. (You are only going "up one level," so no recursion needed.)

So, let me make two suggestions made with an awareness of the fact that, whether you like it or not, what you will get, and what you need, is a vertex split subdivision method: One which is fairly straightforward, and one which relies on distort but may give a better result (but for which I need some IM programming help: I know the internals of IM way more than its API).

Reader beware: These suggestions are likely to give TERRIBLE results for anything but enlargement by a perfect factor of 2 with exactly the default image size convention of IM.

Suggestion 1: Take the most likely candidate filters (Triangle, Hermite, Gaussian, Cubic or even Point, in that order, if you don't want any haloing, Mitchell (the default), Catrom, Lanczos2 and Lanczos if you are OK with some).

One filter at a time, compute ($FILTER is the filter you are trying, $BLUR is the value of the blur you are trying)

Code: Select all

convert in.png -filter $FILTER -define filter:blur=$BLUR -resize 200% out$FILTER$BLUR.png
for blur values between about .5 and 1.5 (depending on the filter, some of the values at either end will suck big time) and inspect them visually.

For example, for moderate enlargements (2x), I kinda like -filter Triangle -define filter:blur=1.33333333333333.

When you've honed in on an OK filter/blur combination, refine by trying blur values close to this value. You'll be amazed at how much difference changing the blur by .1 can make with some filters.

(To do this, you don't need a recent version of IM: It relies on -resize, which has been stable in terms of results, for some time. Lanczos2 is the only "new" thing and you can get it with selecting 2 lobes.)

Suggestion 2:

My favorite vertex-split subdivision method is the midedge method, which I think would work well for your image.

One can emulate midedge using filters by adding .25 times the result of

... -resize -filter Point ...

and adding .75 times the perfectly aligned result of

... +distort -filter Point -define filter:blur=1.6 ...

(Unless I am mistaken RE: the way Point is automatically modified for distort, 1.6 a correct blur because 1.6*.5=.8 is between sqrt(5/8) and sqrt(9/8), which selects the three closest input points for each of the subdivision points. blur=2 should work too.)

If I can find another minute (profs are paid for pontificating, not programming) I'll figure out how to do the full coding.

Note to Anthony (and Fred):

This kind of emulation of one level of a subdivision method with filters is actually kind of cool (if I say so myself). Maybe it should go in the examples (once the exact code is nailed down)? Then, for the special case of enlarging by a factor of 2, people would have access to these kinds of method. Of course, by chaining the operations, they would also get 4, 8...
Last edited by NicolasRobidoux on 2010-11-17T11:23:56-07:00, edited 4 times in total.
User avatar
GreenKoopa
Posts: 457
Joined: 2010-11-04T17:24:08-07:00
Authentication code: 8675308

Re: Best way to upscale pixel art exactly 200%

Post by GreenKoopa »

:shock:
Halle wrote:Best way to upscale pixel art exactly 200%
NicolasRobidoux wrote:(Apologies for using this post to get my thoughts straight about a few things, and most likely being way too technical.)
I guess a lesson to Halle and the rest of us to avoid superlatives in our subject lines.
User avatar
fmw42
Posts: 25562
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: Best way to upscale pixel art exactly 200%

Post by fmw42 »

I don't know if the following are appropriate for you type of image. They are more for cartoon type (graphics gaming) images.

You might take a look at scale2x at http://scale2x.sourceforge.net/index.html (and scale3x at http://web.archive.org/web/200802082151 ... /hq3x.html)

PS Nicholas, what do you think of these kinds of scaling approaches. Anthony is aware of them also.

Fred
NicolasRobidoux
Posts: 1944
Joined: 2010-08-28T11:16:00-07:00
Authentication code: 8675308
Location: Montreal, Canada

Re: Best way to upscale pixel art exactly 200%

Post by NicolasRobidoux »

Fred: I think that scale2x etc are fantastic for their intended use. Even more sophisticated than midedge et al (which has a somewhat different target use).

Halle's images, however, are not quite "MarioBros"ish enough for scale2x. They are more like a combination of smooth text, noiseless "nature", and boxy frames.

I actually think that, with proper alignment fix (no time for that now), one among upsize, upsharp and upsmooth methods I programmed into the competing project nip2/vips may work well. Unlike most of what's found in IM, upsize and upsharp are nonlinear methods, and upsmooth and upsharp use one subdivision step.

On the other hand, I'm really impressed by the IM distort methods, and for this particular problem I think that my advice about using standard resize filters and playing with blur may give Halle what he wants.

I wish I had time to try everything, because I find Halle's problem quite interesting and educational. Full of special features which call for non-generic solutions.
Last edited by NicolasRobidoux on 2010-11-17T15:02:06-07:00, edited 1 time in total.
User avatar
fmw42
Posts: 25562
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: Best way to upscale pixel art exactly 200%

Post by fmw42 »

NicolasRobidoux wrote:I actually think that, with proper alignment fix (no time for that now), one among upsize, upsharp and upsmooth methods I programmed into the competing project nip2/vips may work well. Unlike most of what's found in IM, upsize and upsharp are nonlinear methods, and upsmooth and upsize use one subdivision step.

These sound very interesting. Do you have any links to examples? Do you think at some point in the future, they might make useful enhancements to IM or are they too specialized?
NicolasRobidoux
Posts: 1944
Joined: 2010-08-28T11:16:00-07:00
Authentication code: 8675308
Location: Montreal, Canada

Re: Best way to upscale pixel art exactly 200%

Post by NicolasRobidoux »

fmw42 wrote:
NicolasRobidoux wrote:I actually think that, with proper alignment fix (no time for that now), one among upsize, upsharp and upsmooth methods I programmed into the competing project nip2/vips may work well. Unlike most of what's found in IM, upsize and upsharp are nonlinear methods, and upsmooth and upsharp use one subdivision step.
These sound very interesting. Do you have any links to examples? Do you think at some point in the future, they might make useful enhancements to IM or are they too specialized?
Some information:
http://www.vips.ecs.soton.ac.uk/index.p ... erpolators

Once you have a recent enough install of nip2/vips (you may have to compile from source: the code stabilized late last Spring; not sure exactly how recent a build you need):
http://www.vips.ecs.soton.ac.uk/index.php?title=Stable

Start nip2, load an image, click on the bar next to the image thumnail, and menu yourself to

Toolkits -> Image -> Transform -> Resize -> Scale

(If you don't see upsmooth, upsize and upsharp in the menu with default Bilinear, you have a version of nip2 which is too old.)

Unfortunately for Halle, I think that John Cupitt and I have not carefully fixed the image size convention (yet). One could fix it through the command line vips, but I'm a bit busy right now to assemble the right command line (it's very IM-like in any case).

As to whether they (or their descendents: I'm not quite done improving the methods) should make it in IM, this is a complicated question: The approaches taken are not very resize.c-like. IM, really, is awesomely set up for a linear filtering approach to resampling. These methods are not really compatible with this approach, so they would have to be programmed "separately" (like adaptive resize?).

You can see the source code here:

http://vips.svn.sourceforge.net/viewvc/ ... /resample/

None of these methods are really suited for downsampling. And they all are novel (which is a good and a bad thing).

In the directory, you will see code for:

vsqbs a.k.a. upsmooth is Vertex Split Quadratic B-Spline. It starts by doing a mid-edge subdivision, and finishes off with Quadratic B-Spline smoothing. I am not sure that it really does, in general, better than, say, IM cubic B-Spline smoothing (a.k.a. Cubic). It's a fully 2D linear method which is monotone (no haloing, like B-Spline smoothing, really.) It uses less data than B-Spline smoothing, however (it has an 8 point stencil (3x3 minus one corner) as opposed to a 4x4). If a resampler is set up to draw data from a small cached rectangle, the small footprint makes it faster than a bicubic method, even though it is a fully 2D scheme (as opposed to a tensor method), because less data is drawn from the cache, and the cache patch does not need to be updated quite as often.

lbb is Locally Bounded Bicubic a.k.a. upsize. It's a nonlinear Catmull-Rom with basically non-existent haloing.

nohalo a.k.a. upsharp is one step of a special sharpening subdivision step which does not introduce haloing, finished off with LBB interpolation.

As you probably have figured out already, these methods are for halo-haters.

(I feel compelled to mention some of the people who contributed to these methods: my student Chantal Racette, John Cupitt of Imperial College, and to a lesser extent Minglun Gong of Memorial University, my student Adam Turcotte, Kirk Martinez of U. Southampton, Julien Dompierre of Laurentian University and my student Eric Daoust.)
User avatar
anthony
Posts: 8883
Joined: 2004-05-31T19:27:03-07:00
Authentication code: 8675308
Location: Brisbane, Australia

Re: Best way to upscale pixel art exactly 200%

Post by anthony »

It was the Scale X algorithms I referred to previously!

The best start point in the Wikipedia page, and its links section at the bottom
http://en.wikipedia.org/wiki/Pixel_art_ ... algorithms

There are quite a number of such algorithms. Scale2X HQ2X and even for rotation.
Anthony Thyssen -- Webmaster for ImageMagick Example Pages
https://imagemagick.org/Usage/
Halle
Posts: 4
Joined: 2010-11-12T04:30:11-07:00
Authentication code: 8675308

Re: Best way to upscale pixel art exactly 200%

Post by Halle »

Heh, I had unsubscribed from the thread some time back since it seemed to have quieted down, little did I know :)
I guess a lesson to Halle and the rest of us to avoid superlatives in our subject lines.
Certainly not -- it's great to get so much information and since none of my previous attempts were satisfactory, I'm looking forward to trying out what has been described here when I have some time. Thanks very much for going so in-depth, Nicolas, and thanks to everyone who contributed to this thread.
NicolasRobidoux
Posts: 1944
Joined: 2010-08-28T11:16:00-07:00
Authentication code: 8675308
Location: Montreal, Canada

Re: Best way to upscale pixel art exactly 200%

Post by NicolasRobidoux »

So, let's add "CSI enhance" to Anthony's to do list.

P.S. This was an attempt at a humorous reponse to a post in this thread which appears to have disappeared.
Post Reply