A More Formal Description of the ImageMagick Machine

[Marc Rochkind]

I'm new to ImageMagick, having started using it just a few days ago. In reading the various pages here and some tutorials linked to from here, I was frustrated that I could not find a abstract description of how it works. Only lots of examples.

The page "ImageMagick Command-Line Processing" (http://www.imagemagick.org/script/comma ... essing.php) tries harder than any other reference I looked at, but even it falls short. For example, for "Image Operator" is says "An image operator differs from a setting in that it affects the image immediately as it appears on the command line," and for "Image Sequence Operator" is says "An image sequence operator differs from a setting in that it affects an image sequence immediately as it appears on the command line," implying that the former affects an image and latter affects an image sequence, but this isn't really right, near as I can tell. They BOTH act on the entire preceding image sequence. The difference is that an image operator leaves each processed image where it is in the sequence, so the result is an altered sequence, whereas an image sequence operator eats up the entire sequence and deposits an image in its place. (The names might make more sense if they were reversed: an image operator leaves an image, and an image sequence operator leaves a sequence.)

The operator terminology causes trouble in the explanation of an image stack, in which the page says "Image operators only affect images in the current stack." I'm pretty sure this is true of both image operators and image sequence operators. Or, perhaps there the term "image operator" is meant to apply to both.

Another thing that confused me: The terms "sequence" and "stack" are both used. I guess a stack is a sequence with an operator in parentheses. Or, maybe not, since the operators discussed in the stack section are said to operate on the images in the stack, so perhaps a stack doesn't include the operator. Anyway, I think an improvement is to dispense with the extra term and just talk about a sequence. The so-called stack operators operate on part of the sequence, unlike any others, but not as a stack, since index 0 refers not to the top of the stack, but to the first element in the sequence. Again, as a computer scientist, I would have found it less confusing if the word "stack" had not been used to describe something that doesn't operate as a stack.

Anyway, after reading this page several times, reading some other material, and running many experiments to test out various theories of how the ImageMagick machine works, I think I now understand it. I thought I'd post my understanding here, for two reasons: (1) to elicit corrections to the stuff I've gotten wrong, either stated wrong or genuinely misunderstood, and (2) to help anyone else who's a computer scientist, as I am, and would like a semi-formal, precise, and succinct explanation of how this magnificent machine works.

Actually, everything below is only about the convert command, which is the only one I use and the only one I've worked to figure out so far. So this isn't really about the ImageMagick machine; it's about the convert machine.

In a nutshell, I'd say it like this: The convert machine operates on a postfix expression in which operators operate on the entire preceding sequence of operands and replace the sequence with the result, which may be a sequence or image. The preceding sequence extends leftward only to the matching left parentheses. At the end of processing, all of the images remaining in the sequence are output to one or more files whose names are based on the specified output file name.

(I didn't find it documented, but if you execute:

convert xc:red xc:blue output.jpg

You get the files output-0.jpg and output-1.jpg. So, it appears that convert tries very hard to get all of the image sequence out of the box.)

Breaking the abstract description down, using loose BNF notation, we start with:

command-line ::= 'convert' image-sequence output-image

The interesting part is all in the image-sequence:

image-sequence ::= ( [settings] ( image-expression | element )) ...

The parsing is greedy; the longest possible image sequence is formed and becomes the images acted on by the closet following operator (see below).

Settings means any of the numerous settings arguments (e.g., -size), which can appear prior to any image-expression or element (see below for what they are). They are completely unaffected by parentheses; a setting applies until another setting changes it.

Now for an image-expression:

image-expression ::= image-sequence operator

An operator is any of the image operators or image sequence operators.

So, all of the following are image-sequences:

rose:
xc:red photo.jpg
xc:red photo.jpg rose: +append
-size 100x100 xc:red xc:teal rose: +append xc:blue -rotate 120 +append

It's important to note that an image-sequence may or may not have operators in it. This is what allows everyone's favorite simplest example to work:

convert image.png image.jpg

where the image-sequence is 'image.png'.

Continuing from the BNF for image-sequence a few paragraphs above:

element ::= input-image | '(' image-expression ')'

Here's is where the parentheses come in: They make an image-expression, which has to end in an operator, into an element. In addition, as an operator gobbles up the preceding sequence, the grammar indicates that the gobbling stops when the left parenthesis is encountered.

This concludes the interesting part, and the part that had me confused for days until I understood that this was a postfix machine where all the operators operate on the entire image sequence.

Here's the rest of the grammar I worked up, in case you're interested. Nothing important is going on below; it's just an attempt to account for the various forms of input and output images.

input-image ::= [format :] basic-image [selector]
format ::= /* one of the formats (e.g., rgb) */
basic-image ::= file-ref | stream-ref | built-in-ref
file-ref ::= /* file name, perhaps with globbing */
stream-ref ::= '-' | 'fd:' N /* N is a non-negative integer */
built-in-ref ::= built-in-image | built-in-pattern
built-in-image ::= 'granite' | 'logo' | 'netscape' | 'rose' | 'wizard'
built-in-pattern ::= 'bricks' | 'checkerboard' | /* other patterns */
output-image ::= [format] basic-output-image
basic-output-image ::= file-ref | stream-ref

If you've read this far, thanks for your attention, and please do leave comments. If there's interest, I'll work some more on this approach to explaining the ImageMagick machine and possibly work it up into a very long article or very short book.

Marc Rochkind

[snibgo]

A good description. A couple of observations:

The IM "convert" language is essentially postfix, although (for historical reasons) many operators can also work in the prefix position. Example: "convert -resize 800% rose: bigrose.png". I expect (and hope) these anomonlies will be phased out.

The sequence is a stack in the sense that some operators (notably "-composite") take the top 2 or 3 images and replace them with a new one.

The construct "-remap filename" is lexically awkward. It takes a filename, not an image-expression. I would prefer it to be reversed, thus "image-expression -remap". It would take the top image as a map, apply it to the second from top, and replace both with a new image.

I'm not a computer scientist but:
- I thought (and could be wrong) that an "expression" in a language usually includes the case of a single literal or variable with no operator. Your "expression" requires an operator.
- I notice you don't have a BNF item that represents exactly one image. Or is this "element"?

Here's is where the parentheses come in: They make an image-expression, which has to end in an operator, ...

If you mean that the last thing within parentheses must be an operator, you are mistaken. Eg: "convert rose: ( wizard: ) rw.png". Another weirder example: "convert rose: ( wizard: logo: ) rw.png". I'm not sure if this last one should be allowed, but it is.

I think your effort is very worthwhile. I've thought (for a very long time .. seems like decades .. probably is decades) that IM needs a formal language, and should conform to it.

[fmw42]

Did you see this page? http://www.imagemagick.org/Usage/basics/ It may clarify some things for you, though not as well organized as your summary.

But not everyone is a computer scientist and understands postfix and prefix or BNF syntax, which may appear fairly daunting to the non-computer scientists. They may prefer a more descriptive explanation and/or set of rules.

For those that understand your syntax, it is going to be very helpful.

[glennrp]

Seems to be the start of a useful document.

• input-image ::= [format :] basic-image [selector]

No spaces in "[format:]basic-image[[selector]]" (I'm trying to express
the fact that "selector" is optional and enclosed in actual square brackets.)

• output-image ::= [format] basic-output-image

Colon after "format", and no space after "[format:]"

[Marc Rochkind]

Thanks everyone for these very useful replies.

fmw42: I did read the basic page, which was helpful, but also confusing, because it added another term, "list", to the terms "sequence" and "stack", which I already thought was one too many.

Having reviewed these comments and looked again at my OP, I decided that BNF isn't the right notation to use. Instead, I've come up with the following explanation of the "convert machine":

--------------------------------------

The ImageMagick convert command takes four kinds of arguments:

1. Image expressions,
2. Settings,
3; Operators, and
4. Parentheses.

The convert command operates as follows:

1. There is a single collection of global settings, referred to as the settings, which affect the behavior of operators.

2. There is a stack of image sequences, the top one of which is the current image sequence. Processing begins with an empty current image sequence.

3. Arguments are processed in order, left to right.

4. An image expression is evaluated immediately and causes one or more images to added to the the end of the current image sequence.

5. An operator operates on the entire current image sequence. Some operators remove all of the images and replace them with a single image as a result of the operation; some operate on each image, leaving a modified image in its place in the sequence; and some alter the image sequence by, for example, deleting or swapping images.

6. A left parenthesis pushes a new, empty image sequence onto the image sequence stack, which becomes the current image sequence.

7. A right parenthesis causes the current image sequence to be popped from the stack and its images, if any, to be added to the image sequence that just rose to the top of the stack, which becomes the current image sequence.

8. The last argument is used as a filename template for outputting the current image sequence, which is the only one on the image sequence stack at that point, since all parentheses must balance. Each image is output into a separate file.

(This explanation can be expanded at this point to elaborate on image expressions, settings, and operators.)

[fmw42]

I think your current description and syntax is much better than more formal code structure syntax.

One point about 5) and parenthesis. Parentheses are intended to process only those images, operators and settings within the parentheses. There are some exceptions for which the parentheses do not respect the outside settings. There is also a -respect-parenthesis(es) to try to fix those exceptions, but there are some that are still not respected. (One, I believe, is -compose for which sometimes it is needed to be reset to -compose over before using things like -flatten, if -compose was set to some other option earlier.)

see
http://www.imagemagick.org/Usage/basics ... arenthesis
http://www.imagemagick.org/Usage/basics/#controls

IM 6 has been fixed and patched so many times over many years to make improvements but this causes these kinds of exceptions.

I believe that most if not all of this will be remedied in IM 7 in alpha development right now.

[glennrp]

8. The last argument is used as a filename template for outputting the current image sequence, which is the only one on the image sequence stack at that point, since all parentheses must balance. Each image is output into a separate file.

It is possible that they will be written to a single file, if the output format supports multiple images. To force convert to write separate files, use the "+adjoin" option.

[Marc Rochkind]

fmw42 & glennrp:

Thanks for your comments. That settings don't conform to a simple model insofar as whether they are local to what I call the current image sequence is bothersome.

--Marc

[snibgo]

1. There is a single collection of global settings, referred to as the settings, which affect the behavior of operators.

Some setting affect things that aren't operators. Eg "-density" affects the image formed by reading a PDF file.

[anthony]

The confusion of 'list' vs 'sequence' is due to the internal coding which often uses the terms interchangeably.
Though 'list' is what is favored as meaning multiple images, "sequence" is more commonly used when working with a time sequence of images (GIF animations).

The "Multi-Image List Operators" do NOT always merge all the images in sequence into a single image. Some of the layer operators such as -coalesce and -layers optimize (used for GIF animations) does not, but previous images in the list will effect later images in the list. Most of them however do merge all the images in the list into a single image.

Technically the operators -duplicate, -swap, -insert, and -delete, are more of a List Operators than a Image Stack Operator, (like parenthesis and -clone), and this is reflected in there handing in Internal IM v7 operator processing.

"Simple Image Operators" on the other hand effects all images in the current list but do so individually and independently of each other. They still are processed in sequence though.


I also do not like the operators "-remap" and "tile:" require an image filename rather than a in-memory image (though there are work-rounds for that) and would like to see them become into a new class of operators, which take the last image, and applies that image in some way to all the previous images, individually, like -clut and -hald currently does.

I would love "-composite" to also do this too, but "-composite" currently has a 'three image' (masking or X,Y displacement) handling, making this change incompatible with current use. However the -layers composite (2 list composition) can do this (and more).

And yes there are some hold overs from IMv5. -density is one of those. It is a setting for reading, but it is also a setting for writing, and last year I was shocked to discover that it can also modify images (operator) in very specific situations. Arrrgghhhh.....

[anthony]

IM v7 "magick" command does NOT allow the use of 'pre-fix notion'.

It can't as it is designed to read options form a script or pipeline, and as such operators which work on images, must have actual images to work with, or produce an error. Essentially it will allow better use of IM comamnds from other programs by 'piping' operations and looking at feedback from a background (daemon form) of the command. a technique known as co-processing.

Mind you it also relaxes the requirement that -write (explicit operation, or the implicit write at the end of the command) must have actual images. For example it will let you write zero images to a "null:" file handle. I also want -fx operator (or variant) to be able to run WITHOUT an image, so you can do maths and save results in settings, before images are read.

It also has internal flags to specify whether a explicit -read operator is need when reading in images. This is envisaged to allow the creation of a new replacement "Mogrify" command that will let you apply image list operations to the individual images being given as arguments at the end of a command.

NOTE; in IMv7 "mogrify is still using the old option code, and still needs to be re-written to use the new code. I just have not been able to give any time to it

[Marc Rochkind]

Thanks, Anthony!

[Marc Rochkind]

OK, everyone, I've got that GUI available now as a beta. I call it MagickWrapper. MacOS only. Details at http://basepath.com. Free for anyone who registers during the beta period.