[Oiio-dev] Handling unassociated alpha

[Larry Gritz]

On Dec 18, 2011, at 10:40 AM, Troy Sobotka wrote:

> You said in a post "It's wrong in so many ways.  I can expand on this
> idea if any of you don't know why."
> 
> I'd love to hear your official take on associated versus unassociated
> alpha and why.

Well, this is hardly "official", but here's my take:

If you think of image synthesis as akin to what a camera does, and the image like a photograph where the pixel values are proportional to how many photons hit the detector in each bin, then associated alpha is what you want -- the "color" values really are how bright each pixel should appear.  (If you are using a linear color mapping of value to intensity, which I also recommend under all circumstances.)  "Alpha" in this case is is an extra piece of information, that has no direct analog in photography, and tells you what the "coverage" is for each pixel, in case you want to do compositing of images.  But if you strip the alpha off, the color still makes total sense as the brightness at each pixel, and you can display the image using only the information in the color channels (with proper compensation for the color response for your display).

Math is all linear and easy with associated alpha.  The compositing operation "O = A over B" is very straightforward, and is identical in all channels:

	Or = Ar + (1-Aa) * Br;
	Og = Ag + (1-Aa) * Bg;
	Ob = Ab + (1-Aa) * Bb;
	Oa = Aa + (1-Aa) * Ba;

If you want to express your math as vectors/tuples, 

	O = A + (1-Aa) * B;

Nice!

And because the RGB values represent the amount of light hitting the detector, it makes perfect sense for it to be 0 (no light hit), regardless of how much of the background is covered, and it also makes sense to have values > 1 (there is no limit to how bright it can be) and for that matter, rgb > alpha.  As a simple example, the color (.5, .5, 0, 0) might be used to represent a yellow glow that adds light without blocking any of the background at all.

With the other approach, the RGB channels do not mean much on their own, certainly not anything analogous to light and cameras.  The "alpha" is a mask, so the color of each pixel is actually RGB*A.  This has some strange consequences:

1. The over operation is a little funny now:

	Or = Ar*Aa + (1-Aa) * Br;
	Og = Ag*Aa + (1-Aa) * Bg;
	Ob = Ab*Aa + (1-Aa) * Bb;
	Oa = Aa + (1-Aa) * Ba;

It's both more math (who cares now, but it used to be helpful) but it's also less elegant -- you can't easily express it in vector notation as cleanly, and the math done to get the new alpha is different than what you do for the color channels.  This is good circumstantial evidence that we're not working with as clean an abstraction.

2. Suddenly, it is no longer possible to express the "layer that adds yellow emissive light without blocking what's behind it."  There is no unassociated (r,g,b,0) that can be yellow, since the amount of light is always the "color" multiplied by the "mask".  If the mask is 0, so is the amount of light, by definition.


>  Given the need to unassociate alpha for color
> management and other linear operation, is there an optimal approach
> you can envision?

I'm not sure I know which operations you mean.  Can you give some examples of operations where you'd need to operate on the "unassociated" values?  And how you'd deal with the fact that the set of color/coverage tuples representable by associated values is a superset of what is representable by unassociated, what do these operations do with the values not representable (like my transparent but incandescent yellow)?


I am not completely without sympathy to apps that started out life with unassociated values.  Color+mask is analogous to loading your airbrush with a certain color paint, and spraying it with a mask covering some areas.  So for a painting app designed a long time ago (few bits per pixel, no floating point, and no experience with image synthesis), it was a representation that made some sense, and allows independent manipulation of the mask without losing precision in the color channels.  But it doesn't make much sense these days, IMHO.  Image synthesis used the cleaner abstraction, painting did not.

--
Larry Gritz
lg at larrygritz.com