Ability to split and combine alpha channel

[alexmitchellmus]

[Original title: More Alpha tools & Image Mode Info]

Currently I am working with lots of compositing, which means lots of alpha work. I don’t know of any ‘easy’ way to manipulate alpha channels in Vuo. Thats fine as I end up doing stuff like this in ShaderToy:

void main(void)
{
vec2 uv = (gl_FragCoord.xy/iResolution.xy);
vec4 alpha = texture2D(iChannel0, uv);
vec4 image = texture2D(iChannel1, uv);
gl_FragColor = vec4(image.rgb, 1.0)*alpha.a ;
}

(NB: Shader updated to reflect premultiplied alpha in Vuo)

However nodes like split image RGB & combine image RGB really should have Alpha ports as well to be complete. (This sort of node would be known as an Un-pre-multiply process:I didn’t know until now either

That way we could easily access / manipulate & replace alpha channels in Vuo without having to use ShaderToy.

If there is already a request (I did search but found none) do let me know- and if I am missing something let me know to… I have searched for alpha in library but only found blend modes etc. (which I don’t think have advanced alpha blends- like in FCPX).

UPDATE: After some research, I think that if Vuo is using pre-multiplied alpha, then there is no easy way to extract alpha channel from any images. Can this feature request also include (obviously) un-premultiplied alpha mode?

Found more info here: http://blogs.msdn.com/b/shawnhar/archive/2009/11/06/premultiplied-alpha.aspx

[alexmitchellmus]

I think also there needs to be a way to distinguish between 8 and 16 bpc image outputs, as well as Pre-Multiplied and Non-Pre-Multiplied image cables, (RGB v’s RGBA). Even if this is something that can be set manually in the node (for example if you know that you are programming a node that outputs RGBA then you write it).

I don’t know if they need to be separate types, but surly there must be some way for Vuo to test for these qualities (even if its in the port info box):

[alexmitchellmus]

This article helped me understand a bit more. I believe Vuo is using Pre-multiplied alpha.

premultiplication-demystified

Also this thread was very helpful- albeit about a different issue:
http://stackoverflow.com/questions/18783752/fragment-shader-and-coloring-a-texture

Considering that in Vuo, some Alpha is pre-multiplied, and thus burned into RGB it makes sense why there are little (no) tools for working with Alpha Channels. Is there any thoughts on this from the community? Personally I would like to be able to effect Alpha Channels separately, (especially for complex compositing) otherwise one has to ‘do it all’ at the same time. (Alpha maths all have to be computed together to output only RGB)

[alexmitchellmus]

Just made this test, which clearly shows that make radial gradient image outputs in RGBA format, and that without pre-multiplying (I’m using ShaderToy) you get horrible output:

In the composition you can change the color of the gradient, and see the results of the different channels and pre v’s non-pre-multiplied.

EDIT: Last window is meant to be called: non-pre-multiplied, NOT un-pre-multiplied. Very different.

ImageChannelTester.vuo (11.4 KB)

[jstrecker (Jaymie)]

The Split Image RGB Channels / Combine Image RGB Channels part would certainly be doable. Currently, those nodes apply the alpha to each of the R, G, and B channels. We could add the option to separate out the alpha.

For the Shadertoy code you posted, could you instead use Apply Mask?

What other alpha tools do you have in mind?

mask.vuo (2.09 KB)

[alexmitchellmus]

I did search for alpha in library, can we get “alpha” keyword added to apply mask ?

Also @jstrecker, you mentioned that all of Vuos colour are premultiplied. Then how does the alpha channel get through? For the alpha channel window - I am using shadertoy to extract only alpha.

[alexmitchellmus]

Node suggestions:

• premultiply (convert RGBA → RGB)

• un-premultiply (convert RGB → RGBA)

• invert alpha only (not colour channels: RGBA → RGB,1-A)

• extract alpha channel (RGBA → A or AAAA)

Also - I feel that Vuo in many ways is a “live” compositor… So there are probably many more nodes that are used to work with alpha that I have forgotten out of the list.

[alexmitchellmus]

Guys I’ve realised that it’s probably impossible to keep track of premulitplied vs unpremultiplied alpha. This really is going to have to be a feature that people have to know when it’s wrong.

The issue will come up when trying to color correct (darken etc) footage that is premultiplied. Because the alpha influences the color channels as well- it will cause strange colors in on the edges of image-alpha.

So a simple Un-pre-multiply node would fix that before any color changes.

[jstrecker (Jaymie)]

After some research, I think that if Vuo is using pre-multiplied alpha, then there is no easy way to extract alpha channel from any images

I think you might be misunderstanding an aspect of premultiplication. Premultiplication means multiplying the red, green, and blue values individually by the alpha value. It leaves the alpha value unchanged.

ImageChannelTester.vuo

As demonstrated by your composition, Vuo 1.2 has some inconsistencies in how it handles premultiplication. Those will be fixed by 1.3, at which time premultiplication will work like this (to be included in documentation):

• VuoColors expect and provide non-premultiplied RGB values.
• Images/textures are premultiplied, but framebuffers are not.
  • VuoImages expect and provide premultiplied RGB values in CPU buffers, and textures provide premultiplied RGB values when sampled in GLSL.
  • Colors written to the GLSL framebuffer (gl_FragColor) should be non-premultiplied.
• Always blend premultiplied colors (so transparent colors don’t unduly affect the result).

I think also there needs to be a way to distinguish between 8 and 16 bpc image outputs, as well as Pre-Multiplied and Non-Pre-Multiplied image cables, (RGB v’s RGBA).

We can add the color depth to the port popover. No need to add the premultiplied info, since VuoImage is always premultiplied.

I did search for alpha in library, can we get “alpha” keyword added to apply mask ?

Sure.

premultiply (convert RGBA → RGB)
un-premultiply (convert RGB → RGBA)

As mentioned above, premultiplication preserves the alpha channel. Besides that, nodes to premultiply/unpremultiply wouldn’t fit with Vuo 1.3’s consistent handling of premultiplication.

invert alpha only (not colour channels: RGBA → RGB,1-A)

Because of premultiplication, any RGB values where A=0 in the input image would be 0, so inverting the alpha would result in meaningless colors. It wouldn’t restore the original colors as intended.

extract alpha channel (RGBA → A or AAAA)

Sure, that would be the modification to Split Image RGB Channels / Combine Image RGB Channels that I mentioned in a previous comment.

Looks like that’s the main thing left to do on this feature request, so I’ve adjusted the title and opened it for voting.

[alexmitchellmus]

After further research into premultiplied alpha I think that we would need unpremultiply. However you would need to do whatever effect you were doing (colour correction) then remultiply to achieve correct colour correction only on image. Otherwise both alpha and image will be colour corrected. Does that make sense?

This link discusses colour correction of premultiplied images without unpremultipling first then re-premiultipling: https://www.google.com.au/url?sa=i&source=images&cd=&ved=0ahUKEwjHlOeQo7rLAhVn6aYKHXAPAgYQjhwIBQ&url=http%3A%2F%2Fopticalenquiry.com%2Fnuke%2Findex.php%3Ftitle%3DPremultiplication&psig=AFQjCNG1JVwExLZcSOL0TuoznGyO6emK2g&ust=1457842061967589&rct=j

If team Vuo feels that this isn’t needed- can we at least have the ability to add this in shadertoy? (So that we can play with the RGBA values?)

If we can still play with RGBA then I can do my own maths so disregard my above comment. However I do want to be able to access the RGBA descreetly.

Thanks a million! 1.3 shaping up amazing!

PS: I understand how what I wrote could be misunderstood. There isn’t really an easy way to describe unpremultiplied and premultiplied alpha.

I am interested in using Vuo for realtime effects - 3D - VR - compositing etc. So this sort of functionality is key to me using Vuo.

My understanding of this is that:

Unpremultiplied: 1, 1, 1, .5 (RGBA) =
Premultiplied: .5, .5, .5, .5 (RGBA)

I removed the A from the above text to simplify the logic. Sorry- I can see it didn’t really make sense. Very busy on a project now (using Vuo for 360 VR compositing) so I didn’t have time to fix.

[alexmitchellmus]

This is a graphic I lifted from: Compositing: Science and Nature | Selections: The Key to Compositing

Issue with pre vs unpremultiplied treatment. Left plane correct:

Here is clearer example of the plane added to background incorrectly & correctly:

As long as we are able to still use Vuo to fix these issues (and to be able to color correct pre & unpremultiplied color images then I am happy) :-) Because not too secretly I want to be able to use Vuo as a offline & realtime compositing environment. (Actually I am already doing this!)

[alexmitchellmus]

For inverting alpha request, could this simply be invert mask then? No RGB channels.

[jstrecker (Jaymie)]

For inverting alpha request, could this simply be invert mask then? No RGB channels.

Would this be covered by Invert Image Colors, as in the attached composition?

mask-invert.vuo (6.36 KB)

[jstrecker (Jaymie)]

Vuo 1.2.4 added Opacity Image ports to the Split/Combine Image RGB Channels nodes, as well as the new Split/Combine HSL Channels nodes.

@alexmitchellmus, I’m not sure if that covers everything you mentioned in your most recent comments. In the interest of keeping things organized (1 feature per feature request), I’ll go ahead and mark this “Released”. Please created further FRs if needed.