Performance with Blur Image node's Quality slider

Having a look at the new Blur Image node. Looks great, but seems to take a severe performance hit with Render Image to Window Requested Frame routed back to event input. Bug?

Edit: I’m on OS 10.11.6. Appears to be directly related to the “Quality” slider – at around .3, dragging nodes around in the editor starts to get sluggish. I’m using a jpg, around 1800 x 1300px, nothing else in the comp.

I see that it’s the Disc Blur that performs poorly. The other types do better.

How does the Blur node work out for you guys when connected to the webcam, aka Receive Live Video, at f.e. gaussian blur at 20 ?

For me, even set at a very low quality (0.2), the FPS drops at about 7-10, I get a huge video delay and the whole Editor becomes slow. I’m not even talking about the quality set at 1.

I made a comparative test with QC, it blurs instantly, no delay, still super fast editor.

If it runs fine for you other guys, it may be related to my low end GPU only, then it’s no big deal, but still QC managed to handle my GPU.

Here is a side-by-side screenshot + the compositions.

PS : The blur quality on QC is better because the Vuo one is set to 0.2 quality. However, the gradients in QC seem lower, as if the image was rendered in 8 bit. In Vuo the image popover says it’s at 8 bit too, but looks better. Perhaps that’s the key ?

Blur Webcam - Vuo.vuo (2.51 KB)

Blur Webcam - QC.zip (13.1 KB)

8 bit may be a key…

One way to make a fast blur is to take the raw input image, and downsize it to half or quarter resolution, for example. You take the lower resolution and render it to a quad/sprite/billboard that’s enlarged so that it becomes as big as the input image was.

You then take the image buffer of that and blur it (eg “render in image” type operation).

Since you’re blurring an image that’s already been defocused by OpenGL, it won’t take as high a level of blur. I’ve seen a few variations on this concept over the years.

One variation would be to blur the image while it’s down-res’ed so that the blur is working on less pixels, and then blow it up in OpenGL by enlarging the sprite/quad after. That will look a little grainier though.

I’m not sure if that idea can be successfully applied in VUO or not but it’s worth a shot.

Well, I set this up in Vuo to downsize/upsize, looks like a fairly direct relationship between scaled size and performance.

@jersmi, the blur performance can vary quite a bit depending on the type of blur, radius, quality, image size, and your computer hardware.

Disc blur is the slowest blur (as it says in the node documentation, “most expensive to compute”) because the algorithm is inherently more complex. For the gaussian, linear, and box blurs, blurring in 2D is equivalent to blurring twice in 1D (horizontal then vertical or vice-versa). This is nice because two 1D blurs are a lot cheaper to compute than one 2D blur. For disc blur, you can’t split it up into 1D blurs, you have to do the more expensive 2D blur.

There probably are some optimizations that we could do to improve blur performance in Vuo. However, we wouldn’t be able to get disc blur as fast as the other blurs because of its inherent complexity.

@George_Toledo, do you have a link to any of the downsizing techniques you described? We’d like to learn more.

@Bodysoulspirit, I see a similar disparity in performance between your QC and Vuo compositions on my (also weak) GPU, the NVIDIA GeForce 9400M. It looks like a bit of the slowness comes from Receive Live Video but mostly it’s from Blur Image. Probably Apple has invested a ton of developer time into optimizing Core Image. Maybe that explains why their gaussian blur is faster.

Probably Apple has invested a ton of developer time into optimizing Core Image. Maybe that explains why their gaussian blur is faster.

Mmm yeah. I tried to look for the source code of the blur image QC patch code on internet but couldn’t find it. But you guys as the Kineme team are much more aware of a possible source code availability ;)

What I found however is Vade’s v002 Blur patches (and the Github Source Code).
It’s released under attribution - non commercial so that won’t help, but perhaps some look into the code might give some ideas ?
But Vade says his blur is even better then Apple theirs.

Download the Vade Patches : v002 Blurs 2.0.2 | v002
Github : GitHub - v002/v002-Blurs: v002 Blurs are replacement gaussian, motion and zoom blurs in Quartz Composer, optimized for realtime use. They respect alpha channels and are drop in replacements for standard blurs

EDIT : I later realized you surely mean library code outside the patches with “Optimizing Core Image” + that you already mention the v002 blur on another blur topic. Sorry.  

the blur performance can vary quite a bit depending on the type of blur, radius, quality, image size, and your computer hardware.

Sure. When I actually spent a little time with the node in the downscale/upscale comp, I saw that it works as designed - no bug. I appreciate the image quality with these, nicely put together, imho. I’d vote for some optimization, hope something comes along. The performance drop is pretty harsh right now – unless some tricks come along, might be unusable for real time video work at any HD format I use.

Vade’s optimizations were impressive for the v002 QC blurs. Pretty much blew built-in QC blurs out of the water for speed. VDMX has a nice selection of real-time blurs, too (including the v002’s).  

Although Team Vuo couldn’t directly use Vade’s blur code, or really even look at it for ideas, because of the non-commercial license, if the code is based on work by others (research papers, etc.) we might be able to use those sources.

Or I think it would be OK for someone in the Vuo community to port the v002 blur code to Vuo nodes and share them in the Node Gallery.

For posterity, here’s the blog post with v002 blurs (with vade’s mention of “shortcuts” for speed compared to QC’s blurs): v002 Blurs released. | v002

But that’s 2008… To keep it inside the Vuo circle, just a quick look at the shadertoy.com community with a search for “blur” is a reminder of the the ongoing inventiveness and all the novel approaches… mipmapping, pseudo blurs, bokeh, DOF, etc. (Of course I’m not considering at all where you draw the line with licensing in this context.)  

It’s very hard for me to participate on the technical code points about this.
I don’t know what I should search for, if GLSL or OpenGL or both. Nor what the team already uses for its code.

Talking about Vade’s code, has anyone looked at it at all ? Is he using CoreImage with some side optimizations or has he built some scratch code ? if CoreImage then it’s no use for Vuo if I’m right.

As you say Jersmi, things are evolving so fast, new techniques etc. and that’s the beauty of it. I will try some Shadertoy blurs when the updated syntax is implemented and see how those work.
I see on the web some talks about the Vulkan API with great blurs too.

I also remember Alexander Mitchell’s link to an MIT license blur on the More Blur Nodes topic (Github MIT blur) don’t know if some tests were made but that is also from 2014

The only article I’ve cross read about blurs seem to mention George Toledo’s downscales techniques but I may be wrong.
Surely the team already has read this article ! : https://software.intel.com/en-us/blogs/2014/07/15/an-investigation-of-fast-real-time-gpu-based-image-blur-algorithms

@jersmi and @Bodysoulspirit, thanks for the links. The Intel article provides a nice summary of techniques presented in other articles.