Knob for Faster Expressive Results

[jersmi]

This is about supporting creativity and expression. Vuo lacks tools for working quickly – it can be difficult to set something up and see expressive results right now. In one sense this is a UX/UI issue – modular software is particularly difficult when it can cover so many use cases – how do you decide what to highlight?

Scenario:
say I have a creative thread I wish to follow, a sketch, a seed. Like having a sketchbook close by, I want to be able to set up and explore possibilities ASAP.

To this end I wish I could load a simple ‘knob’ – a Share Value in the form of a knob UI element, could just be a float value normalized to 0 to 1 range. Then I could quickly refine it with Calculate to set up an expressive range.

This idea applies to a majority of Vuo scenarios, here’s a simple example: say I want to play with thin line widths on a 3D Object, say a Make Grid Lines Object – I wish I could load a knob and quickly type an expression into Calculate, knob * knob * .01, connect and go. Two things get in the way – event flow and parameter ranges on modules. The more parameters a module has, the more difficult it is to use as an expressive object. A tool like this would help.

Features (in order of importance):

• Don’t need to consider event flow because it will just work.
• The knob could simultaneously be published (for creating comps to load in other software, VDMX, CoGe, Resolume) and still be used in a running Vuo comp.
• The knob could be assigned to an external source via either OSC or MIDI.

Some think all parameters in modular software should be normalized. In lieu of this, it would be great to have some solution that considers these basics. Pretty sure these concepts have been hashed out a kajillion times by now from various directions…

[useful_design (Alastair)]

And include knobs in a nice macOS GUI window when we go File Menu> Export to App… Bonus points for being about to click on the knob with mouse/stylus and drag up/down or left/right to edit value on the fly… like many Adobe apps and DAW tools.

maybe the knobs could appear horizontally in a dropdown in the Composition runtime window the same way “published to root” inputs did in the QC Viewer window. Multi-position switches would be a welcome addition to sit along side knobs. On/off switches too.  

[jersmi]

Nice ideas, I’d use them! Specifically on the point of immediacy, being able to click and drag value changes like knobs/sliders in other apps, if I didn’t communicate that, should be part of the core FR.

[jmcc (Jean Marie)]

@jersmi,

How about a Fire when Value Edited node? You could then create a protocol composition where you could publish the input port. That way you could change the values by exporting your composition as an FFGL plugin, or importing it directly into VDMX.

You could also use it with Receive OSC Messages or Receive MIDI Events.

If you don’t want to connect/disconnect cables when you switch between in-editor testing (using the Value port’s input editor) and a production mode, you can use a Select Latest node to switch between the two.

Does that meet your needs?

FireWhenValueEdited.zip (4.4 KB)

[useful_design (Alastair)]

Those are good ideas @jmcc for Vuo developer/users but for those of use publishing comps or especially applications for macOS for novice users, external controllers and the like aren’t really an option.

[jersmi]

In the spirit of Vuo right now I think it’s great, I will use this a lot! Thank you!! I appreciate seeing how it was made as well.

Alastair 's point is important, because his point revolves around output (to Mac App, etc.). (Where input is successfully covered for host app/external control).

I guess the next level up would be a larger task, though. For example, I can imagine a subcomposition with n knobs (or sliders), with the knobs (and ports) published and visible in the subcomp UI (thus in the Vuo editor), which could then be published/available as one (or part of a) UI for export to Mac app, etc. Knob position → subcomp UI would need to be addressed – dragging knobs around in some “edit UI” mode would of course be awesome, or just knob xy pos from the right click menu like for published ports. “Skinning” (or just coloring) the knobs for export would also be in play.

[useful_design (Alastair)]

I’m keen to prototype a knobs UI in Vuo and Adobe Illustrator when I get some time for it. Perhaps @MartinusMagneson can help me to wrangle some C code for me to build a Custom Node when I’ve finished a prototype in Vuo.

As you say @jersmi to have a UI module that works both in the Vuo Editor and exported apps we’d need a new Vuo feature class to pull that off.

A UI option for custom and special nodes might be a floating panel such as QC had with the ⌘2 panel for each patch or something on the editor canvas itself appearing as a sub composition with dials and data, but I’m sure @smokris poured ice-cold water over this kind of idea many years ago or in the window bar or dropdown from editor/composition window bar.  

[jersmi]

Haven’t tried much lately, is there an acceptable way now to get UI up and running in a Vuo Mac App?

Regarding knob UI, was wondering about GLSL + “Is Within Circle”. @George_Toledo 's arc generator node is a shader, for ex. Maybe?

At any rate, @jmcc’s solution is nice for a lot of my needs. Thanks again!  

[jmcc (Jean Marie)]

@jersmi,

Our developers created the node. It’s a team effort!

[George_Toledo]

Coincidentally…I was just thinking last night about the way that nodes in Max/MSP can look like sliders, knobs, etc, in the editor, and be hooked to “whatever”.

There is also a mode to the editor where all of the non GUI nodes get hidden, and the end user only sees the GUI node objects. This allows one to code GUI in the editor and then wrap the graph up in app form, all in the same environment.

I had also recently mentioned that I thought custom plugin/node views would be a very good idea for VUO, and this thought about the knob seems very much related.  

[George_Toledo]

Another way that GUI in VUO could work is for users to publish ports they want to hook to GUI.

Another app (a VUO interface builder?), or maybe a built in feature, could cycle through the published ports and programmatically spawn GUI for published ports.

It could be that the user adds a specific key to published name or something to control whether a knob, slider, etc, is created…or they could be presented with a set of options that they then choose from via an interface.

If you look at the sample code for Apple’s QCTV project, and I think QC Performer (or some name like that), you will see a very simple concept along these lines. The GUI is programmatically spawned by the QCTV host app, according to published port types.

That said, I think there is a strong use case for exposing a way to see something like a knob or slider (and other GUI objects) as a node itself, within the VUO editor.  

[jersmi]

@jmcc, your (very nice) example .c node opens up in a text editor when I double click on it from inside the Vuo editor (in my case BB Edit). No problem, really.

Yeah, George, I agree, I think there is a strong use case for seeing GUI objects. Visual feedback is a powerful tool. Like being able to see (and edit, wow!) an audio waveform on a computer – imagine that before and after!

Since we’re using examples of other software – there is a modular audio app called Audulus (currently in beta for version 4) that uses an effective approach with knobs.

Audulus LLC

Audulus was originally directed towards iOS for iPad, then to macOS + AU plugin version. In Audulus users create modules (which would be subcomps in Vuo) for sound stuff then “expose” knobs and other elements from inside the module. Then in the editor (root view) key command an “edit UI” mode to drag knobs around. (Can also cable directly to the knobs to modulate, which makes a lot of sense for modular audio world.)

I like that the dev for Audulus is committed to strong visuals all for UX – he just introduced a shader node for GLSL, and already has for the current beta other clever ways for supporting custom visuals, SVG and PNG support, using demux to animate PNGs specifically for knob UI’s, etc.

Audulus screenshot, users create/customize modules:

 

[jersmi]

So yeah, I do think think the heart of this feature request is for a visual GUI element in the editor that could:

1. Have the same behavior as Fire When Value Edited.
2. Be viewable in the editor.
3. Be viewable in a subcomp UI (with the ability to position them).
4. Export as part of a GUI menu for a Mac App.  

[jersmi]

What about this FR spinning off into one for a “UI protocol”?

Features:

• UI elements could be arranged (and resized?) in the comp loader by dragging UI element bounding boxes with the mouse. (Perhaps isolating Viewer mouse interaction to a UI protocol could relieve some “what about this scenario” confusion?)
• When used in a parent comp the UI comp GUI would show in the editor window. (Maybe a “show UI” true/false toggle in the protocol ports?)
• The UI comp could be saved to modules as a library item.
• Protocol output would be the combined UI layers that could then be used as part of a larger comp view as an overlay or sidebar, or not. (Generating output ports for UI values would obviously be part of it, to connect to other parameters in a parent comp.)  

[useful_design (Alastair)]

I’d have a preference that for a subset of published input ports that represent some of the simple native types could be arranged in a window in horizontal or vertical fashion as a kind of minimum feature set for exported apps.

Those types would be

• bool
• int
• real
• text
• color
• is gradient a native type? if not make it a Vuo type
• enumerated list, not a Vuo type I don’t think would need making (think indexed list in QC)

A user choice of horizontal or vertical arrangement would be the easiest to implement but it’s conceivable more advanced arrangements as discussed above would be possible.

I’m just focusing on exported apps at this point, because some of the good ideas expressed in the last couple of days are for the interface of the Vuo app itself, and there’s overlap with exported apps UI and developer space UI but also differences to the Vuo Editor and how it may offer enhanced for UI design which might be unique to the Vuo and not easily translate to the current automatic app build routine of Vuo.

I think as a minimum for UI in Vuo Editor and development process, the same published input port UI window that gets used for the exported app could be present as real time controls for the Editor though? A lot of coding overlap with the exported app.

I prefer my published ports in one place, but I’m not speaking against any of the ideas to have them in on the editor canvas itself, it could be awesome to have that too. I’m just looking at what might be more easily achieved (I suspect) as a ‘Minimum Viable Product’ UI for exported app developers. And for the Editor UI as an after thought (and what QC had built into the Viewer Window and root level Published Ports viewer pallet from day 1 IIRC).  

[useful_design (Alastair)]

One thing I really want to make clear, I know I’ve talked about before and suspect i have lodged FR around published ports of my own. But I’ll repeat it here because I think it’s of critical importance to understand before any voting or working on this gets done.

The way published ports in Kineme’s Quartz Builder we created as input ports for the application, there was no way to dynamically change their appearance or min/max/step/significant-figure values in the case of sliders. That precludes greying out or hiding irrelevant inputs or changing slider value inputs to be relevant to state, or update the text (and greyed out/hidden status) of enumerated lists.

mimoLive has the most sophisticated example I can think of for how a macOS host application executes binding between a QC file’s published inputs and the host application. They use the file protocol interface of QC where a bunch of protocols which (amongst other mimoLive specific things) organises things like the min/max/step values for sliders and arranging groups of ports into distinct regions of the mimoLive UI which have the little triangle (think folders in macOS Finder) which can flick open more settings and be opened/closed/hidden in mimoLive.

The mimoLive Custom Layer docs explain it pretty well for those interested, it’s stood the test of time at Bonix, though they must be plotting a path away from QC at some point, the main developer of mimoLive is running QC on a virtual machine with an old macOS where QC is stable. I used to make custom layers for mimoLive/BonixTV and using protocols for binding is less than ideal but at least Vuo already has a file protocol system built in (Image Generator, Image Filter). EDIT QC had the additional feature of a protocol definition table, goes Vuo would need some kind of Composition Info dialogue to do that, or a node that loads a text file that accomplishes the same purpose.

In the example images below you can see for “Transition”: when it is a cut there’s no other input, when it’s a transition period is revealed. Crop similarly reveals and hides published input ports according to state of other inputs. (The Vuo composition runtime itself could controlling this state logic in case of exported apps or the Editor UI experience).

It also permits the binding of more sophisticated GUI controls that allow users to interact with geometry directly with mouse clicking/dragging, as in the rotation, scale and position arrangements of a logo in this interface. That’s outside the bounds of what I’m asking for here, it’s not in the set of native macOS controls, but something to think about if the UI of exported apps is enabled some another way, such as incorporating UI builder Xcode template files or something that we, as Vuo developers, can alter to suit our own purposes. That was another post I was going to make but I’ve probably worn out my welcome on this thread for now!

EDIT: Added images again for the third time. Not sure why they keep disappearing?! maybe I need to change the files names after accidentally erasing them with the “remove” button the first time thought it just removed the button appearance at bottom of post but removes the file. Reloaded then came back a day later and gone again. weird.  

[useful_design (Alastair)]

And here’s the FR i made around dynamic values for published port settings

Dynamic shared values node that can dynamically expose/hide and change published value inputs that composition/app users interact with

@TeamVuo have tabled the above FR for now until A UI for storing and editing published input port values but people can still make comments on it . My reading of a comment on the other post by @smokris is that it’s also a FR to have published ports exposed in exported apps in a preferences type window (think Kineme’s Quartz Builder for those who were around in the day).

There’s a lot of overlap with some of these FRs. I’ll leave in @jstrecker’s capable hands to figure out the best taxonomy for all these ideas as discrete FR. I may assist with a listing and tagging exercise at some point soon as it seems to be a hot topic.  

[useful_design (Alastair)]

comment removed  

[jersmi]

Hmm, I think being able to include ports in a UI does seem like a separate issue?

For the FR as stated above, sure, skip bells and whistles – namely window interaction, dragging UI elements around with bounding boxes, etc. Just having x/y position for elements would do. The core of the matter for this one is to be able to show UI elements in the editor as part of user created library module.

With this idea doesn’t it seem possible that a user might be able to design hideable or fold down areas for a UI, perhaps even a window resize (that the user would know would be cropped if not properly arranged in the parent UI)? Though not sure about a min/max slider UI, seems straightforward, maybe that would be a separate FR to add that to the current slider?

@jmcc’s contribution above is a UI element in the viewer after all, no? (Just have to click to open and change the value then it gets closed when not in use…)  

[useful_design (Alastair)]

everything I posted today was clearly outside of this FR. I’ll find somewhere else to post it!