Knob for Faster Expressive Results

Coincidentally…I was just thinking last night about the way that nodes in Max/MSP can look like sliders, knobs, etc, in the editor, and be hooked to “whatever”.

There is also a mode to the editor where all of the non GUI nodes get hidden, and the end user only sees the GUI node objects. This allows one to code GUI in the editor and then wrap the graph up in app form, all in the same environment.

I had also recently mentioned that I thought custom plugin/node views would be a very good idea for VUO, and this thought about the knob seems very much related.  

Another way that GUI in VUO could work is for users to publish ports they want to hook to GUI.

Another app (a VUO interface builder?), or maybe a built in feature, could cycle through the published ports and programmatically spawn GUI for published ports.

It could be that the user adds a specific key to published name or something to control whether a knob, slider, etc, is created…or they could be presented with a set of options that they then choose from via an interface.

If you look at the sample code for Apple’s QCTV project, and I think QC Performer (or some name like that), you will see a very simple concept along these lines. The GUI is programmatically spawned by the QCTV host app, according to published port types.

That said, I think there is a strong use case for exposing a way to see something like a knob or slider (and other GUI objects) as a node itself, within the VUO editor.  

1 Like

@jmcc, your (very nice) example .c node opens up in a text editor when I double click on it from inside the Vuo editor (in my case BB Edit). No problem, really.

Yeah, George, I agree, I think there is a strong use case for seeing GUI objects. Visual feedback is a powerful tool. Like being able to see (and edit, wow!) an audio waveform on a computer – imagine that before and after!

Since we’re using examples of other software – there is a modular audio app called Audulus (currently in beta for version 4) that uses an effective approach with knobs.

Audulus was originally directed towards iOS for iPad, then to macOS + AU plugin version. In Audulus users create modules (which would be subcomps in Vuo) for sound stuff then “expose” knobs and other elements from inside the module. Then in the editor (root view) key command an “edit UI” mode to drag knobs around. (Can also cable directly to the knobs to modulate, which makes a lot of sense for modular audio world.)

I like that the dev for Audulus is committed to strong visuals all for UX – he just introduced a shader node for GLSL, and already has for the current beta other clever ways for supporting custom visuals, SVG and PNG support, using demux to animate PNGs specifically for knob UI’s, etc.

Audulus screenshot, users create/customize modules:

 

2 Likes

So yeah, I do think think the heart of this feature request is for a visual GUI element in the editor that could:

  1. Have the same behavior as Fire When Value Edited.
  2. Be viewable in the editor.
  3. Be viewable in a subcomp UI (with the ability to position them).
  4. Export as part of a GUI menu for a Mac App.  
1 Like

What about this FR spinning off into one for a “UI protocol”?

Features:

  • UI elements could be arranged (and resized?) in the comp loader by dragging UI element bounding boxes with the mouse. (Perhaps isolating Viewer mouse interaction to a UI protocol could relieve some “what about this scenario” confusion?)
  • When used in a parent comp the UI comp GUI would show in the editor window. (Maybe a “show UI” true/false toggle in the protocol ports?)
  • The UI comp could be saved to modules as a library item.
  • Protocol output would be the combined UI layers that could then be used as part of a larger comp view as an overlay or sidebar, or not. (Generating output ports for UI values would obviously be part of it, to connect to other parameters in a parent comp.)  

I’d have a preference that for a subset of published input ports that represent some of the simple native types could be arranged in a window in horizontal or vertical fashion as a kind of minimum feature set for exported apps.

Those types would be

  • bool
  • int
  • real
  • text
  • color
  • is gradient a native type? if not make it a Vuo type
  • enumerated list, not a Vuo type I don’t think would need making (think indexed list in QC)

A user choice of horizontal or vertical arrangement would be the easiest to implement but it’s conceivable more advanced arrangements as discussed above would be possible.

I’m just focusing on exported apps at this point, because some of the good ideas expressed in the last couple of days are for the interface of the Vuo app itself, and there’s overlap with exported apps UI and developer space UI but also differences to the Vuo Editor and how it may offer enhanced for UI design which might be unique to the Vuo and not easily translate to the current automatic app build routine of Vuo.

I think as a minimum for UI in Vuo Editor and development process, the same published input port UI window that gets used for the exported app could be present as real time controls for the Editor though? A lot of coding overlap with the exported app.

I prefer my published ports in one place, but I’m not speaking against any of the ideas to have them in on the editor canvas itself, it could be awesome to have that too. I’m just looking at what might be more easily achieved (I suspect) as a ‘Minimum Viable Product’ UI for exported app developers. And for the Editor UI as an after thought (and what QC had built into the Viewer Window and root level Published Ports viewer pallet from day 1 IIRC).  

1 Like

One thing I really want to make clear, I know I’ve talked about before and suspect i have lodged FR around published ports of my own. But I’ll repeat it here because I think it’s of critical importance to understand before any voting or working on this gets done.

The way published ports in Kineme’s Quartz Builder we created as input ports for the application, there was no way to dynamically change their appearance or min/max/step/significant-figure values in the case of sliders. That precludes greying out or hiding irrelevant inputs or changing slider value inputs to be relevant to state, or update the text (and greyed out/hidden status) of enumerated lists.

mimoLive has the most sophisticated example I can think of for how a macOS host application executes binding between a QC file’s published inputs and the host application. They use the file protocol interface of QC where a bunch of protocols which (amongst other mimoLive specific things) organises things like the min/max/step values for sliders and arranging groups of ports into distinct regions of the mimoLive UI which have the little triangle (think folders in macOS Finder) which can flick open more settings and be opened/closed/hidden in mimoLive.

The mimoLive Custom Layer docs explain it pretty well for those interested, it’s stood the test of time at Bonix, though they must be plotting a path away from QC at some point, the main developer of mimoLive is running QC on a virtual machine with an old macOS where QC is stable. I used to make custom layers for mimoLive/BonixTV and using protocols for binding is less than ideal but at least Vuo already has a file protocol system built in (Image Generator, Image Filter). EDIT QC had the additional feature of a protocol definition table, goes Vuo would need some kind of Composition Info dialogue to do that, or a node that loads a text file that accomplishes the same purpose.

In the example images below you can see for “Transition”: when it is a cut there’s no other input, when it’s a transition period is revealed. Crop similarly reveals and hides published input ports according to state of other inputs. (The Vuo composition runtime itself could controlling this state logic in case of exported apps or the Editor UI experience).

Screen Shot 2021-10-13 at 2.51.36 pm.png Screen Shot 2021-10-13 at 2.51.41 pm.png

It also permits the binding of more sophisticated GUI controls that allow users to interact with geometry directly with mouse clicking/dragging, as in the rotation, scale and position arrangements of a logo in this interface. That’s outside the bounds of what I’m asking for here, it’s not in the set of native macOS controls, but something to think about if the UI of exported apps is enabled some another way, such as incorporating UI builder Xcode template files or something that we, as Vuo developers, can alter to suit our own purposes. That was another post I was going to make but I’ve probably worn out my welcome on this thread for now!

EDIT: Added images again for the third time. Not sure why they keep disappearing?! maybe I need to change the files names after accidentally erasing them with the “remove” button the first time thought it just removed the button appearance at bottom of post but removes the file. Reloaded then came back a day later and gone again. weird.  

1 Like

And here’s the FR i made around dynamic values for published port settings

Dynamic shared values node that can dynamically expose/hide and change published value inputs that composition/app users interact with

@TeamVuo have tabled the above FR for now until A UI for storing and editing published input port values but people can still make comments on it :slightly_smiling_face:. My reading of a comment on the other post by @smokris is that it’s also a FR to have published ports exposed in exported apps in a preferences type window (think Kineme’s Quartz Builder for those who were around in the day).

There’s a lot of overlap with some of these FRs. I’ll leave in @jstrecker’s capable hands to figure out the best taxonomy for all these ideas as discrete FR. I may assist with a listing and tagging exercise at some point soon as it seems to be a hot topic.  

1 Like

comment removed  

Hmm, I think being able to include ports in a UI does seem like a separate issue?

For the FR as stated above, sure, skip bells and whistles – namely window interaction, dragging UI elements around with bounding boxes, etc. Just having x/y position for elements would do. The core of the matter for this one is to be able to show UI elements in the editor as part of user created library module.

With this idea doesn’t it seem possible that a user might be able to design hideable or fold down areas for a UI, perhaps even a window resize (that the user would know would be cropped if not properly arranged in the parent UI)? Though not sure about a min/max slider UI, seems straightforward, maybe that would be a separate FR to add that to the current slider?

@jmcc’s contribution above is a UI element in the viewer after all, no? (Just have to click to open and change the value then it gets closed when not in use…)  

1 Like

everything I posted today was clearly outside of this FR. I’ll find somewhere else to post it!  

Well, what do I know, don’t you think it worth the effort to add focus? I love the conversation and trading of ideas, and abs no disrespect for your ideas, clearly born of experience! It’s all Vuo, after all… and who knows, maybe the overlapping ideas will spark something for Team Vuo.  

2 Likes

Is this about a knob on the editor surface or a knob in a rendering window? I thought this specific conversation was more about GUI widgets on the editor.

1 Like

GUI elements in the editor window is the original idea, yes. But above I asked about spinning this off into a FR for a “UI protocol”, so users could generate their own modules as library items of UI elements that would be viewable in the editor and also be available in the parent comp as GUI if so desired. Dumb idea?  

2 Likes

Personally, I think it makes plenty of sense.

I think that getting a knob and/or slider actually working in the editor as distinct nodes, but with the appearance of GUI widget interface, would just be great first steps that could probably make plenty of workflows much easier.

It feels like getting something basic like that inevitably reveals areas where performance enhancements need to happen after initial implementation. User feedback on the real world thing becomes non-hypothetical.

It’s hard to gauge the complexity of implementation of various tiers of this idea because the editor is closed source (maybe older versions of editor source got published at some point?). But it seems very useful workflow wise.

If more complex versions of the idea seemed possible to address initially, by all
means, would be great.

2 Likes

Ok, sorry I misread the original FR as being in a compiled Vuo composition runtime window.

I might try and map out all the different UI/UX FRs and sort them into sets or tag them so we can see where there are synergies between ideas and where they are completely orthogonal/unrelated.

it’s not a dumb idea @jersmi and it might not surprise you to hear that George and I (IIRC hope you agree, @George_Toledo) had discussions with Kosada about this kind of thing in the very early pre-beta maybe even pre-alpha days of Vuo. I think it took some flack from one of the core dev team, especially the idea user definable skins and stuff like a music Digital Audio Workspace effects rack (breaking with the Vuo brand look and feel i guess is the concern). At any rate Vuo was never going to have this in early versions so it’s good to visit the discussion now that Vuo is getting some maturity in it’s feature set and native data types.  

I missed the point of this FR because i didn’t watch the GIF image in the OP attachments.

is this something that happens for all Real type input ports (some have no bounds other than the system min/max) or is it something that only happens when you drop the Fire When Value Changes node onto the canvas @jersmi?

If I understand your question, the Fire When Value Changes module that @jmcc presented above is a special .c module that works in the editor like the sliders on many modules. It requires the min/max in the module to do its job. Real type input ports do not do this as designed – need to publish then set min/max to access a slider.

1 Like

Ah it actually works! (Doh)

I downloaded but only looked at the C code and the GIF… thought it was a mock up not actual thing.