User interface (UI) node set — buttons, sliders, text/number boxes, menus

Paul,

We are interested in knowing how you would use a text input UI node. As a workaround, perhaps a TouchOSC implementation might work (as in Simple TouchOSC Typewriter). Once we know what you need, we could give you an estimate on commissioning it.

For my current use case I have realtime graphics being created by Vuo on a secondary screen that is projected, this is what the audience sees. I wanted the operator (teacher) to be able to feed text questions into the realtime graphics.

I too would like to see a text/number input UI node. This would be very helpful for the ‘lighting visualizers’ I create in VUO. These would not need to be complex, just a graphic box that accepts either text or numbers when highlighted, and fires when ‘enter’ is hit.

Not a super high priority, but very helpful for interactive compositions.

William

Paul, William,

Perhaps you could use Display Console Window as a workaround for now. As the node documentation says, “Typed Line — When a linebreak is typed in the window, this port fires an event with the line just completed.” So you can enter a line of text in the Console Window and output it.

@jmcc The user interface IS the product sometimes, displaying a console window is not a workaround. It’s clearly a sought after and useful thing, presumably not that difficult(?). It’s a six year old feature request now that was chosen to be implemented 4 years ago. Should we give up hope?

1 Like

I agree with @bLackburst. Console window is not a viable workaround from real-time operation of anything.

Years ago I spend countless hours making a multi-bank switcher in QC for QC that used spooky patches along with some pretty simple JS to sample data at any point of the composition and record that value (bool, int, real, string, array or complex object) and then save that to a patch like an old analogue synthesiser (think Roland synths like the Juno 60). Then at any time you could restore any of these patches and those data values, including objects would replace the value on the noodle coming out of it. If you changed the value coming into the patchbank node then it would change the value again just like moving a slider on an old analogue synth would edit the current patch. The JS code for the controller app was hundreds of lines long and became too much of a headache for me when i realised I needed to refactor it using an improved coding style to get the bugs out (I was teaching myself JS along the way!)

Anyhow, I put in a feature request to put the framework for that concept into Vuo. Basically the ability to sniff state and write state. Either in dedicated Vuo nodes or as a new API interface for all new Vuo nodes.

But another work-around is another thing I requested. A websocket node. Websockets allow two way binding between web pages and other programming tools. there was a 3rd part websocket tool i used with QC that allowed for reasonably sophisticated UIs to be built competitively effortlessly with HTML & CSS and a tiny bit of JS just to implement the websocket binding for each button, slider, enumerated list etc.

EDIT> FR links added  

Lost me, but it feels the spirit was there somewhere.

I’ll say it more simply, @bLackburst!

Team Vuo clearly has a capacity issue in delivering FRs and updates for what many consider to be core-need Vuo features. Therefore we need to make it as easy as possible for Team Vuo in terms of development hours and integration complexity with the existing Vuo codebase and code roadmap. Not to mention bug fixing down the track.

UI tools are inherently behind the times because expectations are always moving on to the next big thing. In my experience using QC, it takes a long time to smooth all the rough edges and corners making generalised tools that work for any window size and play nice with resizable windows etc etc.

In my view deploying the websocket API offers the most bang for buck for @teamVuo’s effort to provide a workable UI solution. All they need to do is build a bridge using the tried and true websocket framework. All the UI tools are already there in internet-land. Anybody who can teach themselves VUO can learn to make a few buttons, radio buttons, sliders and drop down lists and arrange them on an HTML/CSS page with other text and images. And i can guarantee you whatever UI tools Team Vuo can release in the next year, some people will find them lacking compared to what they’re used to in other apps.

One down side is that it will be outside of a compiled Vuo app that we might want to distribute to other people, but a browser based UI will still be able to talk to the Vuo app via websocket and we might even be able bundle the HTML/CSS/JS files and to script the webpage to launch?

If Team Vuo think that they can do a set of custom UI features with less time invested in dev and polishing than it takes to make a websocket API implementation inside Vuo then go for it but I’m, extremely doubtful. Remember that a Vuo UI feature set will require all the logic processing nodes to do the view/controller/data back-end of the screen elements also. To me, websocket is the best option for step one down this path.

Also i found this thing Mozilla developed called HumbleNet: A cross-platform networking library that works in the browser. It consists of a C wrapper around WebSockets and WebRTC that abstracts away cross-browser differences, facilitating the creation of multi-user networking functionality for games and other apps. It might be a shortcut for Team Vuo to get websockets and the WebRTC low-latency protocol happening ASAP, although I’m not sure how much it allows for websocket exclusive use, seems ot be used just for the authorisation phase of the connection.  

Not to be an donkey, but text input has been available for some time. See my post here to get a primer.

You should also search for “vuo.ui”, “vuo.mouse” and “receive” in the sidebar. Most of what’s asked for in this thread is available through the nodes listed there. Some will have to be built more from scratch than others - but I had working sliders from those nodes in 1.1 or 1.2 iirc.

2 Likes

If you guys want to spend time coding websocket guis and Rube Goldberg typing machines knock yourselves out. Vuo needs a text field gui element.

I for one don’t believe a web socket GUI solution would work well for me. I currently use the existing GUI buttons along with live video feeds and live graphics in a couple of my projects with almost 0 latency which is important to how I use VUO. I also have a number of ideas for future projects and applications that will require a built-in GUI as part of the exported app and so really want these as non-websocket controls.

So I am very interested in these new GUI features that will eventually be coming out and believe they will be worth waiting for, I would also be really disappointed in a websocket only implementation. I’m not saying websockets can’t be useful but I don’t want to see it as the only solution to this feature request.  

1 Like

@MartinusMagneson! This is awesome! Thanks for providing not only a working example of text input, but also a tutorial on how you built it.

Until there’s a textfield node built in to Vuo, people can reuse your composition in their own compositions. At the end of your tutorial, I like how you explain how to turn it into a subcomposition for easier reuse.

@bLackburst, please keep in mind the community agreement, specifically about generalized negative statement. Ideas about alternative ways to accomplish a task in Vuo are always welcome on this forum. Criticisms are also welcome, but they need to focus on specific points that could be improved.

To all who’ve asked about a textfield node — We’re still planning to add one. We actually started work on a textfield node while developing the Vuo 2.0 release. I would not have guessed this, but it is surprisingly, fiendishly difficult to make a textfield with customizable appearance that behaves like a proper macOS textfield. After a lot of hours spent, we decided that it wasn’t worth holding up the release for this, and we would come back to it as soon as we could. Right now our focus is on getting Vuo ready for the macOS 11 release and Apple’s ARM processors. Once those are under control, we plan to pick up where we left off with the textfield node (which, based on the comments on this issue, is resoundingly the most wanted of the remaining potential UI nodes).

@bLackburst Well, thank you for the insult I guess. Mr Goldberg did some pretty nifty drawings. When that is said, I hardly think 7 nodes qualifies for a Golberg-esque typing machine, especially when it comes to text input. I do understand that there are different views on how node based systems should work, and that we probably are on different sides of the table there though.

Instead of sitting around waiting for a feature for years, I would advocate trying out what’s possible with the current nodes, finding roadblocks there, and suggest improvements to the existing nodes to achieve results. This could for instance be a “text buffer” output in the “Receive Keyboard Typing” node, which would reduce the amount of required nodes to 2 for this application - and would probably be a lot less work-intensive task than making a fully themed UI textbox.

If you read my post I also provided a downloadable composition along with an explanation of what’s what in it, and how to scrap useless parts and save it as a user library node. With a MIT license. This means you can store it, have one node do what you want, and claim it as your own without having to neither pay for it or acknowledge my existence in any way. No Goldberging needed on your part, only a tiny bit of effort!

Also applicable XKCD (which pretty much sums up all of Team Vuo’s comments in this thread):

XKCD: Tasks

(Also, also, thank you for the kind words @jstrecker! Nice to be of help!)

3 Likes

Thanks Martinus for the compositions and the reversed queue node !

I got a bit lost between the different subcombs and how to use them, but thanks to you, I was able to tweak your original composition to match my needs, use some buttons as some text fields (clicking a text field resets the field however) (see joined composition below).

One thing I wonder though reading Jaymie’s comment about the work needed for an UI text input, is I wonder if under the hood some changes could not be made to Vuo to make these kind of things easier to implement.
Is it because of the push only system it’s so hard to code ?
I’m no code expert at all, but UI text fiels are basically everywhere, internet, apps etc, do under the hood they all have major code ?
I was thinking about this also because you can’t create lists of buttons since they need the window port, and this isn’t supported within process list loops.
I wonder if it will be possible with the “Allow single ports to accept lists” feature request to create a list of buttons on the fly, with each of them be clickable and editable.  

UI Text Fields.zip (60.1 KB)

Would another workaround until implementation of the text UI field also be using some applescript to prompt a text field and retrieve it into Vuo ?

Joined is some composition with some code I grabbed from internet to test.
Works for me, but not sure how that would work for distribution on recent macOS versions due to security restrictions ?

PS : for now the code retrieves an error when canceled.
PS2 : It’s also possible to use Applescript to retrieve files and folder paths, for example to load an image, but you have to modify the Execute Shell Command node to allow the access for the user folder, by default it’s forbidden for security reasons.


 

Applescript Prompt.vuo (4.89 KB)

3 Likes

@bLackburst and @Chris when a websocket plugin was made available for QC i had an low-latency interface with a self hosted (same Mac) GUI that was used to drive giant video screens in front of audiences of 100,000+ people. I built the GUI i wanted with drop down lists etc and push button preview and live states in a few hours. If you want to be dismissive of such solutions, bLackburst then i think that’s more on you than anything else. I’m only suggesting that this is a low-effort implementation for Team Vuo, given that they seem to be running into significant hurdles on the UI element nodes.

@Chris I can understand why you want stock VUO tools to do it. It’s a shame it’s so hard to make them using Vuo itself (I haven’t ever tried but I’d have thought someone would have by now b/c they are so essential for production tools if you aren’t using VDMX or similar). Many people made such things for QC over the journey. I made an advanced GUI inside QC using JS and it had 30 or so buttons and several modes of operation, flashing LED animations when moving between modes and two digital readouts for numbers (it wasn’t any improvement on latency than Websocket though).

There’s a VUO node for loading an HTML page that could be embedded in your app. VUO may even get an HTML server node one day since Kineme built one for QC, in which case it could be all bundled inside the app.  

Alastair, so far I’ve been able to do a lot with the existing buttons and slider nodes, I’ve never used VDMX or anything else with VUO. Admittedly I’m really looking forward to a native textfield node (Magneson thanks for MM.ListTools, and the great example of how to use it). I would also love to see a dropdown menu node from a list become a thing as well.

As far as the difficulty in building GUI using the existing button nodes I haven’t had any real issues and have had great working GUIs that scale well in a few hours. I built the GUI for this project in about 4-5hrs it has 36 buttons; 42 now, and 4 shortcut keys to switch project modes so the same buttons call up different graphics. The backend took much longer but the GUI was pretty easy. I think it’s all in the planning, having a good idea of what you want before you start. I use to design webpages early in my career and still work with HTML from time-to-time, so I find the idea of using websockets interesting but I don’t want to see it as the only implementation of the GUI features being considered; and that latency concern I have for the live video and graphics previews and program is a big one for me.

1 Like

One thing I wonder though reading Jaymie’s comment about the work needed for an UI text input, is I wonder if under the hood some changes could not be made to Vuo to make these kind of things easier to implement.
Is it because of the push only system it’s so hard to code ?
I’m no code expert at all, but UI text fiels are basically everywhere, internet, apps etc, do under the hood they all have major code ?

If you’re building, say, an iPhone app or a webpage, it’s easy to pop in a textfield because the app or webpage is built on top of a platform that provides the widget. For iPhone, there’s a UITextField class within the UIKit framework that you just pop into your code. For webpages, the HTML standard defines an <input type="text"> element that all browsers implement.

For Vuo, we have to build the underlying platform that provides a textfield widget that’s cross-platform-compatible and has a customizable appearance. Luckily, we don’t have to start from scratch. We’re building on top of the open-source stb_textfield. Even so, there’s a ton of tweaking needed for the appearance and behavior. Textfields have a lot of little behaviors that most people don’t really think about but would miss if they weren’t there, or if they behaved differently from typical macOS textfields (cursor, highlight, copy-paste, etc.).

Our next step on the textfield node will be to reassess where we left off and come up with a minimum set of changes, with the goal of not letting the perfect get in the way of the good. Then we can release a textfield node that is hopefully decent, and over time tweak the behaviors that the community considers most important based on bug reports.

2 Likes

Hi Jaymie

100% about lots of little behaviours. Even two browsers on macOS implement text fields differently. Eg Safari has macOS text substitution (something I miss a lot making comments on science related websites when I’m in Firefox), macOS system spelling correction engine and user dictionary, word/phrase meaning dictionary “lookup” (helpfully always top item in the contextual menu when you right-click), user programable “Services” and more. While Firefox implements it’s own spell-checker, it’s own user dictionary, and none of those other things I mentioned.

I’m wondering, given that Vuo already uses the ‘Qt’ cross platform UI framework, wouldn’t it be useful in this context? It’s made for this kind of situation and extremely versatile and powerful from what I can tell.

Or would there be too many licensing issues? It’s used in all manner of platforms including embedded systems even, so if the popular FR for Vuo on Rassberry Pi ever wins selection to implement then Qt will go the distance, no problems I’d have thought.

Qt do have a free community licence for non-commercial app builds which might apply to people making app for self use. And if selling their apps then the pro version of Vuo could implement the royalties/licensing fees perhaps?

ps
I just spent five minutes trying to find the Vuo code base dependencies/licence. I know there’s a page on this site somewhere where they are all listed but I couldn’t find it even doing google searches of the Vuo.org site and naming various dependencies I thought I could remember the names of. I ended up on GitHub and confirmed you are using Qt already in seconds on GitHub.

Can I ask that the 3rd party framework/licence listing page be moved into the FAQ because this is not the first time I haven’t been able to find it on this site?  

In Vuo 2.2.0, we added Make Text Field, Make Number Field, Make Text Field Theme (Rounded), Display "Open" Window, and Display "Save" Window.

Known limitations of Make Text Field and Make Number Field:

  • right-to-left languages (such as Arabic) aren’t currently supported
  • 4-byte glyphs (e.g., :100:) aren’t currently supported; they show up as a question mark in a box
  • keyboard shortcut to show the character picker (control-command-space) isn’t currently supported
  • System Preferences > Keyboard > Dictation isn’t currently supported
  • System Preferences > Accessibility > VoiceOver isn’t currently supported
  • tabbing from one field to the next isn’t currently supported
  • context menu isn’t currently supported (though the shortcut keys for common operations (cut/copy/paste/undo/redo) work)
  • cursor doesn’t flash
  • spell/grammar-checking isn’t currently supported
  • phrase-substitutions, smart quotes, and auto-capitalization (System Preferences > Keyboard > Text) aren’t currently supported
  • should scroll when the text width exceeds the widget width, rather than expanding the widget
  • double-click-drag should select by word, but currently it selects by character

We’re going to call this feature request “released”. Lesson learned: it’s easier to track feature requests that are smaller and more focused :) If there’s another UI widget you’d like to see, or limitation of a UI node that you’d like us to prioritize, please create a separate feature request.

1 Like