Happy to see new versions popping up! I hope VUO still has a bright future ahead: )
Would it even be possible to integrate ChatGPT to perhaps write a prompt of what to create? Or at least get a head start on a project…
When you say “write a prompt”, are you thinking it would generate a suggestion on what to make (a phrase or sentence of text, similar to a creative writing prompt)? Or were you thinking something more involved?
I was thinking more in terms of the LLM to have insight and knowledge of witch nodes to incorporate, and therefor also set them up so that you would get a great starting point for your project, or even better, a final solution. Kind of how https://makereal.tldraw.com creates mockups of apps, from images and text → working prototype
: )
Not sure if you’ve given the current ChatGPT 3.5 a try but it already has a fair bit of knowledge regarding Vuo and its nodes (up to Jan 2022). It’s able to recommend possible node suggestions to help solve a proposed problem. I only played with it a couple times, asking some fairly basic questions to test it out. I found its answers were fairly accurate and if you go into it understanding that it might hallucinate a bit it could still be a useful in working through a problem.
It was more accurate than what it suggested for a simple Blender script to make a linked chain. That one took 10 tries and me finally suggesting a specific way to do it before it was finally able to get close to what I asked. Haha! It did however do a nice job of reading a badly written and slightly broken script; it cleaned it up, simplified it, and fixed the broken part. That I was actually quite impressed by, though I did have to tell it what the script was supposed to be doing.
Have also used GPT to help plot coordinates in vuo. Would be cool to see some sort of LLM nodes. Local LLM or API.
@cwilms-loyalist and @krezrock, interesting to hear of your experiences asking ChatGPT for help with Vuo. If either of you happen to have the time & energy to make a longer writeup or video, I’m guessing people would be quite interested to see some working (and not-working) examples.
Coming back to the feature request — if it gets a significant number of votes, we can check the state of the technology at that point and see how difficult it would be. With ChatGPT 3.5, we were able to obtain a set of steps that came close to solving a small and specific problem (“Create a Vuo composition that increases the contrast of live video”). But there’s a gap between the ChatGPT output and a working composition, which includes translating the steps from English text to an actual Vuo composition and working around mistakes such as the wrong node/port names that ChatGPT emitted during our test. Those are things a human can do well (we’re quite fault-tolerant) but would be hard to automate in terms of writing an integration between Vuo and ChatGPT (if ChatGPT’s instructions aren’t precisely correct in all details, it may not produce a meaningful or even syntactically correct composition). ChatGPT or its successors would have to fill more of the gap for this to be feasible. There’s also the question of how well ChatGPT would do with more complex and interesting problems.
As another avenue toward providing starting points for projects, I would also mention that the example compositions are meant to help with this — so if you have any ideas for examples that you think would be commonly useful, please feel free to create a feature request or share a composition you’d like to see incorporated into Vuo.
Hi Jaymie,
My use of Chat GPT is very limited. My use involved asking it to help solve some integration issues with Ableton and Touch Designer. And plotting some grid points for me in Vuo coordinates. It’s really hit or miss, and it’s questionable if it’s helpful or not.
Question. Is Vuo in a good position with its front end to implement Stable Diffusion? Similar to Comfy UI. If it is. That may be a good FR.