Have you seen OpenPose?

Can you do anything with it for Vuo?

Yeah, looks like it provides nice human feature detection.

The commercial license is $25,000/year, which is a wee bit over Team Vuo’s budget ;) However, if someone else were to implement some OpenPose nodes for Vuo non-commerically, they could use the OpenPose library for free.

The documentation for the OpenPose Demo says that it can output JSON. So if you were to record a video and write a JSON file from it, you could then play back the video and the JSON data in Vuo.

Or if someone has created an app using OpenPose that outputs OSC messages, Vuo could use the tracking data in real time.

if someone else were to implement some OpenPose nodes for Vuo non-commerically, they could use the OpenPose library for free.

What does this mean exactly? Non-commercial implementations could be used for commercial end-user applications, or not? It just can’t be Kosada that devs it? Anyone using it to make money has to pay the $25,000? I don’t get it.  

Those are good questions. I’m sure that making it built-in to Vuo would count as commercial since Vuo is a commercial product, but I’m unsure of all the implications of a node developed by a third party. If you happen to find anything in the OpenPose license agreement or other documentation that clears this up, please follow up here.

Well I guess this line from the ‘license’ says what we don’t want to read:
‘DERIVATIVES: You may create derivatives of or make modifications to the Software, however, You agree that all and any such derivatives and modifications will be owned by Licensor and become a part of the Software licensed to You under this Agreement. You may only use such derivatives and modifications for your own noncommercial internal research purposes, and you may not otherwise use, distribute or copy such derivatives and modifications in violation of this Agreement.’

But I’m contacting one of the dev’s to clarify anyhoo: https://www.gineshidalgo.com/  

1 Like

I received a reply within minutes which was a nice surprise, but I’m still not 100% clear on the answer or that they were totally clear on what I was asking:

‘A point set generated from OpenPose software wouldn’t be considered a derivative. (A derivative would be modifications to the code itself, not products of the code). So, that point set could be distributed.
In order to generate that code set for commercial purposes though, a commercial license would be required.’

I suggested that they comment here if they would care to do so.

@Scratchpole, thanks for checking.

I had some further correspondence with Ginés:
'So far OpenPose only outputs 2D keypoint locations. With 3-D cameras yes, it would be able to output 3-D keypoints (there is a 3-D demo inside OpenPose), but we still don’t provide joint angles (not sure if it’s a requirement for you).

About the output, I understand it should be easy to modify the output to write OSC files rather than JSON files if your developers have knowledge on how to write those kind of files in C++ (I am not familiar how hard this OSC format is). If they add that functionality for their own OpenPose version for Vuo, they can definitely contact me again and I can try to officially join it to OpenPose so you would be able to use always the most updated version of OpenPose (not your custom version).

About licensing I am just an engineer (so not an expert at all), so not sure what your idea of its use is, but if it’s released for free I think it should be fine (but check the license and previous answer from David to be sure).’

OpenPose can now be used as part of OpenCV: https://opencv.org/opencv-4-0-0.html

Thanks for the tip. We looked into OpenPose in OpenCV to see if it would be compatible with Vuo. Although the OpenPose model dataset is still noncommercial, there’s an alternative dataset with a license that should work for us. They say that macOS support is coming sometime in 2019. With macOS support, we could consider OpenCV for the built-in skeletal tracking feature request.

Here’s the next game changer for skeleton tracking: http://gvv.mpi-inf.mpg.de/projects/XNect/

2 Likes

Just found this PoseOSC app: https://github.com/LingDong-/PoseOSC
Should do the job, just need to get it working/parsing the OSC…

1 Like

From macOS 11 there is system support for pose estimation. Wondering if this might be the better option?
https://developer.apple.com/documentation/vision/detecting_human_body_poses_in_images

2 Likes

Nice find @keithlang thanks

1 Like

Yeah starting to see several options online for pose.
Nice find to externally root into Vuo Joe.

Had been asking also about TensorFlow here : CoreML integration Commment