Multi-camera Mocap?

I'm seeing more and more of this. Sites like easyMocap on Github demonstrate how to accomplish this, but there is ALOT of setup and configuration to do on the client side (and it's ALL client side). If I could upload synchronized videos from 2 - 3 different cameras it should go along way to helping your algorithms more accurate understand the motion in the video.

Any thoughts on this?

6replies Oldest first
  • Oldest first
  • Newest first
  • Active threads
  • Popular
  • I just opened the Forum to write the exact same post. I tried this service and it looks amazing at least when looking at the animation from the front, when looking at the animation from the side it's not so accurate.

    I understand that add a multi camera support is not that easy and you will need to re-create your ML model from scratch, but I've the feeling that if the AI has at least the side video too, it could produce an amazing result.

    Like 1
  • I also think adding the option to include 2 or more cameras would be amazing and put DeepMotion in a really good spot to compete with others on the market!  Does anyone from DeepMotion read or respond to these posts?

    Like
    • Ryan Klotz Hi Ryan, yes we do! What makes us different from other services is that we do not require any hardware, cameras, or suits. Our AI is trained on a single video input and our goal is to generate the best animation possible from a single video. Requiring multiple inputs changes the scope of our service dramatically, decreases accessibility, and moves toward a studio setup. Studio setups with multiple angles are not something we are pursuing at this time. 

      Like 1
    • DeepMotion_Admin Thanks for the reply!  And thanks for being honest that it's not on your roadmap.  I get it that it adds complexity to the setup and you wouldn't want to require it, but I'm thinking of it as an optional setting to increase accuracy if you have limbs being occluded behind the body, etc.  What if I recorded from two camera angles and I synced them before uploading, then ran them each separately.  Could there be a "merge" option to take the most accurate data from each synthesis and throw out the bad data?

      Like
    • Ryan Klotz It's unfortunately not as easy as just allowing another upload. As mentioned, our AI is trained with large data sets on specific inputs for optimized tracking from a single video. This is why we recommend certain capture guidelines for generating the best animation possible. The more inputs added, the more datasets we need to build and models we need to train for added angles. This would significantly change the scope of our service away from what we have built. We are focused on refining single video motion capture while developing features like Face Tracking, Hand Tracking, and most recently our Rotoscope Joint Editor. We are working on Multi-Person Tracking as we speak with many other items in the pipeline!

      Like
    • DeepMotion_Admin thanks for explaining.  It makes sense that your focus is single video capture.

      Like
Like3 Follow
  • 3 Likes
  • 3 wk agoLast active
  • 6Replies
  • 65Views
  • 5 Following