Skeletron: Human POSE Prediction in 3d
Press
The Next Web: "Programmers use TensorFlow AI to turn any webcam into Microsoft Kinect"
TechRadar: "Two AI developers have turned a simple webcam into a system that behaves just like a Microsoft Kinect using machine learning"
About
Developed with the fabulous Or Fleisher, Skeletron is a system that predicts joints and human skeleton position in 3d from real-time video taken by any (cheap) RGB camera, such as a webcam. The system sends the data about the position of the human body to Unity, a 3D game development engine, to allow engineers, artists, and creative technologists to use it to develop digital experiences.
This demo was developed thanks to the NYU ITP Xstory grant.
ingredients
Machine Learning Model - http://gvv.mpi-inf.mpg.de/projects/VNect/
TensorFlow port - https://github.com/timctho/VNect-tensorflow
Model prediction - Google TensorFlow
Data representation in 3D - https://unity3d.com/