Demo teleoperation mediapipe

[charlesmadjeri]

Description

Using mediapipe and lucy, control thais robot hand moving your own hand

Delivery

• Install and discover mediapipe, skeleton models especially
• Get and parse mediapipe fingers movement positions
• Use TypeScript to integrate in our web demo this tool
• Send converted positions to rosbridge/ros topics

Technical context

Demo online:
https://pjbelo.github.io/mediapipe-js-demos/hands.html
Demo with code:
https://codepen.io/mediapipe-preview/pen/gOKBGPN

Main github:
https://github.com/google-ai-edge/mediapipe

Installation for web:
https://ai.google.dev/edge/mediapipe/solutions/setup_web
https://ai.google.dev/edge/mediapipe/solutions/vision/hand_landmarker/web_js

Hand documentation:
https://ai.google.dev/edge/mediapipe/solutions/vision/hand_landmarker

[Arcod7]

Working react mediapipe tracking with holistic (every model, can be very useful for later)
https://medium.com/@mamikonyanmichael/what-is-media-pipe-and-how-to-use-it-in-react-53ff418e5a68