Managing a robot arm with leap motion

Last year, I put together a roboruk , wrote a “driver” for linux for it, and taught me how to hold different objects. And somehow I forgot about her. But yesterday they delivered leap motion to me and came up with a new idea - to learn how to control a robot arm using the movements of its hands.

What you need to manage

Roboruki have:

• base - can rotate clockwise and counterclockwise;
• shoulder - move up and down;
• elbow - move up and down;
• wrist - move up and down;
• claws - open and close.

For all this, the “driver” provides a simple api:

from roboarm import Arm
arm = Arm()
arm.base.rotate_clock(3)
arm.elbow.up(1)
arm.grips.open(2)

First implementation

Leap motion can detect hands , fingers and their direction.
Therefore, I decided to attach movements to the fingers:

Part of Roboruki	My hand	Number of fingers	Axis
Base	Right	4 or 5	X
Shoulder	Left	4 or 5	Y
Cubit	Right	2 or 3	Y
Wrist	Any	1	Y
Claws	Left	2 or 3	X

Nothing came of it. Leap motion correctly determined when there were 1-2 fingers on the hand, but when more - a random value from 1 to 5.

Second implementation

Leap motion gives the coordinates of the hands in certain boundaries, I have about -200 to 200 in X and from 0 to 400 in Y. I divided this area into ten equal parts and attached actions to them:

Clockwork	Shoulder up	Elbow up	Wrist up	Open claws
Anti-clockwork base	Shoulder down	Elbow down	Wrist down	Close claws

This implementation turned out to be working =)

What happened

Video with the result, where the robot is trying to raise the box:



Video where hand movements are better seen:

References

Roboruk owi robotic arm.
Leap motion.
"Driver".
The source code of the implementation.