MDGPS with Baxter

[Cristian Beltran]

Dear Chelsea and Sergey,

I have been trying to use MDGPS algorithm with the robot Baxter using torque control. I implemented the functionality for an agent in the GPS API.

I am testing the task of moving the right arm to a given position (based on the end effector position):

State: {Joints angle, joints velocities, end effector position, end effector velocity}

Action: joints torque

Only one condition, initial and target position are fixed.

Cost function: forward kinematics cost function

However, I have been unable to make the robot learn a good policy in simulation, the robot learns to move towards the target, but the error at the end of the episode is still considerably large. While the learning curve seems to improve after some iterations (first 3 or 5 iterations), after that, the learning seems to stop, it does not improve even after 10 o 20 more iterations. I have been playing with the hyperparameters, which leads to slightly better results but still not good enough. I have also tried using velocity control but the results are similar.

I would appreciate if you could share some heuristics on how to tweak the hyperparameters to improve the performance and avoid local optimum. 

Best Regards,

Cristian Beltran