Applying gps for a simple robotic arm

497 views
Skip to first unread message

Ahmed E.A Abdalla

unread,
Apr 25, 2016, 7:09:52 PM4/25/16
to gps-help
I'm currently unsure of how to start implementing gps to my custom made robot arm.
I intend to make it grasp objects. I have a xtion pro live camera.
How do I proceed with implementing gps?
Thanks in advance.

Ethan Keller

unread,
Apr 29, 2016, 3:59:30 PM4/29/16
to gps-help
I am working on applying GPS to an arm of Rethink Robotic's Baxter arm. The documentation suggests this should be easy... "The code was written to be modular, to make it easy to hook up your own robot. To do so, either use one of the existing agent interfaces (e.g. AgentROS), or write your own."... However I am finding it quite hard to divorce the code from the pr2 api in order to use the baxter topics directly.

Chelsea Finn

unread,
Apr 30, 2016, 1:49:36 AM4/30/16
to Ethan Keller, gps-help, Justin Fu
Hi Ethan,

Hooking up a Baxter will likely be a bit different from the PR2 because the effort-control interface is different. I'd recommend either adding to or modifying the existing ROS agent interface, or writing a new agent entirely. You shouldn't need to modify any of the python algorithm code though.

Generally, we'd love to have a Baxter interface in the repo! If you have any specific questions about the current PR2 interface, Zoe or I would be happy to answer them, though we have a major deadline coming up in a few weeks, so we might not be super responsive.

Justin Fu, CC'ed, has some experience with controlling a Baxter, though I'm not sure if he'll have time to assist you / answer questions.

Chelsea

--
You received this message because you are subscribed to the Google Groups "gps-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gps-help+u...@googlegroups.com.
To post to this group, send email to gps-...@googlegroups.com.
 To view this discussion on the web visit https://groups.google.com/d/msgid/gps-help/a1249f30-4159-4afe-8b50-309da5e25313%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Ethan Keller

unread,
Apr 30, 2016, 11:27:05 AM4/30/16
to cbf...@eecs.berkeley.edu, gps-help, Justin Fu
Hi Chelsea, 

So glad to hear from you! I am currently working on a school project (I'm an M. Eng. in computer science at Cornell) to apply GPS to Baxter, so I am certainly interested in working through this implementation and potentially adding it to your GPS repo. 

I'm currently trying to subclass the robotplugin class with a baxterplugin class, but I am finding it difficult to write the baxter code in CPP because it doesn't have a CPP API (meaning I have to access the topics directly, meaning no synchronization built in, and I would have to manually implement the FK solver (I think).

Alternatively, if I write the baxter plugin node as a python node, I can use the Baxter API. My proposed approach is to launch a python baxterplugin node from a baxter launch file, and that node will talk to the gps node over ROS topics. Is there more detailed documentation somewhere that would outline which topics need to get fed what information at what time? Basically the Ros Agent API?

Thanks for the help! 
-Ethan
--
774-258-0866

Sergey Levine

unread,
Apr 30, 2016, 12:29:46 PM4/30/16
to Ethan Keller, Dylan Hadfield-Menell, cbf...@eecs.berkeley.edu, gps-help, Justin Fu
+ Dylan, who also was asking about this

Are you running on board the Baxter or offboard with torque commands via ROS topic? If the latter, it's best to write your own agent in Python based off our ROS agent (which should really be called a PR2 agent...), but directly send torque commands each time step like we do in the MuJoCo agent.

If running onboard, then you would need to subclass the robot plugin class and figure out the Rethink API. This is probably more complex, and I recommend the other option (just time your controller carefully in Python to make sure it runs at the right rate).

Baxter safeties will interfere with direct torque control. You could turn them off, but then you might need to take care that it doesn't bang against the hard joint limits, since it doesn't enforce soft joint limits with the safeties off.

Be sure to have the estop handy. If you have not worked with low level robot control before, probably a good idea to find someone who has to oversee the first few experiments.

Chelsea et al: we should ask Clemens at some point to see if he can share some of his agents to integrate into the main branch. He did this at least twice (once for soft hand and once for Barrett WAM).

SL

Ethan Keller

unread,
Apr 30, 2016, 5:26:13 PM4/30/16
to Sergey Levine, Dylan Hadfield-Menell, cbf...@eecs.berkeley.edu, gps-help, Justin Fu
Hello everyone and thanks for the advice!

I am wanting to run offboard with torque commands sent via ROS topics, so I plan to start looking at the MuJoCo agent later today. 

It would be great to see another example of an integrated agent as I don't have the ability to run the MuJoCo because I don't have a license. 

I'll be sure to test it out on the simulator first and write a small program to monitor the soft limits before I take it to the hardware. 

Thanks,
EK
--
774-258-0866

Marvin Zhang

unread,
May 1, 2016, 9:53:18 PM5/1/16
to Ethan Keller, Sergey Levine, Dylan Hadfield-Menell, cbf...@eecs.berkeley.edu, gps-help, Justin Fu
Perhaps you could take a look at the Box2D agent? Box2D is free and open-source, so you can download and play around with it pretty easily.

Marvin

To view this discussion on the web visit https://groups.google.com/d/msgid/gps-help/CA%2BN%2Bvwtiuc8s8s%3DWwnv9Kudt4_R-NGyyGJqTNVLBizYNt0ifEg%40mail.gmail.com.

algo...@gmail.com

unread,
May 5, 2016, 10:21:26 PM5/5/16
to gps-help
Fisrt,You can obtain a trial version by touch with this eamil .After 30 days,you can not use it .For some bugs with this software,you can only change your computer system time to fix it.
在 2016年5月2日星期一 UTC+8上午9:53:18，Marvin Zhang写道：
To unsubscribe from this group and stop receiving emails from it, send an email to gps-help+unsubscribe@googlegroups.com.

To post to this group, send email to gps-...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "gps-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gps-help+unsubscribe@googlegroups.com.

To post to this group, send email to gps-...@googlegroups.com.

Mitch Adler

unread,
May 18, 2016, 7:51:45 PM5/18/16
to gps-help, svle...@cs.washington.edu

Hey Sergey,


We are adapting gps to work with a UR5 robot in simulation based on your comments above. We are operating under the assumption that we should not have to subclass pr2plugin in c++, and simply have to load the plugin and interface with it via the python code in gps_main.


We are having an issue with the initial launch file called as part of gps_agent_pkg. We have modified it to load and run UR5 related code from the universal robots ROS package and patterned it after the gps pr2 launch files.  We have tried using both controller_manager and  pr2_controller_manager (based on pr2_only_controller.launch) to spawn a GPSPR2Plugin controller and run into issues in both cases.

  • When using pr2_controller_manager the code hangs at the function rospy.wait_for_service inside the pr2 controller interface file.

  • When using controller_manager we get an error that “gps_agent_pkg/GPSPR2Plugin” does not exist because its type does not exist


Should we be using the unmodified pr2plugin here?  Is there something else that we are missing?  


Thanks for your help!
To unsubscribe from this group and stop receiving emails from it, send an email to gps-help+unsubscribe@googlegroups.com.

To post to this group, send email to gps-...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "gps-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gps-help+unsubscribe@googlegroups.com.

Sergey Levine

unread,
May 19, 2016, 12:47:21 AM5/19/16
to Mitch Adler, gps-help
Hi Mitch, see my original response: your choice is to either write everything in Python if your arm can listen to torque commands over a ROS topic, or subclass the robot plugin class and figure out the API for your robot. If your robot is just going to listen to torque commands over a topic, there is minimal benefit to writing the controller in C vs Python. Our PR2 controller runs in the real time loop of the PR2 robot controller, which is begged at 1 KHz, hence it's in C. You should not attempt to use a PR2 controller with another robot, it uses a whole bunch of PR2-specific code. Better to just subclass RobotPlugin if you're running onboard.

Sergey

P.S. If you're using an arm that does not have mechanical gravity compensation/mitigation like the PR2 or Baxter, be sure to figure out how you'll be doing gravity compensation, otherwise you'll find that it spends a lot of time interacting with the floor.


To unsubscribe from this group and stop receiving emails from it, send an email to gps-help+u...@googlegroups.com.

To post to this group, send email to gps-...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "gps-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gps-help+u...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "gps-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gps-help+u...@googlegroups.com.

To post to this group, send email to gps-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gps-help/8a003efb-6697-43fe-9a05-58d5d9f98662%40googlegroups.com.

Mitch Adler

unread,
May 27, 2016, 7:14:30 PM5/27/16
to gps-help, svle...@cs.washington.edu, cbf...@eecs.berkeley.edu
Thanks Sergey!  That makes sense, we've got the communication between the robot and gps up and running.  

We are now encountering an error stating, "ValueError: Failed to find PD solution even for very large eta (check that dynamics and cost are reasonably well conditioned!"  The examples specify PR2_GAINS in the hyperparams.py files (eg. here). Are those gains related to this error?  If so what do they represent and how might we go about acquiring them for a UR5?  If not, what other things do we need to modify (eg guess_dynamics)?

Nice work on the software btw :)

Mitch
To unsubscribe from this group and stop receiving emails from it, send an email to gps-help+unsubscribe@googlegroups.com.

To post to this group, send email to gps-...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "gps-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gps-help+unsubscribe@googlegroups.com.

Sergey Levine

unread,
May 27, 2016, 7:26:18 PM5/27/16
to Mitch Adler, gps-help, CHELSEA FINN
That error could mean a few different things, but often it means that the states are somehow malformed or bad -- for example, if your state contains NaN or Inf, you will see this error.

I would recommend starting with something simple like controlling one joint of your robot to verify that everything is working.

The gains correspond to the PD gains for the initial controller before learning starts. This controller tries to keep the robot at the initial state. You can set gains to zero and you will simply have random motor commands on the first iteration. For some robots, this is OK. For others, it might cause damage (e.g. if it's a big heavy arm that can swing around and damage itself). In that case, treat these as PD gains, it takes some tuning to set reasonable values. Remember that you do want some random motion (as much as possible ideally, subject to your safety tolerance).

Sergey



To unsubscribe from this group and stop receiving emails from it, send an email to gps-help+u...@googlegroups.com.
To post to this group, send email to gps-...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "gps-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gps-help+u...@googlegroups.com.
To post to this group, send email to gps-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gps-help/CA%2BN%2BvwsD3zjuJ9fYA1x1CfxRr0YuJ8A2TgrykX053s1Bd24Uew%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "gps-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gps-help+u...@googlegroups.com.
To post to this group, send email to gps-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gps-help/8a003efb-6697-43fe-9a05-58d5d9f98662%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "gps-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gps-help+u...@googlegroups.com.
To post to this group, send email to gps-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gps-help/f8412116-9682-4a62-a435-7ff7cf22254a%40googlegroups.com.

Ethan Keller

unread,
May 31, 2016, 12:32:06 PM5/31/16
to Marvin Zhang, Sergey Levine, Dylan Hadfield-Menell, cbf...@eecs.berkeley.edu, gps-help, Justin Fu
Hello all,

I've been creating the Baxter GPS plugin based off the MuJoCo example (a very helpful example indeed) and I need some advice. So far, I've been able to replace all muJoCo parameter sample.set calls to protobuf with the sensor data obtained from the Baxter API. I've also been able to route the robot joint commands into the Baxter simulator. So at the moment, Baxter is waving around haphazardly and those motions are captured and calculated in the GPS algorithm. 

However, I'm not out of the woods yet, because in the simple example of trying to learn a target pose (not involving any visual feedback) I am not able to get a diminishing cost over multiple iterations. The obvious culprit is that each iteration does not 'reset' the Baxter world as it is set to do in the MuJoCo agent code here. Would anyone know how I can implement a similar step using the Gazebo simulator? One option would be to re-spawn the baxter model with each iteration/condition. This is ugly because the Baxter urdf is currently craching Gazebo when spawned with a rosrun call (a known issue), and more importantly, this solution has no correspondence in the physical world for when we try it on the real Baxter. How is this handled in the PR2 implementation?

My second issue involves adjusting the size of the sensor dimensions specified in the hyperparams file. In the mjc_example,the END_EFFECTOR_POINTS is passed a length 6 array that contains a point for each finger (I think) like: [x0,y0,z0,x1,y1,z1] however, Baxter only returns one point for a single limb's end effector position as well as a quarternion to specify the pose. When I try to reduce the EE points dimension to 3 and re-specify the target position to accommodate the change, I get a matrix dimension mismatch error in here because d1 is now size (100,3) rather than (100,6), however the size of my Jacobian didn't change. My main question is: how can I pass the end-effector quaternion to the algorithm? and also what things should I be changing to re-specify the endpoint position dimension as length 3?

Thanks for all the help thus far and I look forward to a working GPS algorithm on Baxter soon!

Cheers,
-Ethan Keller
--
774-258-0866

Sergey Levine

unread,
Jun 1, 2016, 2:10:17 AM6/1/16
to Ethan Keller, Marvin Zhang, Dylan Hadfield-Menell, cbf...@eecs.berkeley.edu, gps-help, Justin Fu
Hi Ethan,

For the reset, you would probably want to implement something with the Baxter position control instead of resetting Gazebo, because you can't reset the real world with a rosrun call. For the PR2, you will notice there is a special message that is sent to get the robot to reset via the onboard controller. Typically this message just tells the robot to move back to the initial joint angles and zero out the velocities.

For the end effector, you would probably want to use that quat to transform some end effector points into the wrist frame. You may be able to use the quats directly, but I've never done this and I'm not sure how well this would work (the unit norm constraint would also not be respected, though it's not clear how bad this would be locally).

Sergey

Chelsea Finn

unread,
Jun 1, 2016, 7:30:29 PM6/1/16
to Sergey Levine, Ethan Keller, Marvin Zhang, Dylan Hadfield-Menell, gps-help, Justin Fu
This is where the PR2 reset is called: https://github.com/cbfinn/gps/blob/master/python/gps/agent/ros/agent_ros.py#L127

A message is sent with the desired joint angles to move the arm to.

The robot plugin listens for the message and the position controller executes the commands (e.g. see robotplugin.cpp and positioncontroller.cpp here)

To view this discussion on the web visit https://groups.google.com/d/msgid/gps-help/CAHwucCo-XXCiVEikzNx4kG4scxrTPyBLK58MEvxGdcVusytTLA%40mail.gmail.com.

Ethan Keller

unread,
Sep 6, 2016, 11:21:35 AM9/6/16
to Dylan Hadfield-Menell, gps-help
Hi Dylan,

So glad to hear you're interested in working on this. I graduated this Spring and since then have not had a chance to work on it. The code I worked on is on the baxter branch of the gps repo here: https://github.com/ethan92429/gps/tree/baxter You can browse through all the commit history to see the exact changes I made... some of which took me a very long time to finish! As I recall, my general strategy was to replace the direct torque control of the PR2 with the direct torque control of the PR2, leaving everything else the same. However, I was seeing really erratic arm behavior, perhaps because I wasn't sure how to record the end effector position so the feedback was off? To be honest, I'm forgetting a lot of the nitty gritty! As the code is in that branch, it compile and run on the baxter simulator, but the arm will do some funky stuff. 

Let me know how I can help more!
-Ethan

On Sat, Sep 3, 2016 at 2:25 PM, Dylan Hadfield-Menell <d...@eecs.berkeley.edu> wrote:
Hi Ethan,

I wanted to check-in with you to see if you'd made some progress on this and if you'd be willing to share your code.

I'm working on a project and I'd like to execute GPS on Baxter. If there's a way I can help out with development let me know.

Thanks!

-Dylan


To unsubscribe from this group and stop receiving emails from it, send an email to gps-help+unsubscribe@googlegroups.com.

To post to this group, send email to gps-...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "gps-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gps-help+unsubscribe@googlegroups.com.

To post to this group, send email to gps-...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "gps-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gps-help+unsubscribe@googlegroups.com.

To post to this group, send email to gps-...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "gps-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gps-help+unsubscribe@googlegroups.com.

To post to this group, send email to gps-...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "gps-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gps-help+unsubscribe@googlegroups.com.

To post to this group, send email to gps-...@googlegroups.com.



--
774-258-0866

Rishabh Jangir

unread,
Feb 14, 2018, 10:07:53 AM2/14/18
to gps-help
Hello, I'm trying to port GPS code to a Barret WAM, and I saw in the comments that Dr. Sergey was talking about asking Clemens about sharing his Barret WAM agent, can someone please update me on this, I am trying to work in simulation for now.
Reply all
Reply to author
Forward
0 new messages