LQRs are not really converging

[Manuel Kaspar]

Hi together,

we tried porting GPS for using it at a KUKA LBR iiwa. For easy stuff it is working quite well. Now, we set up a experiment, where we try to insert a cube into a hole. But we don't suceed. 

Currently we are using 2 conditions (ca. 5cm away from each other) and 5 samples per condition.

The system is pretty unstable, sometimes we manage to come pretty close to the hole ~1cm. But then we can't see a real improvement towards the hole. Mostly it drives down to the boarder and then stops there, but is not really 'searching' into the hole.

Does anybody have an idea, why we don't have a real convergence here? We are using the cost function like normal, penalizing to high forces, distance to goal, last timestep ramp up. 

Thanks a lot!

[Manuel Kaspar]

Ah - we are using mdgps off policy