GSoC 2014 Project: GUI based plugin for creating dynamics systems interactively

[TARUN GABA]

Hi all,

I am drafting my GSoC 2014 application here:

https://github.com/pydy/pydy/wiki/GSoC-2014-Application:-Tarun-Gaba-(GUI-based-plugin-for-creating-dynamic-systems-interactively)

Please have a look and provide suggestions on how this can be effectively implemented.

Thanks

[Aaron Meurer]

Are you planning on using the new IPython interactive widgets?

Aaron Meurer

> --
> You received this message because you are subscribed to the Google Groups
> "PyDy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pydy+uns...@googlegroups.com.
> To post to this group, send email to py...@googlegroups.com.
> Visit this group at http://groups.google.com/group/pydy.
> For more options, visit https://groups.google.com/d/optout.

[TARUN GABA]

This wont have an IPython dependency.  I am planning on using html5's new canvas features for making an interactive UI.

[Jason Moore]

Tarun,

Some comments:

The idea of a GUI that lets you build systems out of 3D shapes and add constraints is cool project. You'll need to spend time thinking about a System or World class that allows you build up multibody systems at the "object" level. Chris Dembia has the start of a system like this for making simple robot arms, or tree like linkages with no constraints. You'll need a Python API that allows you to build systems, add/remove constraints, add mass/inertial properties, etc. Once you have that you can make a thin layer to Javascript land so that you can drag and drop rigibodies into a 3D arena and then simulate them.

I'd highly recommend you use the IPython framework for this. IPython 2.0 now provides a Python kernel, the webserver connections, and the javascript widget capabilities that can easily allow your JS code to manipulate Python objects. I think it is a better idea to use their framework than inventing your own.

Some notes about pydy-viz:

There is also still a ton of work to do for pydy-viz. You could easily fill an entire summer turning pydy-viz from a prototype app for simple visualization to a full fledged tool comprehensive tool for the same purpose. Here are some things I'd like to see pydy-viz do:

- Run seamlessly in a cell in the IPython notebook (no separate tabs).

- Utilize the JS widget functionality in IPython 2.0 to allow you to change the system's constants, initial conditions, and specified inputs and then rerun the simulation all within the GUI.

- Write a JS based numerical integrator so that you can can system parameters via the GUI and get real time changes in the simulation (the JS integrator would provide real time simulation capabilities, or at least as fast as it can go).

- Make it easier to configure the orientation, size, etc of shapes.

- Make it easier to build up collections of shapes for a single rigid body.

- Make it possible to load STL files or blender files into the visualization space as shapes, so that we can have more realistic looking visualizations.

- Allow for scene backgrounds to be used.

Jason
moorepants.info
+01 530-601-9791

[Jason Moore]

One more note on pydy-viz:

mgview is now BSD/MIT licensed. Both pydy-viz and mgview doe the same thing on the JS side. MGView does things better on the JS side as far as I can see. Why not create an MGView output method on the Scene class so that you can use MGView. We should compare both MGView and pydy-viz and collaborate with Adam if we can. He sounds willing to collaborate. Better to have 2 or more heads than one writing the JS side of things.

Jason
moorepants.info
+01 530-601-9791

[Chris Dembia]

My work on "pydy-linkage", or an API for building models, is here: https://github.com/chrisdembia/pydy-linkage/

The objective was to make it as simple as possible to build a model. That is, the fewest lines. The big stumbling block was handling properties (mass, inertia, distances). Right now, in pydy, a user manually specifies these. I wanted the symbols to be created automatically. However, I didn't figure out a way to do this that I was happy with.

I would enjoy talking more about this.

[TARUN GABA]

So either I can work on an entirely new module(GUI plugin) or I can work on pydy-viz improvement. Both look equally promising to me.

But responses from the community suggests that I should work on refining pydy-viz rather than creating a new module.

Additionally I can work on creating a numerical integrator in Javascript, so that simulations can be relaunched after numerical integration using javascript code only. So If I am going to draft a proposal around pydy-viz, what all can be included (apart from that mentioned by Jason) further in the project?

I would like to know more about Chris's ideas, and how they can be used from visualization point of view.

Since we might be using a javascript integrator, Chris's idea of quickly(in fewest lines) building models using an abstraction, which can then be passed on to javascript for numerical integration directly might be helpful.

[Aaron Meurer]

I would also strongly encourage you to look at the new IPython GUI
stuff. It solves a lot of problems for you, and, more importantly,
people are going to start using it everywhere. Your vis stuff is going
to feel out of touch if it doesn't use IPython.

Aaron Meurer

[Chris Dembia]

Tarun:

I haven't kept up with pydy-viz, so I don't know helpful I can be. Maybe give me a better idea of what you want to know?

I imagine this hierarchy:

GUI model building, on top of:

modeling language / abstraction layer, on top of:

the current pydy interface.

Pydy-linkage was my first attempt at a modeling language. It was only going to work for linkages (i.e., chains) at first, but it seems now that that limitation should be lifted. Further, it assumed the underlying dynamics engine would be symbolic. Given recent discussion, it sounds like we would want to allow multiple underlying dynamics engines. It could take some work to create a nice interface that played well with various engines. 

See github.com/chrisdembia/pydy-linkage to see my sketch of this. I haven't thought much about how this works with visualization. I imagine the GUI would maintain an instance of a sys =  MultibodySystem() object, and would make calls to methods like

sys.body_new(Body('link1'))

sys.body('link1').mass = 5

sys.body('ink1').shapes += [pydy_viz.Shape(...)]

sys.body('link1').origin = [symbol, symbol, symbol]

...

sys.controller_new(MyLQRController(K=python.contol.lqr(...)))

sys.visualize()

sys.integrate(time=5.0)

It might get confusing, because there are some differences I think in how symbolic vs numerical libraries expect a description of a model. We could get some good ideas from the interface of Simbody (for modeling and integration) and OpenSim (for controllers).

Ideally, the modeling language would allow serialization via JSON or XML. However, it seems that Tarun already accomplished this for pydy-viz. So maybe we could just use/adapt that.

Also, people want to do more than just visualize their system. They want to get time history data out (coordinates, tracking errors, controller signals). The MultibodySystem could provide this data, and the GUI application could display graphs of the data.

The GUI would be a client of the modeling layer and of pydy. This is similar to how OpenSim is an application on top of the Simbody library.

I'm happy to lay out more ideas with anyone who's interested.

[Chris Dembia]

See the Robot Modeling Language (SDF) used in Gazebo (for serialization). http://gazebosim.org/sdf/dev.html

[Jason Moore]

We should probably add a python physics abstraction layer idea to the ideas list as a separate item. As this would be a tough project on it's on. And it probably should be a prelude to a GUI tool that allows you to build models, especially if we want that to generalize beyond what pydy already does (symbolics + numerical integration).

Jason
moorepants.info
+01 530-601-9791

[TARUN GABA]

After considering the suggestions, I have decided to change my proposal to enhancing the existing codebase for pydy-viz. 

I feel that instead of creating new and incomplete modules, I should focus more on improving the existing one.

I have started working on a new application here:

https://github.com/pydy/pydy/wiki/GSOC-2014%3A-Tarun-Gaba%28PyDy%29---PyDy-Viz-Improvements-and-Enhancements

Major ideas include:

 - Support for IPython notebooks

 - Integration with MGView.

 - creating visualizations for all existing pydy_examples, and creating a web-app for posting them on org. website as demonstrations, similar to matplotlib gallery.

http://matplotlib.org/gallery.html

http://matplotlib.org/examples/

Apart from that I have ideas for some other minor, but useful features, which I would keep posting on pydy wiki(application).

[Jason Moore]

Some thoughts:

- At the bare minimum you want to simply embed the JS webgl viewer in a webpage. This does not require a server or to be a "web app" in the traditional sense. Write now pydy-viz creates JS that is indepent of the python side and simply plays through the data.

- I think we should move torwards using IPython as our server as opposed to introducing a web framework. I'd much rather make a hard dependency on IPython than Flask or something similar. 99% of people doing scientific computing with Python have IPython and they have lots of resources making it awesome. In particular, the JS to Python bridge they have is much better for what we want to do than what any web frameworks offer. If we create a "web app" it should fundamentally run on IPython. Also, we should be able to export "static" visualizations that can be embedded in a static web page (i.e. no server side stuff).

- What is the integration with MGView? Do you plan to simply offer an export feature so you can open data in MGView? Or do you plan to make MGView a dependency and use it for the default visualizations? MGView has more features than what the JS stuff in pydy-viz has? Why create them both when the do the fundamental same thing?

Jason
moorepants.info
+01 530-601-9791

[TARUN GABA]

I am in agreement with the IPython part. Making ipython a dependency is a good option (over flask or bottle or like).

About MGview, either we can simply add a function to export simulation data in mgview compatible format, or we can use mgview UI as our default.

I am inclined towards the latter option, but I am not sure what other features mgview can offer us. I should start a conversation with its author.

[TARUN GABA]

About the visualization part, I think we can completely move away from "creating our own server" part. We can make a hard dependency on IPython, and a soft dependency on MGView in the sense that:

1. We can provide a MGview compatible output json file. If a user doesnot want to use IPython, he can use MGview to visualize.

2. A complete IPython support along-with Javascript widgets. 

We can employ javascript based numerical integrators for the same.

Our current server is a very basic one. It will not be able to handle the complicated tasks of creating dynamic systems interactively(which is on our roadmap for future) and rerunning the simulations after changing parameters. Instead of working on improving the existing server(will require a lot of input), we can simply switch to IPython kernel and MGView output(It uses a node.js server as far as I know).

[Jason Moore]

Jason
moorepants.info
+01 530-601-9791

On Sat, Mar 15, 2014 at 3:55 AM, TARUN GABA <tarun...@gmail.com> wrote:

About the visualization part, I think we can completely move away from "creating our own server" part. We can make a hard dependency on IPython, and a soft dependency on MGView in the sense that:

1. We can provide a MGview compatible output json file. If a user doesnot want to use IPython, he can use MGview to visualize.

Does MGView include a server?

I think there are two options:

1. Don't use MGView and conitnue developing the JS stuff in pydy-viz.

2. Use MGView completely but work with Adam so that the project meets both of our needs, if he is willing.

If you do #1 you can still output an MGview compatible json file if someone wants to play with their sim in that framework. But only do #2 if you are committed to work with Adam on a collaborative project and he is also willing. I personally like the idea of making MGView a generic visualization tool from simulation from any dynamic engine, where pydy can be one. But this may be a bad idea unless people are committed to it. The fact that pydy-viz (the js part) does the same thing as MGView is fine but it's always nice to have more people solving a common problem that get a better solution than a bunch of poor solutions from many people. Keep in mind that this would only work if you seriously engage Adam Leeper in this conversation.

[Sahil Shekhawat]

tHi Chris,
I want to work on this as my GSoC project. I have looked at pydy-linkage and am working on a demo.
Here, you said "it sounds like we would want to allow multiple underlying dynamics engines"
Can you please shed light on multiple dynamics engines?

[Jason Moore]

I'm not sure what Chris meant in the code. But a "dynamics engine" is something that computes x'=f(x) for multibody systems. PyDy generates "engines" for particular multibody systems.

Jason
moorepants.info
+01 530-601-9791

[Chris Dembia]

Yes, PyDy is currently a symbolic engine that creates x' = f(x) for a single system. That's one type of engine. Other engines would be the order-n gsoc project. We could also tie into existing (c++) physics engines: box2d, bullet, ode, simbody, etc. Hopefully this interface for defining multibody systems is independent from the engine. You definitely don't want to worry about implementing other engines.

In my mind, there are roughly two ways to define multibody systems in software. With symbolic solvers (pydy, motiongenesis) you explicitly define the position and velocity of points and bodies. With numeric solvers (what are typically called physics engines: ode, bullet), you define bodies and joints. Thus, I've defined two interfaces: the interface that typically goes with symbolic solvers (specify positions and velocities) and the interface that typically goes with numeric solvers (bodies and joints). PyDy, of course, currently has the symbolic interface. It's somewhat difficult to use PyDy's interface to define an n-link pendulum currently (or at least I think there is a better way): by specifying bodies and joints. So the whole point of pydy-linkage was to create the "bodies and joints" interface for a symbolic solver.

Then, with one single "bodies and joints" interface, its possible to use either symbolic or numeric solvers (physics engines). 

The issue is that the "bodies and joints" interface is not as general as the "points and velocities" interface.

There are perhaps two choices for your project then:

A. Create a single "bodies and joints" interface that allows the use of symbolic and numeric solvers, but only for a limited set of systems (well, linkages).

B. Create an interface that allows specification of "body and joints", but ALSO of "points and velocities".

My idea with pydy-linkage was "A". If your system is not well-described as a linkage, then you must use PyDy's original interface, which is fine I think. My recent email on pydy was regarding "B". I guess I only thought of "B" because Sahil's example code seemed more along the lines of "B". That is, in pydy-linkage, I did not imagine that methods like "set_ang_vel" could exist (since that is part of a "points and velocities" interface).

B sounds really hard and not well-defined, though I haven't thought much about it. B Sounds more like an improvement of PyDy's existing interface rather than a new interface that wraps the existing one.

Sorry if this was all extremely confusing.