How can we improve 3d graphics in sage?

[Greg Laun]

I have a student who specializes in 3d graphics (in particular writing very fast physics and game engines) who has expressed interest in contributing to Sage.  From the sound of it, it would be very simple for him to implement fast, responsive interactive 3d graphics, which I for one would definitely like to see.  I realize we have jmol, but it currently has a long load time.

The only issue is I don't know enough about the status of sage's 3d graphics to know where to start.  Is there a 3d graphic wish list, or any tickets that describe what features a 3d graphics system needs?  Are separate native implementations in Windows, OSX and Linux okay, or do we need something like Java that is truly platform agnostic? 

As a user, my ideal would be 3d and 2d graphics that update in real time with the interact command.  Mathematica has the edge here, and it would go a long way toward drawing some users away from Mathematica if we could compete in this arena.

Thanks,

Greg

[Michael Orlitzky]

On 09/13/2013 02:10 PM, Greg Laun wrote:
> I have a student who specializes in 3d graphics (in particular writing
> very fast physics and game engines) who has expressed interest in
> contributing to Sage. From the sound of it, it would be very simple for
> him to implement fast, responsive interactive 3d graphics, which I for
> one would definitely like to see. I realize we have jmol, but it
> currently has a long load time.
>

We don't even have basic 3d plots (you have to drop down into
matplotlib) -- that would be a good place to start.

Something like this should be possible:

sage: f(x,y) = x + y
sage: f.plot((x,0,1), (y,0,1))

Right now the best you can do is,

sage: p = plot3d(f, (x,0,1), (y,0,1))
sage: p.show(viewer='tachyon')

which is ugly and has precisely zero options.

Matplotlib *does* produce nice output, but it takes a page of code to do
what the two lines above do.

[Volker Braun]

Matplotlib doesn't have real 3d, only isometric projection. Often good enough, but nothing where you can really rotate the scene.

My whishlist would be

  * opengl

  * ability to render scene as 2d vector pdf for publications, possibly with gl2ps

Writing a cross-platform opengl viewer with some controls to rotate is the easy part. For really smooth 3d animations/interacts you need a different way of plotting than what we do now. Instead of plot3d() spitting out a 3d screne every time you call it, you need to once generate the 3d scene and then progressively update the existing OpenGL objects as you animate (or change the interact).

[David Joyner]

I wonder if https://github.com/nilesjohnson/sage_animate
could be expanded upon?

> --
> You received this message because you are subscribed to the Google Groups
> "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at http://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/groups/opt_out.

[Vee Kay]

Hello,

On Friday, September 13, 2013 7:10:16 PM UTC+1, Greg Laun wrote:

I have a student who specializes in 3d graphics (in particular writing very fast physics and game engines) who has expressed interest in contributing to Sage.  From the sound of it, it would be very simple for him to implement fast, responsive interactive 3d graphics, which I for one would definitely like to see.  I realize we have jmol, but it currently has a long load time.

as a maths student and a gamer, this is definitely something I would like to see. From an educational point at least; for research, I have the impression pre-rendered plots are preferable.
That said, gaming requires graphic card(s) where hardware acceleration is done. Most of my professors and fellow students have rigs incapable of running modern 3d game engines.

The only issue is I don't know enough about the status of sage's 3d graphics to know where to start.  Is there a 3d graphic wish list, or any tickets that describe what features a 3d graphics system needs?  Are separate native implementations in Windows, OSX and Linux okay, or do we need something like Java that is truly platform agnostic?

OpenGL would seem the most natural choice as it is cross-platform with bindings in numerous languages, including C and python.

Albert

[Travis Scrimshaw]

Hey all,

   I have some (Java [for Android]) code for rending 2 -> 1 and 3 -> 1 plots that uses OpenGL ES if anyone would like to use that as a starting point.

Big +1 for interactive plots with simple interfaces.

Best,

Travis

[Jason Grout]

On 9/13/13 1:10 PM, Greg Laun wrote:
> As a user, my ideal would be 3d and 2d graphics that update in real time
> with the interact command.

I think it would also be cool to add the ability to pick and interact
with objects in the 3d scene, not just control them with external sliders.

Thanks,

Jason

[Eric Gourgoulhon]

Hi,

Le vendredi 13 septembre 2013 22:50:59 UTC+2, Volker Braun a écrit :

Writing a cross-platform opengl viewer with some controls to rotate is the easy part. For really smooth 3d animations/interacts you need a different way of plotting than what we do now. Instead of plot3d() spitting out a 3d screne every time you call it, you need to once generate the 3d scene and then progressively update the existing OpenGL objects as you animate (or change the interact).

 
What about using Blender ? It is a mature free and open-source 3d/4d graphics software of professional level. Moreover, it has a Python API (Python 3 however, while Sage is still in Python 2). The 3d rendering and animation of Sage objects in Blender scenes would allow one to explore geometric objects in a way far beyond what Mathematica or Maple can offer.

Best,

Eric.

[Volker Braun]

Being able to interface with blender would be intriguing, but I don't think that it would be useful for the majority of users. For starters, blender is so packed with features that its basically unusable for any casual user. There are probably as many Mathematicians that can use blender as there are graphics designers that use Sage ;-)

[Jonathan]

Brief status of Jmol 3-D.  

1) The javascript only implementation is now working. (Some simple examples mostly molecular can be seen at: http://www.uwosh.edu/faculty_staff/gutow/Orbitals/Cl/Cl_AOs.shtml and http://chemapps.stolaf.edu/jmol/jsmol/test2.htm).  However, I have not had time to do the work to couple it to Sage.  If this student wants to help I would really like it.

2) The things people would like to do here are all things Jmol already does.  Somebody simply has to implement the right javascript and backend python for people to use it.  Much of the difficulty surrounds communicating with the Sage server via asynchronous calls. Again I would love somebody who has time to work on it help with this.  I know a lot about Jmol and how to make it do things, but have found making the connections to Sage workable time consuming.

3) Some things Jmol does that I think might be useful that I haven't seen people ask for: slicing off parts of surfaces, density plots for 3-D functions,  3-D animations that you can spin while they change.  These are all available now, we just need to manpower to couple them to Sage in a useful way.

4) I suspect it would be an unnecessary duplication of effort to start from scratch on 3-D viewing using either Java or javascript now that Jmol has one code base that generates both.

Anyway, if somebody can implement something that works better than Jmol I think that would be great.  We developed Jmol for molecular visualization, which requires visualizing many 3-D objects and surfaces simultaneously, but it was certainly not originally intended for function plots.  Note there is a stub in the code for plots with range specification  that nobody has had time to work on.  So I suggest that anybody who has time to work on this start by contacting me, learning what Jmol can do and deciding if their time would be better spent writing something from scratch or helping to couple what Jmol already provides to Sage in a better way.

Jonathan

[Nils Bruin]

On Saturday, September 14, 2013 3:07:45 AM UTC-7, Eric Gourgoulhon wrote:

What about using Blender ? It is a mature free and open-source 3d/4d graphics software of professional level. Moreover, it has a Python API (Python 3 however, while Sage is still in Python 2). The 3d rendering and animation of Sage objects in Blender scenes would allow one to explore geometric objects in a way far beyond what Mathematica or Maple can offer.

Using, or at least exporting to, blender is easy: we can already write to x3d, which is understood by almost all 3d software. It hasn't been used much, so you might find that the x3d conversion is buggy and missing some features, but fixing that is straightforward.

Concerning interactive manipulations: I'd think that being able to limit the rendering to a half space could be a useful one to look "inside" an object, and Jonathan's remark suggests this should be straightforward to expose in jmol.

For more complicated things, e.g., providing an interact to plot x^2+y^2-z^2-t=0, with a slider for t, I don't immediately see how you can do significantly better than just shipping a new scenery to the renderer.

[Jonathan]

On Saturday, September 14, 2013 12:32:31 PM UTC-5, Nils Bruin wrote:

For more complicated things, e.g., providing an interact to plot x^2+y^2-z^2-t=0, with a slider for t, I don't immediately see how you can do significantly better than just shipping a new scenery to the renderer.

This is where Jmol's object oriented approach may help.  It is possible to send Jmol just the new object and not the other information about the scene, such as axes, background color, etc...In theory that would make network based updates faster.  At present Sage rebuilds the whole scene each time. 

[mmarco]

Well, i don't know if there is any graphic designer that uses Sage, but i do use blender.

I agree with you, Blender is definitely overkill for sage graphics. Some .stl or .ply exporter for sage would be nice though (an much easyer to implement).

[Greg Laun]

 Hey everyone,

Thanks for all the input.  I'll share the discussion with my student and I'll send an update to this thread once we've decided what direction to go in.

- Greg

[Robert Bradshaw]

Being able to display graphics in the browsers is key, and things have
improved a *lot* since we settled on jmol (which has served us well
and was the best option at the time).

I'm surprised no one's yet mentioned
https://plus.google.com/115360165819500279592/posts/WkY7mQ6ju2u which
is extremely relevant.

[Jason Grout]

On 9/14/13 12:32 PM, Nils Bruin wrote:
> Using, or at least exporting to, blender is easy: we can already write
> to x3d, which is understood by almost all 3d software. It hasn't been
> used much, so you might find that the x3d conversion is buggy and
> missing some features, but fixing that is straightforward.

IIRC, we also export Wavefront .obj format, which is also common. The
other day, I exported to .obj and imported into some 3d printer software.

Jason

[Volker Braun]

On Tuesday, September 17, 2013 4:11:41 AM UTC+1, Robert Bradshaw wrote:

I'm surprised no one's yet mentioned
https://plus.google.com/115360165819500279592/posts/WkY7mQ6ju2u which
is extremely relevant

Where is the code?

Presumably that is three.js... which doesn't work outside of the browser. And doesn't give us a reasonable export to 2d vector pdf for publications, the svg/pdf renderers for three.js are mostly proof-of-concept not production-quality code.

[Greg Laun]

Hey everyone,

I wanted to give a brief update.  We decided that the most important thing right now is to have graphics classes where the transform(), rotate() and related functions return matrices that can be passed to the graphics card rather than returning the matrix applied to the object.  The idea is that the graphics card should hold a representation of the object and it's the job of the graphics card to apply those transformations, not the CPU.  I haven't looked into the source code too much, but the current tentative plan is to fork the current 3d graphics classes and give them a new name to maintain backwards compatibility while we work out how the new system might work.

My student is also rather excited by the idea of making sage interface with Blender.  I know a few people were asking for that.

Greg

On Friday, September 13, 2013 2:10:16 PM UTC-4, Greg Laun wrote:

[Jason Grout]

On 9/17/13 5:19 AM, Volker Braun wrote:
> On Tuesday, September 17, 2013 4:11:41 AM UTC+1, Robert Bradshaw wrote:
>
> I'm surprised no one's yet mentioned
> https://plus.google.com/115360165819500279592/posts/WkY7mQ6ju2u
> <https://plus.google.com/115360165819500279592/posts/WkY7mQ6ju2u> which
> is extremely relevant
>
>
> Where is the code?
>

https://github.com/sagemath/sagecell/commit/241a75a9f404a9b1ac72d5900c841c193760888d

> Presumably that is three.js... which doesn't work outside of the
> browser. And doesn't give us a reasonable export to 2d vector pdf for
> publications, the svg/pdf renderers for three.js are mostly
> proof-of-concept not production-quality code.
>
>

It is using three.js. This is just a renderer specifically for the
browser, and it works great for that. It would be cool if someone also
worked on something that could render to pdf/svg or a better solution
outside of a browser. I've kept my eye on gl2ps
(http://www.geuz.org/gl2ps/)---that might be a good direction to go for
rendering 2d vector projections.

Thanks,

Jason

[Volker Braun]

Sounds great. I agree with the basic premise that the lifetime of the GPU data needs to be coupled to the lifetime of the Python object. On the Python side, we then implement a mutable interface. Changes in Python (like rotations, but e.g. also changing a sphere radius) are propagated to the GPU. Then it is easy to do 3d animations, say.