High Level 3D

[Adam Kowalski]

Are there any plans to support any libraries such as Three or Babylon?  I would really like to use Elm, but right now it seems like the only way to work with 3D is to get down and dirty with elm-webgl and elm-linear-algebra.  While both of those libraries are fantastic, I cannot figure out how I would for example, load in a mesh from an external file?  I am not sure I would be able to write my own loader, unless this is not as difficult as it seems.  On the other hand, these other libraries seem to abstract away a lot of the complexity, allowing you to focus on adding features to your app, rather than worrying about the low level details.  Would it maybe be possible to completely wrap one of them using ports?

[Adam Kowalski]

Would it be a good idea to essentially create a wrapper for Three.JS using ports, and essentially do something similar to a virtual DOM?  The way I imagine it, is that I would have an in memory representation of what should be shown; this could be compared to what is actually shown, and it would imperatively either add or remove it from the scene.  This means that I would be using fast imperative management of what is shown, but I could wrap it in pure functions in Elm through ports.  Am I on the right track here?

[Evan Czaplicki]

Sorry for the delay Adam!

I recall Hassan worked on this project which aimed to give a bunch of high-level tools. I'm not sure if it has been updated for 0.15 yet (elm-webgl just got updated recently) but that may be a good place to start. Does it seem to fit your case?

My main concern with wrapping up something like three.js is that the API mismatch may be enough between Elm and JS that it's not so simple to do. I don't think that should discourage you, but definitely keep it in mind that lots of those APIs probably use mutation, which would not be allowed in Elm.

On Tue, Jun 9, 2015 at 2:21 PM, Adam Kowalski <adam.kowa...@gmail.com> wrote:

Would it be a good idea to essentially create a wrapper for Three.JS using ports, and essentially do something similar to a virtual DOM?  The way I imagine it, is that I would have an in memory representation of what should be shown; this could be compared to what is actually shown, and it would imperatively either add or remove it from the scene.  This means that I would be using fast imperative management of what is shown, but I could wrap it in pure functions in Elm through ports.  Am I on the right track here?

--
You received this message because you are subscribed to the Google Groups "Elm Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elm-discuss...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Hassan Hayat]

I recall Hassan worked on this project which aimed to give a bunch of high-level tools. I'm not sure if it has been updated for 0.15 yet (elm-webgl just got updated recently) but that may be a good place to start. Does it seem to fit your case?

Yeah, I haven't gotten around to updating it. Sorry about that. There are a few changes I have to do to before I can get ready for 0.15 (mainly an issue related to shaders getting recompiled).

My main concern with wrapping up something like three.js is that the API mismatch may be enough between Elm and JS that it's not so simple to do. I don't think that should discourage you, but definitely keep it in mind that lots of those APIs probably use mutation, which would not be allowed in Elm.

Yeah, prior to that project I had tried to wrap ThreeJS but mutation is a really big problem. It can be done but you'd need to treat the ThreeJS scene graph as Virtual DOM treats the DOM. If you do it naively, you'd end up blowing the entire scene graph on each frame (which is bad cuz you'd miss out on all the optimizations that make ThreeJS go fast). 

In my opinion, the best/easiest course of action would be to wrap elm-webgl  and make a high level 3D library out of it (as I attempted). The hard part is constructing the shaders. I haven't fully nailed that part yet, and dynamically constructing shaders from strings can easily cause runtime errors that are super hard to debug. 

[John Mayer]

Hassan,

Can you comment @johnpmayer a few examples with bad shader performance in your library on GitHub?

[Hassan Hayat]

Sure, but it's my fault (I think). It was the old thing about recompiling shaders on each frame. This weekend I'll look into it and give back some feedback on the elm-webgl repo if necessary.

[Adam Kowalski]

So it seems like in general because of the mutations that Three.js performs, essentially I would have to create a React (does elm html work this way too?) like diffing algorithm.  This seems like an ok approach because you would gain all of the functionality of Three.js.  However, I think that the negatives outweigh the gains in this case.  

This leads me to believe that Hassan has the right idea about using elm webgl as the starting point and building slowly on top of this.  I like this idea too, because I think it will really force me to learn how graphics are rendered on the screen.  Graphics engine seems like an amazing package as well, so I will definitely keep an eye on that.

Can any of you recommend good resources about learning the concepts behind 3D graphics (down to the webgl level)?  I would really like to be able to contribute in some way and help advance WebGL rendering for Elm, but I don't feel I am quite knowledgeable enough to be of much use at the moment.

[John Mayer]

I learned using the Opera webgl 101 video. In the "learn the hard way" style, I followed along the live coding video with my own editor. Before this, I had no formal 3d graphics experience.

--

[Max Goldstein]

Yeah, I think a 3D engine would be neat, but this stuff is complicated. Choosing the right abstraction to present to client code will be tricky.

In 2D, I know there was a D3 wrapper that seems to have been abandoned. D3 is built around enter/update/exit selections, basically informing you of the state of the DOM and asking how you want to manipulate it. The react-style diffing seems to be at odds with this. That being said, 2D is easier than 3D...

[Adam Kowalski]

Wait WebGL can be used for performant 2D graphics as well right?  Maybe I could start learning about WebGL and can start by doing something simpler.  It could be an interesting project to try to make an alternative to Graphics.Collage.  Have the same api, but instead of using the regular canvas, we could have hardware accelerated 2D in WebGL.  What do you guys think?

[Dobes Vandermeer]

For some reason I have this idea webgl is not available in browsers reliably right now, compared to canvas.  I'm not sure if that's a concern still?

--

[Hassan Hayat]

Wait WebGL can be used for performant 2D graphics as well right?  Maybe I could start learning about WebGL and can start by doing something simpler.  It could be an interesting project to try to make an alternative to Graphics.Collage.  Have the same api, but instead of using the regular canvas, we could have hardware accelerated 2D in WebGL.  What do you guys think?

WebGL is a 2D graphics framework, not a 3D one, which is logical because screens are two dimensional. So, yeah, you could totally use it to re-implement collage. You could even do one better by having two renderers (one in normal canvas and one in webgl) and decide on which based on the browser support. But don't worry too much about that, WebGL has pretty good support these days.

The big problem with WebGL is that it can get super hard to do certain things with it, especially the easy ones. If you're interested in this, go for the simple stuff. Make a triangle. Then make a rectangle out of two triangles. Then, make a circle out of a bunch of triangles. And so on...

[Adam Kowalski]

I found this course on Udacity - Interactive 3D Graphics.  It seems to be ok, but the series is done using three.js.  Most of it seems like it applies to WebGL in general.  For one of the examples they have you implement a function that can create a polygon of n sides (ngon).  It then tries to reduce the number of vertices by creating a "fan" of triangles.  It seems like you could make a circle by increasing the sides to a fairly large number too.  I will try to reimplement this in Elm.

[Adam Kowalski]

Also if we had a api that was identical to the api Evan used in Canvas, it should be relatively straightforward for other users to swap them out for each other.  Unless you are suggesting to do that on the library side, and have it detect whether it is supported using ports or something, otherwise use the canvas fallback.  Then it could be merged with Collage, and nobody would notice a difference, development wise, but could still gain a speed boost for no extra effort.

[Corey Trampe]

If you really want to go down that road, I recommend studying Pixi.js. However, I don't think it would be very practical to aim for parity between the two renderers. It made a lot of sense for Pixi at the time, but WebGL is so widely supported now that I don't think it would be worth the effort for a new library.

Personally, I think that if you want to use WebGL for 2D graphics, and you want to draw with an API like Collage... then the best approach would be to draw a sprite sheet using Collage and then feed that texture to a batching sprite renderer. I've used this technique for a couple of games, and it works very well. Your sprites are pixel-perfect, drawn once on the CPU, stored in graphics memory, and then batch rendered each frame in a single draw call.

I hope that makes sense.

On Wed, Jun 10, 2015 at 11:38 PM, Adam Kowalski <adam.kowa...@gmail.com> wrote:

Also if we had a api that was identical to the api Evan used in Canvas, it should be relatively straightforward for other users to swap them out for each other.  Unless you are suggesting to do that on the library side, and have it detect whether it is supported using ports or something, otherwise use the canvas fallback.  Then it could be merged with Collage, and nobody would notice a difference, development wise, but could still gain a speed boost for no extra effort.

--

[Adam Kowalski]

Yes this makes sense, however, my ultimate goal is to learn 3D.  I just thought that it would be much simpler to start there and learn the basic theory behind how everything gets rendered onto the screen.  Then I could start diving deeper into linear algebra and 3D, as well as learning about things like lighting, perspective, and other more complicated things necessary to sell the illusion to the end user.