Techniques for drawing lines, with mouse, on 3D objects in WebGL or OpenGL?

[Lindsay Kay]

Hi,

Can anyone point me to techniques for drawing lines, with the mouse, on 3D objects in WebGL or OpenGL?

Can't seem to find anything so far.

Here's the effect I'm looking for, a cool demo by SpaceGoo on WebGL: http://www.spacegoo.com/cadillac/

I can imagine the general idea - projecting a ray onto the surface, then perhaps using normals and UVs, transform the intersections into UV-space and drawing the line on a texture, which gets mapped onto the object.

Would love to find some literature, tutorials or education code on it though.

cheers

Lindsay

[Theo Armour]

Hi Lindsay

Is it OK for me to point you toward some Three.js examples?

If so, have a look at:

http://mrdoob.github.io/three.js/examples/#webgl_decals

http://mrdoob.github.io/three.js/examples/#webgl_interactive_raycasting_pointcloud

Theo

--
You received this message because you are subscribed to the Google Groups "WebGL Dev List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to webgl-dev-lis...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Diego]

Check this out.
http://www.cartelle.nl/deathpaint/study/

[Lindsay Kay]

Thanks guys, I'm getting an idea of how to approach this now. 

Also, I should post what Xavier at SpaceGoo recommended, when I asked him how he did Cadillac Ranch ( http://www.spacegoo.com/cadillac/ ):

It is all WebGL without a framework.
There is an octree for the cadillacs, it is used to compute the
intersection between the mouse ray and the cadillac. Cadillac are mapped
with texture without overlapping (I used blender and a custom python
script to export the mesh).
When I have the point of intersection between the ray and the cadillac, I
also have the triangle face intersected and its summit ABC. And I have the
UV coordinates of A,B,C.
I use barycentric coordinates
(http://en.wikipedia.org/wiki/Barycentric_coordinate_system) to compute
the UV coordinates of the point. Then I use gl.texSubImage2D to draw the
brush on the texture (mipmap level 0), and I recompute mipmaps.
Every X seconds, I send with AJAX the modification to the server. The
server do the same thing using image magick, and you should also use a
filelock to avoid that the texture will be modified at the same time.

cheers,

Lindsay

On 9 May 2015 at 15:17, Diego <diego....@gmail.com> wrote:

Check this out.
http://www.cartelle.nl/deathpaint/study/

[Theo Armour]

Hi Lindsay

I could not resist trying to come up with an attempt at responding to your fun question.

The 'compare and contrast' is that you are looking for first the 3D faces and I went for the bitmap first.

I've been looking a lot at bitmaps as heightmaps recently, so editing the bitmap to edit the surface seemed a natural thing to do.

The usual bitmaps are for terrain, but that seemed boring so it turns out that there is a bitmap image of a Lindsay Kay on LinkedIn.

So I used that bitmap to vary the elevations on a plane but human faces are more round than flat, so wrapping the height map around a sphere seemed more interesting.

So now having a variegated 3D surface, I invoke Mr.Doob's Raycaster.setFromCamera to locate where the cursor touches the mesh.

This gives a 3D point from which you can calculate the equivalent position back on the bitmap.

Edit the pixels on the bitmap/heightmap causes the mesh to update.

Bingo! You are drawing on a 3D object in 3D in real time. Sculpey in 200 lines of code.

Warning: both/or the image of you and the code may cause nightmares.

Drawing on Heightmap

Theo

 

[Lindsay Kay]

That's the spirit Theo!

I shall treasure this.

Thanks!

[Theo Armour]

Hi Lindsay

Whew and Yay!

Thank you for not being offended by the defamation of image and non-use of scene.js - and for the nice Tweet. ;-)

So what's the next "Can you do this?" question?

Theo

[Cindy Sherman Bishop]

hey Lindsay- how did you proceed on your project?  I am looking at doing a similar thing, but with primitives (so the uvs are pretty straightforward)

thanks!

Cindy

[Theo Armour]

'Decals' - built with Three.js

http://mrdoob.github.io/three.js/examples/#webgl_decals

--

[Jaume Sánchez]

The decals in three.js (THREE.DecalGeometry) have their own repo here https://github.com/spite/THREE.DecalGeometry with a bit more info. I did them for a project couple years ago, and used them a few times (last time, for last Halloween http://itsamessage.com/diademuertos2015/ ). Decals are mostly for projecting textures; free-drawing with them gets a bit tricky and expensive.

I've remembered seeing this some time ago: http://www.cartelle.nl/deathpaint/study/ http://www.cartelle.nl/skullpaint

[Lindsay Kay]

Hi, 

I got it done in the end - drawing on meshes in xeoEngine (which is still in beta): http://xeoengine.org/examples/#interaction_picking_drawing 

What I'm doing is:

1. First pass - pick the object, using color-indexed picking (ie. render each object with unique color, map the picked pixel back to object)

2. Second pass - pick the triangle on the object, again with color-indexed picking (ie. render each triangle with unique color, map the picked pixel color back to triangle)

3. Do ray-intersect with the triangle, find barycentric coordinate at intersection, then use that to find UV coordinate within triangle

4. Draw on hidden canvas at that UV location, copy the canvas into a texture that's applied to the object.

For step (2), I generate, on-the-fly, positions, vertex colors and indices arrays that allow each triangle to be rendered with flat color for picking, where the three vertices of each triangle have the same color.

Here's the entry point for picking, with the mathsy bits in case they are useful. And here's the bits deeper inside the renderer, where the draw list is executed and the pick buffer read and mapped to the object and triangle. 

While a little bit convoluted, this technique scales to huge numbers of objects and triangles, since I'm only doing a ray-intersect with one triangle, rather than iterating over each triangle in every mesh, to test for ray-intersection on each one.

I also added the technique to SceneJS, so steps 1,2,3 are now used for 3D picking there as well (just so we can get that precious UV coordinate, which wasn't possible with the old picking technique). 

Hope some of that is useful!

[Theo Armour]

Hi Lindsay

Interaction Picking Drawing looks good.

Comments

Pressing down the spacebar while holding down the mouse button is not easy to do on a phone or tablet. Have you looked into using the oninput event?

https://developer.mozilla.org/en-US/docs/Web/API/GlobalEventHandlers/oninput

It's like mousemove and touchend all wrapped up in a single bundle.

Also I like the user experience on the previously mentioned

http://mrdoob.github.io/three.js/examples/#webgl_decals

Click on the object: draws. Click outside the object: rotates.

In any case, there's a ton of cool demos on http://xeoengine.org/. Nice work!

Theo

 

[Lindsay Kay]

Thanks Theo!

I'll fix that interaction up. The list of pre-release bugs is getting pretty short now, should have a release after the new year.

cheers

Lindsay