Ray tracing for complex geometry

[kleinsplash]

I was wondering if someone could help me out with a decision/offer an opinion:

I need a ray tracer that deals with complex geometry (a fast ray tracer that can create 1000's of point clouds in minimal time) 

Python has methods: http://pyopengl.sourceforge.net/ that I could get to grips with. But I want to stick with Julia. 

I have found these resources: 

https://github.com/JuliaGL/ModernGL.jl - not sure if this has a ray tracing option

http://www.cs.columbia.edu/~keenan/Projects/QuaternionJulia/ - looks crazy complicated

https://github.com/JuliaLang/julia/blob/master/test/perf/kernel/raytracer.jl - looks like only handles simple geometry

Could someone point me in the right direction?

 

[DNF]

I can't really help you, but I think that http://www.cs.columbia.edu/~keenan/Projects/QuaternionJulia/  does not have anything to do with Julia the programming language, but is related to Julia sets.

[Tom Breloff]

Could you describe a little more about your use-case?  I'm not sure that ray-tracing is necessarily what you want if you're displaying point clouds.  I would check out GLVisualize.jl as a first step.

[Steve Kelly]

How is your geometry defined? If it is an implicit function, ShaderToy.jl (built on GLVisualize) has a raymarching example. https://github.com/SimonDanisch/ShaderToy.jl/blob/master/examples/rayprimitive.frag

The method can be generalized to generating distance fields, but I haven't gotten to it yet. I'd also recommend taking a look at the link in the comments. Inigo has some great stuff on ray tracing techniques for the GPU.

I've been working on a solid modeler that makes describing primitives as functions much easier. https://github.com/FactoryOS/Descartes.jl/tree/master/examples The eventual goal is to get all the geometric realization code on the GPU (SDFs and Dual Contours).

[kleinsplash]

Thanks a stack. The geometry is defined in a mesh file .obj. Consisting of vertices and faces. The objects are complex, so any primitive ray-tracing is most likely not going to work. My challenge is that my obj files are just not dense enough. If they were I could use the vertices no problem. So I either increase the density of the mesh, or ray-trace.

My outline:

load obj file
set camera angle(random range something like 0-2/pi)
set focal length(random range)
ray-trace(~1000 points - much greater than the number of vertices I have)
plot the view of the object(just for visual confirmation)
hold 3D matrix of view (x,y,z) or (x,y,ind)

[kleinsplash]

Not sure if you get other comments, please forgive me if this generates two mails in your inbox. Please see my answer to Steve Kelly below.

[Simon Danisch]

I'm still confused about what you want.
Google says:
"In computer graphics, ray tracing is a technique for generating an Image by tracing the path of light through pixels in an image plane and simulating the effects of its encounters with virtual objects." which is pretty much what I understand under ray tracing.
So what is your goal? Usually ray tracing is used when you want a realistically lit render(plot) of a 3d scene, or if you have some implicitely defined geometry ( like steve kelly pointed out).
Usually you have a mesh in an obj file, which you would only ray trace when you need photo realism (since it's way slower then any other displaying method).
I'm really unsure, why you want to get out a 3d matrix in the end. Do you want to compute a lightfield?! Or do you want to have an interpolated voxel representation of that mesh in a dense volume? This is probably not what you really want and also not what a ray tracer produces.
The last thing I can imagine is, that you actually have only vertices(points) in your obj file without any faces, which then wouldn't really show up in a plot, so that you started assuming it's not dense enough. I wouldn't call pointclouds a complex geometry though, since its one of the most simplest.
If that's the case, you should just visualize them as particles.
Here is how you would do that in GLVisualize:
Using FileIO, GLVisualize
obj = load("pathtoobj.obj")
points = vertices(obj)
w,r = glscreen()
view(visualize(points))
r()

If you have problems with that example, please file an issue on github under GLVisualize.jl.

Best,
Simon

[kleinsplash]

Thank you.

Apologies, I struggle trying to explain myself at the best of times.

I have a number of virtual objects (*.obj) I want to get ~1000 depth images from x angles. I used the pinpoint camera approximation only to find that the obj files had so view vertices that getting reasonable depth information was not feasible. So I can do two things - figure out how to make my mesh more dense (surface fitting or re-sampling) or do ray tracing so that I can obtain points wherever my rays intersect with the faces of my virtual object.

I will then be using these depth images as input for a convolutional neural net.

[Simon Danisch]

I'm not sure what you mean by virtual objects. Obj is in the context of 3D objects is usually the wavefront format.

If you have an object database with *.obj's in it, the probability is very high, that you don't have pointclouds whatsoever.

You can try this, to confirm my hypothesis:

using GLVisualize, FileIO

obj = load("file.obj")

w,r = glscreen()

view(visualize(obj))

r()

Or just download any obj viewer from the interwebzz and look at that thing. 

If you have nice smooth surfaces, you're getting it all wrong with the pointclouds and ray tracing.

I could give you some hacky way of extracting depth images with GLVisualize, if that's what you're after.

In that case, just try the example above and if that works, open an issue at GLVisualize that you want depth images. Then we can take it from there.

If by any chance you DO have pointclouds stored in an obj file, things are more complicated since you then need to approximate the surface of that cloud.

Still, raytracing wouldn't be your friend ;) If you have infinitely small points, there is no magic that lets a ray hit these points any better then some other visualization algorithm.

Even if it's really dense, you still have infinitely small points. You can treat the points as particles, to give them some "body" that you can see, but then it's not really a surface anymore.

Just google for pointcloud surface approximation and see where that gets you.

I'm guessing here, that you have some sensor that outputs depth images and you want to recognize objects in these depth images.

To train your depth image classifier, you need depth images from a lot of perspectives from a lot of random 3D objects, which is why you searched for a 3D object database, which got you to the obj files of random 3D objects.

It'd have been a lot easier, if you just stated this in your problem description, probably even with links to the obj database.

Best,

Simon

[kleinsplash]

Nope. I have obj files. Your hypothesis is correct. I have attached one of them. Your script works just fine (is there an easy way to save this image?).

As a side note: I do collect point clouds using V-REP, and I can generate pointclouds (pcd) using pcl - but I want to work with the obj mesh files because the clouds are too sparse.

I probably could have explained myself better, point taken. I will aim to try harder next time, I feel horrid when I am asking basic questions and on top of that writing an essay.

The only other person I know who uses the term interwebz is Richard on Fast and Loud - I am an avid supporter.

[Simon Danisch]

Yeah, now worries! This is pretty much the XY problem, which easily happens to the best of us.

Just be sure to include enough context information. 

You can make a screenshot with glvisisualize with: screenshot(window, path="screenshot.png")

Screenshot of the depth map is slowly on its way... Don't want to promise anything here!

Best would be, if you file an issue, so we don't have to missuse julia-users for that discussion!

Am Freitag, 20. November 2015 16:18:46 UTC+1 schrieb kleinsplash:

[Steve Kelly]

I think most of the people working on mesh related geometry are doing it part-time and building these libraries is part of the learning experience. Asking questions is important, since it is the only way to gain insight. If you have more questions about Meshes, I'd be happy to help on the github bug tracker.

My senior thesis is on Polyhedra, and I thought it would be easy since I have been working with mesh types for over a year. It turned out I was wrong, and the needs of the application revealed a whole set of types (and more importantly relations between them) that are very important for combinatorial geometry. Stretching the application of libraries is really important, and I am sure in time you will be contributing back if you stick with it!

[Andre Bieler]

I dont know if this is still needed. But I think I did a similar thing a while ago.

One difference being I used .ply files instead of .obj files.
From what I understand it should be straight forward to transform .obj to .ply
with something like paraview or meshlab.
(It has to be ascii .ply for my example to work)

Attached is an example script that computes the intersection point of a line of sight
and a collection of triangles (your surface mesh I assume)
it returns the index of the triangle which is intersected and the coordinates of the
point of intersection.

If multiple triangles are intersected it returns the index and coordinates of the closest
triangle.

if no intersection is found it returns -1 as index and [0,0,0] as the intersection point
(this is to have the function return type stable)

I attached a zip archive containing the necessary julia file and an example
.ply file of a triangulated cow :)

All you have to define is the starting point of the line of sight "pStart"
and the direction towards it is pointing "r".
"r" has to be a unit vector


Any comments are welcome.

Cheers
Andre

[kleinsplash]

Hi,

@Simon Danish. So posted issue as suggested:  new issue. Have tried the screenshot(window, path="screenshot.png") and get ArgumentError: function screenshot does not accept keyword arguments. Where window = visualize(obj). Obviously I need to go back and look at the documentation.

@Andre Bieler this is awesome! thank you - just what I was looking for - you should add it to something, anything. Not sure if that be Meshes.jl, MeshIO.jl or Meshing.jl.

As a side note: I was directed toward using Barycentric_coordinate_system something like accurate-point-in-triangle-test.html  But will be looking at yours first.
@Steve Kelly thanks - needed that pep talk. And it was easy we probably wouldn't be doing it.

[Simon Danisch]

Sorry, that was the method definition, not the actual call!

So it's just an optional second argument!

Ah yeah, if implementing it from scratch is an option, ray tracing in its simple form is probably the most approachable (although still slow compared to visualizing it with OpenGL).

Am Freitag, 20. November 2015 16:18:46 UTC+1 schrieb kleinsplash:

[Andre Bieler]

@Simon

GLVisualize looks great! I ll try using it as soon as there

is an opportunity. (and then bother you with questions.. ;) )

@kleinsplash

Not sure if those meshing libraries are the right place to put my stuff.

I didnt even know about all these packages until I posted my answer

here. I will get rid of my custom mesh loading prcedures and start

using whats already available.

On a somewhat related note. I do have an Octree implementation

in Julia which I use for meshing/partitioning of 3D domains, maybe

that could be interesting for someone?

It generates an octree around triangulated surface meshes.

E.g. here

https://github.com/abieler/dsmc/blob/master/doc/cow_mesh2.png

If there is interest I d be happy to share/clean up my code.

[kleinsplash]

I think thats useful, especially if you are doing object detection and avoidance on a mobile platform. I wont be able to use it right this minute, but pretty sure the guys at MoveIt! could.