Heightfield Bug

268 views
Skip to first unread message

Oleh Derevenko

unread,
Mar 27, 2011, 11:24:03 AM3/27/11
to ODE Group
Guys,
 
Some time ago I've been contacted via e-mail by Erik Schuitema <e.sch...@tudelft.nl>. He told me he has identified and fixed the bug in heightfield everybody has been talking about for so long but he only made implementation for spheres (the only kind of geojmetry he needed) and did not have spare time and enthusiasm to finish all the work and/or become ODE developer. So he suggested if I had time to look at his changes and integrate them into ODE having made corrections as necessary and having finished what was missing from those changes.
However during the work on the sources we have come to a problem we need to consult with the community about and so I suggested (and Erik agreed) to publish our conversations here in mailing list and ask everybody to tell what are their thoughts on it.

I include two mail threads (letter order has been reversed for convenient reading) along with pictures that were included in them.
I also include original source changes by Erik. I've already started making some minor corrections/improvements to those but suspended the work until it is decided how to proceed.
Please review the conversations, look at the source changes and let us know what you think would be the best approach to solving this.
 
============== Begin of Thread 1 ======================
------------------ Letter 1 ------------------
----- Original Message -----
From: "Erik Schuitema" <...>
To: "Oleh Derevenko" <...>
Sent: Thursday, February 17, 2011 6:35 PM
Subject: ODE heightfield patch

Hi Oleh,

I created a working version for heightfield-sphere collisions in ODE. In
v0.11.1, a sphere would fall through the heightfield when it passes a
convex edge. The larger the angle between triangles, the greater the
virtual 'gap'. But I fixed it now.

Unfortunately, I don't have enough time to create a nice patch or become
an active ODE developer, while you seem to be working on the heightfield
code (and other parts of ODE) nowadays.

Can I send/discuss my patch with you so that you can integrate it in a
nice way so that it will also work (at least not harm) collisions with
other objects? Or should I perhaps just submit it as a patch through the
sourceforge tracker and mail my discussion to the mailing list?

Best regards,
Erik

Erik Schuitema, PhD candidate
Delft University of Technology, The Netherlands
http://www.dbl.tudelft.nl/

------------------ Letter 2 ------------------
From: "Oleh Derevenko" <...>
To: <...>
Sent: Thursday, February 17, 2011 6:40 PM
Subject: Re: ODE heightfield patch

I'm not very familiar with collision detection mathematics - I just fix
obvious algorythmic bugs. However, if it is not large code change and there
is a chance I could be able to figure out how to integrate it - send it to
me and I'll have a look at it.

Oleh Derevenko
-- Skype with underscore
 
--------------------------------------------
[Several letters of no interest been skipped here. Erik has first sent me initial version of his fix but then continued making fixes to the sources and so I discard those initial files from here.]
[Attached Readme: README_ODE_heightfield_patch.txt]
 
------------------ Letter 3 ------------------
From: "Erik Schuitema" <...>
To: "Oleh Derevenko" <...>
Sent: Monday, March 07, 2011 6:24 PM
Subject: Re: ODE heightfield patch

Hi Oleh,

I thoroughly tested the heightfield after some more changes, and it
really works fine for spheres. However, I find it difficult to predict
whether it will work fine for other geoms too, because of the lack of
good edge collider functions. Currently the code uses ray colliders but
that's not what we want. But at least, it works for one type of geoms now.

I think I won't make any further changes, since it works sufficiently
for my purposes. I will clean up the code (remove debug messages) and
send it to you soon.

Erik
------------------ Letter 4 ------------------
From: "Erik Schuitema" <...>
To: "Oleh Derevenko" <...>
Sent: Wednesday, March 09, 2011 7:58 PM
Subject: Re: ODE heightfield patch

Hi Oleh,

Hereby I send you the current status. I don't have more time
unfortunately to extend the heightfield further than this. As far as I
can see, it works flawlessly for spheres. I put as much comments in the
code as I could to explain what I did and what remains to be done.

Please contact me (you or the author of heightfield) if you have any
further questions or remarks. I put a lot of thought in it and I'd be
happy to further illustrate my code decisions or future directions.

Best regards,
Erik
[Attached Source: heightfield.cpp]
[Attached Header: heightfield.h]
 
============== End of Thread 1 ======================
 
 
 
 
============== Begin of Thread 2 ======================
------------------ Letter 1 ------------------
From: "Oleh Derevenko" <...>
To: <...>
Sent: Saturday, March 12, 2011 6:22 PM
Subject: Re: ODE heightfield patch

Hi Erik,

Why was not it possible to continue using ray colliders? I guess a ray as a
generalization of line segment can always be reduced back. For exanple if
ray returns its contacts way too far (further than the length of segment
from ray's origin) you can always discard them and act as if there were no
contacts. But this way you would not have to reinvent colliders for all the
types of geometries.

Oleh Derevenko
-- Skype with underscore

------------------ Letter 2 ------------------
From: "Erik Schuitema" <...>
To: "Oleh Derevenko" <...>
Sent: Sunday, March 13, 2011 2:37 AM
Subject: Re: ODE heightfield patch

Hi Oleh,

The purpose of ray colliders is to find the intersection point (either
entrance or exit) of a ray and a geom. A ray collider "shoots a hole"
into the geom and returns that point. However, what I find necessary
with triangle edges collisions is to find the point of maximum
penetration of the line segment into the geom, which is different. The
easiest way to see the difference is to examine a sphere penetrating a
segment. While the ray collider would return a contact somewhere on the
side of the sphere and bounce the sphere back in an odd way (probably
making the sphere rotate), the segment collider returns the point of
maximum penetration, which is closest to the sphere's center. This
contact will nicely bounce the sphere off the contact in the direction
perpendicular to the segment, with the force pointing right through the
center of the sphere.

Best regards,
Erik


------------------ Letter 3 ------------------
From: "Oleh Derevenko" <...>
To: <...>
Sent: Sunday, March 13, 2011 5:03 PM
Subject: Re: ODE heightfield patch

Erik,

It seems to me you have wrong inderstanding how ODE contacts work. ODE does
not rely on a single contact. It rather would gather all the contacts of all
the rays on the surface of the sphere and then they together would add up
their forces to move sphere out of collision. Surely, if you use ray
collider the contact's normal need to be corrected to be a bissector of
angle between faces and perpendicular to the edge itself. That's because the
ray itself "does not know" you use it for triange edges. But if you only use
contact location and generate direction manually that should give you
correct forces in result.

As for your approach to check whether a segment is inside of the sphere -
that violates ODE priniples altogether. Because nowhere in ODE contacts are
generated if one object is entirely contained in another. Contacts are
generated on surface intersections only.

I'll try to make corrections to your code to use rays and then send you back
what I obtain for you to test and let me know if it still works for you.

Oleh Derevenko
-- Skype with underscore


------------------ Letter 4 ------------------
From: "Erik Schuitema" <...>
To: "Oleh Derevenko" <...>
Sent: Sunday, March 13, 2011 8:38 PM
Subject: Re: ODE heightfield patch

Hi Oleh,
Comments within your text..

On 13/3/2011 15:03, Oleh Derevenko wrote:
> Erik,
>
> It seems to me you have wrong inderstanding how ODE contacts work. ODE
> does not rely on a single contact. It rather would gather all the
> contacts of all the rays on the surface of the sphere and then they
> together would add up their forces to move sphere out of collision.
I know very well how ODE works. After my university background in
Applied Physics, I've used it for 5 years now. So it's probably just a
misunderstanding between us.

I attached an illustration of what I meant in my previous email.
Although technically, the entrance and exit contact of the ray (you
would have to calculate them both to begin with) would together push the
sphere out of the line segment, the physical effect is different from a
single contact. The spring-damper properties of specific values for the
contact's ERP and CFM have been carefully chosen in all my simulations.
When suddenly adding an extra contact when the sphere rolls from a plane
onto an edge, the stifness doubles, which causes a false discontinuity
in the simulation's behavior. So you cannot just add an extra contact
just because you pass a line. The best way to test this is to set ERP
and CFM to represent a fairly low stifness (and thus significant
penetration of the sphere into the heightfield) and check the continuity
when passing an edge. I've run extensive tests like this.

Multiple contacts are of course allowed - and often necessary - but only
when there's a good reason for it: the geom must touch multiple planes,
edges or vertices.

> Surely, if you use ray collider the contact's normal need to be
> corrected to be a bissector of angle between faces and perpendicular
> to the edge itself. That's because the ray itself "does not know" you
> use it for triange edges. But if you only use contact location and
> generate direction manually that should give you correct forces in
> result.
Unfortunately, this is not true. When a sphere rolls over an inclining
plane, reaching an edge, and going down afterwards, the contact normal
on the edge continuously changes and always points exactly through the
center of the plane - not in a fixed direction, e.g., the bisector of
the angle of the two bordering planes. I will make a movie of the
correct behavior on monday, when I'm back at work.
> As for your approach to check whether a segment is inside of the
> sphere - that violates ODE priniples altogether. Because nowhere in
> ODE contacts are generated if one object is entirely contained in
> another. Contacts are generated on surface intersections only.
I think we have a misunderstanding again. I meant to say that the
segment penetrates the sphere, and we need to find the point of maximum
penetration. ODE contacts rely on penetration. I didn't mean full
enclosure of the segment.
>
> I'll try to make corrections to your code to use rays and then send
> you back what I obtain for you to test and let me know if it still
> works for you.
Unfortunately it will not work. I'll send you a movie of the correct
physical behavior tomorrow.

A 'hack' could be to use ray colliders and calculate both the entrance
and exit contact, and merge the contacts. For spheres, this could work
well if you manage to correctly calculate the normal. For other geoms,
it's a not-so-good approximation, but better than nothing maybe.

Best regards,
Erik

[Attached Picture: heightfield_segment_illustration.png]
 
 
------------------ Letter 5 ------------------
From: "Oleh Derevenko" <...>
To: <...>
Sent: Sunday, March 13, 2011 10:17 PM
Subject: Re: ODE heightfield patch

Hi Erik

> Unfortunately, this is not true. When a sphere rolls over an inclining
> plane, reaching an edge, and going down afterwards, the contact normal
> on the edge continuously changes and always points exactly through the
> center of the plane - not in a fixed direction, e.g., the bisector of
> the angle of the two bordering planes. I will make a movie of the
> correct behavior on monday, when I'm back at work.

Well, if you generated contact force always towards the object's centre you
could easily achieve the situation when you would drop a cube with a
diagonal down exactly onto a convex heightfield edge and it would stop like
that and never fall.

See attachment for illustration.

Oleh Derevenko
-- Skype with underscore
[Attached Picture: cube_on_edge.png]
 
 
------------------ Letter 6 ------------------
From: "Erik Schuitema" <...>
To: "Oleh Derevenko" <...>
Sent: Monday, March 14, 2011 9:28 PM
Subject: Re: ODE heightfield patch

Hi Oleh,

On 03/13/2011 08:17 PM, Oleh Derevenko wrote:
> Hi Erik
>
> Well, if you generated contact force always towards the object's
> centre you could easily achieve the situation when you would drop a
> cube with a diagonal down exactly onto a convex heightfield edge and
> it would stop like that and never fall.
It is only coincidence that for the sphere it points through the center
(so my implementation is not generic). I tried to capture the general
idea in the attached illustration of a box with exaggerated penetration.
Let me know if it's clear (my painting skills suck) and whether you
would agree.

Apart from that, if your cube is exactly in balance on the edge (as it
would be on a needle tip), the angle of the planes forming the edge is
irrelevant and the cube would indeed never fall.

I created a movie of a rolling sphere, again with exaggerated
penetration. Please note the continuity of the contact - no jumps in
penetration depth or position: http://www.essd.nl/tu/heightfield_correct.avi
I also created the behavior of the original, flawed heightfield (but we
already agreed it was flawed :)):
http://www.essd.nl/tu/heightfield_flawed.avi

Best regards,
Erik
 
[Attached Picture: heightfield_illustration02.png]
 
 
------------------ Letter 7 ------------------
From: "Oleh Derevenko" <...>
To: <...>
Sent: Wednesday, March 16, 2011 1:51 PM
Subject: Re: ODE heightfield patch

Hi Erik,

I guess I understand your idea. But to implement colliders for edge and
vertex against all the possible shapes would be a fairly complex task (if
not impossible at all) and nobody would support it. Generally speaking you
need to estimate shape of penetrating part, find the component which
penetrates the most (would it be a face, an edge or a vertex) and generate
contact from it to the current erge or verted of heightfield.
I have a different idea which could be easier to implement. If face (plane)
collisions are fine, and it is only necessary to "cover the gap" of
disconuity between face angles during transition over the edge it could be
usible to use plane colliders alone.
It's necessary to implement iterative process by generating virtual
rectangular faces along the edge of interest. First you start with tilt
angle being average of two edge's neighbour faces. You canclulate the
contact point, project it onto virtual face and then increase or decrease
tilt of virtual face by divide-by-half descending angles into the direction
of conatct projection to obtain contact point as close to the edge as
possible. That should have similar effect and work well by the cost of
excessive collision checks.
If vertex collisions are OK they could be left as they are or similar
virtual plave iterations (but in 2D) could be implemented for them too.

Let me know what you think about this.

Oleh Derevenko
-- Skype with underscore
 
------------------ Letter 8 ------------------
From: "Erik Schuitema" <...>
To: "Oleh Derevenko" <...>
Sent: Thursday, March 17, 2011 1:37 AM
Subject: Re: ODE heightfield patch

Hi Oleh,

I think we shouldn't need to reinvent the wheel for the heightfield. The
problems we face should be similar to the ones solved in trimesh
colliders. For example, the code of the edge-sphere collider in my
heightfield version is very similar to
gim_geometry.h:CLOSEST_POINT_ON_SEGMENT() from ODE's own trimesh code.
Hopefully you can find more solutions or hints in the trimesh code.

Please note that if you would use a 'fake plane' to detect collisions
with an edge, the drawn example from my previous email would return two
contacts - the corners of the box - and none above the edge...

I think the depth getters for vertex collision detection should be fine,
although I haven't really tested them besides the one for spheres.

Best regards,
Erik

------------------ Letter 9 ------------------
Sent: Sunday, March 20, 2011 5:04 PM
Subject: Re: ODE heightfield patch

Hi Erik,
 
----- Original Message -----
From: "Erik Schuitema" <...>
 
> I think we shouldn't need to reinvent the wheel for the heightfield. The
> problems we face should be similar to the ones solved in trimesh
> colliders. For example, the code of the edge-sphere collider in my
> heightfield version is very similar to
> gim_geometry.h:CLOSEST_POINT_ON_SEGMENT() from ODE's own trimesh code.
> Hopefully you can find more solutions or hints in the trimesh code.
 
Unfortunately I'm not so strong in collision detection theory to reassemble anything from pieces of other colliders. So, I would not be able to do it (with reasonable time and work efforts). And my personal impression is that the problem of finding a single contact application entity (a face, an edge or a vertex) for a heightfield edge segment is not a well defined problem. You are just lucky to only need the simplest shape - the sphere. And for more complex shapes there will be ambiguities regarding what is to be chosen as a contact force application point.
And I'm personally not sure if it is reasonable to implement whole set of colliders for heightfield edge segments. I would prefer reusing existing colliders and merging/culling contacts they generate into a single point if necessary/possible (if that is of great importance).
So, what I can suggest is to make our letters available to public at ODE mailing list (I can do that if you don't object) and listen what other people would say.

> Please note that if you would use a 'fake plane' to detect collisions
> with an edge, the drawn example from my previous email would return two
> contacts - the corners of the box - and none above the edge...
 
That means that the tilt angle of the plane has been chosen wrong and must be adjusted until one of the corners gets below the edge. You just don't have any warranty you'll get the deepest penetration corner there. So for such cases of multiple contact points generated by plane it could be necessary to think of some additional means to obtain the correct point. For example you may chose to get all the contacts and manually select the contact point with largest distance from erge (those distances should not change while you alter plane tilt). Then you just iterate tilt to get that chosen point under the edge. Well, it actually even not necessary to iterate at all - the tilt angle could be calculated directly. The iteration would only be needed in assumption that contact point may travel over the geometry while plane tilt changes. So, this problem needs some more thinking about.
 
> I think the depth getters for vertex collision detection should be fine,
> although I haven't really tested them besides the one for spheres.

Oleh Derevenko
-- Skype with underscore

 
------------------ Letter 10 ------------------
Sent: Sunday, March 20, 2011 7:42 PM
Subject: Re: ODE heightfield patch

Hi Oleh,

On 3/20/2011 3:04 PM, Oleh Derevenko wrote:
Unfortunately I'm not so strong in collision detection theory to reassemble anything from pieces of other colliders. So, I would not be able to do it (with reasonable time and work efforts). And my personal impression is that the problem of finding a single contact application entity (a face, an edge or a vertex) for a heightfield edge segment is not a well defined problem. You are just lucky to only need the simplest shape - the sphere. And for more complex shapes there will be ambiguities regarding what is to be chosen as a contact force application point.
For me, physical correctness is a very important feature of ODE - it shouldn't become a 'game engine' type of environment that just looks good, but is actually not. I agree that it can be a tough job to do it correctly, but I think that's the only way to get the heightfield widely accepted as a geom. My feeling is that there is no ambiguity in how geoms should collide with edges and vertices.
And I'm personally not sure if it is reasonable to implement whole set of colliders for heightfield edge segments. I would prefer reusing existing colliders and merging/culling contacts they generate into a single point if necessary/possible (if that is of great importance).
So, what I can suggest is to make our letters available to public at ODE mailing list (I can do that if you don't object) and listen what other people would say.
I think that's a very good idea to make our letters available, please do.

That means that the tilt angle of the plane has been chosen wrong and must be adjusted until one of the corners gets below the edge. You just don't have any warranty you'll get the deepest penetration corner there. So for such cases of multiple contact points generated by plane it could be necessary to think of some additional means to obtain the correct point. For example you may chose to get all the contacts and manually select the contact point with largest distance from erge (those distances should not change while you alter plane tilt). Then you just iterate tilt to get that chosen point under the edge. Well, it actually even not necessary to iterate at all - the tilt angle could be calculated directly. The iteration would only be needed in assumption that contact point may travel over the geometry while plane tilt changes. So, this problem needs some more thinking about.
I'm not sure if I understand you correctly. I assumed you meant collision with a plane that stretches to infinity and has no edges?

Best regards,
Erik
 
------------------ Letter 11 ------------------
Sent: Sunday, March 20, 2011 8:10 PM
Subject: Re: ODE heightfield patch

Hi
----- Original Message -----
I'm not sure if I understand you correctly. I assumed you meant collision with a plane that stretches to infinity and has no edges?
Yes, a plane. But that's a plane which passes through the edge we are checking collision against and it therefore has just one degree of freedom - a rotation angle around edge's axis. And if the plane cuts two corners of a box and they are generated as collision points the contact directions should be by normal to the plane regardless of the actual plane's tilt angle value. And if you choose the plane angle so that the contact point projects by its direction onto the edge, that should be nearly the same effect as that of collision you are willing to implement. It looks like you do a kind of filet for the faces' angle cutting it and replacing with a series of planes which gradually transit from one face to another, having that way a singe discontinuity replaced with multiple smaller discontinuities.

Oleh Derevenko
-- Skype with underscore
 
============== End of Thread 2 ======================
 
 
Thank you very much for spending your time to read all the conversation (if you did). ;)
Your feedback would be appreciated.

Oleh Derevenko
-- Skype with underscore
 
 
README_ODE_heightfield_patch.txt
heightfield.cpp
heightfield.h
heightfield_segment_illustration.png
cube_on_edge.png
heightfield_illustration02.png

Martijn Buijs

unread,
Mar 29, 2011, 11:05:38 AM3/29/11
to ode-...@googlegroups.com
On 27-Mar-11 5:24 PM, Oleh Derevenko wrote:
> Guys,
> Some time ago I've been contacted via e-mail by Erik Schuitema <e.sch...@tudelft.nl
> <mailto:e.sch...@tudelft.nl>>. He told me he has identified and fixed the bug in heightfield

> everybody has been talking about for so long but he only made implementation for spheres (the only
> kind of geojmetry he needed) and did not have spare time and enthusiasm to finish all the work
> and/or become ODE developer. So he suggested if I had time to look at his changes and integrate them
> into ODE having made corrections as necessary and having finished what was missing from those changes.
> However during the work on the sources we have come to a problem we need to consult with the
> community about and so I suggested (and Erik agreed) to publish our conversations here in mailing
> list and ask everybody to tell what are their thoughts on it.
>
> I include two mail threads (letter order has been reversed for convenient reading) along with
> pictures that were included in them.
> I also include original source changes by Erik. I've already started making some minor
> corrections/improvements to those but suspended the work until it is decided how to proceed.
> Please review the conversations, look at the source changes and let us know what you think would be
> the best approach to solving this.

Interesting.

I haven't read everything in detail, but from what I understand this seems to be happening?

http://www.bytehazard.com/_temp/terrain_edge.jpg

a - small angle difference, small gap
b - bigger angle difference, bigger gap
c - how it should be, no gap

The contacts are projected to the plane properly, but the "point-in-triangle" test is flawed because
the triangle vertices (thus edges) are not correctly projected onto the plane?

/Martijn

Oleh Derevenko

unread,
Apr 5, 2011, 4:24:21 AM4/5/11
to ODE Group
Yes, as far as I understand that is the effect. However I did not dig to the
roots of the problem to understand it entirely.
If somebody makes a fix for that I can make minor corrections/optimizations
afterwards. Or I can try to implement iterative "divide-by-2" algorithm with
plane colliders integrated into code by Erik (that would be quite a
sophisticated thing though, since iteration target may change on each step
and it would be necessary to work on logic to avoid obtaining
inconsistent/intermittent solutions). In any case, don't ask me "is that the
wrong thing?" since I don't know the theory well enough.

Oleh Derevenko
-- Skype with underscore

Reply all
Reply to author
Forward
0 new messages