I'm posting this because I don't understand the purpose of the code in question and I have nothing to compare it to to determine it's correctness.
The lines in question are these inside if dtObstacleAvoidanceQuery::processSample:
// RVO
float vab[3];
dtVscale(vab, vcand, 2);
dtVsub(vab, vab, vel);
dtVsub(vab, vab, cir->vel);
I get the last line which is getting the relative velocity of the obstacles.
What I don't understand is why vcand (the candidate velocity) is being scaled by 2 and then having the velocity of the agent subtracted from it.
This *appears* like an ad hoc way to combine vcand with the current velocity.
But just subtracting these velocities doesn't seem like a valid way to blend them.
Is there some mathematical explanation for this code? I'm reasonably familiar with RVO math and I can't find anything analogous with the conventional formulations.
In the case of a velocity lateral to the vcand why would it switch the velocity to the opposite side of the agent?
It seems quite easy to show it wrong in the case of a vcand of 0,0,0 which results in a combined velocity opposite to the current velocity - which is why I discovered this - I was having sweepCircleCircle detect a potential collision between objects that were moving away from each other and the agent in question was wanting to remain still (vcand was 0,0,0).
I tested another scenario where I got an agent to run through the origin and created a procedural obstacle to avoid at the origin (a dtObstacleCircle at the origin with zero velocity and setup the side vectors just like dtObstacleAvoidanceQuery::prepare), the original code above had great difficulty navigating around this simple stationary circle.
If I simplified the code to
// RVO
float vab[3];
dtVsub(vab, vcand, cir->vel);
It passed the simple stationary circle avoidance perfectly. However this results in pretty terrible avoidance in the general case.
The original code seems to work in most cases though and has been around for a long time. So I'm terribly confused.
Thanks for any assistance.