Wim et al,
You might want to check (plot) the force.alpha value; all force layout experiments I did which exhibited a 'hard to catch with the mouse' behaviour had a problem due to the .alpha being 'reset' to 0.1 by the drag behaviour addition to the force. (On retrospect it's not a 'problem' but merely an 'artifact'. Which can be very obnoxious some times.)
You can 'debug' the alpha (and other) parameter of the force layout by plotting it as if you're doing a classic 'logger' engine (sorry, I fail at providing the proper English technical name for this; anyone who's ever been interested in his [mechanical] engineering history may recognize this one instantly and feel obliged to provide the proper general name for this little 'grapher / logger' engine:
http://www.utsic.org/2011/09/bendix-hygro-thermograph-mechanical.html ; the OG will definitely have handled these buggers and their 'infinite' graph paper rolls in research and controlled environment plants anywhere... maybe you need to ask your real OG ;-) )
For an example of this debugging, see
http://bl.ocks.org/3637711 where the top bright-blue line in the background (note the debug slider at top-left) is the force.alpha value plotted over time; wait a short while to see what happens when it hits the right edge. This is the way you debugged equipment before there was modern fancy stuff around like logic analyzers and computers in every gadget.
When your problem is force.drag behaviour related, you may also note a certain 'heartbeat'-like visual behaviour when you drag a force node around.
Anyway, the problem may be similar or identical to this scenario:
you move the mouse to 'touch' (mouseover) a moving force node. This node gets marked 'fixed' immediately, but since human movement is not robot-precise, a slight jitter exists and you happen to move outwards a pixel without noticing and the node gets un'fixed' right away; at the same moment the force.drag behaviour, which picked up your mouseover before, has already 'updated'/'reset' the alpha to 0.1 so at this mouseout the 'unfixed' node gets a copious amount of force applied, including gravity, so it 'jumps away', making it feel like it's almost 'rejecting' a mousepointer that seems to radiate a force of its own (incorrect, it's just the alpha change kicking in in a single animation frame and making the force layout 'jump' due to increased influence of forces all around).
Irrespective of whether I hit the spot here or not, start by graphically debugging your force input signals such as that force.alpha in a way similar to the referred gist; I'm sure you're in for a few surprises there. ;-)