That doesn't surprise me, though it's obviously undesirable. I have done no testing so far of Canviz with malformed, truncated, or garbage documents, and would not be surprised if Canviz does not handle them gracefully, even to the extent that you've observed, running into an infinite loop trying to process data that isn't there. Some browsers, including Firefox, include protection from infinite loops and other runaway JavaScript code, and prompt you asking if you would like to interrupt the script; if you do your testing in such a browser, and interrupt the script at that point and open e.g. Firebug, you would be able to see exactly what it was doing. But it also probably wouldn't be difficult to construct some test cases of bad xdot documents and see how Canviz reacts to them. It's definitely something I still plan to do eventually.
My initial eyeballing makes me think you should be using .onSuccess() not onComplete() in Ajax.Request(). I suspect that would cope with non-20x status codes, at least, but not necessarily with malformed data. I suspect the latter is relatively unlikely for dot generated .xdot.
Yes, changing onComplete() -> onSuccess works().