Not quite. The problem is that there's also a Jacobian floating around from the transformation from x to cos(x - mu). In fact, there are an infinite number of Jacobians because the transformation has to be evaluated at every zero, in this case cos(x - mu + 2pi), so you have to increment lp an infinite number of times. This is actually a known problem in the circular statistics literature, and it arises because we sample from a non-compact space (R^{n}) where as the von Mises lives on a compact space (S^{1}). Any transformation between the two topologies causes problems that either end up requiring things like complex numbers.
Fortunately Ben and Marcus realized that you could sample from S^{1} uniformly by using a trick that involves sampling two gaussians (actually, it can be two of any variate defined on all of R) and then normalizing. You should be able to just use the code they were developing for the unit_vector type (did that ever get implemented?) in 2D and augment with a the Radon-Nikodym derivative between the von-Mises and uniform-on-a-circle distribution. Which in this case I think just reduces to the density you quoted above.