So I don't know if the problem is Math.net Numerics, NancyFX, bad ram, or missing server patches, but when I hit a rest service I wrote multiple times on a particular 2008R2 server the process crashes
If anyone has the time I'd appreciate if they can try it on other 2008R2 servers. I'll try to find other ones or build a VM.
As Holmes would say, the game is afoot!