Hi Ben,
Just my take, and possibly different than others...
I think the more slim you are the more the sweaters run true to size. That said, being not as slim as I would like, I sized up and that approach ended up working really well for me (from an XL to 2XL). I normally wear XL shirts and t-shirts.
Only you can decide where you fall on the slim scale, or how you prefer things to fit, so advice can only go so far. That said, the sweaters are beautiful, warm and of real quality.
Bob