Hello TJ,
Thanks for your interest in Open edX.
Unfortunately, the codebase has inconsistent support for HTML emails. As you mention, bulk email is HTML, however most of the other workflow emails (e.g. activation, password reset, etc.) are plain text.
I'm not sure if this is on edX's near-term roadmap to address, but - if you have the ability to write Python/Django - it should be relatively straightforward to add this functionality yourself (and contribute it back :-) )
Best wishes,
Chris Dodge