Thanks everyone for the correct and prompt responses. I'm a little
smarter about TeX fonts now as a result.
The answer, as pointed out in the responses, is that the default font
defines character 92 (the default escape character) as a double left
quote. The easy solution is to select a font where character 92 is
the backslash, for example the cmtt10 font which can be used via the
\tt control sequence. Therefore {\tt\string\TeX} does the trick.
The $\backslash$ prints a backslash because math mode has different
fonts which include the backslash.
regards,
eds