I think the '+' is not as prevalent as you might think.
MATLAB: ['foo' 'bar']
R: paste('foo', 'bar', sep="")
Python/C++/Ruby: "foo" + "bar"
Perl/PHP: "foo" . "bar"
C: strcat(bufcontainingfoo, "bar")
Mathematica: "foo" <> "bar"
People coming from MATLAB and R (probably a very large portion of those trying Julia) are already used to something else. There may be a lot of Python programmers as well, but Julia isn't trying to target Python users specifically.
I think having a +(String, String) defined with a helpful error would be a good idea, and I don't think it needs to be confined to the REPL (unless something in the language depends on seeing method missing errors).