That does mostly fix selection (with some word-separators tweaks), but the pane switching on click behavior is problematic for me. I wish they hadn't made mouse support all-or-nothing.
It would be nice if Terminal could use the same word break behavior as other common terminals like gnome-terminal/iterm/macos terminal, but I'm sure there are people that have gotten used to the current behavior as well so changing it might not be easy.
I think the best fix for me is to just use the extension and customize word-break-match-*.