Thanks for using shadowDOM.
To get caret position in shadow tree, you need to use ShadowRoot.caretPositionFromPoint(x, y) instead of Document.caretPositionFromPoint(x, y)
However, current Blink doesn't implement caretPositionFromPoint() on Document and ShadowRoot. Note: caretRangeFromPoint() is replaced by caretPointFromPoint() in current CSSOM specification.
Please start following bug in Blink issue tracker to raise priority of implementing caretPositionFromPoint()
Thanks!
-yosi