Last time I checked the SpringBoot + React example (disclaimer : it was months ago), they didn't use JSX transforming (they kinda "cheated" in the GitHub repo). But they may have found a good way to handle this since :)
In my humble opinion, such an approach can be viable, at least it's worth the try. I'm worried about performance though. You mentionned one of my repos earlier in this post (the one of the registry, not using JSX iirc), I had huge performance issues while evaluating JS files. (but there was a huge amount of data to handle... so...).
And also, one key point to keep in mind is that some kind of things won't be possible, by design. For instance, I've seen people using isomorphic/universal rendering with XMLHttpRequest. That means you don't pass an initial set of props to React the first time you render the page, you simply let it make an http request. This is possible with V8/nodejs, this isn't possible with Nashorn.
I'm not sure whether such an approach is common, or if it makes sense, but this is definitely something nashorn will not be able to do. I prefer passing a set of determined props for server-side rendering. Also I'm not sure how to interact with a flux architecture like redux. In a Single Page Application, people will tend to use such an architecture. So the main idea for isomorphic rendering would be to fulfill stores with data coming from the server I guess... Not sure such thing is possible with nashorn.
Injecting data is also a potential pitfall. You have to pass a Map of objects Javascript can understand.
So... Lots of things to dig through in order to answser the "is it viable" question :)