Yup, as long as the water isn't too high it's a nice spot to wade. Tons of ledges and boulders in the section farthest upriver so watch your step if you go. If you had downstream a bit from the canoe launch, it's a bit easier wade and fishes just as well. In the summer, it definitely gets crowded.
I think if you have a full day to fish, the most efficient way to fish that area would be to do a float trip from Front Royal Canoe or run your own shuttle if possible.