Aden -
I've done this one time, at the Swain's Lock (#21) campground. It's an unremarkable but totally solid camping spot, with sites for probably 6 or 7 groups. When I went I had no trouble finding a spot.
They have running water and a few porta-johns up by the canal, although the NPS site currently has a not saying "water is unavailable until further notice", so I'm not sure what the deal is with that.
Here's a bit more info:
The coolest thing about these campgrounds IMO is that they are accessible via a reasonable bike ride from downtown DC, so I was able to leave my house in Mt. Pleasant by bike and get to a place to camp in about 1.5 hours. When I went the Potomac was really high but i still had a good time picking off bluegills in the canal.
Would be curious to explore some of the spots further out.