you can go way bigger on A23, (or most other clincher rims for that matter)
I also have a sam hillborne, and it also came with 36h atlas hoops. I had a shop build a "lighter" 32h velocity dyad wheelset for me for the same reason. there was absolutely no noticeable difference changing wheels on that bike. i now have a roadini and sam, and swap the wheels between bikes for random reasons, boredom, fenders, tires, etc.
if you're doing it just for convenience of not needing to swap tires for different rides, that's legit. if you're goal is trying to save weight, I'd say save your money instead. tire swaps are cheaper and make a huge difference in ride feel + effeciency