There's no real reason not to- they have more than enough power (65W vs max of about 15W for Pi, which will never go near that ceiling, although it might get close if there are peripherals drawing power). So long as you use a good buck converter and you respect both the power limitations of the buck converter (mostly current draw) and thermal limits. Thermals are the biggest "downside" in that buck converters can hot, so you may need a heat sink- easiest to suck it and see for your particular situation.
The only other reason not to is that it's kind of a lot of work when USB power supplies are pretty cheap nowadays. I can find 65W GaN chargers with an AU plug on Aliexpress for about $6 (and they'll be good chargers, not pieces of crap). I'll pay that to just not to have to fiddle around with the soldering.
Jenna