For troubleshooting your issue, can you photograph your setup or post your model somewhere? Is the position of the hole misaligned with respect to other features that were milled in the same setup?
If you need to bore a hole in a specific location based on existing stock geometry, you could treat the Pocket NC as a "3+0" machine. Zero our your G54 A-axis offset once you've manually commanded the table to face the spindle. Then you'd use whatever edge- or feature-finding setup you like, zero out the rest of your DROs, and run a program with an origin and work coordinate system set up as though you were running a 3-axis mill.
More generally, the Fusion 360 Bore command interpolates a (bigger) hole with a (smaller) mill, so you don't need an infinite number of drill sizes. The movement in X and Y axes are expected.
As far as rotating B goes, that would only help if the center of the hole or circular feature were exactly in line with the center of rotation of the table. If you had repeatable fixtures, it could work, but you'd have to move the stock to change the position of the hole. It's pretty niche for Fusion to do, but you ought to be able to program such a thing manually. It would be one continuous move of Z and B. Using G93 inverse time feedrate mode would be a good choice.