There are two related problems with drilled holes based on how the board is placed into the drilling machine:
1) the per-board registration on the holes might not be reliably consistent between boards, and
2) the relationship between the holes and the pads is not predictable - ie, the drill hole center line cross might not be the same as the pad's center line cross.
I've had boards where the drill holes were off center "north/west" on the top left of a board and off center "south/west" on the bottom right - the board was slightly skewed clockwise and offset on the drill machine. Other boards differed within a batch, as they presumably were cut out of larger boards drilled on different machines/runs - the relationship between board edges and drill holes is not always as precise as one could hope. As for silk screen on some, I've seen graffiti on a warehouse with better fidelity!
All these boil down to poor registration tolerances in the low cost fast as possible board fab lines we use :-) To be fair, if you are placing 0805's or larger, the holes-for-registration may be good enough; my experience says you will end up being frustrated when you move to smaller components and higher pitch devices.
On the other hand, the fab process for pads and etching on a board is very precise, which is why Fids/Marks are often specified as copper circles with concentric copper rings, with no solder mask or silk or drill activity to muck them up.