NumericalCalculus`ND uses Richardson extrapolation to estimate the derivative, so e.g. ND[Sin[x], x, 5] has the following equivalent one-line implementation in Mathematica:
Part of this is due to extra overheads for ND like processing user input and options. Even taking this into account, however, the InterpolatingPolynomial-based implementation above is still about twice as fast. I've modfied my NDerivative package to use this method instead of NumericalCalculus`ND and I've also added a drop-in replacement for ND, called FastND.