Hi Rahul,
The DITA OT's own code uses new Saxon API and you simply cannot replace the Saxon libraries with older versions of themselves and expect for the publishing engine to work anymore.
With the new Saxon libraries, in order to call custom Java methods from the XSLT code you will need to create Saxon extension function definitions in your Java code.
As an example:
My plugin.xml contributes an extra JAR library. The JAR library contains inside a Saxon extension function class:
The "latexToSVG.jar" library also contains inside a META-INF/services folder with a file defining that class as a Saxon extension.
You can look inside the XSLTs to see how the class is further called.
Regards,
Radu