Thank you for the pointer.
Apparently you can just add .jar file path to sys.path and it picks up from there. I ended up writing following code(Kotlin)
this.javaClass.classLoader.getResources("mypackage/__init__.py").asSequence().forEach { url ->
if (url.protocol == "file") {
jepConfig.addIncludePaths(Paths.get(url.path).parent.parent.toString())
} else if (url.protocol == "jar") {
val path = Paths.get(url.path).parent.parent.toString().trimEnd('!')
jepConfig.addIncludePaths(path)
}
}
Works like a charm.