Issue 159 in xdocreport: Can not find DOCX converters in OSGi container.

[xdocr...@googlecode.com]

Status: New
Owner: ----
Labels: Type-Defect Priority-Medium

New issue 159 by wilkinso...@gmail.com: Can not find DOCX converters in
OSGi container.
http://code.google.com/p/xdocreport/issues/detail?id=159

Hi,

I have a felix osgi container that has the following relevant bundles
active and fragments resolved.

-> find itext
START LEVEL 60
ID State Level Name
[ 15] [Active ] [ 30] Lowagie iText (2.0.8)
-> find xdocreport
START LEVEL 60
ID State Level Name
[ 52] [Active ] [ 30] fr.opensagres.xdocreport.converter (0.9.8)
[ 53] [Active ] [ 30] fr.opensagres.xdocreport.core (0.9.8)
[ 54] [Active ] [ 30] fr.opensagres.xdocreport.document (0.9.8)
[ 55] [Resolved ] [ 30] fr.opensagres.xdocreport.document.docx (0.9.8)
[ 56] [Active ] [ 30] fr.opensagres.xdocreport.template (0.9.8)
[ 57] [Resolved ] [ 30] fr.opensagres.xdocreport.template.freemarker
(0.9.8)

I have some code that in another active bundle that calls

Options options =
Options.getFrom(DocumentKind.DOCX).getTo(ConverterTypeTo.PDF).via(ConverterTypeVia.ITEXT);
report.convert(context, options, fos);

To convert a docx file to pds via itext.

However when report.convert(..) is called there is an exception.

2012-10-02 17:30:28,366 159360 [Thread-14] ERROR
fr.opensagres.xdocreport.core.registry.AbstractRegistry - Cannot find
converters from=DOCX
2012-10-02 17:30:28,381 159375 [Thread-14] ERROR
com.experian.eda.component.workflow.test.runtime.builder.WorkflowTestRuntimeBuilder
- Document Template component Accept:encountered content which could not be
converted correctly
fr.opensagres.xdocreport.converter.XDocConverterException: Cannot find
converters from=DOCX
at
fr.opensagres.xdocreport.converter.ConverterRegistry.internalFindConverter(ConverterRegistry.java:121)
at
fr.opensagres.xdocreport.converter.ConverterRegistry.findConverter(ConverterRegistry.java:74)
at
fr.opensagres.xdocreport.document.AbstractXDocReport.getConverter(AbstractXDocReport.java:645)
at
fr.opensagres.xdocreport.document.AbstractXDocReport.convert(AbstractXDocReport.java:662)


Which implies that no DOCX converters have been found.

However the the relevent xdocreport bundles are active and so is iText.

Looking at the code it is trying to instantiate the xdoc converter which
depends on the itext bundle using ServiceLoader

http://code.google.com/p/xdocreport/source/browse/core/fr.opensagres.xdocreport.core/src/main/java/fr/opensagres/xdocreport/core/registry/AbstractRegistry.java?name=xdocreport-0.9.8

Now following comment in the code indicates that it should work in an OSGi
container.
// getClass().getClassLoader() to work under OSGi context
Iterator<Discovery> discoveries = ServiceRegistry.lookupProviders(
registryType, getClass().getClassLoader() );

This page
http://code.google.com/p/xdocreport/wiki/DocxReportingJavaMainConverter
indicates that we should be using itext 2.1.7 but the most recent osgi
itext far that i can find in itext 2.0.8. If I were to use 2.1.7 i would
have to wrap itext myself.

There are no error messages that indicate that itext 2.0.8 shouldn't work.

Could you tell me if this is a bug, or I have I missed something and just
need to do some more configuration to get it to work in osgi container.

Regards
Phil Wilkinson.

[xdocr...@googlecode.com]

Updates:
Status: Accepted
Owner: angelo.z...@gmail.com

Comment #1 on issue 159 by angelo.z...@gmail.com: Can not find DOCX

converters in OSGi container.
http://code.google.com/p/xdocreport/issues/detail?id=159

Hi Phil,

At first I know that our documentation is very bad about our converter. As
soon as we will release 1.0.0, I will update it.
https://oss.sonatype.org/content/repositories/snapshots/fr/opensagres/xdocreport/)
I suggest you to use 1.0.0-SNAPSHOT (use maven for that or download at hand
on the for docx converter because teh converter is very very lot improved
(I'm improving again). The docx converter 0.9.8 is very bad.


Lik eexplained at
http://code.google.com/p/xdocreport/wiki/DocxReportingJavaMainConverter you
need iText and our iText extension, POI, our POI converter and frament
fr.opensagres.xdocreport.converter.docx.xwpf which register the POI
converter in the ServiceRegistry.

Pay attention, if you wish to use 1.0.0-SNAPSHOT you need :

1) Docx Converter JARs

fr.opensagres.xdocreport.converter.docx.xwpf =>
https://oss.sonatype.org/content/repositories/snapshots/fr/opensagres/xdocreport/fr.opensagres.xdocreport.converter.docx.xwpf/1.0.0-SNAPSHOT/

2) Docx Converter JARs

org.apache.poi.xwpf.converters.core =>
https://oss.sonatype.org/content/repositories/snapshots/fr/opensagres/xdocreport/org.apache.poi.xwpf.converters.core/1.0.0-SNAPSHOT/

org.apache.poi.xwpf.converters.pdf=>
https://oss.sonatype.org/content/repositories/snapshots/fr/opensagres/xdocreport/org.apache.poi.xwpf.converters.pdf/1.0.0-SNAPSHOT/

fr.opensagres.xdocreport.itext.extension =>
https://oss.sonatype.org/content/repositories/snapshots/fr/opensagres/xdocreport/fr.opensagres.xdocreport.itext.extension/1.0.0-SNAPSHOT/

4) POI JARs

I know it's complicated for the moment, but once release will be done it
will have zip with docx converter with the whole JARs.

But if you use maven it's will be easy to use 1.0.0-SNAPSHOT

Regards Angelo

[xdocr...@googlecode.com]

Comment #2 on issue 159 by wilkinso...@gmail.com: Can not find DOCX

converters in OSGi container.
http://code.google.com/p/xdocreport/issues/detail?id=159

Thanks for you're reply Angelo.

Can I just ask you to be specific, will 0.9.8 work in an OSGi felix
container converting docx to pdf via itext, and what are the runtime
bundles\fragments\jars it needs to work.

(are they....

Lowagie iText (2.0.8)
fr.opensagres.xdocreport.converter (0.9.8)
fr.opensagres.xdocreport.core (0.9.8)
fr.opensagres.xdocreport.document (0.9.8)
fr.opensagres.xdocreport.document.docx (0.9.8)
fr.opensagres.xdocreport.template (0.9.8)
fr.opensagres.xdocreport.template.freemarker (0.9.8)
fr.opensagres.xdocreport.converter.docx.xwpf (0.9.8)
fr.opensagres.xdocreport.itext.extension (0.9.8)
org.apache.poi.xwpf.converters.core (0.9.8)
org.apache.poi.xwpf.converters.pdf (0.9.8)

.. are there more runtime dependencies?)

if 0.9.8 doesn't work then will 1.0.0?
and if we need to work with 1.0.0, when will it be released?

The itext bundle, can I use 2.0.8 from springsource, or do I have to wrap
it myself, or possibly put it in the osgi containers lib directory so it
goes on the osgi container class path (the nuclear option :( ).

Really sorry for all the questions, but we've spent upwards of 30 hours
trying to get get this working, and my boss is telling me to look at at
possible libre office solution. but i feel we are really close to getting
it xdocreport working.

Regards
Phil W.

[xdocr...@googlecode.com]

Comment #3 on issue 159 by angelo.z...@gmail.com: Can not find DOCX

converters in OSGi container.
http://code.google.com/p/xdocreport/issues/detail?id=159

> Can I just ask you to be specific, will 0.9.8 work in an OSGi felix
> container >converting docx to pdf via itext, and what are the runtime
> bundles\fragments\jars it >needs to work.

It should work on felix because we have JUnit Test which use OSGi container
(felix perhaps? It's Pascal who has developped that).

In 0.9.8, it's org.apache.poi.xwpf.converters (0.9.8) and not

org.apache.poi.xwpf.converters.core (1.0.0)
org.apache.poi.xwpf.converters.pdf (1.0.0)

> if 0.9.8 doesn't work then will 1.0.0?

it should work.

> and if we need to work with 1.0.0, when will it be released?

I don't know when we do that (we develop XDocReport on our spare time and I
would like improve more docx->pdf converter).

As I said you, if you maven it will be easy for you (it will download the
whole dependencies)

> The itext bundle, can I use 2.0.8 from springsource

I don't know.

Pascal do you know that?

I suggest you to download this sample zip which contains the whole JARs for
docx reporting with velocity (just remove velocity jars because it seems
that you use Freemarker) and convert PDF/XHTML =>

http://code.google.com/p/xdocreport/downloads/detail?name=docxandvelocity.converters-0.9.8-sample.zip&can=2&q=#makechanges

Regards Angelo

[Pascal Leclercq]

Hi,

becuse we developped a eclipse/RAP (http://xdocreport-rap.opensagres.cloudbees.net/xdocreport?startup=fr.opensagres.xdocreport.eclipse.ui.application)  demo application we needed to have all the jars required by xdcreport under a OSGi format 

Some of these jars are not provided as OSGi by default so I decide to create them "statically"

You can find them here.

http://code.google.com/p/xdocreport/source/browse?repo=p2#git%2Fexternals%2Fplugins


They are stoded under a p2 layout repo format but you can download them separetly.

Hope you enjoy xdocreport.

AFAIK you are the first that ask question on xdocreport under OSGi environment : your feedback is welcome !!!

Best regards.

2012/10/3 <xdocr...@googlecode.com>

--
Pascal Leclercq

[xdocr...@googlecode.com]

Comment #4 on issue 159 by wilkinso...@gmail.com: Can not find DOCX

converters in OSGi container.
http://code.google.com/p/xdocreport/issues/detail?id=159

Hi,

The good news is that I’ve got the xdocreport working creating a pdf with
itext.
The bad news is that I had to modify your docx.xwpf-0.9.8 bundle to get it
working.

To get it working I took your list of jars from the zip file
(docxandvelocity.converters-0.9.8-sample.zip) excluding the velocity and
adding freemaker. [thanks for providing that by the way]

I used the following bundles in the OSGi container (in brackets if not
already an osgi bundle)

commons-codec-1.5.jar (com.springsource.org.apache.commons.codec-1.4.0.jar)
commons-collections-3.2.1.jar
(com.springsource.org.apache.commons.collections-3.2.1.jar)
commons-lang-2.4.jar (com.springsource.org.apache.commons.lang-2.6.0.jar)
dom4j-1.6.1.jar (com.springsource.org.dom4j-1.6.1.jar)
fr.opensagres.xdocreport.converter-0.9.8.jar
fr.opensagres.xdocreport.converter.docx.xwpf-0.9.8.jar
(modified-fr.opensagres.xdocreport.converter.docx.xwpf-0.9.8- modified
MANIFEST.MF to import-package fr.opensagres.xdocreport.core.utils)
fr.opensagres.xdocreport.core-0.9.8.jar
fr.opensagres.xdocreport.document-0.9.8.jar
fr.opensagres.xdocreport.document.docx-0.9.8.jar
fr.opensagres.xdocreport.itext.extension-0.9.8.jar
fr.opensagres.xdocreport.template-0.9.8.jar
fr.opensagres.xdocreport.template.freemarker-0.9.8.jar
itext-2.1.7.jar (wrapper-itext-2.1.7.jar - created custom shade wrapper)
ooxml-schemas-1.1.jar (poi-osgi-wrapper-3.8.jar - created custom shade
wrapper)
org.apache.poi.xwpf.converter-0.9.8.jar
oro-2.0.8.jar (com.springsource.org.apache.oro-2.0.8.jar)
poi-3.8.jar (poi-osgi-wrapper-3.8.jar - created custom shade wrapper)
poi-ooxml-3.8.jar (poi-osgi-wrapper-3.8.jar - created custom shade wrapper)
stax-api-1.0.1.jar (com.springsource.javax.xml.stream-1.0.1.jar)
xml-apis-1.0.b2.jar (com.springsource.org.apache.xmlcommons-1.3.4.jar)
xmlbeans-2.3.0.jar (modified-com.springsource.org.apache.xmlbeans-2.4.0.jar
modified MANIFEST.MF to import-package org.w3c.dom)

Note that I had to modify
fr.opensagres.xdocreport.converter.docx.xwpf-0.9.8.jar to add an
import-package for fr.opensagres.xdocreport.core.utils, in
fr.opensagres.xdocreport.converter.docx.xwpf as it needed StringUtils.

Caused by: java.lang.NoClassDefFoundError:
fr/opensagres/xdocreport/core/utils/StringUtils
at
fr.opensagres.xdocreport.converter.docx.poi.itext.XWPF2PDFViaITextConverter.toPDFViaITextOptions(XWPF2PDFViaITextConverter.java:93)
at
fr.opensagres.xdocreport.converter.docx.poi.itext.XWPF2PDFViaITextConverter.convert(XWPF2PDFViaITextConverter.java:66)
at
fr.opensagres.xdocreport.document.AbstractXDocReport.convert(AbstractXDocReport.java:685)
... 34 more
Caused by: java.lang.ClassNotFoundException:
fr.opensagres.xdocreport.core.utils.StringUtils
at
org.apache.felix.framework.ModuleImpl.findClassOrResourceByDelegation(ModuleImpl.java:814)

If you could comment on if you think this is valid and whether you could
modify this yourselves and when then that would be great, else I’ll just
have to create a patched version and deploy to our maven repository.

I also had to modify com.springsource.org.apache.xmlbeans-2.4.0.jar
modified MANIFEST.MF to import-package org.w3c.dom to prevent..
Caused by: java.lang.NoClassDefFoundError
at org.apache.xmlbeans.XmlBeans.class$(XmlBeans.java:43)
at org.apache.xmlbeans.XmlBeans.buildNodeMethod(XmlBeans.java:195)
at org.apache.xmlbeans.XmlBeans.buildNodeToCursorMethod(XmlBeans.java:232)
at org.apache.xmlbeans.XmlBeans.<clinit>(XmlBeans.java:131)
at
org.openxmlformats.schemas.wordprocessingml.x2006.main.DocumentDocument$Factory.parse(Unknown
Source)
at
org.apache.poi.xwpf.usermodel.XWPFDocument.onDocumentRead(XWPFDocument.java:134)
at org.apache.poi.POIXMLDocument.load(POIXMLDocument.java:159)
at org.apache.poi.xwpf.usermodel.XWPFDocument.<init>(XWPFDocument.java:123)
at
fr.opensagres.xdocreport.converter.docx.poi.itext.XWPF2PDFViaITextConverter.convert(XWPF2PDFViaITextConverter.java:65)
at
fr.opensagres.xdocreport.document.AbstractXDocReport.convert(AbstractXDocReport.java:685)
... 34 more
Caused by: java.lang.ClassNotFoundException: org.w3c.dom.Node
at
org.apache.felix.framework.ModuleImpl.findClassOrResourceByDelegation(ModuleImpl.java:814)

As this is just a wrapper, I will create my own wrapper, re-wrapping the
original xmlbeans library with the required org.w3c.dom import.

I had to wrap all the apache.poi libraries as I couldn't locate osgi
bundles for these. (poi-3.8.jar, poi-ooxml-3.8.jar and
ooxml-schemas-1.1.jar all wrapped in my wrapper ooxml-schemas-1.1.jar).
(attached poi.pom.xml).

I had to wrap the itext libraries to get the 2.1.7 version of itext, as I
could only locate a springsource version at 2.0.8.
(attached itext.pom.xml)

I had to ensure that org.w3c.dom was being exported from the java runtime
via the felix container by adding org.w3c.dom to the
org.osgi.framework.system.packages property in felix.properties.

Finally I had to increase the permgen heap size from its default (64m) to
128m by adding -XX:MaxPermSize=128m to the batch file we use to lanch the
osgi container, to prevent..
fr.opensagres.xdocreport.converter.XDocConverterException:
java.lang.OutOfMemoryError: PermGen space
at
fr.opensagres.xdocreport.document.AbstractXDocReport.convert(AbstractXDocReport.java:720)
at
com.experian.eda.component.workflow.runtime.engine.handler.DocumentTemplateActivityBehaviour.mergeDoc(DocumentTemplateActivityBehaviour.java:286)
at
com.experian.eda.component.workflow.runtime.engine.handler.DocumentTemplateActivityBehaviour.executeInternal(DocumentTemplateActivityBehaviour.java:187)
at
com.experian.eda.component.workflow.runtime.engine.handler.AbstractActivityBehaviour.execute(AbstractActivityBehaviour.java:57)


Regards,
Philip Wilkinson.


Attachments:
itext-pom.xml 3.4 KB
poi-pom.xml 3.8 KB

[xdocr...@googlecode.com]

Comment #5 on issue 159 by angelo.z...@gmail.com: Can not find DOCX

converters in OSGi container.
http://code.google.com/p/xdocreport/issues/detail?id=159

Hi Phil,

Wow congratulation for your work. I don't understand why you had had so
many problems because we have a JUnit which works with OSGi?

@Pascal :I think we should provide several zip sample like we have provided
(reporting+freemarker, reporting+velocity, reporting+converter+freemarker
for docx and odt, etc) with OSGi. Those zip will contain a basic sample
with Activator which generate report and contains the whole JARs with well
MANIFEST.MF.

What do you think about that?
If you are OK, could you start this task (if you have time). I could help
you if you wish.

Regards Angelo

[xdocr...@googlecode.com]

Comment #6 on issue 159 by wilkinso...@gmail.com: Can not find DOCX

converters in OSGi container.
http://code.google.com/p/xdocreport/issues/detail?id=159

Hi,
your junit tests that I think you are refereing to are here.
http://code.google.com/searchframe#cLr_AEi-h-g/integrationtests/fr.opensagres.xdocreport.osgi.integrationtests/src/test/java/fr/opensagres/xdocreport/osgi/integrationtests/ConverterTest.java
You use pax exam to wrap all the 3rd party jars.
pax exam uses bnd but from what I can see specifies no parameters so
everything is imported and everything is exported.
I've had problems in past with this where I've had conflicts between
different bundles exporting the same packages, which should be internal. So
I use a maven shade plugin and are more explicit about what gets exported
and what gets imported.
I think your tests work because, they are not complete, i.e. they don't use
all the jars I needed, they don't produce a pdf, and they run in a
dedicated container whose only purpose is to run the tests (so no other
possibility of conflict).
Also I wanted to use springsource provided wrappers where ever possible
(not on-the fly bnd created 3rd party bundles) as it makes the who
configuration simpler.
Regards,
Philip Wilkinson.

[xdocr...@googlecode.com]

Comment #7 on issue 159 by wilkinso...@gmail.com: Can not find DOCX

(not on-the fly bnd created 3rd party bundles) as it makes the whole

[xdocr...@googlecode.com]

Comment #8 on issue 159 by angelo.z...@gmail.com: Can not find DOCX

converters in OSGi container.
http://code.google.com/p/xdocreport/issues/detail?id=159

Hi Philip,

> your junit tests that I think you are refereing to are here.
http://code.google.com/searchframe#cLr_AEi-h-g/integrationtests/fr.opensagres.xdocreport.osgi.integrationtests/src/test/java/fr/opensagres/xdocreport/osgi/integrationtests/ConverterTest.java

yes it that.

Many thank's for your great information.
I tell me if you will be OK to improve our OSGi JUnit and avoid having your
big problems that you have done. Are you interested to help us?

Many thanks for your help.

Regards Angelo

[Pascal Leclercq]

Hi Philip,

First of all I have to say I'm very enthusiast to find users of xdocreport under OSGi.

I admit our current Pax Exam are fairly naive and must be improved.

I use to hesitate of using itext bundle  from springsource because only 2.0.8 is available :

http://ebr.springsource.com/repository/app/bundle/version/detail?name=com.springsource.com.lowagie.text&version=2.0.8&searchType=bundlesByName&searchQuery=itext

Using xmlbeans bundle from spring source or fuse source os great and should be done so.

I have difficulties with POI because we need 2 differents jars (poi and poi-ooxml) and both share some packages (org.apache.poi for example).

Apache POI doesn't provide any OSGi bundle (yet). A case has been create recently : https://issues.apache.org/bugzilla/show_bug.cgi?id=53866

Do you want to contribute to xdocreport on that part ?

Do you have a patch to submit or even a modified "ConverterTest" te replace ours ?

Best regards.

2012/10/10 <xdocr...@googlecode.com>

--
Pascal Leclercq

[xdocr...@googlecode.com]

Comment #9 on issue 159 by sethhels...@googlemail.com: Can not find DOCX

converters in OSGi container.
http://code.google.com/p/xdocreport/issues/detail?id=159

Has this been installed anywhere on a Maven Repo, in its working form?

--
You received this message because this project is configured to send all
issue notifications to this address.
You may adjust your notification preferences at:
https://code.google.com/hosting/settings

[Pascal Leclercq]

Can you precise what you are looking for ?

The ConverterTest can be found here :

https://code.google.com/p/xdocreport/source/browse/#git%2Fintegrationtests%2Ffr.opensagres.xdocreport.osgi.integrationtests%2Fsrc%2Ftest%2Fjava%2Ffr%2Fopensagres%2Fxdocreport%2Fosgi%2Fintegrationtests

and It is not deployed to any maven repo.

Best regards

2013/7/15 <xdocr...@googlecode.com>

You received this message because you are subscribed to the Google Groups "xdocreport" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xdocreport+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Pascal Leclercq

[xdocr...@googlecode.com]

Comment #10 on issue 159 by angelo.z...@gmail.com: Can not find DOCX

converters in OSGi container.
http://code.google.com/p/xdocreport/issues/detail?id=159

Is it again this problem with 1.0.5-SNAPSHOT?