Issue 238 in xdocreport: PDF Images lost at xdocreport 1.0.0 in felix osgi container

[xdocr...@googlecode.com]

Status: New
Owner: ----
Labels: Type-Defect Priority-Medium

New issue 238 by Mr.M.McM...@googlemail.com: PDF Images lost at xdocreport
1.0.0 in felix osgi container
http://code.google.com/p/xdocreport/issues/detail?id=238

What steps will reproduce the problem?

Docx to PDF conversion of images loses image.

What is the expected output? What do you see instead?

The PDF contains no image: the image should be in the PDF.

What version of the product are you using? On what operating system?

xdocreport 1.0.0 in felix OSGi on Windows 7: some jars are re-wrapped.

Please provide any additional information below.

When generating from docx to pdf, images are not being written to the
generated pdf, but only when I am running in a felix osgi container.
Running outside of this enviroment generates images successfully when using
the same jars. So there appears to be a config issue with the bundle
wrappers. Some of the bundles I am using are wrapped by you, some by me.

I was previously running xdocreport 0.9.8 as and wrapped several bundles in
order to get up and running in osgi, as per:
http://code.google.com/p/xdocreport/issues/detail?id=159. Images were being
successfully generated.

Migrating to xdocreport 1.0.0 involved:

. modifying:

- my existing poi wrapper which wraps:
org.apache.poi:poi:3.8
org.apache.poi:poi-ooxml:3.8
org.apache.poi:ooxml-schemas:1.1
in order to export
org.openxmlformats.schemas.drawingml.x2006.wordprocessingDrawing.

. wrapping (to modify imports/exports to coexist with my config):

- fr.opensagres.xdocreport.converter-1.0.0.jar
- fr.opensagres.xdocreport.converter.docx.xwpf-1.0.0.jar (I also wrapped
this at 0.9.8)

. using unmodified:

- fr.opensagres.xdocreport.core-1.0.0.jar
- fr.opensagres.xdocreport.document-1.0.0.jar
- fr.opensagres.xdocreport.document.docx-1.0.0.jar
- fr.opensagres.xdocreport.itext.extension-1.0.0.jar
- fr.opensagres.xdocreport.template-1.0.0.jar
- fr.opensagres.xdocreport.template.freemarker-1.0.0.jar
- org.apache.poi.xwpf.converter.core-1.0.0.jar
- org.apache.poi.xwpf.converter.pdf-1.0.0.jar
- org.apache.poi.xwpf.converter.xhtml-1.0.0.jar

The problem is related to xmlbeans representing the image as XmlAnyTypeImpl
object when it should be a CTPicture.

The line it fails on is
org.apache.poi.xwpf.converter.core.XWPFDocumentVisitor.visitGraphicalObject(..)
line 1223 (org.apache.poi.xwpf.converer.core-1.0.0.jar). The object is an
XmlAnyTypeImpl object rather than a CTPicture object so it skips some
significant code. I cannot debug the getObject() call.

XmlObject o = c.getObject();
if ( o instanceof CTPicture ) { // test fails as o is XmlAnyType
CTPicture picture = (CTPicture) o;
// extract the picture if needed
IImageExtractor extractor = getImageExtractor();

Note that prior to this in
org.apache.poi.xwpf.usermodel.XWPFRun.getCTPictures(...) line 150 it caters
for an XmlAnyTypeImpl object and creates a CTPicture from it.

if(pict instanceof XmlAnyTypeImpl) {
// Pesky XmlBeans bug - see Bugzilla #49934
try {
pict = CTPicture.Factory.parse(pict.toString());
} catch(XmlException e) {
throw new POIXMLException(e);
}
}

Applying the same check and fix in
XWPFDocumentVisitor.visitGraphicalObject(..) resolves my problem. However,
given that this only occurs in the osgi environment (I get a CTPicture

"org.openxmlformats.schemas.drawingml.x2006.picture.impl.CTPictureImpl" at
both points when running outside osgi) I believe this is not the correct
fix - there must be some additional config required. There is nothing in
the log files which supplies any clues regarding what to do.

I noticed that the pom for your
org.apache.poi.xwpf.converter.core-1.0.0.jar contains an exclusion which
hints at some problems in this area:

<dependency>
<groupId>org.apache.poi</groupId>
<artifactId>poi-ooxml</artifactId>
<version>3.8</version>
<exclusions>
<!-- surprisingly this artifect contains a subset of
org.apache.poi:ooxml-schemas:1.1 -->
<exclusion>
<artifactId>poi-ooxml-schemas</artifactId>
<groupId>org.apache.poi</groupId>
</exclusion>
</exclusions>
</dependency>

And there is also the issue referenced in the commment in the XWPFRun code
which again hints at similar problems:
https://issues.apache.org/bugzilla/show_bug.cgi?id=49934.

I have tinkered with the import/exports in the bundles that I have wrapped
to try expose the packages which contain the CTPictures related class files
but without success. I'm basically working by guesswork.

Can you provide any suggestions regarding what is occurring here and how to
resolve it?



--
You received this message because this project is configured to send all
issue notifications to this address.
You may adjust your notification preferences at:
https://code.google.com/hosting/settings

[xdocr...@googlecode.com]

Updates:
Status: Accepted
Owner: pascal.leclercq

Comment #1 on issue 238 by angelo.z...@gmail.com: PDF Images lost at

xdocreport 1.0.0 in felix osgi container
http://code.google.com/p/xdocreport/issues/detail?id=238

Hi,
At first, why you have wrapping XDocReport bundles :

- fr.opensagres.xdocreport.converter-1.0.0.jar
- fr.opensagres.xdocreport.converter.docx.xwpf-1.0.0.jar (I also wrapped
this at 0.9.8)

If there are a problem with thoses JARs in OSGi context, it should be fixed.

@Pascal: could you see this issue please?

Many thank's

[Jan Hahnisch]

Hi,

I'm working on conversation of odt to PDF. There are some problems with images due to position of the image in header footer or other position relativ to an other element. Maybe that's also the problem in your case.

Regards jan

> You received this message because you are subscribed to the Google Groups "xdocreport" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to xdocreport+...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Angelo zerr]

Hi Jan,

No I think it's not the same problem. I think we have not managed that in the ODT converter. If you have problem, please creat ea new isue with your ODT problems (create one isuses per problems with ODT attached).

Regards Angelo

2013/3/21 Jan Hahnisch <hahni...@yahoo.de>

[Jan Hahnisch]

Hi Angelo, 

There are already issues 223 and 224 for my problems, but I wrote you that I'm fixing that issues myself and let you the solution via patch. 

It was a hint. If it is not the same for docx so forget my message:-) 

Jan 

[xdocr...@googlecode.com]

Comment #2 on issue 238 by Mr.M.McM...@googlemail.com: PDF Images lost at

xdocreport 1.0.0 in felix osgi container
http://code.google.com/p/xdocreport/issues/detail?id=238

At 0.9.8 I followed the advice at
http://code.google.com/p/xdocreport/issues/detail?id=159 to run in OSGi.
Specifically the wrapper is needed, as reported in 159, to avoid:

java.lang.NoClassDefFoundError:
fr/opensagres/xdocreport/core/utils/StringUtils

At 1.0.0 for similar reasons I again wrapped the jar, ie in order to import
fr.opensagres.xdocreport.core.utils which is missing from your version of
the wrapper.

Note that this is only required when executing the conversion via:

IXDocReport report = ...
report.convert(context,
Options.getTo(ConverterTypeTo.PDF).via(ConverterTypeVia.XWPF), os);

but it is not required when executing the conversion via:

XWPFDocument document = ...
PdfConverter.getInstance().convert(document, os, PdfOptions.create())


The other wrapper was modified in error by me. In accounting for the
addition of the import of org.apache.poi.xwpf.converter.core to
fr.opensagres.xdocreport.converter.docx.xwpf-1.0.0, I wrapped
fr.opensagres.xdocreport.converter-1.0.0 and imported it there. So there
was no need to wrap fr.opensagres.xdocreport.converter-1.0.0.

I have now rectified this situation so that I only now wrap
fr.opensagres.xdocreport.converter.docx.xwpf-1.0.0.jar to reolve the
StringUtils problem. However, the same lost image issue occurs when
converting a docx to a PDF: ie XmlAnyTypeImpl object rather than a
CTPicture object so image is lost.

I also followed the wrapping advice from
http://code.google.com/p/xdocreport/issues/detail?id=159 for POI (using the
that issue's attached pom.xml). When reusing this with your
org.apache.poi.xwpf.converter.core 1.0.0 there was a problem starting that
bundle:

org.osgi.framework.BundleException: Unresolved constraint in bundle
org.apache.poi.xwpf.converter.core [89]: package;
(package=org.openxmlformats.schemas.drawingml.x2006.wordprocessingDrawing)

hence, my exposing
org.openxmlformats.schemas.drawingml.x2006.wordprocessingDrawing in my POI
wrapper.

My suspicion is that more of these org.openxmlformats.schemas are required.
I tried exporting the CTPictures related ones but it made no difference.
Should the org.apache.poi.xwpf.converter.core wrapper perhaps import some
of these other schema packages?

Can you provide any suggestions regarding what is occurring here and how to
resolve it?

[Pascal Leclercq]

Hi,

thanks for your feedback.

I was unaware org.opensagres.xdocreport.core.utils Import-Package was missing

I'm going to add It and check if some other are missing.

You may already had a look but we have a RAP application working here :

http://xdocreport-rap.opensagres.cloudbees.net/xdocreport?startup=fr.opensagres.xdocreport.eclipse.ui.application

So far we don't have that much tests on felix.

You may want to try our hand made POI osgi bundle (here included).

We do expect a "real" POI osgi bundle too....

I will try also to update our Pax Exam test to take in account your case.

Best regards

2013/3/21 <xdocr...@googlecode.com>

You received this message because you are subscribed to the Google Groups "xdocreport" group.

To unsubscribe from this group and stop receiving emails from it, send an email to xdocreport+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
Pascal Leclercq

[xdocr...@googlecode.com]

Comment #3 on issue 238 by Mr.M.McM...@googlemail.com: PDF Images lost at

xdocreport 1.0.0 in felix osgi container
http://code.google.com/p/xdocreport/issues/detail?id=238

Hi there.

Do you have any thoughts about what is happening here or how to resolve
this issue?

Thanks.

[xdocr...@googlecode.com]

Comment #4 on issue 238 by pascal.leclercq: PDF Images lost at xdocreport

1.0.0 in felix osgi container
http://code.google.com/p/xdocreport/issues/detail?id=238

Hi,

on the dev branch i add the missing package on the converter fragments
(fr.opensagres.xdocreport.core.utils).
If you want to test, our jars are deployed to
https://oss.sonatype.org/content/repositories/snapshots/.

Not sure I get your point with POI packages but I suspect Bnd may miss
Import-package on POI.
I suggest to add
DynamicImport-Package: org.apache.poi.* on xdocreport XMPF converter.

what do u think ?

[xdocr...@googlecode.com]

Comment #5 on issue 238 by Mr.M.McM...@googlemail.com: PDF Images lost at

xdocreport 1.0.0 in felix osgi container
http://code.google.com/p/xdocreport/issues/detail?id=238

I tried the DynamicImport-Package as suggested but the image was still lost.

Do you understand the behaviour? Is the image objcet created as an
XmlAnyType rather than a CTPicture because some package is incorrectly
exported in the POI wrapper (from
http://code.google.com/p/xdocreport/issues/detail?id=159) or because it is
incorrectly imported by xdocreport 1.0.0? Why did it work with xdocreport
0.9.8?

[xdocr...@googlecode.com]

Comment #6 on issue 238 by angelo.z...@gmail.com: PDF Images lost at

xdocreport 1.0.0 in felix osgi container
http://code.google.com/p/xdocreport/issues/detail?id=238

I have doen a big refactoring in XDocReport 1.0.0 for docx->pdf converter
for images to improve it (manage aboslute image, etc)

In 0.9.8, pictures are visited like this :

---------------------------------------------------------------------
protected void visitPictures( XWPFRun run, T parentContainer )
throws Exception
{
List<XWPFPicture> embeddedPictures = run.getEmbeddedPictures();
for ( XWPFPicture picture : embeddedPictures )
{
visitPicture( picture, parentContainer );
}
}
---------------------------------------------------------------------

See
https://code.google.com/p/xdocreport/source/browse/thirdparties-extension/org.apache.poi.xwpf.converter/src/main/java/org/apache/poi/xwpf/converter/internal/XWPFElementVisitor.java?name=xdocreport-0.9.8

In 1.0.0, it is visited like this :

------------------------------------------------------------------
private void visitGraphicalObject( T parentContainer, CTGraphicalObject
graphic, Float offsetX,
STRelFromH.Enum relativeFromH, Float
offsetY, STRelFromV.Enum relativeFromV )
throws Exception
{
if ( graphic != null )
{
CTGraphicalObjectData graphicData = graphic.getGraphicData();
if ( graphicData != null )
{
XmlCursor c = graphicData.newCursor();
c.selectPath( "./*" );
while ( c.toNextSelection() )
{

XmlObject o = c.getObject();
if ( o instanceof CTPicture )
{

CTPicture picture = (CTPicture) o;
// extract the picture if needed
IImageExtractor extractor = getImageExtractor();

if ( extractor != null )
{
XWPFPictureData pictureData = getPictureData(
picture );
if ( pictureData != null )
{
try
{
extractor.extract( WORD_MEDIA +
pictureData.getFileName(), pictureData.getData() );
}
catch ( Throwable e )
{
LOGGER.log( Level.SEVERE,
"Error while extracting the
image " + pictureData.getFileName(), e );
}
}
}
// visit the picture.
visitPicture( picture, offsetX, relativeFromH,
offsetY, relativeFromV, parentContainer );
}
}
c.dispose();
}
}
}
-----------------------------------------

See
https://code.google.com/p/xdocreport/source/browse/thirdparties-extension/org.apache.poi.xwpf.converter.core/src/main/java/org/apache/poi/xwpf/converter/core/XWPFDocumentVisitor.java?name=xdocreport-1.0.0

If I remember I do that to keep the oroder of image, text, image text etc.

Perhaps we should apply your patch :

-------------------------------------

if(pict instanceof XmlAnyTypeImpl) {
// Pesky XmlBeans bug - see Bugzilla #49934
try {
pict = CTPicture.Factory.parse(pict.toString());
} catch(XmlException e) {
throw new POIXMLException(e);
}
}

-------------------------------------

I don't know?

[xdocr...@googlecode.com]

Comment #7 on issue 238 by gha...@gmail.com: PDF Images lost at xdocreport

1.0.0 in felix osgi container
http://code.google.com/p/xdocreport/issues/detail?id=238

I'm trying to convert docx to pdf, I'm not seeing the images in header
section converted in pdf. Can you please help? I'm using xdocreport 1.0.0
version

[xdocr...@googlecode.com]

Comment #8 on issue 238 by angelo.z...@gmail.com: PDF Images lost at

xdocreport 1.0.0 in felix osgi container
http://code.google.com/p/xdocreport/issues/detail?id=238

At first last version is 1.0.2. Please test your docx with this version.

Are you in OSGi env? If no, your problem is not linked to this issue, so
please create a new issue if 1.0.2 doesn't resolve your problem by
attaching your docx.

[xdocr...@googlecode.com]

Comment #9 on issue 238 by rohit.mc...@gmail.com: PDF Images lost at

xdocreport 1.0.0 in felix osgi container
http://code.google.com/p/xdocreport/issues/detail?id=238

org.apache.poi.xwpf.converter.core.XWPFConverterException:
java.lang.NullPointerException
at
org.apache.poi.xwpf.converter.pdf.PdfConverter.doConvert(PdfConverter.java:59)
at
org.apache.poi.xwpf.converter.pdf.PdfConverter.doConvert(PdfConverter.java:37)
at
org.apache.poi.xwpf.converter.core.AbstractXWPFConverter.convert(AbstractXWPFConverter.java:45)
at
com.medikm.util.PdfConversion.convertDocToPDF(PdfConversion.java:131)
at
com.medikm.util.PdfConversion.convertFileToPDF(PdfConversion.java:405)
at
com.medikm.servlet.ResourceUploadServlet.doPost(ResourceUploadServlet.java:134)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:641)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:304)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at com.medikm.filter.UserAuthFilter.doFilter(UserAuthFilter.java:43)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
com.medikm.filter.DispatcheFilter.doFilter(DispatcheFilter.java:133)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:224)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:169)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:929)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:405)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:964)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:515)
at
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:302)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
Caused by: java.lang.NullPointerException
at
org.apache.poi.xwpf.converter.core.utils.XWPFTableUtil.getWidth(XWPFTableUtil.java:273)
at
org.apache.poi.xwpf.converter.core.utils.XWPFTableUtil.computeColWidths(XWPFTableUtil.java:284)
at
org.apache.poi.xwpf.converter.core.utils.XWPFTableUtil.computeColWidths(XWPFTableUtil.java:127)
at
org.apache.poi.xwpf.converter.core.XWPFDocumentVisitor.visitTable(XWPFDocumentVisitor.java:828)
at
org.apache.poi.xwpf.converter.core.XWPFDocumentVisitor.visitBodyElements(XWPFDocumentVisitor.java:246)
at
org.apache.poi.xwpf.converter.core.XWPFDocumentVisitor.start(XWPFDocumentVisitor.java:194)
at
org.apache.poi.xwpf.converter.pdf.PdfConverter.doConvert(PdfConverter.java:55)
... 29 more



how to solve this problem