NegativeArraySizeException: Stress Testing Islandora Using Hubble's Greatest TIFFs

Brad Spry

unread,

Oct 28, 2014, 5:16:29 PM10/28/14

to isla...@googlegroups.com

Conducting stress testing using Hubble's greatest TIFFs:

http://hubblesource.stsci.edu/sources/illustrations/

Hubble's Sharpest View of the Orion Nebula:

http://imgsrc.hubblesite.org/hu/db/images/hs-2006-01-a-full_tif.tif

(385.09 MB, 18000 X 18000 resolution)

Iridescent Glory of Nearby Helix Nebula:

http://imgsrc.hubblesite.org/hu/db/images/hs-2003-11-a-full_tif.tif

(285.99 MB, 16000 X 16000 resolution)

Ingest is completed, but then system throws "NegativeArraySizeException" errors, and derivatives such as TECHMD, TN, JPG, and JPG2 are not generated:

ERROR 2014-10-28 16:33:22.272 [http-8443-10] (FedoraObjectResource) Unexpected error fulfilling REST API request

java.lang.NegativeArraySizeException: null

ERROR 2014-10-28 16:33:22.274 [http-8443-10] (WebApplicationImpl) Internal server error

javax.ws.rs.WebApplicationException: java.lang.NegativeArraySizeException

The only clue I've been able to find:

https://jira.duraspace.org/browse/FCREPO-944

No Fedora out of memory errors, no PHP errors, just seeing the aforementioned "NegativeArraySizeException" errors...

Any assistance/clue would be greatly appreciated!

Brad Spry

Atkins Library

UNC Charlotte

Brad Spry

unread,

Oct 29, 2014, 12:41:43 PM10/29/14

to isla...@googlegroups.com

Successfully ingested and fully processed Hubble's third biggest TIFF:

The Secret Lives of Galaxies Unveiled in Deep Survey

http://imgsrc.hubblesite.org/hu/db/images/hs-2003-18-d-full_tif.tif

(154.08 MB, 8003 X 11894 resolution)

Working on a 200 MB test image next.

Brad Spry

unread,

Oct 29, 2014, 5:20:00 PM10/29/14

to isla...@googlegroups.com

I'm observing tuque/HttpConnection.php running out of PHP memory now, upping and re-testing.

Brad Spry

unread,

Oct 31, 2014, 5:00:55 PM10/31/14

to isla...@googlegroups.com

I worked on this issue for approximately three days, tuning my system's generous resources for maximum effectiveness.

Now, I have absolutely no indication in any log file that the issue is caused by low memory or other related resources.

I do have a specific file size however, in which it rears it's ugly head: between 250 MB and 300MB...

The issue happens regardless if file is ingested via web browser or command line batch, the error result is the same:

Command line batch error:

WD islandora: Failed to ingest object: islandora:492code: 500message: javax.ws.rs.WebApplicationException:                                [error]
java.lang.NegativeArraySizeException

---

Fedora.log:

INFO 2014-10-31 14:41:48.886 [http-8443-7] (DefaultDOManager) New object PID is islandora:492
INFO 2014-10-31 14:41:48.903 [http-8443-7] (DefaultDOManager) Committing addition of islandora:492
INFO 2014-10-31 14:41:48.903 [http-8443-7] (DefaultDOManager) Getting managed datastream from internal uploaded location: uploaded://1997
INFO 2014-10-31 14:42:47.542 [http-8443-7] (DefaultDOManager) Replaced managed datastream location with internal id: islandora:492+OBJ+OBJ.0
INFO 2014-10-31 14:43:37.707 [http-8443-7] (DefaultDOManager) Adding to ResourceIndex
INFO 2014-10-31 14:43:37.723 [http-8443-7] (DefaultDOManager) Updating dissemination index
INFO 2014-10-31 14:43:37.723 [http-8443-7] (DefaultDOManager) Updating FieldSearch index
INFO 2014-10-31 14:43:37.753 [http-8443-7] (DefaultManagement) Completed ingest(objectXML, format: info:fedora/fedora-system:FOXML-1.1, encoding: UTF-8, pid    : islandora:492, logMessage: )
INFO 2014-10-31 14:43:37.901 [http-8443-5] (DefaultManagement) Completed export(pid: islandora:492, format: info:fedora/fedora-system:FOXML-1.1, exportContext: public, encoding: UTF-8)
INFO 2014-10-31 14:50:01.585 [http-8443-7] (DefaultManagement) Completed export(pid: islandora:492, format: info:fedora/fedora-system:FOXML-1.1, exportContext: archive, encoding: UTF-8)
ERROR 2014-10-31 14:50:01.587 [http-8443-7] (FedoraObjectResource) Unexpected error fulfilling REST API request
java.lang.NegativeArraySizeException: null
    at org.apache.commons.codec.binary.Base64.encodeBase64(Base64.java:258) [commons-codec-1.3.jar:1.3]
    at org.apache.commons.codec.binary.Base64.encodeBase64(Base64.java:178) [commons-codec-1.3.jar:1.3]
    at org.fcrepo.utilities.Base64.encode(Base64.java:27) [fcrepo-common-3.5.jar:na]
    at org.fcrepo.utilities.Base64.encode(Base64.java:47) [fcrepo-common-3.5.jar:na]
    at org.fcrepo.utilities.Base64.encodeToString(Base64.java:79) [fcrepo-common-3.5.jar:na]
    at org.fcrepo.server.storage.translation.FOXMLDOSerializer.appendDatastreams(FOXMLDOSerializer.java:369) [fcrepo-server-3.5.jar:na]
    at org.fcrepo.server.storage.translation.FOXMLDOSerializer.serialize(FOXMLDOSerializer.java:118) [fcrepo-server-3.5.jar:na]
    at org.fcrepo.server.storage.translation.DOTranslatorImpl.serialize(DOTranslatorImpl.java:99) [fcrepo-server-3.5.jar:na]
    at org.fcrepo.server.storage.translation.DOTranslatorModule.serialize(DOTranslatorModule.java:139) [fcrepo-server-3.5.jar:na]
    at org.fcrepo.server.storage.SimpleDOReader.Export(SimpleDOReader.java:203) [fcrepo-server-3.5.jar:na]
    at org.fcrepo.server.management.DefaultManagement.export(DefaultManagement.java:330) [fcrepo-server-3.5.jar:na]
    at sun.reflect.GeneratedMethodAccessor119.invoke(Unknown Source) [na:na]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [na:1.6.0_45]
    at java.lang.reflect.Method.invoke(Method.java:597) [na:1.6.0_45]
    at org.fcrepo.server.messaging.NotificationInvocationHandler.invoke(NotificationInvocationHandler.java:68) [fcrepo-server-3.5.jar:na]
    at com.sun.proxy.$Proxy6.export(Unknown Source) [na:na]
    at org.fcrepo.server.management.ManagementModule.export(ManagementModule.java:279) [fcrepo-server-3.5.jar:na]
    at org.fcrepo.server.rest.FedoraObjectResource.getObjectExport(FedoraObjectResource.java:92) [fcrepo-server-3.5.jar:na]
    at sun.reflect.GeneratedMethodAccessor138.invoke(Unknown Source) [na:na]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [na:1.6.0_45]
    at java.lang.reflect.Method.invoke(Method.java:597) [na:1.6.0_45]
    at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:175) [jersey-bundle-1.0.3.1.jar:1.0.3.1]
    at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:67) [jersey-bundle-1.0.3.1.jar:1.0.3.1]
    at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:163) [jersey-bundle-1.0.3.1.jar:1.0.3.1]
    at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:111) [jersey-bundle-1.0.3.1.jar:1.0.3.1]
    at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:71) [jersey-bundle-1.0.3.1.jar:1.0.3.1]
    at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:111) [jersey-bundle-1.0.3.1.jar:1.0.3.1]
    at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:63) [jersey-bundle-1.0.3.1.jar:1.0.3.1]
    at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:689) [jersey-bundle-1.0.3.1.jar:1.0.3.1]
    at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:647) [jersey-bundle-1.0.3.1.jar:1.0.3.1]
    at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:638) [jersey-bundle-1.0.3.1.jar:1.0.3.1]
    at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:309) [jersey-bundle-1.0.3.1.jar:1.0.3.1]
    at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:425) [jersey-bundle-1.0.3.1.jar:1.0.3.1]
    at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:590) [jersey-bundle-1.0.3.1.jar:1.0.3.1]
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) [servlet-api.jar:na]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) [catalina.jar:6.0.29]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) [catalina.jar:6.0.29]
    at org.fcrepo.server.security.servletfilters.FilterRestApiFlash.doFilter(FilterRestApiFlash.java:79) [fcrepo-server-3.5.jar:na]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) [catalina.jar:6.0.29]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) [catalina.jar:6.0.29]
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:368) [spring-security-web-3.0.5.RELEASE.jar:3.0.5.RELEASE]
    at org.fcrepo.server.security.jaas.AuthFilterJAAS.doFilter(AuthFilterJAAS.java:330) [fcrepo-security-jaas-3.5.jar:na]
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:380) [spring-security-web-3.0.5.RELEASE.jar:3.0.5.RELEASE]
    at org.springframework.security.web.access.channel.ChannelProcessingFilter.doFilter(ChannelProcessingFilter.java:109) [spring-security-web-3.0.5.RELEASE.jar:3.0.5.RELEASE]
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:380) [spring-security-web-3.0.5.RELEASE.jar:3.0.5.RELEASE]
    at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:169) [spring-security-web-3.0.5.RELEASE.jar:3.0.5.RELEASE]
    at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237) [spring-web-3.0.5.RELEASE.jar:3.0.5.RELEASE]
    at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167) [spring-web-3.0.5.RELEASE.jar:3.0.5.RELEASE]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) [catalina.jar:6.0.29]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) [catalina.jar:6.0.29]
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) [catalina.jar:6.0.29]
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) [catalina.jar:6.0.29]
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) [catalina.jar:6.0.29]
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) [catalina.jar:6.0.29]
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) [catalina.jar:6.0.29]
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) [catalina.jar:6.0.29]
    at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857) [tomcat-coyote.jar:6.0.29]
    at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) [tomcat-coyote.jar:6.0.29]
    at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) [tomcat-coyote.jar:6.0.29]
    at java.lang.Thread.run(Thread.java:662) [na:1.6.0_45]
ERROR 2014-10-31 14:50:01.631 [http-8443-7] (WebApplicationImpl) Internal server error
javax.ws.rs.WebApplicationException: java.lang.NegativeArraySizeException
    at org.fcrepo.server.rest.BaseRestResource.handleException(BaseRestResource.java:172) [fcrepo-server-3.5.jar:na]
    at org.fcrepo.server.rest.FedoraObjectResource.getObjectExport(FedoraObjectResource.java:99) [fcrepo-server-3.5.jar:na]
    at sun.reflect.GeneratedMethodAccessor138.invoke(Unknown Source) [na:na]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [na:1.6.0_45]
    at java.lang.reflect.Method.invoke(Method.java:597) [na:1.6.0_45]
    at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:175) [jersey-bundle-1.0.3.1.jar:1.0.3.1]
    at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:67) [jersey-bundle-1.0.3.1.jar:1.0.3.1]
    at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:163) [jersey-bundle-1.0.3.1.jar:1.0.3.1]
    at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:111) [jersey-bundle-1.0.3.1.jar:1.0.3.1]
    at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:71) [jersey-bundle-1.0.3.1.jar:1.0.3.1]
    at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:111) [jersey-bundle-1.0.3.1.jar:1.0.3.1]
    at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:63) [jersey-bundle-1.0.3.1.jar:1.0.3.1]
    at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:689) [jersey-bundle-1.0.3.1.jar:1.0.3.1]
    at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:647) [jersey-bundle-1.0.3.1.jar:1.0.3.1]
    at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:638) [jersey-bundle-1.0.3.1.jar:1.0.3.1]
    at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:309) [jersey-bundle-1.0.3.1.jar:1.0.3.1]
    at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:425) [jersey-bundle-1.0.3.1.jar:1.0.3.1]
    at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:590) [jersey-bundle-1.0.3.1.jar:1.0.3.1]
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) [servlet-api.jar:na]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) [catalina.jar:6.0.29]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) [catalina.jar:6.0.29]
    at org.fcrepo.server.security.servletfilters.FilterRestApiFlash.doFilter(FilterRestApiFlash.java:79) [fcrepo-server-3.5.jar:na]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) [catalina.jar:6.0.29]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) [catalina.jar:6.0.29]
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:368) [spring-security-web-3.0.5.RELEASE.jar:3.0.5.RELEASE]
    at org.fcrepo.server.security.jaas.AuthFilterJAAS.doFilter(AuthFilterJAAS.java:330) [fcrepo-security-jaas-3.5.jar:na]
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:380) [spring-security-web-3.0.5.RELEASE.jar:3.0.5.RELEASE]
    at org.springframework.security.web.access.channel.ChannelProcessingFilter.doFilter(ChannelProcessingFilter.java:109) [spring-security-web-3.0.5.RELEASE.jar:3.0.5.RELEASE]
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:380) [spring-security-web-3.0.5.RELEASE.jar:3.0.5.RELEASE]
    at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:169) [spring-security-web-3.0.5.RELEASE.jar:3.0.5.RELEASE]
    at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237) [spring-web-3.0.5.RELEASE.jar:3.0.5.RELEASE]
    at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167) [spring-web-3.0.5.RELEASE.jar:3.0.5.RELEASE]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) [catalina.jar:6.0.29]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) [catalina.jar:6.0.29]
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) [catalina.jar:6.0.29]
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) [catalina.jar:6.0.29]
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) [catalina.jar:6.0.29]
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) [catalina.jar:6.0.29]
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) [catalina.jar:6.0.29]
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) [catalina.jar:6.0.29]
    at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857) [tomcat-coyote.jar:6.0.29]
    at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) [tomcat-coyote.jar:6.0.29]
    at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) [tomcat-coyote.jar:6.0.29]
    at java.lang.Thread.run(Thread.java:662) [na:1.6.0_45]
Caused by: java.lang.NegativeArraySizeException: null
    at org.apache.commons.codec.binary.Base64.encodeBase64(Base64.java:258) [commons-codec-1.3.jar:1.3]
    at org.apache.commons.codec.binary.Base64.encodeBase64(Base64.java:178) [commons-codec-1.3.jar:1.3]
    at org.fcrepo.utilities.Base64.encode(Base64.java:27) [fcrepo-common-3.5.jar:na]
    at org.fcrepo.utilities.Base64.encode(Base64.java:47) [fcrepo-common-3.5.jar:na]
    at org.fcrepo.utilities.Base64.encodeToString(Base64.java:79) [fcrepo-common-3.5.jar:na]
    at org.fcrepo.server.storage.translation.FOXMLDOSerializer.appendDatastreams(FOXMLDOSerializer.java:369) [fcrepo-server-3.5.jar:na]
    at org.fcrepo.server.storage.translation.FOXMLDOSerializer.serialize(FOXMLDOSerializer.java:118) [fcrepo-server-3.5.jar:na]
    at org.fcrepo.server.storage.translation.DOTranslatorImpl.serialize(DOTranslatorImpl.java:99) [fcrepo-server-3.5.jar:na]
    at org.fcrepo.server.storage.translation.DOTranslatorModule.serialize(DOTranslatorModule.java:139) [fcrepo-server-3.5.jar:na]
    at org.fcrepo.server.storage.SimpleDOReader.Export(SimpleDOReader.java:203) [fcrepo-server-3.5.jar:na]
    at org.fcrepo.server.management.DefaultManagement.export(DefaultManagement.java:330) [fcrepo-server-3.5.jar:na]
    at sun.reflect.GeneratedMethodAccessor119.invoke(Unknown Source) [na:na]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [na:1.6.0_45]
    at java.lang.reflect.Method.invoke(Method.java:597) [na:1.6.0_45]
    at org.fcrepo.server.messaging.NotificationInvocationHandler.invoke(NotificationInvocationHandler.java:68) [fcrepo-server-3.5.jar:na]
    at com.sun.proxy.$Proxy6.export(Unknown Source) [na:na]
    at org.fcrepo.server.management.ManagementModule.export(ManagementModule.java:279) [fcrepo-server-3.5.jar:na]
    at org.fcrepo.server.rest.FedoraObjectResource.getObjectExport(FedoraObjectResource.java:92) [fcrepo-server-3.5.jar:na]
    ... 42 common frames omitted

I would appreciate any eyeballs, thoughts, and suggestions, especially on how to debug it further, now that's not a resources issue...

https://jira.duraspace.org/browse/FCREPO-944
...was reportedly solved way back in 2011 for Fedora 3.6; I'm dreading the thought of upgrading Fedora and all it's mated components, but if that's what I have to do, then that's what I will have to do.

Advice??   I know someone out there is reading this... :-)

Sincerely,

Nick Ruest

unread,

Oct 31, 2014, 5:08:20 PM10/31/14

to isla...@googlegroups.com

What are your JAVA_OPTS and PHP memory limits?

I've ingested files up to 15G without an issue.

-nruest

> --
> For more information about using this group, please read our Listserv
> Guidelines: http://islandora.ca/content/welcome-islandora-listserv
> ---
> You received this message because you are subscribed to the Google
> Groups "islandora" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to islandora+...@googlegroups.com
> <mailto:islandora+...@googlegroups.com>.
> Visit this group at http://groups.google.com/group/islandora.
> For more options, visit https://groups.google.com/d/optout.

Brad Spry

unread,

Oct 31, 2014, 8:15:25 PM10/31/14

to isla...@googlegroups.com

JAVA_OPTS=-Xms10g -Xmx14g -XX:MaxPermSize=1024m -XX:+UseParNewGC -XX:+CMSParallelRemarkEnabled -XX:+UseConcMarkSweepGC -XX:ParallelGCThreads=4

memory_limit = 8192M

Found these cool settings here:

https://github.com/Islandora/islandora_deployments/blob/master/York%20University.md

Happy Halloween /\/\òó/\/\

Adam Vessey

unread,

Nov 1, 2014, 1:06:44 PM11/1/14

to isla...@googlegroups.com

I'm suspecting you have islandora_bagit installed? The stack trace you provided indicates it's failing when exporting the object in "archive" format, and the "archive" format does in fact Base64 encode all the binary datastreams... So yeah, looks like FCREPO-944 is indeed rearing its head.

Digging around on Github, I found: https://github.com/Islandora/islandora_bagit/blob/7.x/islandora_bagit.module#L903-L917
And I'm assuming that at some point, it hits: https://github.com/Islandora/islandora_bagit/blob/7.x/islandora_bagit.module#L903-L917

I'm thinking it comes down to either updating Fedora if you wish to use Bagit, or disabling Bagit... Or possibly patch Bagit such that it doesn't actually have to perform the "archive" export... Not sure that I'd recommend it, but hypothetically, one might export for "migrate" and jam in the Base64'd datastreams PHP-side (or somewhere else outside of Fedora, anyway) to effectively transform the "migrate" FOXML into "archive" FOXML. Alternatively, a try/catch bit around the export and drupal_set_message()-ing something may suffice?

- Adam

To unsubscribe from this group and stop receiving emails from it, send an email to islandora+...@googlegroups.com.

Nick Ruest

unread,

Nov 3, 2014, 6:47:25 AM11/3/14

to isla...@googlegroups.com

Looks like we need to create a ticket for BagIt?

-nruest

>> > an email to islandora+...@googlegroups.com <javascript:>
>> > <mailto:islandora+...@googlegroups.com <javascript:>>.

>> > Visit this group at http://groups.google.com/group/islandora

>> <http://groups.google.com/group/islandora>.

>> > For more options, visit https://groups.google.com/d/optout

>> <https://groups.google.com/d/optout>.

Adam Vessey

unread,

Nov 3, 2014, 7:30:49 AM11/3/14

to isla...@googlegroups.com

Probably... Actually, it looks like you might've run into this before,
Nick:
https://github.com/Islandora/islandora_bagit/issues/8#issuecomment-37398323
:P

Not actually sure which approach would be best... Possibly trying the
"archive" export, falling back to the "migrate" "enrichment" (not sure
what to call it), possibly falling back to throw a message? Or just let
the exception through if the "migrate" stuff fails?... I'm... hesitant
to suggest this, but this could possibly be done in Tuque, as it would
potentially be an issue anywhere else we might happen to export in the
"archive" format... Yeah, dunno.

- Adam

Nick Ruest

unread,

Nov 3, 2014, 10:28:56 AM11/3/14

to isla...@googlegroups.com

Oh, hey! Would ya look at that. Nice catch Adam!

Hopefully Mark Jordan will notice this thread. If not, I'll just prod
him into noticing :-)

-nruest

Mark Jordan

unread,

Nov 3, 2014, 1:42:31 PM11/3/14

to isla...@googlegroups.com

Yes, nice catch Adam. Not sure what the best solution is either, other than flag it as a known issue for versions of Fedora less than 3.6 (which is the version that had the commit resolved https://jira.duraspace.org/browse/FCREPO-944).

Mark

To unsubscribe from this group and stop receiving emails from it, send an email to islandora+...@googlegroups.com.

Brad Spry

unread,

Nov 3, 2014, 2:51:11 PM11/3/14

to isla...@googlegroups.com

Thank You All for your eyes, thoughts, and feedback!

Any advice for upgrading Fedora from 3.5 to 3.6?

Here's a tiny bit of documentation:

https://wiki.duraspace.org/display/FEDORA36/Upgrading+from+3.x

Brad

Brad Spry

unread,

Nov 3, 2014, 5:11:56 PM11/3/14

to isla...@googlegroups.com

Grievously, I just filed:

Proposal to eliminate Fedora 3.5 as Supported Version from all official Islandora documentation, on the sole basis of Blocker bug:
https://jira.duraspace.org/browse/FCREPO-944

Brad Spry

unread,

Nov 3, 2014, 5:18:16 PM11/3/14

to isla...@googlegroups.com

My proposal:

https://jira.duraspace.org/browse/ISLANDORA-1128

Brad Spry

unread,

Jan 8, 2015, 5:37:19 PM1/8/15

to isla...@googlegroups.com

This issue I was experiencing is isolated to the generation of bags containing FOXML using islandora_bagit's plugin_object_foxml.inc

The issue can be nullified by discontinuing use of the aforementioned object plugin.

<B

Reply all

Reply to author

Forward