DSpace 6.3. Internal system error sending resource. Heisenbug

[Кирилл Гайдук]

Good day!

There is a repository based on DSpace 6.3 and Ubuntu 16.04. Previously, there were no problems, but after transferring to a new server, errors appeared when sending a new resource. Namely:

1. "Internal system error" after clicking the "Next" button on the file selection page. The file selection page corresponds to the choose-file.jsp file from dspace/webapps/jspui/submit - there are no problems here. After clicking "Next", upload-file-list.jsp starts executing - an error occurs here. More precisely, an error occurs in the following place in the code - if the list of bitstreams is (for unknown reasons) empty.

List<Bitstream> bitstreams = ContentServiceFactory.getInstance().getItemService().getNonInternalBitstreams(context, subInfo.getSubmissionItem().getItem());
List<Bundle> bundles = null;

if (bitstreams.get(0) != null) {
    bundles = bitstreams.get(0).getBundles();
}

The following is displayed in /var/log/tomcat8/localhost.[date].log:

20-Nov-2019 13:58:30.373 SEVERE [http-nio-8080-exec-23] org.apache.catalina.core.ApplicationDispatcher.invoke Servlet.service() for servlet jsp threw exception
 java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
    at java.util.ArrayList.rangeCheck(ArrayList.java:657)
    at java.util.ArrayList.get(ArrayList.java:433)
    at org.apache.jsp.submit.upload_002dfile_002dlist_jsp._jspService(upload_002dfile_002dlist_jsp.java:414)
    at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
    at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:438)
    at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:401)
    at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:345)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:292)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
    at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:720)
    at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:466)
    at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:391)
    at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:318)
    at org.dspace.app.webui.util.JSPManager.showJSP(JSPManager.java:64)
    at org.dspace.app.webui.submit.JSPStepManager.showJSP(JSPStepManager.java:370)
    at org.dspace.app.webui.submit.step.JSPUploadStep.showUploadFileList(JSPUploadStep.java:580)
    at org.dspace.app.webui.submit.step.JSPUploadStep.showUploadPage(JSPUploadStep.java:498)
    at org.dspace.app.webui.submit.step.JSPUploadStep.doPostProcessing(JSPUploadStep.java:452)
    at org.dspace.app.webui.submit.JSPStepManager.doStepEnd(JSPStepManager.java:407)
    at org.dspace.app.webui.submit.JSPStepManager.processStep(JSPStepManager.java:250)
    at org.dspace.app.webui.servlet.SubmissionController.doStep(SubmissionController.java:505)
    at org.dspace.app.webui.servlet.SubmissionController.doDSPost(SubmissionController.java:443)
    at org.dspace.app.webui.servlet.DSpaceServlet.processRequest(DSpaceServlet.java:122)
    at org.dspace.app.webui.servlet.DSpaceServlet.doPost(DSpaceServlet.java:80)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:648)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:292)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
    at org.dspace.app.webui.filter.RegisteredOnlyFilter.doFilter(RegisteredOnlyFilter.java:66)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
    at org.dspace.utils.servlet.DSpaceWebappServletFilter.doFilter(DSpaceWebappServletFilter.java:78)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:494)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
    at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:616)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:522)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1095)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:672)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1520)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1476)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Thread.java:748)

2. "Internal system error" after accepting the license agreement (the page corresponds to the show-license.jsp file). The problem is due to the presence of incorrect characters in the header of the HTTP request, as evidenced by the output in /var/log/tomcat8/catalina.[date].log

20-Nov-2019 16:18:33.882 INFO [http-nio-8080-exec-30] org.apache.coyote.http11.AbstractHttp11Processor.process Error parsing HTTP request header
 Note: further occurrences of HTTP header parsing errors will be logged at DEBUG level.
 java.lang.IllegalArgumentException: Invalid character found in the request target. The valid characters are defined in RFC 7230 and RFC 3986
    at org.apache.coyote.http11.AbstractNioInputBuffer.parseRequestLine(AbstractNioInputBuffer.java:286)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1009)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:672)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1520)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1476)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Thread.java:748)

Both errors are floating: may occur, may not occur. If you click the "Back" button, and try to repeat the request, it sometimes skips further. Sometimes it helps to go to "My resource archive", where a previously downloaded file may be displayed. If you delete the resource and try again, errors may not occur. But usually arise. No patterns were revealed.

Errors are not related to server load: more than enough memory and disk space. An error can occur even if you try to add a resource in the middle of the night.

Rebooting the server / tomcat / postgresql does not solve the problem.

I couldn’t enable debug mode, so there is no more detailed information.

[
What was done to enable debugging mode:
In the log4j.properties file
loglevel.dspace = DEBUG
loglevel.other = DEBUG
Reboot tomcat.
Nothing is output to the dspace.log. [Date] file.
]

I would be grateful for ideas / suggestions / suggestions.