Fwd: Build failed in Jenkins: okapi #2112

Chase Tingley

unread,

Jan 3, 2018, 12:02:55 AM1/3/18

to okapi...@googlegroups.com

The build is getting that "Xvfb stopping" issue which we've seen occasionally in the past -- currently while running XLIFFFilterTest, which I just added an extra test to. (That test file is very large, probably larger than is healthy for a single file in terms of both code and static data, so maybe that's related.) This condition has gone away on its own in the past so I'm going to let it sit for a bit. I can't reproduce the failure except in Jenkins.

---------- Forwarded message ----------
From: <jenkins-...@cloudbees.com>
Date: Tue, Jan 2, 2018 at 9:58 PM
Subject: Build failed in Jenkins: okapi #2112
To: okapi-...@googlegroups.com, jhar...@lingotek.com, tin...@gmail.com

See <https://okapi.ci.cloudbees.com/job/okapi/2112/display/redirect>

[....]

-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running net.sf.okapi.filters.xliff.XLIFFFilterCtypeTest
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.251 sec - in net.sf.okapi.filters.xliff.XLIFFFilterCtypeTest
Running net.sf.okapi.filters.xliff.XLIFFFilterLengthConstraintsTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.009 sec - in net.sf.okapi.filters.xliff.XLIFFFilterLengthConstraintsTest
Running net.sf.okapi.filters.xliff.XLIFFFilterSDLPropTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.02 sec - in net.sf.okapi.filters.xliff.XLIFFFilterSDLPropTest
Running net.sf.okapi.filters.xliff.XLIFFFilterBalancingTest
[main] WARN net.sf.okapi.filters.xliff.XLIFFFilter - The source language declared in <file> is 'de' not 'en'.
[main] WARN net.sf.okapi.filters.xliff.XLIFFFilter - The target language declared in <file> is 'en' not 'fr'. 'en' will be used.
[main] WARN net.sf.okapi.filters.xliff.XLIFFFilter - The source language declared in <file> is 'de' not 'en'.
[main] WARN net.sf.okapi.filters.xliff.XLIFFFilter - The target language declared in <file> is 'it' not 'fr'. 'it' will be used.
[main] ERROR net.sf.okapi.filters.xliff.XLIFFFilter - The <seg-source> content for the entry id='1' is different from its <source>. The un-segmented content of <source> will be used.
[main] WARN net.sf.okapi.filters.xliff.XLIFFFilter - The source language declared in <file> is 'de' not 'en'.
[main] WARN net.sf.okapi.filters.xliff.XLIFFFilter - The target language declared in <file> is 'it' not 'fr'. 'it' will be used.
[main] ERROR net.sf.okapi.filters.xliff.XLIFFFilter - The <seg-source> content for the entry id='1' is different from its <source>. The un-segmented content of <source> will be used.
[main] WARN net.sf.okapi.filters.xliff.XLIFFFilter - The source language declared in <file> is 'de' not 'en'.
[main] WARN net.sf.okapi.filters.xliff.XLIFFFilter - The target language declared in <file> is 'it' not 'fr'. 'it' will be used.
[main] WARN net.sf.okapi.filters.xliff.XLIFFFilter - The source language declared in <file> is 'de' not 'en'.
[main] WARN net.sf.okapi.filters.xliff.XLIFFFilter - The target language declared in <file> is 'it' not 'fr'. 'it' will be used.
[main] WARN net.sf.okapi.filters.xliff.XLIFFFilter - The source language declared in <file> is 'de' not 'en'.
[main] WARN net.sf.okapi.filters.xliff.XLIFFFilter - The target language declared in <file> is 'en' not 'fr'. 'en' will be used.
[main] WARN net.sf.okapi.filters.xliff.XLIFFFilter - The source language declared in <file> is 'de' not 'en'.
[main] WARN net.sf.okapi.filters.xliff.XLIFFFilter - The target language declared in <file> is 'it' not 'fr'. 'it' will be used.
[main] WARN net.sf.okapi.filters.xliff.XLIFFFilter - The source language declared in <file> is 'de' not 'en'.
[main] WARN net.sf.okapi.filters.xliff.XLIFFFilter - The target language declared in <file> is 'it' not 'fr'. 'it' will be used.
[main] WARN net.sf.okapi.filters.xliff.XLIFFFilter - The source language declared in <file> is 'de' not 'en'.
[main] WARN net.sf.okapi.filters.xliff.XLIFFFilter - The target language declared in <file> is 'it' not 'fr'. 'it' will be used.
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.064 sec - in net.sf.okapi.filters.xliff.XLIFFFilterBalancingTest
Running net.sf.okapi.filters.xliff.SdlXliffConfLevelTest
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.003 sec - in net.sf.okapi.filters.xliff.SdlXliffConfLevelTest
Running net.sf.okapi.filters.xliff.XLIFFFilterEquivTextTest
Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.044 sec - in net.sf.okapi.filters.xliff.XLIFFFilterEquivTextTest
Running net.sf.okapi.filters.xliff.XLIFFFilterTest
[main] WARN net.sf.okapi.filters.xliff.XLIFFFilter - The source language declared in <file> is 'en-US' not 'en'.
[main] WARN net.sf.okapi.filters.xliff.XLIFFFilter - The source language declared in <file> is 'en-US' not 'en'.
[main] WARN net.sf.okapi.filters.xliff.XLIFFFilter - The source language declared in <file> is 'en-US' not 'en'.
[main] WARN net.sf.okapi.filters.xliff.XLIFFFilter - The source language declared in <file> is 'en-US' not 'en'.
[main] WARN net.sf.okapi.filters.xliff.XLIFFFilter - The source language declared in <file> is 'en-US' not 'en'.
[main] WARN net.sf.okapi.filters.xliff.XLIFFFilter - The source language declared in <file> is 'en-US' not 'en'.
[main] WARN net.sf.okapi.filters.xliff.XLIFFFilter - The source language declared in <file> is 'en-US' not 'en'.
[main] WARN net.sf.okapi.filters.xliff.XLIFFFilter - The source language declared in <file> is 'en-US' not 'en'.
[main] WARN net.sf.okapi.filters.xliff.XLIFFFilter - The target language declared in <file> is 'es' not 'fr'. 'es' will be used.
[main] WARN net.sf.okapi.filters.xliff.XLIFFFilter - The target language declared in <file> is 'fr' not 'es'. 'fr' will be used.
[main] WARN net.sf.okapi.filters.xliff.XLIFFFilter - The target language declared in <file> is 'es' not 'fr'. 'es' will be used.
[main] WARN net.sf.okapi.filters.xliff.XLIFFITSFilterExtension - ITS Localization Quality Issue data category is to be used only on the main source and target and on mrk.
[main] WARN net.sf.okapi.filters.xliff.XLIFFFilter - The source language declared in <file> is 'en-US' not 'en'.
[main] WARN net.sf.okapi.filters.xliff.XLIFFFilter - The target language declared in <file> is 'es-ES' not 'fr'. 'es-ES' will be used.
[main] WARN net.sf.okapi.filters.xliff.XLIFFFilter - The source language declared in <file> is 'en' not 'en-US'.
[main] WARN net.sf.okapi.filters.xliff.XLIFFFilter - The target language declared in <file> is 'de' not 'es-ES'. 'de' will be used.
[main] WARN net.sf.okapi.filters.xliff.XLIFFFilter - The source language declared in <file> is 'en' not 'en-US'.
[main] WARN net.sf.okapi.filters.xliff.XLIFFFilter - The target language declared in <file> is 'de' not 'es-ES'. 'de' will be used.
[main] WARN net.sf.okapi.filters.xliff.XLIFFFilter - The target language declared in <file> is 'fr-FR' not 'es-ES'. 'fr-FR' will be used.
[main] WARN net.sf.okapi.filters.xliff.XLIFFFilter - The target language declared in <file> is 'fr-FR' not 'es-ES'. 'fr-FR' will be used.
Xvfb stopping
ERROR: Maven JVM terminated unexpectedly with exit code 137

Mihai Nita

unread,

Jan 4, 2018, 3:04:58 AM1/4/18

to Group: okapi-devel

Triggered a build, worked fine. Now is running the integration tests.

I didn't have to do anything, just started the build.

One of those... :-(

Mihai

--
You received this message because you are subscribed to the Google Groups "okapi-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to okapi-devel+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Mihai Nita

unread,

Jan 4, 2018, 6:30:05 PM1/4/18

to Group: okapi-devel

Troubles... builds just fail, and fail.

I don't think it has anything to do with Xvfb, the "Xvfb stopping" seem to be the normal message when Xvfb is stopped at the end of the build. I have set "Log Xvfb output" and it looks fine.

I think that the real cause is this:

   ERROR: Maven JVM terminated unexpectedly with exit code 137

That looks like it is about not having enough memory on the Jenkins machine.

I have spent a lot of time on it...

Executed shell scripts as pre / post build steps, listed permissions on folders, listed all running tasks, dump environment, show free memory...

* I didn't start the Xvfb at all

* I have tried setting Xmx in MAVEN_OPTS to nothing and to 1024m and to 2048m

(before all this is used to be 512m, not it is back to 512m)
It looks like the machine a memory of 1024 MB, so 2048 is pointless (or starts swapping?)

* I have tried using Oracle JDK instead of Open JDK

* I have restarted Jenkins (but that does not restart the machine, so I don't know if it helps much)
* I have changed the default DEV@cloud Agent Template from lxc-fedora17 to lxc-fedora25
* I have changed the "Default Size" of lxc-fedora25 from "small" to "standard"

Of course, most of this stuff I reverted :-)

Nothing seem to work...

It breaks in slightly different places, depending on how the wind blows, or the temperature in Tokyo, or something...

And (very rarely) succeeds...

Most posts about Maven JVM ... exit code 137 claim that this can't be controlled with MAVEN_OPTS,
and all you can do is "add more memory" or "reboot the machine" (for a temporary solution)

    https://issues.jenkins-ci.org/browse/JENKINS-12035
    https://groups.google.com/forum/#!topic/jenkinsci-dev/V3wLEbDR2Sw (looks like this one is from the Cloudbees guys)
    https://stackoverflow.com/questions/24989653/jenkins-maven-build-137-error
    https://plus.google.com/+I88Ca/posts/F3pTY8vx7wE

Tonight I can also try creating and using a Jenkins pipeline job, maybe that takes less memory...

But we will still be on the edge...

Mihai

Chase Tingley

unread,

Jan 4, 2018, 6:40:28 PM1/4/18

to okapi...@googlegroups.com

I think we're still running with -TC8 during the builds -- maybe that's subdividing the JVM enough that some of the bigger tests can't run concurrently?

Mihai Nita

unread,

Jan 4, 2018, 8:35:58 PM1/4/18

to Group: okapi-devel

Got home and checked.

We don't do that anymore.

The current command (for the okapi jobs) is
clean install -DskipITs -q -U -P release_or_snapshot

I have another idea: uninstall a bunch of Jenkins plugins.

There are a few of plugins that we don't use (not that many though)

Don't know how smart Jenkins is, if unused plugins get loaded or not.

Mihai

Mihai Nita

unread,

Jan 5, 2018, 1:05:49 PM1/5/18

to Group: okapi-devel

I am completely blocked...I don't know what to do anymore...

I have created a pipeline for the integration tests (that's the simplest job, because there is no publishing, no dependent jobs, etc)

Triggered by hand only.

And still fails, with the same error as the older integration tests job.

I have enabled all the logging possible, disabled all parallel processing.
I tried various Xmx settings, no visible difference

It fails in different places, but it fails, both with small and large executors

So it might be some resource exhaustion, but not necessarily memory

My suspects:

* Some commit in Okapi changed some test consume more memory or to leak resources
* Maybe the move from JDK 7 to JDK 8 is the cause (although I don't see any message to make me believe that...)

But thins started to fail before that...

* Some updates to Jenkins done by Cloudbees?

Things work fine on my side (also on Linux, also with JDK 8)

But I don't see anything, in any logs...

If anyone want to take a look, the pipeline job (okapi-integration-tests-pipeline) has full logging enabled (mvn run with -X and -e)

The error is below.

The "starting" point is the "Killed" line. The place where this happens is not fixed.

Mihai

===============

[main] INFO net.sf.okapi.simplifier.integration.RoundTripSimplifyIdmlTkitsIT - sample_flyer1.idml
[main] INFO net.sf.okapi.common.pipelinedriver.PipelineDriver - Input: /scratch/jenkins/workspace/okapi-integration-tests-pipeline/target/test-classes/idml/sample_flyer1.idml
[main] INFO net.sf.okapi.common.pipelinedriver.PipelineDriver - Input: /scratch/jenkins/workspace/okapi-integration-tests-pipeline/target/test-classes/idml/sample_flyer1.idml.simplify_xliff
[main] INFO net.sf.okapi.common.pipelinedriver.PipelineDriver - Input: /scratch/jenkins/workspace/okapi-integration-tests-pipeline/target/test-classes/idml/sample_flyer1.idml
[main] INFO net.sf.okapi.common.pipelinedriver.PipelineDriver - Input: /scratch/jenkins/workspace/okapi-integration-tests-pipeline/target/test-classes/idml/sample_flyer1.idml.simplify_xliff
[main] INFO net.sf.okapi.simplifier.integration.RoundTripSimplifyIdmlTkitsIT - sample_photobook2.idml
[main] INFO net.sf.okapi.common.pipelinedriver.PipelineDriver - Input: /scratch/jenkins/workspace/okapi-integration-tests-pipeline/target/test-classes/idml/sample_photobook2.idml
[main] INFO net.sf.okapi.common.pipelinedriver.PipelineDriver - Input: /scratch/jenkins/workspace/okapi-integration-tests-pipeline/target/test-classes/idml/sample_photobook2.idml.simplify_xliff
[main] INFO net.sf.okapi.common.pipelinedriver.PipelineDriver - Input: /scratch/jenkins/workspace/okapi-integration-tests-pipeline/target/test-classes/idml/sample_photobook2.idml
/bin/sh: line 1:   968 Killed                  /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.141-1.b16.fc25.x86_64/jre/bin/java -jar /scratch/jenkins/workspace/okapi-integration-tests-pipeline/target/surefire/surefirebooter3459363605574156290.jar /scratch/jenkins/workspace/okapi-integration-tests-pipeline/target/surefire/surefire2886268142684985990tmp /scratch/jenkins/workspace/okapi-integration-tests-pipeline/target/surefire/surefire_02103021228150129213tmp

Results :

Tests run: 48, Failures: 0, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 24:30 min
[INFO] Finished at: 2018-01-05T11:44:12-05:00
[INFO] Final Memory: 35M/758M
[INFO] ------------------------------------------------------------------------
[INFO] [jenkins-event-spy] Generated /scratch/jenkins/workspace/okapi-integration-tests-pipeline@tmp/withMavenb63be6ea/maven-spy-20180105-111941-9615076719317886241306.log
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-failsafe-plugin:2.19:integration-test (default) on project okapi-integration-tests: Execution default of goal org.apache.maven.plugins:maven-failsafe-plugin:2.19:integration-test failed: The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
[ERROR] Command was /bin/sh -c cd /scratch/jenkins/workspace/okapi-integration-tests-pipeline && /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.141-1.b16.fc25.x86_64/jre/bin/java -jar /scratch/jenkins/workspace/okapi-integration-tests-pipeline/target/surefire/surefirebooter3459363605574156290.jar /scratch/jenkins/workspace/okapi-integration-tests-pipeline/target/surefire/surefire2886268142684985990tmp /scratch/jenkins/workspace/okapi-integration-tests-pipeline/target/surefire/surefire_02103021228150129213tmp
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-failsafe-plugin:2.19:integration-test (default) on project okapi-integration-tests: Execution default of goal org.apache.maven.plugins:maven-failsafe-plugin:2.19:integration-test failed: The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
Command was /bin/sh -c cd /scratch/jenkins/workspace/okapi-integration-tests-pipeline && /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.141-1.b16.fc25.x86_64/jre/bin/java -jar /scratch/jenkins/workspace/okapi-integration-tests-pipeline/target/surefire/surefirebooter3459363605574156290.jar /scratch/jenkins/workspace/okapi-integration-tests-pipeline/target/surefire/surefire2886268142684985990tmp /scratch/jenkins/workspace/okapi-integration-tests-pipeline/target/surefire/surefire_02103021228150129213tmp
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:224)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
	at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
	at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
	at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
	at org.apache.maven.cli.MavenCli.execute(MavenCli.java:862)
	at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:286)
	at org.apache.maven.cli.MavenCli.main(MavenCli.java:197)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
	at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
	at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.PluginExecutionException: Execution default of goal org.apache.maven.plugins:maven-failsafe-plugin:2.19:integration-test failed: The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
Command was /bin/sh -c cd /scratch/jenkins/workspace/okapi-integration-tests-pipeline && /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.141-1.b16.fc25.x86_64/jre/bin/java -jar /scratch/jenkins/workspace/okapi-integration-tests-pipeline/target/surefire/surefirebooter3459363605574156290.jar /scratch/jenkins/workspace/okapi-integration-tests-pipeline/target/surefire/surefire2886268142684985990tmp /scratch/jenkins/workspace/okapi-integration-tests-pipeline/target/surefire/surefire_02103021228150129213tmp
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:145)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
	... 20 more
Caused by: java.lang.RuntimeException: The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
Command was /bin/sh -c cd /scratch/jenkins/workspace/okapi-integration-tests-pipeline && /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.141-1.b16.fc25.x86_64/jre/bin/java -jar /scratch/jenkins/workspace/okapi-integration-tests-pipeline/target/surefire/surefirebooter3459363605574156290.jar /scratch/jenkins/workspace/okapi-integration-tests-pipeline/target/surefire/surefire2886268142684985990tmp /scratch/jenkins/workspace/okapi-integration-tests-pipeline/target/surefire/surefire_02103021228150129213tmp
	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:591)
	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:461)
	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:229)
	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:201)
	at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1014)
	at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:850)
	at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:739)
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
	... 21 more
[ERROR] 
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
[Pipeline] }
[withMaven] junitPublisher - Archive test results for Maven artifact net.sf.okapi:okapi-integration-tests:1.0.0 generated by maven-surefire-plugin:test (default-test): target/surefire-reports/*.xml
Recording test results
None of the test reports contained any result
[withMaven] junitPublisher - Archive test results for Maven artifact net.sf.okapi:okapi-integration-tests:1.0.0 generated by maven-failsafe-plugin:integration-test (default): target/failsafe-reports/*.xml
Recording test results
[withMaven] Skip 'Open Task Scanner Publisher' disabled by configuration
[Pipeline] // withMaven
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
ERROR: script returned exit code 1
Finished: FAILURE

Mihai Nita

unread,

Jan 5, 2018, 2:18:28 PM1/5/18

to Group: okapi-devel

The okapi-integration-tests pipeline version succeeds if I switch it back to JDK 7.

The only differences between build #8 and #10 are

* java source and target in pom.xml changed from 1.8 to 1.7
* the pipeline script changed from
withMaven(jdk: 'OpenJDK 8 (latest)', maven: 'Maven 3.3.3', ...
to
withMaven(jdk: 'OpenJDK 7 (latest)', maven: 'Maven 3.3.3', ...

I will try another build with the pom 1.7 and build with 1.8, see if it fails...

Don't know if / how that might help, but I don't have other ideas :-)

Mihai

Jim Hargrave

unread,

Jan 5, 2018, 9:40:19 PM1/5/18

to okapi...@googlegroups.com, Mihai Nita

I changed the okapi version to 0.36-SNAPSHOT (built with 1.8). Then changed the compiler back to 1.8 for the integration tests. Not sure if this fixes the problem. Also changed the repository enable flag from truefalse to true

Jim

--
You received this message because you are subscribed to the Google Groups "okapi-devel" group.

To unsubscribe from this group and stop receiving emails from it, send an email to okapi-devel...@googlegroups.com.

Mihai Nita

unread,

Jan 5, 2018, 11:01:31 PM1/5/18

to Jim Hargrave, Group: okapi-devel

Unfortunately no, still failing, with the same error.

Playing with the pipeline version I have tried to narrow down the case.

So I tried to combine thw jdk versions with the versions in the pom.

jdk pom result

8 1.8 fails

8 1.7 fails

7 1.8 can't generate 1.8 code with a 7 compiler

7 1.7 success

So it looks like it is something about the jdk 8, maybe in combination with something in the cloudbees environment. Nothing fails in my docker Jenkins.

Over the weekend I want to setup a git repo at home so that I stop messing with the public one.

And I will also try to build a "small" instance (1gb of ram, few cores, and reduced storage space) using either docker or a virtual machine.

Mihai

To unsubscribe from this group and stop receiving emails from it, send an email to okapi-devel+unsubscribe@googlegroups.com.

Yves Savourel

unread,

Jan 7, 2018, 4:04:33 PM1/7/18

to okapi...@googlegroups.com

FWIW:

I get the main okapi project to build OK.

But I do get 4 errors with the integration-tests.

However they look like “normal” errors (i.e. differences with the gold files) not weird issues like on Jenkins.

-ys

--

You received this message because you are subscribed to the Google Groups "okapi-devel" group.

To unsubscribe from this group and stop receiving emails from it, send an email to okapi-devel...@googlegroups.com.

Jim Hargrave

unread,

Jan 8, 2018, 3:45:15 PM1/8/18

to okapi...@googlegroups.com, Yves Savourel

Are we at a point where I can remove the jackson code? Didn't want to add more complexity. I can also fix the integration tests.

BTW: I will be moving the jackson code to the sandbox.

Jim

Mihai Nita

unread,

Jan 8, 2018, 5:22:58 PM1/8/18

to Group: okapi-devel, Yves Savourel

The integrations test still fail to build :-(

I have spent some time over the week-end, but with nothing much to show for it.

I have created a Jenkins server in a Virtual machine and played with the memory.

At 1 GB and 2 GB it fails with the same stack dump as the Cloudbees one.

At 4 GB it succeeds.

So it really looks like a memory problem.

Java 7 also worked with 1 GB. So it is very much like Cloudbees.

I have also spent some time in visualvm.

There are several "humps", one of them when the sdlxliff tests start.

At the same time the number of loaded classes "jumps"

The max heap (as shown by visualvm) is between 2.5GB and 3GB

I have then tried to tell maven to not be so memory hungry.

Tried both Xmx and XX:MaxMetaspaceSize (Metaspace replaces PermGen)

And tried that both as MAVEN_OPTS, and as a setting for maven-surefire-plugin

(each separate and both in the same time)

But it does not seem to have any effect, visualvm shows the heap still being somewhere between 2.5GB and 3GB

Slowly running out of ideas...

I am thinking to split the one pom running all integrations it separate modules,
with separate poms, called from the main one.

My hope is that this will reduce the number of loaded classes (if each module starts with a clean jvm)

What still puzzles me:

* that Xmx and XX:MaxMetaspaceSize don't seem to have any visible effect

* that Cloudbees still fails even if I use a standard executor, which has 4 GB of memory

Mihai

To unsubscribe from this group and stop receiving emails from it, send an email to okapi-devel+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "okapi-devel" group.

To unsubscribe from this group and stop receiving emails from it, send an email to okapi-devel+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "okapi-devel" group.

To unsubscribe from this group and stop receiving emails from it, send an email to okapi-devel+unsubscribe@googlegroups.com.

Mihai Nita

unread,

Jan 8, 2018, 5:25:38 PM1/8/18

to Group: okapi-devel, Yves Savourel

To answer the jackson code part: I think you can do it at any time...

Should not affect the jdk 7 - 8 migration.

Except that is means that the integration does not run, so you are doing it without a safety net :-)

Mihai

On Mon, Jan 8, 2018 at 12:45 PM, Jim Hargrave <jhargr...@gmail.com> wrote:

To unsubscribe from this group and stop receiving emails from it, send an email to okapi-devel+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "okapi-devel" group.

To unsubscribe from this group and stop receiving emails from it, send an email to okapi-devel+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "okapi-devel" group.

To unsubscribe from this group and stop receiving emails from it, send an email to okapi-devel+unsubscribe@googlegroups.com.

Mihai Nita

unread,

Jan 8, 2018, 5:27:03 PM1/8/18

to Group: okapi-devel, Yves Savourel

> Except that is means that the integration does not run, so you are doing it without a safety net :-)

Let me rephrase that: you can run the integration tests locally, if you have more than 4 GB or RAM (and who doesn't ? :-)

M.

Reply all

Reply to author

Forward