JMeter plugin runs out of heap space, while JMeter GUI doesn't

[Martijn Vos]

Hi,

I'm new to JMeter and load testing in general. I was under the impression that a good load test

should be done without the JMeter GUI, so I used the maven jmeter plugin (hoping that it could

eventually be integrated into our build server, and maybe the jmeter analysis maven plugin

could analyze the results, because I have no idea how else to do that).

Problem is: maven runs out of heap space. When I increase the heap space to 2 GB, it can't

allocate it (my work PC isn't that big). At 1 GB, heap space error. I reduced the size of the

load test by more than half, and I still get that heap space error. Yet the JMeter GUI had no

problem running the original test (though it took a long time).

Interestingly, maven says it completed the test just before it throws the OutOfMemoryError.

So it seems to be after the real test that it runs out of memory.

So what's the problem here? Why would it run out of memory after the test is done? And why

does it need so much more memory than the GUI?

How can I fix this?

Added detail: my heaviest test stated out as 60 loops with 100 threads on 11 different URLs.

I have now reduced it to 10 loops with 100 threads.

Regards,

Martijn Vos.

[Mark Collin]

Are you using the latest version of the plugin?

If you are using the latest version of the plugin are you configuring the JVM that the JMeter process runs in?

https://github.com/Ronnie76er/jmeter-maven-plugin/wiki/Configuring%20the%20JVM%20that%20the%20JMeter%20process%20runs%20in

By default it will only set and -Xms and -Xmx of 512Mb, this may be what is causing you issues.

--
You received this message because you are subscribed to the Google Groups "Maven JMeter Plugin Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to maven-jmeter-plugi...@googlegroups.com.
To post to this group, send an email to maven-jmeter...@googlegroups.com.
Visit this group at http://groups.google.com/group/maven-jmeter-plugin-users.
For more options, visit https://groups.google.com/groups/opt_out.

[Martijn Vos]

I'm using 1.9.0. I initially used the maven config you linked to to give it more memory, but that seemed to be ignored. Then I used the settings of my IDE which worked, but at 1 GB it still ran out of memory, and at 2 GB it couldn't allocate any. I guess I could try killing everything on my PC and then running it with more memory, but why does it even need so much more memory than the GUI version does?

mcv.

[Mark Collin]

We are running JMeter in a separate JVM so you will have memory used by maven and the memory used by the new JVM we spawn.  Are you sure you are measuring memory usage correctly, if you aren’t measuring the memory used by the spawned JVM, but just maven that would explain why the memory settings appeared to be ignored.

[Martijn Vos]

That would certainly help me understand what's going on. But then it's not JMeter that's running out of memory (because changing the plugin's config didn't help), but the plugin itself (because changing the IDE's settings did help).

And the plugin mentioned that JMeter's tests finished correctly. So what does the plugin do after JMeter finishes? Could that be something that eats memory?

Or, also a possibility: I added this to an existing pom. I wonder if there's something the pom might trigger after the plugin finishes. Though I'd expect that a jmeter:jmeter target really only runs jmeter.

mcv.

--
You received this message because you are subscribed to a topic in the Google Groups "Maven JMeter Plugin Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/maven-jmeter-plugin-users/GKaEOrnj13c/unsubscribe.
To unsubscribe from this group and all of its topics, send an email to maven-jmeter-plugi...@googlegroups.com.

[Mark Collin]

How big are the results files being generated? the plugin will scan through the results files looking for a failure so it can fail the build/report errors so if the file is very big that could use up memory.

[Martijn Vos]

The largest result file is a 900 MB XML. The next are 370 MB and 37 MB (strange that the first difference isn't a factor 10). The largest CSV file is 1.5 MB.

So the XML is pretty big. Could that be the problem? Is there a way to reduce its size? The first few times I ran JMeter (from the GUI) it didn't generate XML at all; just a lot of identical CSV files.

mcv.

[Ardesco]

Have a look here to see the options for configuring the test results:

https://github.com/Ronnie76er/jmeter-maven-plugin/wiki/Test-Results-File-Format#wiki-4

To unsubscribe from this group and all of its topics, send an email to maven-jmeter-plugin-users+unsub...@googlegroups.com.

[Martijn Vos]

Thanks. I'm afraid I do want to stick to XML rather than CSV, because I'm planning to use the analysis plugin too. But maybe I could skip the error checking. Although if the error checking takes too much memory, I guess the analysis will take even more.

Perhaps the best option is to reduce the number of results by reducing the duration of the test. I started with looping 60 times, which with 100 threads means 6000 results. If instead I loop 20 times with 50 threads, I'd have only 1000 results. Not quite the load test I wanted, but it might seriously cut down on memory.

mcv.

To unsubscribe from this group and all of its topics, send an email to maven-jmeter-plugi...@googlegroups.com.

[Ardesco]

Do you do anything else with maven?

It may not be the plugin using up all the memory, just the plugin finishing it off.  I only ask this because I haven't heard of anybody else who is using it having memory problems (maybe nobody has said anything and they do see these issues).

To unsubscribe from this group and all of its topics, send an email to maven-jmeter-plugin-users+unsubsc...@googlegroups.com.

To post to this group, send an email to maven-jmeter...@googlegroups.com.

Visit this group at http://groups.google.com/group/maven-jmeter-plugin-users.
For more options, visit https://groups.google.com/groups/opt_out.

[Martijn Vos]

I originally added the jmeter plugin to an existing pom. Moving it to a pom of its own solved the memory issue. That is, it solved the memory issue until I added the jmeter-analysis-plugin, but that was solved by giving maven a bit more heapspace.

I'm now stuck at the next step, though: the jmeter-analysis-plugin gives me this error:

[ERROR] Failed to execute goal com.lazerycode.jmeter:jmeter-analysis-maven-plugin:1.0.4:analyze (default-cli) on project gCDS4OWATest-JMeter: The parameters 'source' for goal com.lazerycode.jmeter:jmeter-analysis-maven-plugin:1.0.4:analyze are missing or invalid -> [Help 1]

But I am specifying the source parameter, and I've taken the example directly from the github wiki. I've played around a bit by more precisely specifying the file, adding or removing 'target' in the path, but nothing works.

Here's what's in my pom:

            <plugin>

                <groupId>com.lazerycode.jmeter</groupId>

                <artifactId>jmeter-analysis-maven-plugin</artifactId>

                <version>1.0.4</version>

                <executions>

                    <execution>

                        <goals>

                            <goal>analyze</goal>

                        </goals>

                        <configuration>

<source>${project.build.directory}/jmeter/results/*.jtl</source>

<targetDirectory>${project.build.directory}/results</targetDirectory>

Any idea why this is somehow not legal?

Regards,

mcv.

To unsubscribe from this group and all of its topics, send an email to maven-jmeter-plugi...@googlegroups.com.

[Martijn Vos]

Having run mvn -X, it seems the maven-jmeter-analysis-plugin is ignoring the configuration. In several places in the debug log I see stuff like:

[DEBUG] Configuration: <?xml version="1.0" encoding="UTF-8"?>

<configuration>

  <maxSamples default-value="50000"/>

  <preserveDirectories default-value="false"/>

  <processAllFilesFound default-value="false"/>

  <targetDirectory default-value="${project.build.directory}"/>

</configuration>

and:

[DEBUG] Configuring mojo 'com.lazerycode.jmeter:jmeter-analysis-maven-plugin:1.0

.4:analyze' with basic configurator -->

[DEBUG]   (f) maxSamples = 50000

[DEBUG]   (f) preserveDirectories = false

[DEBUG]   (f) processAllFilesFound = false

[DEBUG]   (f) targetDirectory = C:\workspace\project\test\target

Which looks like default config, and not the config I specified. How is this possible?

mcv.

[Mark Collin]

I would suggest raising a bug, off the top of my head I have no idea.

You received this message because you are subscribed to the Google Groups "Maven JMeter Plugin Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to maven-jmeter-plugi...@googlegroups.com.

[Mark Collin]

I would suggest raising a bug, off the top of my head I have no idea.

On 4 Mar 2014, at 15:31, Martijn Vos <mcv...@gmail.com> wrote:

You received this message because you are subscribed to the Google Groups "Maven JMeter Plugin Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to maven-jmeter-plugi...@googlegroups.com.

[Arne Franken]

Hi Martijn,

I can't reproduce the behaviour you're describing.

Using the example workspace (https://github.com/afranken/jmeter-maven-plugin-example), I ran the test as-is:

[DEBUG] Configuring mojo 'com.lazerycode.jmeter:jmeter-analysis-maven-plugin:1.0.4:analyze' with basic configurator -->

[DEBUG]   (f) maxSamples = 50000

[DEBUG]   (f) preserveDirectories = false

[DEBUG]   (f) processAllFilesFound = false

[DEBUG]   (f) source = /Users/_/dev/github/jmeter-maven-plugin-example/target/jmeter/results/*

[DEBUG]   (f) targetDirectory = /Users/_/dev/github/jmeter-maven-plugin-example/target/reports

and after adding

<maxSamples>10000</maxSamples>

<preserveDirectories>true</preserveDirectories>

<processAllFilesFound>true</processAllFilesFound>

[DEBUG] Configuring mojo 'com.lazerycode.jmeter:jmeter-analysis-maven-plugin:1.0.4:analyze' with basic configurator -->

[DEBUG]   (f) maxSamples = 10000

[DEBUG]   (f) preserveDirectories = true

[DEBUG]   (f) processAllFilesFound = true

[DEBUG]   (f) source = /Users/_/dev/github/jmeter-maven-plugin-example/target/jmeter/results/*

[DEBUG]   (f) targetDirectory = /Users/_/dev/github/jmeter-maven-plugin-example/target/reports

I can only guess that somehow your configuration doesn't take when you're running the plugin.

Are you sure that you put the plugin configuration into the right POM? Maybe you're triggering the wrong profile, execution ID, ...

Cheers,

Arne

To post to this group, send email to maven-jmeter...@googlegroups.com.

[Martijn Vos]

Hi Arne,

I was working from the example at https://github.com/afranken/jmeter-analysis-maven-plugin, which I'm executing simply with `mvn jmeter-analysis:analyze`. (The JMeter results are already where they need to be, and I don't want to rerun the entire test while experimenting with the analysis plugin.)

I'm not entirely sure what the differences are, other than that your example is executed with `mvn verify -Pperformance` and defines a pile of stuff in the pom.xml which I think I have in the .jmx I created with the JMeter GUI.

But clearly with the way I'm using it, it doesn't seem to pick up the configuration from the pom, when I think it should. Shouldn't it?

mcv.

To unsubscribe from this group and all its topics, send an email to maven-jmeter-plugi...@googlegroups.com.
To post to this group, send email to maven-jmeter...@googlegroups.com.

[Arne Franken]

Hi Martijn,

I think that the problem is the direct invocation of the plugin combined with the way you put the configuration into your POM.

In the example workspace, I bound the plugin execution to the lifecycle phase "post-integration-test" which is executed when running "mvn verify".

This example configuration works for me with direct invocation:

<build>

    <plugins>

      <plugin>

        <groupId>com.lazerycode.jmeter</groupId>

        <artifactId>jmeter-analysis-maven-plugin</artifactId>

        <version>1.0.4</version>

        <configuration>

          <source>${project.build.directory}/jmeter/results/*</source>

          <targetDirectory>${project.build.directory}/reports</targetDirectory>

          <maxSamples>1000</maxSamples>

        </configuration>

      </plugin>

    </plugins>

  </build>

output:

[DEBUG] Configuring mojo 'com.lazerycode.jmeter:jmeter-analysis-maven-plugin:1.0.4:analyze' with basic configurator -->

[DEBUG]   (f) maxSamples = 1000

[DEBUG]   (f) preserveDirectories = false

...

Cheers,

Arne

[Martijn Vos]

Hi Arne,

I've run the test with your previous example (https://github.com/afranken/jmeter-maven-plugin-example), and it seems to pick up the configuration correctly. However, I seem to have two different problems:

1) When I run it together with the jmeter-maven-plugin, maven still runs out of memory after the jmeter test, before the analysis plugin starts.

2) When I run the analysis plugin on its own (by commenting out the jmeter-plugin) it works, but it picks up only the oldest .jtl (from a previous test) and not the latest one. I've got two different test plans that I want to run, and it'd be nice if it picked up both or just the most recent one.

But I'll try it again with your new example config.

Regards,

Martijn.

For more options, visit https://groups.google.com/d/optout.

[Martijn Vos]

Hi Arne,

I got it working with this configuration, which works quite nice. I can run just jmeter with `mvn jmeter:jmeter`, or just the analysis with `mvn jmeter-analysis:analyze`. I still can't run them together, because mvn still runs out of memory when I do that.

Unfortunately, it turns out jmeter-analysis doesn't quite analyze it the way I'd hoped: it throws the results of my different test sets in one big heap. I'd hoped it might compare the results of one test set with another, or at least show them side by side, but it just throws them all together. I guess I'll have to make a separate test plan for every test set.

(I've got two test plans: one has test sets with 1 thread, 3 threads, 5 threads and 10 threads, getting pages as fast as they can. The other is the same, but with the ConstantThroughputTimer added. I was hoping to see some graphs of how response time deteriorated as throughput went up, or something,)

Regards,

Martijn.

For more options, visit https://groups.google.com/d/optout.

[miguel.p...@gmail.com]

Hi, I am having this same issue... and do not know what am I doing wrong.

The parameters 'source' for goal com.lazerycode.jmeter:jmeter-analysis-maven-plugin:1.0.2:analyze are missing or invalid -> [Help 1]

I have tried with all the versions of the plugin, but none seems to pick the configurations.

Any help would be really appreciated.

[Mark Collin]

The plugin now runs JMeter in a separate process, the current version is 1.10.0. Details on how to configure the JVM that JMeter runs inside are available on the wiki:

https://github.com/jmeter-maven-plugin/jmeter-maven-plugin/wiki/Configuring-the-jvm-that-the-jmeter-process-runs-in

> --
> You received this message because you are subscribed to the Google Groups "Maven JMeter Plugin Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to maven-jmeter-plugi...@googlegroups.com.
> To post to this group, send an email to maven-jmeter...@googlegroups.com.
> Visit this group at http://groups.google.com/group/maven-jmeter-plugin-users.

> For more options, visit https://groups.google.com/d/optout.