FlexibleFileWriterPlugin throws OverlappingFileLockException

[venkatd...@gmail.com]

We are using FlexibleWriter plugin.
When the jMeter tests run in a distributed manner, we are seeing OverlappingFileLockException

Should flexible plugin write to files on the jMeter servers (slaves) in a dist mode.

2015/03/25 01:16:16 ERROR - jmeter.JMeter: Uncaught exception:  java.nio.channels.OverlappingFileLockException
        at sun.nio.ch.SharedFileLockTable.checkList(FileLockTable.java:255)
        at sun.nio.ch.SharedFileLockTable.add(FileLockTable.java:152)
        at sun.nio.ch.FileChannelImpl.lock(FileChannelImpl.java:1011)
        at java.nio.channels.FileChannel.lock(FileChannel.java:1052)
        at kg.apc.jmeter.reporters.FlexibleFileWriter.syncWrite(FlexibleFileWriter.java:246)
        at kg.apc.jmeter.reporters.FlexibleFileWriter.sampleOccurred(FlexibleFileWriter.java:239)
        at org.apache.jmeter.samplers.RemoteSampleListenerImpl.processBatch(RemoteSampleListenerImpl.java:96)
        at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
        at sun.rmi.transport.Transport$1.run(Transport.java:177)
        at sun.rmi.transport.Transport$1.run(Transport.java:174)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
        at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:556)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:811)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:670)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
        at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
        at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
        at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
        at org.apache.jmeter.samplers.RemoteSampleListenerImpl_Stub.processBatch(Unknown Source)
        at org.apache.jmeter.samplers.AsynchSampleSender$Worker.run(AsynchSampleSender.java:152)


JMEter version: apache-jmeter-2.11

[Andrey Pokhilko]

Hi,

Are there any other exceptions before this in the log? 

--

Andrey

среда, 25 марта 2015 г., 6:01:21 UTC+3 пользователь venkatd...@gmail.com написал:

[vdeva...@yahoo-inc.com]

Hi Andrey,

There are no other exceptions before this log.

I am starting jmeter like this:

apache-jmeter-2.11/bin/jmeter -n -t /tmp/custom_local.jmx -X -Jsample_variables=var1,var2 -Gvar1=aaa -Gvar2=bbb -R loadgen1.somehost.com:4965

Here is the relevant section from the jmeter template xml:

<kg.apc.jmeter.reporters.FlexibleFileWriter guiclass="kg.apc.jmeter.reporters.FlexibleFileWriterGui" testclass="kg.apc.jmeter.reporters.FlexibleFileWriter" testname="jp@gc - Flexible File Writer" enabled="true">

    <stringProp name="filename">/tmp/flex-test.log</stringProp>

    <stringProp name="columns">endTime| var1=|variable#0| var1=|variable#1|\r\n</stringProp>

    <boolProp name="overwrite">false</boolProp>

    <stringProp name="header"></stringProp>

    <stringProp name="footer"></stringProp>

</kg.apc.jmeter.reporters.FlexibleFileWriter>

<hashTree/>

<Arguments guiclass="ArgumentsPanel" testclass="Arguments" testname="User Defined Variables" enabled="true">

    <collectionProp name="Arguments.arguments">

        <elementProp name="teamName" elementType="Argument">

        <stringProp name="Argument.name">var0</stringProp>

        <stringProp name="Argument.value">${__P(var0)}</stringProp>

        <stringProp name="Argument.metadata">=</stringProp>

        </elementProp>

        <elementProp name="testName" elementType="Argument">

          <stringProp name="Argument.name">var1</stringProp>

          <stringProp name="Argument.value">${__P(var1)}</stringProp>

          <stringProp name="Argument.metadata">=</stringProp>

        </elementProp>

    </collectionProp>

</Arguments>

When I run my tests on a distributed mode, this happens on the jmeter server and stop load generation (running samples).

In a distributed mode, should the flexiblewriter plugin execute on jMeter server?

Here is the full exception:

2015/03/25 18:35:43 ERROR - jmeter.JMeter: Uncaught exception:  java.nio.channels.OverlappingFileLockException

        at sun.nio.ch.SharedFileLockTable.checkList(FileLockTable.java:255)

        at sun.nio.ch.SharedFileLockTable.add(FileLockTable.java:152)

        at sun.nio.ch.FileChannelImpl.lock(FileChannelImpl.java:1030)

        at java.nio.channels.FileChannel.lock(FileChannel.java:1053)

        at kg.apc.jmeter.reporters.FlexibleFileWriter.syncWrite(FlexibleFileWriter.java:246)

        at kg.apc.jmeter.reporters.FlexibleFileWriter.sampleOccurred(FlexibleFileWriter.java:239)

        at org.apache.jmeter.samplers.RemoteSampleListenerImpl.processBatch(RemoteSampleListenerImpl.java:96)

        at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:483)

        at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:323)

        at sun.rmi.transport.Transport$1.run(Transport.java:200)

        at sun.rmi.transport.Transport$1.run(Transport.java:197)

        at java.security.AccessController.doPrivileged(Native Method)

        at sun.rmi.transport.Transport.serviceCall(Transport.java:196)

        at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568)

        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826)

        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$87(TCPTransport.java:683)

        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$$Lambda$1/639230025.run(Unknown Source)

        at java.security.AccessController.doPrivileged(Native Method)

        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

        at java.lang.Thread.run(Thread.java:745)

        at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:276)

        at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:253)

        at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:162)

[Andrey Pokhilko]

Well, the function is so simple, and I have no idea what can go wrong there...

See https://github.com/undera/jmeter-plugins/blob/master/standard/src/kg/apc/jmeter/reporters/FlexibleFileWriter.java#L241

--

Andrey

среда, 25 марта 2015 г., 22:52:30 UTC+3 пользователь venkatd...@gmail.com написал:

[venkatd...@gmail.com]

Do you think that this could be due to the -G option and the way sample_variable are being used?

thanks

[Andrey Pokhilko]

I don't think so... There are no other places that write into the file, so I'm curious why this is happening.

четверг, 26 марта 2015 г., 0:03:16 UTC+3 пользователь venkatd...@gmail.com написал:

[venkatd...@gmail.com]

When I run my tests on a distributed mode, this happens on the jmeter server and stop load generation (running samples).

My question is that while running in distributed mode, should the flexiblewriter plugin execute on jMeter server?

thanks

[Andrey Pokhilko]

Yes, I think it should

четверг, 26 марта 2015 г., 19:13:20 UTC+3 пользователь venkatd...@gmail.com написал:

[venkatd...@gmail.com]

We downloaded the latest nightly developer snapshot (that seems to have a fix for the overlapping issues) from here
http://jmeter-plugins.org/downloads/file/nightly/JMeterPlugins-Standard-1.2.2_4b2fd7e129.zip

We see the following errors:

at java.nio.channels.FileChannel.lock(FileChannel.java:1053)
at kg.apc.jmeter.reporters.FlexibleFileWriter.syncWrite(FlexibleFileWriter.java:246)
at kg.apc.jmeter.reporters.FlexibleFileWriter.sampleOccurred(FlexibleFileWriter.java:239)

When using this plugin in distribute mode with n servers, the exception happens in n-1 servers randomly and only one server continues to work without exceptions.

[venkatd...@gmail.com]

I noticed one more thing that looks odd:

This commit (https://github.com/undera/jmeter-plugins/commit/a8c8044ef64e93b3f16296d64aa87df60213ec3b)  is trying to fix the overlapping lock issue.

However, the line numbers dont match with the exception thrown and
https://github.com/undera/jmeter-plugins/blob/a8c8044ef64e93b3f16296d64aa87df60213ec3b/standard/src/kg/apc/jmeter/reporters/FlexibleFileWriter.java

Does this zip file
http://jmeter-plugins.org/downloads/file/nightly/JMeterPlugins-Standard-1.2.2_4b2fd7e129.zip
have the latest code (this was taken from nightly developer builds)

thanks

[Andrey Pokhilko]

It does, but you'll need to replace all plugin sets you have, since versions up to 1.2.1 had a problem with classes shipping.

--

Andrey

четверг, 26 марта 2015 г., 23:25:09 UTC+3 пользователь venkatd...@gmail.com написал: