Jmeter with Selenium-Testng

[sun...@gmail.com]

We used Selenium with Testng for functional testing  and we are planning to reuse the existing  functional testcases for Jmeter Load testing.  looks like Webdriver is currently supports only Junit ,any plug-ins for integrating Jmeter with TestNG ?

Thanks

Sunil Kumar

[sun...@gmail.com]

Can someone please help me if there are any plug-in that can run test-ng test cases with Jmeter.

Thanks

Sunil Kumar

[gli...@gmail.com]

Hi Sunil,

I don't see any sense in running TestNG testcases using JMeter as TestNG provides both pass/fail reporting and possibility to run tests in parallel. You may want to consider Selenium Grid which is designed to scale by distributing tests on several machines via parallel execution. It can also be used to execute the same test scenarios by multiple Selenium instances providing the flavor of load testing. 

However if you need to trigger TestNG tests suite execution from JMeter you can use Beanshell Sampler which allows arbitrary Beanshell code execution. In order to execute your Selenium TestNG tests from JMeter Beanshell Sampler you'll need the following:

1. Add testng.jar to JMeter classpath (drop it to /lib folder of your JMeter installation)
2. Do the same for your test classes and their dependency jar libraries
3. Add a Beanshell Sampler to your JMeter Test plan with the following code:

import org.testng.TestListenerAdapter;

import org.testng.TestNG;

TestNG testng = new TestNG();
testng.setOutputDirectory("/path/to/results/folder");
TestListenerAdapter tla = new TestListenerAdapter();
testng.setTestClasses(new Class[]{com.company.selenium.SeleniumTestSuite.class});
testng.addListener(tla);
testng.run();

 I also expect that you'll need to provide separate port for each thread associated with Selenium. 

See How to use BeanShell: JMeter's favorite built-in component guide for more details on Beanshell scripting in Apache JMeter. 

However I would recommend consider using Selenium Grid instead. 

Hope this helps.  

[sun...@gmail.com]

thanks a lot reply.. I want use Jmeter only take advantages of graph and charts provided by pulg-ins .. Does Selenium Grid provide same kind graphs and charts ( like response time..)

Thanks

Sunil Kumar

[garcia...@gmail.com]

I am trying to solve the same problem. After copying the jaRs I tried your sample code but the import cannot find the classes

[Jon Thor Austen]

I am a heavy user of Selenium and TestNG as well as Jmeter.    In my opinion, the ability to run Selenium tests as multi-threaded has not existed so far but when the 1.2.1  version of the selenium plugin pack comes out, I think  there is a new RemoteWebDriver  handler that should work fine with a Selenium Grid, which gives you the capability you are looking for.  That being the case, with Jmeter handling the multi-threading,  you should NOT NEED testng nor JUnit for parallel execution.    Jmeter is able to generate a HTML report by doing a transform with the .xsl style sheet included with jmeter.   Also, performance information generally outputs to the standard out console.


-Jon

[rohit1...@gmail.com]

Hi gil....,your post is helpful but JMETER is not able to find class name i am getting this error 



2015/06/14 10:25:21 INFO  - jmeter.util.BSFTestElement: Registering JMeter version of JavaScript engine as work-round for BSF-22
2015/06/14 10:25:21 INFO  - jmeter.protocol.http.sampler.HTTPSamplerBase: Cannot find .className property for htmlParser, using default
2015/06/14 10:25:21 INFO  - jmeter.protocol.http.sampler.HTTPSamplerBase: Parser for text/html is 
2015/06/14 10:25:21 INFO  - jmeter.protocol.http.sampler.HTTPSamplerBase: Parser for application/xhtml+xml is 
2015/06/14 10:25:21 INFO  - jmeter.protocol.http.sampler.HTTPSamplerBase: Parser for application/xml is 
2015/06/14 10:25:21 INFO  - jmeter.protocol.http.sampler.HTTPSamplerBase: Parser for text/xml is 
2015/06/14 10:25:21 INFO  - jmeter.protocol.http.sampler.HTTPSamplerBase: Parser for text/vnd.wap.wml is org.apache.jmeter.protocol.http.parser.RegexpHTMLParser
2015/06/14 10:25:21 INFO  - jmeter.gui.util.MenuFactory: Skipping org.apache.jmeter.protocol.http.control.gui.WebServiceSamplerGui
2015/06/14 10:25:22 INFO  - jmeter.gui.util.MenuFactory: Skipping org.apache.jmeter.protocol.http.modifier.gui.ParamModifierGui
2015/06/14 10:25:22 INFO  - jorphan.exec.KeyToolUtils: keytool found at 'keytool'
2015/06/14 10:25:22 INFO  - jmeter.protocol.http.proxy.ProxyControl: HTTP(S) Test Script Recorder SSL Proxy will use keys that support embedded 3rd party resources in file E:\apache-jmeter-2.13\bin\proxyserver.jks
2015/06/14 10:25:22 INFO  - jmeter.samplers.SampleResult: Note: Sample TimeStamps are START times
2015/06/14 10:25:22 INFO  - jmeter.samplers.SampleResult: sampleresult.default.encoding is set to ISO-8859-1
2015/06/14 10:25:22 INFO  - jmeter.samplers.SampleResult: sampleresult.useNanoTime=true
2015/06/14 10:25:22 INFO  - jmeter.samplers.SampleResult: sampleresult.nanoThreadSleep=5000
2015/06/14 11:48:14 INFO  - jmeter.services.FileServer: Default base='E:\apache-jmeter-2.13\bin'
2015/06/14 11:48:14 INFO  - jmeter.services.FileServer: Set new base='E:\apache-jmeter-2.13\bin'
2015/06/14 11:48:14 INFO  - jmeter.save.SaveService: Testplan (JMX) version: 2.2. Testlog (JTL) version: 2.2
2015/06/14 11:48:14 INFO  - jmeter.save.SaveService: Using SaveService properties file encoding UTF-8
2015/06/14 11:48:14 INFO  - jmeter.save.SaveService: Using SaveService properties file version 1656252
2015/06/14 11:48:14 INFO  - jmeter.save.SaveService: Using SaveService properties version 2.8
2015/06/14 11:48:14 INFO  - jmeter.save.SaveService: All converter versions present and correct
2015/06/14 11:48:14 INFO  - jmeter.engine.StandardJMeterEngine: Listeners will be started after enabling running version
2015/06/14 11:48:14 INFO  - jmeter.engine.StandardJMeterEngine: To revert to the earlier behaviour, define jmeterengine.startlistenerslater=false
2015/06/14 11:48:14 INFO  - jmeter.engine.StandardJMeterEngine: Running the test!
2015/06/14 11:48:14 INFO  - jmeter.samplers.SampleEvent: List of sample_variables: []
2015/06/14 11:48:14 INFO  - jmeter.samplers.SampleEvent: List of sample_variables: []
2015/06/14 11:48:14 INFO  - jmeter.gui.util.JMeterMenuBar: setRunning(true,*local*)
2015/06/14 11:48:14 INFO  - jmeter.engine.StandardJMeterEngine: Starting ThreadGroup: 1 : Thread Group
2015/06/14 11:48:14 INFO  - jmeter.engine.StandardJMeterEngine: Starting 1 threads for group Thread Group.
2015/06/14 11:48:14 INFO  - jmeter.engine.StandardJMeterEngine: Thread will continue on error
2015/06/14 11:48:14 INFO  - jmeter.threads.ThreadGroup: Starting thread group number 1 threads 1 ramp-up 1 perThread 1000.0 delayedStart=false
2015/06/14 11:48:14 INFO  - jmeter.threads.JMeterThread: jmeterthread.startearlier=true (see jmeter.properties)
2015/06/14 11:48:14 INFO  - jmeter.threads.JMeterThread: Running PostProcessors in forward order
2015/06/14 11:48:14 INFO  - jmeter.threads.ThreadGroup: Started thread group number 1
2015/06/14 11:48:14 INFO  - jmeter.engine.StandardJMeterEngine: All thread groups have been started
2015/06/14 11:48:14 INFO  - jmeter.threads.JMeterThread: Thread started: Thread Group 1-1
2015/06/14 11:48:14 INFO  - jmeter.threads.JMeterThread: Thread is done: Thread Group 1-1
2015/06/14 11:48:14 INFO  - jmeter.threads.JMeterThread: Thread finished: Thread Group 1-1
2015/06/14 11:48:14 INFO  - jmeter.engine.StandardJMeterEngine: Notifying test listeners of end of test
2015/06/14 11:48:14 INFO  - jmeter.gui.util.JMeterMenuBar: setRunning(false,*local*)
2015/06/14 11:48:16 INFO  - jmeter.engine.StandardJMeterEngine: Running the test!
2015/06/14 11:48:16 INFO  - jmeter.samplers.SampleEvent: List of sample_variables: []
2015/06/14 11:48:16 INFO  - jmeter.gui.util.JMeterMenuBar: setRunning(true,*local*)
2015/06/14 11:48:16 INFO  - jmeter.engine.StandardJMeterEngine: Starting ThreadGroup: 1 : Thread Group
2015/06/14 11:48:16 INFO  - jmeter.engine.StandardJMeterEngine: Starting 1 threads for group Thread Group.
2015/06/14 11:48:16 INFO  - jmeter.engine.StandardJMeterEngine: Thread will continue on error
2015/06/14 11:48:16 INFO  - jmeter.threads.ThreadGroup: Starting thread group number 1 threads 1 ramp-up 1 perThread 1000.0 delayedStart=false
2015/06/14 11:48:16 INFO  - jmeter.threads.ThreadGroup: Started thread group number 1
2015/06/14 11:48:16 INFO  - jmeter.engine.StandardJMeterEngine: All thread groups have been started
2015/06/14 11:48:16 INFO  - jmeter.threads.JMeterThread: Thread started: Thread Group 1-1
2015/06/14 11:48:16 INFO  - jmeter.threads.JMeterThread: Thread is done: Thread Group 1-1
2015/06/14 11:48:16 INFO  - jmeter.threads.JMeterThread: Thread finished: Thread Group 1-1
2015/06/14 11:48:16 INFO  - jmeter.engine.StandardJMeterEngine: Notifying test listeners of end of test
2015/06/14 11:48:16 INFO  - jmeter.gui.util.JMeterMenuBar: setRunning(false,*local*)
2015/06/14 11:48:46 INFO  - jmeter.engine.StandardJMeterEngine: Running the test!
2015/06/14 11:48:46 INFO  - jmeter.samplers.SampleEvent: List of sample_variables: []
2015/06/14 11:48:46 INFO  - jmeter.gui.util.JMeterMenuBar: setRunning(true,*local*)
2015/06/14 11:48:47 INFO  - jmeter.engine.StandardJMeterEngine: Starting ThreadGroup: 1 : Thread Group
2015/06/14 11:48:47 INFO  - jmeter.engine.StandardJMeterEngine: Starting 1 threads for group Thread Group.
2015/06/14 11:48:47 INFO  - jmeter.engine.StandardJMeterEngine: Thread will continue on error
2015/06/14 11:48:47 INFO  - jmeter.threads.ThreadGroup: Starting thread group number 1 threads 1 ramp-up 1 perThread 1000.0 delayedStart=false
2015/06/14 11:48:47 INFO  - jmeter.threads.ThreadGroup: Started thread group number 1
2015/06/14 11:48:47 INFO  - jmeter.engine.StandardJMeterEngine: All thread groups have been started
2015/06/14 11:48:47 INFO  - jmeter.threads.JMeterThread: Thread started: Thread Group 1-1
2015/06/14 11:48:47 INFO  - jmeter.threads.JMeterThread: Thread is done: Thread Group 1-1
2015/06/14 11:48:47 INFO  - jmeter.threads.JMeterThread: Thread finished: Thread Group 1-1
2015/06/14 11:48:47 INFO  - jmeter.engine.StandardJMeterEngine: Notifying test listeners of end of test
2015/06/14 11:48:47 INFO  - jmeter.gui.util.JMeterMenuBar: setRunning(false,*local*)
2015/06/14 11:48:50 INFO  - jmeter.engine.StandardJMeterEngine: Running the test!
2015/06/14 11:48:50 INFO  - jmeter.samplers.SampleEvent: List of sample_variables: []
2015/06/14 11:48:50 INFO  - jmeter.gui.util.JMeterMenuBar: setRunning(true,*local*)


2015/06/14 11:48:50 INFO  - jmeter.engine.StandardJMeterEngine: Starting ThreadGroup: 1 : Thread Group
2015/06/14 11:48:50 INFO  - jmeter.engine.StandardJMeterEngine: Starting 1 threads for group Thread Group.
2015/06/14 11:48:50 INFO  - jmeter.engine.StandardJMeterEngine: Thread will continue on error
2015/06/14 11:48:50 INFO  - jmeter.threads.ThreadGroup: Starting thread group number 1 threads 1 ramp-up 1 perThread 1000.0 delayedStart=false
2015/06/14 11:48:50 INFO  - jmeter.threads.ThreadGroup: Started thread group number 1
2015/06/14 11:48:50 INFO  - jmeter.engine.StandardJMeterEngine: All thread groups have been started
2015/06/14 11:48:50 INFO  - jmeter.threads.JMeterThread: Thread started: Thread Group 1-1
2015/06/14 11:48:50 INFO  - jmeter.threads.JMeterThread: Thread is done: Thread Group 1-1
2015/06/14 11:48:50 INFO  - jmeter.threads.JMeterThread: Thread finished: Thread Group 1-1
2015/06/14 11:48:50 INFO  - jmeter.engine.StandardJMeterEngine: Notifying test listeners of end of test
2015/06/14 11:48:50 INFO  - jmeter.gui.util.JMeterMenuBar: setRunning(false,*local*)
2015/06/14 11:52:55 INFO  - jmeter.engine.StandardJMeterEngine: Running the test!
2015/06/14 11:52:55 INFO  - jmeter.samplers.SampleEvent: List of sample_variables: []
2015/06/14 11:52:55 INFO  - jmeter.gui.util.JMeterMenuBar: setRunning(true,*local*)
2015/06/14 11:52:55 INFO  - jmeter.engine.StandardJMeterEngine: Starting ThreadGroup: 1 : Thread Group
2015/06/14 11:52:55 INFO  - jmeter.engine.StandardJMeterEngine: Starting 1 threads for group Thread Group.
2015/06/14 11:52:55 INFO  - jmeter.engine.StandardJMeterEngine: Thread will continue on error
2015/06/14 11:52:55 INFO  - jmeter.threads.ThreadGroup: Starting thread group number 1 threads 1 ramp-up 1 perThread 1000.0 delayedStart=false
2015/06/14 11:52:55 INFO  - jmeter.threads.ThreadGroup: Started thread group number 1
2015/06/14 11:52:55 INFO  - jmeter.engine.StandardJMeterEngine: All thread groups have been started
2015/06/14 11:52:55 INFO  - jmeter.threads.JMeterThread: Thread started: Thread Group 1-1
2015/06/14 11:52:55 INFO  - jmeter.threads.JMeterThread: Thread is done: Thread Group 1-1
2015/06/14 11:52:55 INFO  - jmeter.threads.JMeterThread: Thread finished: Thread Group 1-1
2015/06/14 11:52:55 INFO  - jmeter.engine.StandardJMeterEngine: Notifying test listeners of end of test
2015/06/14 11:52:55 INFO  - jmeter.gui.util.JMeterMenuBar: setRunning(false,*local*)
2015/06/14 11:54:04 INFO  - jmeter.engine.StandardJMeterEngine: Running the test!
2015/06/14 11:54:04 INFO  - jmeter.samplers.SampleEvent: List of sample_variables: []
2015/06/14 11:54:04 INFO  - jmeter.gui.util.JMeterMenuBar: setRunning(true,*local*)
2015/06/14 11:54:04 INFO  - jmeter.engine.StandardJMeterEngine: Starting ThreadGroup: 1 : Thread Group
2015/06/14 11:54:04 INFO  - jmeter.engine.StandardJMeterEngine: Starting 1 threads for group Thread Group.
2015/06/14 11:54:04 INFO  - jmeter.engine.StandardJMeterEngine: Thread will continue on error
2015/06/14 11:54:04 INFO  - jmeter.threads.ThreadGroup: Starting thread group number 1 threads 1 ramp-up 1 perThread 1000.0 delayedStart=false
2015/06/14 11:54:04 INFO  - jmeter.threads.ThreadGroup: Started thread group number 1
2015/06/14 11:54:04 INFO  - jmeter.engine.StandardJMeterEngine: All thread groups have been started
2015/06/14 11:54:04 INFO  - jmeter.threads.JMeterThread: Thread started: Thread Group 1-1
2015/06/14 11:54:04 INFO  - jmeter.threads.JMeterThread: Thread is done: Thread Group 1-1
2015/06/14 11:54:04 INFO  - jmeter.threads.JMeterThread: Thread finished: Thread Group 1-1
2015/06/14 11:54:04 INFO  - jmeter.engine.StandardJMeterEngine: Notifying test listeners of end of test
2015/06/14 11:54:04 INFO  - jmeter.gui.util.JMeterMenuBar: setRunning(false,*local*)
2015/06/14 12:00:14 INFO  - jmeter.engine.StandardJMeterEngine: Running the test!
2015/06/14 12:00:14 INFO  - jmeter.samplers.SampleEvent: List of sample_variables: []
2015/06/14 12:00:14 INFO  - jmeter.gui.util.JMeterMenuBar: setRunning(true,*local*)
2015/06/14 12:00:14 INFO  - jmeter.engine.StandardJMeterEngine: Starting ThreadGroup: 1 : Thread Group
2015/06/14 12:00:14 INFO  - jmeter.engine.StandardJMeterEngine: Starting 1 threads for group Thread Group.
2015/06/14 12:00:14 INFO  - jmeter.engine.StandardJMeterEngine: Thread will continue on error
2015/06/14 12:00:14 INFO  - jmeter.threads.ThreadGroup: Starting thread group number 1 threads 1 ramp-up 1 perThread 1000.0 delayedStart=false
2015/06/14 12:00:14 INFO  - jmeter.threads.ThreadGroup: Started thread group number 1
2015/06/14 12:00:14 INFO  - jmeter.engine.StandardJMeterEngine: All thread groups have been started
2015/06/14 12:00:14 INFO  - jmeter.threads.JMeterThread: Thread started: Thread Group 1-1
2015/06/14 12:00:14 INFO  - jmeter.threads.JMeterThread: Thread is done: Thread Group 1-1
2015/06/14 12:00:14 INFO  - jmeter.threads.JMeterThread: Thread finished: Thread Group 1-1
2015/06/14 12:00:14 INFO  - jmeter.engine.StandardJMeterEngine: Notifying test listeners of end of test
2015/06/14 12:00:14 INFO  - jmeter.gui.util.JMeterMenuBar: setRunning(false,*local*)


Please revert me back with this issue

[jche...@gmail.com]

I'm not an expert at JMeter, but I thought I'd offer some things I have learned about it for consideration.  JMeter's advantage in its ability to load test applications comes from the fact that it can simulate browser requests to the web server WITHOUT using browsers.  This is because JMeter tests are engineered to send HTTP requests directly to the server, an action which can be scaled very easily from a single machine, or at least a small number of machines.

JUnit/TestNG tests using Selenium should be focused on Web Site functionality, NOT load testing.  Selenium's value is in its ability to interface with various types of browsers, which allows you to run the same tests against each type without having to custom design your tests for each type.  Selenium Grid, by extension, helps you to expand the range of your tests to include browsers that don't play well running on the same machine with other browsers.  This is especially useful for testing against multiple versions of the same browser.  For example, I.E. 11 will not allow you to run I.E. 10 on the same machine.  Trying to install it on the same machine triggers an upgrade.  Selenium grid allows you to have 4 machines with 4 different versions of I.E. installed on it.

On the other hand, browsers include a LOT of overhead.  So the only way to get close to creating enough client requests to push a server's load capacity to the max is by using Selenium Grid to employ 10's 100's or maybe even 1000's of machines to hit the server simultaneously.  Very inefficient.  

JMeter tests may need to use multiple machines to challenge a high powered server, and it has utilities for setting up grids similar to Selenium grids for just that purpose.  However, because those machines would all send HTTP requests without using browsers, the same load can be sent to the server with a JMeter grid using a fraction of the machine power that a Selenium grid would use.

If you want to utilize the rich plethora of tests you have set up using Selenium using JMeter, I would recommend you tap into JMeter's recording capabilities.  Set up JMeter to listen to a browser that your TestNG w/ Selenium script is hitting, and run all of your tests through that one browser single threaded.  JMeter will record all of the various HTTP requests the browser sends to the server.  You can then take that recording and with a little editing, produce load tests with the rich set of request types you are looking for.

Another advantage of this approach is that, if you find that the server has a bottleneck, you can more easily break apart the different types of tests into smaller modules to pinpoint the exact commands that are causing the server trouble.  This, in turn, will allow you to write bug reports that are much more valuable to your developers.

Hope that helps folks in the future. 

[John Chesshir]

Sunil,

Not last I looked, and I never found any indication that it was being considered.  

However, I would recommend that you merely transfer all of the interfacing use cases directly into JMeter.  Once that is done you can then use those use cases much more efficiently for load testing than if you tried to run a JUnit script using Selenium through JMeter.

You can hopefully do this fairly quickly, especially if you have many test cases that utilize the same Selenium connection to a browser.  Run your TestNG script through a debugger and break the execution right after the lines where Selenium has set up its browser.  Then attach JMeter's recording utility to the browser instance that Selenium created.  Start back up the TestNG script and let JMeter record as TestNG and Selenium run your test cases against the browser.

Hope that helps!

John Chesshir

--
You received this message because you are subscribed to a topic in the Google Groups "jmeter-plugins" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jmeter-plugins/MYerC8iIJCA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jmeter-plugins+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[vineet...@gmail.com]

Hi Sunil ,

I have same scenario as  yours , want to use  Selenium with Testng for functional testing for performance testing .

how did you do this ?

Thanks in advance

On Monday, June 30, 2014 at 9:48:52 AM UTC-5, sun...@gmail.com wrote:

[rajes...@gmail.com]

We are in 2019 

Team ,

Can any please provide for below question . All my test cases are designed using Testng framework how i can convert them into Junit to use Jmter .

Please help me out...

Thanks

Raj

On Monday, June 30, 2014 at 9:48:52 AM UTC-5, sun...@gmail.com wrote:

[Andrey Pokhilko]

Hi,

The only advice I can give is to use Taurus tool to run your TestNG as-is. I know no tool to convert TestNG to JMeter.

Andrey Pokhilko

--
You received this message because you are subscribed to the Google Groups "jmeter-plugins" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jmeter-plugin...@googlegroups.com.