Using browsermob concurrently

[Vasyl Vaskul]

Hello, 

i am using browsermob on the same machine with selenium grid hub connected to selenium grid nodes (see Grid2) using xvfb as soon as I had problems with defining display number etc.

I launch multiple tests in parallel and for every test I create new proxy with unique port assigned.  All tests concurrently share hashmap (ConcurrentHashMap) of references to the these created proxies. 

The sense of the test is to capture ajax GET for a given url. Sometimes proxies have har logs with zero entries though the tests fails though when I check in the real browser it works. I retry getHar() 6 times with delay 10 seconds. 

My questions is does getHar() function is thread-safe and could be used in the configuration I described?

In the ProxyServer class i found the following part :

 public Har getHar() {

        // Wait up to 5 seconds for all active requests to cease before returning the HAR.

        // This helps with race conditions but won't cause deadlocks should a request hang

        // or error out in an unexpected way (which of course would be a bug!)

        boolean success = ThreadUtils.waitFor(new ThreadUtils.WaitCondition() {

            @Override

            public boolean checkCondition(long elapsedTimeInMs) {

                return requestCounter.get() == 0;

            }

        }, TimeUnit.SECONDS, 5);

        if (!success) {

            LOG.warn("Waited 5 seconds for requests to cease before returning HAR; giving up!");

        }

        return client.getHar();

    }

Could you please also explain the glitches behind this pause of 5 seconds?

Thanks for your help.

[Patrick Lightbody]

I guess the question is: what do you expect getHar() to do? Right now it tries to return whatever activity has run through the proxy at that exact moment of time, waiting for up to 5 seconds for any active requests to complete. It seemed like a good compromise to me. I could make it return immediately, but what if there is an active request coming through? We could make getHar() optionally take a timeout value for how long you should wait.

As for BMP being used concurrently, just a reminder: you should only use one proxy/port per browser. If you are sending multiple streams of independent traffic (ex: 2+ Selenium browsers) through the same port, you're going to have problems. Instead, that's why BMP supports multiple in-container proxies, identified by port.

Patrick

On Aug 30, 2013, at 1:49 AM, Jerome Tang <jerom...@gmail.com> wrote:

I also noted the issue, for the same BMP instance, getHar() seems not thread-safe.

--
 
---
You received this message because you are subscribed to the Google Groups "BrowserMob Proxy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to browsermob-pro...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Vasyl Vaskul]

Hello Patrick, 

thank you very much for your answer.

Could please specify how could I make use of browsermob in-container proxies in case I use java wrapper?

For your information I am assigning unique port for every ProxyServer I create like this

ProxyServer server = new ProxyServer(UNIQUE_PORT);

server.start();

and because of parallel mode with 10 threads I could have up to 10 active proxies each pointing to its own port.

Selenium nodes that execute tests and proxies are separated physically between different machines, I hope that shouldn't be the problem and could just impact latency of tests.

Finally, the really strange thing going on is that when running tests in parallel mode (multiple proxies at a time each pointing to its own port) I am having problems with empty har files (even after n retries)

though while re-running the same tests in sequential mode (with one proxy at a time)

it works OK without any problems.

Thanks

Vasyl

[Vasyl Vaskul]

As soon as I am retrying getHar() many times could it be smth with cache control? 

Are there any options to force BrowserMob not to cache requests?

[Patrick Lightbody]

BMP does not have anything to do with caching. The behavior of caching is entirely determined by the browser and the server.

--

[Vasyl Vaskul]

Finally the issue has been resolved. 

The problem was due to bug in the way my application was running tests concurrently.

Thank you Patrick for your help.

[motorhead]

Hello,

Can you pls tell me how you're doing the parallel execution with respect to browsermob?

any code is appreciated

Thanks!

[Vasyl Vaskul]

You can use testng parallel execution capabilites.

See my gist https://gist.github.com/basilboli/7212357

[Jatin Sethi]

Patrick, Do we have an option to override this time to more number of seconds? Thinking of a scenario where A-Synch, and Synch AJAX calls are still being sent to the server, and those need to be recorded as part of the HAR files?