How to get all network calls while performing selenium automation?

64 views
Skip to first unread message

ankit gupta

unread,
May 29, 2018, 8:02:39 AM5/29/18
to Selenium Users
While performing selenium automation, I also want to get the all the APIs call so that I can get the exact failure issue either it is on Website side or on the API's side.

Joe Ward

unread,
May 29, 2018, 8:41:18 AM5/29/18
to seleniu...@googlegroups.com
No way to do it with Selenium, you can probably use the JavaScriptExecutor to get what's produced by Window.performance if that's what you really wanted, though.


On 29 May 2018 at 13:02, ankit gupta <ankit...@gmail.com> wrote:
While performing selenium automation, I also want to get the all the APIs call so that I can get the exact failure issue either it is on Website side or on the API's side.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to selenium-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/c9a88dc2-d672-4b8a-839d-e7fe9a25a112%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Ankit Gupta

unread,
May 29, 2018, 10:01:23 AM5/29/18
to seleniu...@googlegroups.com
Thanks a lot, Joe.

But I want to fetch all API's call and their request data and response data after click() or some other functionality.

On Tue, May 29, 2018 at 6:10 PM, Joe Ward <ddl...@gmail.com> wrote:
No way to do it with Selenium, you can probably use the JavaScriptExecutor to get what's produced by Window.performance if that's what you really wanted, though.

On 29 May 2018 at 13:02, ankit gupta <ankit...@gmail.com> wrote:
While performing selenium automation, I also want to get the all the APIs call so that I can get the exact failure issue either it is on Website side or on the API's side.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to selenium-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/CAJPyTs1vSm_aFApPOo%2BvBwD0eJrtsaH91urrcGCVMSchvZVbbw%40mail.gmail.com.

Krishnan Mahadevan

unread,
May 29, 2018, 10:27:23 AM5/29/18
to seleniu...@googlegroups.com

Route your browser traffic through a proxy server such as BrowserMobProxy or try to retrieve the logs via the Beta level performance APIs in the Selenium WebDriver code (Sample shown as below)

 

import java.util.Iterator;

import java.util.logging.Level;

import org.openqa.selenium.chrome.ChromeDriver;

import org.openqa.selenium.chrome.ChromeOptions;

import org.openqa.selenium.logging.LogEntries;

import org.openqa.selenium.logging.LogEntry;

import org.openqa.selenium.logging.LogType;

import org.openqa.selenium.logging.LoggingPreferences;

import org.openqa.selenium.remote.CapabilityType;

 

public class Sample {

 

  public static void main(String[] args) {

    LoggingPreferences preferences = new LoggingPreferences();

    preferences.enable(LogType.PERFORMANCE, Level.ALL);

    ChromeOptions options = new ChromeOptions();

    options.setCapability(CapabilityType.LOGGING_PREFS, preferences);

    ChromeDriver driver = new ChromeDriver(options);

    try {

      driver.get("http://wwww.facebook.com");

      LogEntries logs = driver.manage().logs().get(LogType.PERFORMANCE);

     Iterator<LogEntry> iterator = logs.iterator();

      while (iterator.hasNext()) {

        LogEntry entry = iterator.next();

        System.err.println(entry.toString());

      }

 

    } finally {

      driver.quit();

    }

  }

}

 

 

PS: I know for sure that this works in Chrome, but am not sure if it would work the same way in other browser flavors.

 

 

Thanks & Regards

Krishnan Mahadevan

 

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

 

From: <seleniu...@googlegroups.com> on behalf of Ankit Gupta <ankit...@gmail.com>
Reply-To: <seleniu...@googlegroups.com>
Date: Tuesday, May 29, 2018 at 7:31 PM
To: <seleniu...@googlegroups.com>
Subject: Re: [selenium-users] How to get all network calls while performing selenium automation?

 

Thanks a lot, Joe.

 

But I want to fetch all API's call and their request data and response data after click() or some other functionality.

On Tue, May 29, 2018 at 6:10 PM, Joe Ward <ddl...@gmail.com> wrote:

No way to do it with Selenium, you can probably use the JavaScriptExecutor to get what's produced by Window.performance if that's what you really wanted, though.

 

On 29 May 2018 at 13:02, ankit gupta <ankit...@gmail.com> wrote:

While performing selenium automation, I also want to get the all the APIs call so that I can get the exact failure issue either it is on Website side or on the API's side.

--

You received this message because you are subscribed to the Google Groups "Selenium Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.


To post to this group, send email to seleniu...@googlegroups.com.


For more options, visit https://groups.google.com/d/optout.

 

--

You received this message because you are subscribed to the Google Groups "Selenium Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.


To post to this group, send email to seleniu...@googlegroups.com.

Ankit Gupta

unread,
May 30, 2018, 2:49:46 AM5/30/18
to seleniu...@googlegroups.com
Thanks,  Krishnan

But it also does not have data of all the API's calls 

--

To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to selenium-users@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to selenium-users@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to selenium-users@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to selenium-users@googlegroups.com.
 To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/7AD72B34-D623-4973-B45C-DCE83F02FAA5%40gmail.com.

⇜Krishnan Mahadevan⇝

unread,
May 30, 2018, 3:32:45 AM5/30/18
to seleniu...@googlegroups.com
What did you try ? Can you elaborate ?

--

To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/7AD72B34-D623-4973-B45C-DCE83F02FAA5%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/CAKrer0DthdOE5AtZ__jSni83YdftO%3DgrB1jPAXRtHBA4%2BgXnew%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.
--

Cassian Raja Thomas

unread,
May 30, 2018, 3:38:02 AM5/30/18
to seleniu...@googlegroups.com
Did you try with fiddler?

To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/CANikZLmWuaa1eKO%2BZF6faVCpOR2GpxADTG8otgnpRsWcS%3DBvjQ%40mail.gmail.com.

Ankit Gupta

unread,
May 30, 2018, 3:41:21 AM5/30/18
to seleniu...@googlegroups.com
The same code which you shared, 
It does not have API's (GET, POST,PUT) request and response data.

On Wed, May 30, 2018 at 1:01 PM, ⇜Krishnan Mahadevan⇝ <krishnan.ma...@gmail.com> wrote:
What did you try ? Can you elaborate ?

On Wed 30 May, 2018, 12:19 Ankit Gupta, <ankit...@gmail.com> wrote:
Thanks,  Krishnan

But it also does not have data of all the API's calls 

--

To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to selenium-users@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to selenium-users@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to selenium-users@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to selenium-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/7AD72B34-D623-4973-B45C-DCE83F02FAA5%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to selenium-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/CAKrer0DthdOE5AtZ__jSni83YdftO%3DgrB1jPAXRtHBA4%2BgXnew%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
--

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to selenium-users@googlegroups.com.
 To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/CANikZLmWuaa1eKO%2BZF6faVCpOR2GpxADTG8otgnpRsWcS%3DBvjQ%40mail.gmail.com.

⇜Krishnan Mahadevan⇝

unread,
May 30, 2018, 3:47:29 AM5/30/18
to seleniu...@googlegroups.com
I believe I shared two alternatives. The log API in selenium is "Beta" so it's still work in progress and may not give you all aspects. 

What happens when you try using a proxy server such as browsermobproxy or fiddler ?

On Wed 30 May, 2018, 13:11 Ankit Gupta, <ankit...@gmail.com> wrote:
The same code which you shared, 
It does not have API's (GET, POST,PUT) request and response data.
On Wed, May 30, 2018 at 1:01 PM, ⇜Krishnan Mahadevan⇝ <krishnan.ma...@gmail.com> wrote:
What did you try ? Can you elaborate ?

On Wed 30 May, 2018, 12:19 Ankit Gupta, <ankit...@gmail.com> wrote:
Thanks,  Krishnan

But it also does not have data of all the API's calls 

--

To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/7AD72B34-D623-4973-B45C-DCE83F02FAA5%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.
--

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

 --
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/CAKrer0Bp9B8P2OBP4KFedCZHMZEKiuTXswUdWYgc3A7DmAs3HQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

Joe Ward

unread,
May 30, 2018, 4:27:29 AM5/30/18
to seleniu...@googlegroups.com
What issue are you actually trying to solve by doing this? Selenium doesn't really have a programmatic way to do what you're asking. Fiddler and BrowserMob are probably OK for exploratory testing but a bit outside what this group is "for". If you explained your reasons for wanting to do this then maybe we can suggest alternatives?

On 30 May 2018 at 08:46, ⇜Krishnan Mahadevan⇝ <krishnan.ma...@gmail.com> wrote:
I believe I shared two alternatives. The log API in selenium is "Beta" so it's still work in progress and may not give you all aspects. 

What happens when you try using a proxy server such as browsermobproxy or fiddler ?
On Wed 30 May, 2018, 13:11 Ankit Gupta, <ankit...@gmail.com> wrote:
The same code which you shared, 
It does not have API's (GET, POST,PUT) request and response data.

--

To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to selenium-users@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to selenium-users@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to selenium-users@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to selenium-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/7AD72B34-D623-4973-B45C-DCE83F02FAA5%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to selenium-users@googlegroups.com.
--

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

 --
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to selenium-users@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to selenium-users@googlegroups.com.
--

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to selenium-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/CANikZLkyzJ%2BrQHb2xJRg%2B1q_2Xaen6ahY1eETbv%3Dv82a4YB5CA%40mail.gmail.com.

Ankit Gupta

unread,
May 30, 2018, 5:10:14 AM5/30/18
to seleniu...@googlegroups.com
I got the answer and Question is same as previous, To capture all the API's call while performing the test with selenium.

Using Fiddler and BrowserMob, the problem would solve completely.


Thanks a lot, Joe and Krishnan 

On Wed, May 30, 2018 at 1:56 PM, Joe Ward <ddl...@gmail.com> wrote:
What issue are you actually trying to solve by doing this? Selenium doesn't really have a programmatic way to do what you're asking. Fiddler and BrowserMob are probably OK for exploratory testing but a bit outside what this group is "for". If you explained your reasons for wanting to do this then maybe we can suggest alternatives?

--

To post to this group, send email to seleniu...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.

To post to this group, send email to seleniu...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/7AD72B34-D623-4973-B45C-DCE83F02FAA5%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.
--

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

 --
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.
--

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to selenium-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/CAJPyTs2%2BZA9OEsfSruahJJQSvMVUkqt5bbtMhqr%3D_gGgfp8iTA%40mail.gmail.com.

Ankit Gupta

unread,
May 30, 2018, 7:09:36 AM5/30/18
to seleniu...@googlegroups.com
Hi Krishnan,

Did you try with  BrowserMobProxy to get the logs? because it is not working for HTTPS website.
Could you please share the code if you did.

Reply all
Reply to author
Forward
0 new messages