Network / Fetch access to multiple tabs

[Cary FitzHugh]

Hello -

My goal is to be able to record traffic in multiple tabs via the CDP.

According to the documentation, I believe I should be able to have:

1. A main CDP connection process which handles new target discovery (Target.targetCreated, etc)
2. For each new page-target:
   1. Connect directly to that target with CDP
   2. Enable Fetch, Network, Page
   3. Listen and capture traffic

I understand there are some race conditions with capturing traffic at the very start of a target's life. I have some ideas to work around those limitations. 

The main issue I am seeing , is that I seem unable to consistently watch the network traffic of two different targets.  What are the subtle constraints on listening to / pausing traffic?

I have tried to have one CDP connection, which detects targets and "attaches" to them.  This will let me record traffic on the initial tab, but subsequent tabs do not send Network.* or Fetch.* events (is there a way to make that work?)

I have tried multiple CDP connections, one per page-target, and that does not seem to consistently work either.  Sometimes it does work for the new tab to receive traffic events, but then the other CDP listener clients no longer receive events.

Is it true that only one event stream can be active from chrome browser at a time?
And if so, is there a way to get the event stream for *all* tabs at once?

Or possibly there are constraints I do not see in the documentation, because *sometimes* I can get events on two CDP clients.  But nothing consistent yet.

Thank you for any tips!
Cary FitzHugh

[Alex Rudenko]

Hi,

you probably need to use Target.setAutoAttach with waitForDebuggerOnStart=true. Then while the target is pauses you need to configure the Network/Fetch domains. 

To enable events for the target you need to call Network.enable/Network.fetch. All of this should happen in a single CDP connection but you would have multiple CDP sessions, one for every target. You need to include the right sessionId to configure domains for the new targets.

Best regards,
Alex

[Cary FitzHugh]

Hi Alex - thanks for the suggestions...

some pseudo-code:

initialize CDP connection

  Target.enable()

  Target.setDiscoverTargets(...)

  Target.setAutoAttach(autoAttach=T, waitForDebuggerOnStart=T, flatten=T)

  Network.enable()

  Fetch.enable()

wait for "Target.targetCreated" ?

wait for "Target.attachedToTarget" ?

It doesn't seem to work as I thought it should, based on your description.

The targetCreate events show targets which are not attached. (but autoAttach is true?)

How could I interact with a target which is paused for debugger startup?  I don't see this concept in the API.

I have used attachToTarget - which returns me a SessionID - but I see no API endpoints that use sessionID as an input?

Running Chromium Version 113.0.5672.126 (Official Build) snap (64-bit)

Thank you for any help!