How to add headers in Selenium with CDP?

[Abdul Isik]

Hello all,

I need to send some headers to a test, mainly for testing geolocation for GDPR purposes. Though, there are some other uses of headers for us other than geolocation. Adding geolocation headers through a browser add-on does work for our tests.

Now, I've tried adding headers by using Network.setExtraHTTPHeaders, but apparently it applies to all the requests on the page, which breaks functionality and thus doesn't work. I've tried using the same method to update the headers with empty values after the initial page load, but it didn't work, and isn't a good idea to begin with.

Considering that the filter request to chromium team that I've linked above is 3+ years old at this point, so I'm in search of a different way to achieve this. I came across Network Interceptor in CDP documentation, but the Java code does not work, and is marked as "Disabled, not working yet". What can I do to make this feature work, in order to send extra headers for specific domains?
Or, is there another way of sending headers without affecting all the requests?

[⇜Krishnan Mahadevan⇝]

Abdul,

Can you please check if something like below would work for you ?

This sample did fudge my geo location [ Original Location: Bangalore, Karnataka, , Faked Location: San Francisco, USA] 

package org.example;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriverException;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.devtools.DevTools;
import org.openqa.selenium.devtools.HasDevTools;
import org.openqa.selenium.devtools.v119.emulation.Emulation;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.testng.Assert;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;

import java.util.Map;
import java.util.Optional;

//Reference: https://gist.github.com/ShamaUgale/552df64db476f26459a38e69fa02cfbe
public class GeoLocationTest {

    private RemoteWebDriver driver;

    @BeforeClass
    public void setup() {
        ChromeOptions options = new ChromeOptions();
        Map<String, Object> prefs = Map.of(
                "profile.default_content_settings.geolocation", 1,
                "googlegeolocationaccess.enabled", true
        );
        options.setExperimentalOption("prefs", prefs);
        driver = new ChromeDriver(options);
    }

    @Test
    public void testMethod() {
        double expectedLatitude =  37.615223;
        double expectedLongitude = -122.389977;
        DevTools devTools = ((HasDevTools) driver).getDevTools();
        devTools.createSession();
        devTools.send(Emulation.setGeolocationOverride(Optional.of(expectedLatitude),
                Optional.of(expectedLongitude), Optional.of(100)));
        driver.get("https://mycurrentlocation.net/");
        WebElement btn = driver.findElement(By.xpath("//button[@class='btn btn-warning mt-3 main-button']"));
        btn.click();
        dismissAlertsSilently();
        double actualLatitude = Double.parseDouble(driver.findElement(By.id("detail-latitude")).getText());
        System.err.println("Latitude = " + actualLatitude);
        double actualLongitude = Double.parseDouble(driver.findElement(By.id("detail-longitude")).getText());
        System.err.println("Longitude = " + actualLongitude);
        Assert.assertEquals(expectedLatitude, actualLatitude);
        Assert.assertEquals(expectedLongitude, actualLongitude);
    }

    @AfterClass
    public void cleanup() {
        driver.quit();
    }

    private void dismissAlertsSilently() {
        try {
            driver.switchTo().alert().dismiss();
        } catch (WebDriverException e) {
            System.err.println(e.getMessage());
        }

    }
}

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribblings @ https://rationaleemotions.com/

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/570b28f8-caa1-49e0-9e8e-41c59eeda3bcn%40googlegroups.com.

[Abdul Isik]

Hey Krishnan,

Thanks for your answer. I'm already using `Emulation.setGeolocationOverride`, and I'll add those extra Chrome Options, but I do need to send headers regardless.

You received this message because you are subscribed to a topic in the Google Groups "Selenium Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/selenium-users/JYCt1X1Tfjw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to selenium-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/CANikZLnpxjKe94b6UW%3DO%3DEq6ruyfGiwap_UQEWeb5jNg8rMkyA%40mail.gmail.com.

[Abdul Isik]

Following up on this. I've tried using Network Interception to replace headers, but cannot get it to work.

I'm hitting the exact same error from this Stack Overflow post: https://stackoverflow.com/questions/77372202/selenium-cdp-connection-error-on-fetch-chrome-devtools

In Selenium docs, Java example for Request Interception specifically says “Not working yet” as I mentioned earlier. Even though C# example and Ruby example for the same thing does not have that comment, I'm unable to test those though.

I can't find enough documentation on this. I keep scratching my head, thinking how come not too many people needed to add headers using Chrome Dev Tools Protocol or Selenium before?

As a last resort, I'll try to programmatically use ModHeaders extension to add headers, even though I despise the extension as they started to inject ads recently.

[⇜Krishnan Mahadevan⇝]

Abdul,

I think you should basically create an issue on the selenium github page and share a sample that can be used to reproduce the problem. Maybe someone more well versed on this than me, can help out ?

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribblings @ https://rationaleemotions.com/

To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/708fafcb-e243-432a-a71e-8b34e51e8a95n%40googlegroups.com.