"[WARNING] Invalid cookie header" when using driver.manage().addCookie() on selenium webdriver

[stan...@fmlogistic.com]

Hello, 

I get this error when launching this command to server having following geolocation on browserstack : 

"browserstack.geoLocation": "US",
"browserstack.timezone": "New_York"

when launching from "Paris", "FR", there is no warning

(I launch my test from france, so I should have format fr_FR on my system)

[2019-03-05 06:14:22] [WARNING] Invalid cookie header: "Set-Cookie: tracking_id=44b2a268-25bf-4736-993e-0e9ba8628ed4; domain=.browserstack.com; path=/; expires=Tue, 05 Mar 2024 05:14:22 -0000". Invalid 'expires' attribute: Tue, 05 Mar 2024 05:14:22 -0000 

according to following stackoverflow post (https://stackoverflow.com/questions/13302245/getting-set-cookie-header), this warning seem due to  'Set-Cookie' header with the RFC 2965 compliant spec, whereas RFC 2965 accepts 'Set-Cookie2' headers only.

The cookie in question is malformed. It contains non-standard 'expires' attribute, which, to make matters worse, contains a reserved character (comma) without enclosing quote marks. However, given it is a very common protocol violation HttpClient should be able to parse this cookie using 'best_match', 'browser_compatibility' or 'netscape_draft' policies.

Do you know if there is a way to select the 'best_match' policy when using driver.manage().addCookie() as suggested in this post ?

[Mark Collin]

I would expect the "-0000" bit to be "GMT" as per https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Date.

Can you show us your code?

[stan...@fmlogistic.com]

Hello, 

thank for your answer, yes I can tell a little bit more  :)

I use a custom method which allows to set a previously saved cookie : 

boolean cookieOk = CookieManagement.restoreCookies(driver, cookieFileName);

if cookie is not valid, I authenticate with login/password, and then I save cookie with method : 

CookieManagement.saveCookies(driver, cookieFileName);

Then I will use the cookie file to call restoreCookies() the next time. 

You can find complete code bellow.

Another usefull information could be an example of cookies sent to driver.manage().addCookie(ck) in my application : 

"_ga=GA1.2.183258443.1551780145; expires=Thu, 04 Mar 2021 11:02:24 CET; path=/; domain=.fistic.com"
"userSession=tot...@gmail.com; expires=Wed, 06 Mar 2019 11:02:28 CET; path=/; domain=training.fistic.com"
"PHPSESSID=52rurv4ij0ehgsebd7eai6pcb7; expires=Tue, 05 Mar 2019 11:02:11 CET; path=/; domain=training.fistic.com"
"BCSI-AC-210c81e1b576644d=2B87A0B800000305K3RnJ0dXg9UF42LJRYSdlsUtuXMRAAAABQMAAGmEDwDAqAAAAgAAAK4zAAA=; path=/; domain=.fistic.com"
"_gat=1; expires=Tue, 05 Mar 2019 11:03:24 CET; path=/; domain=.fistic.com"
"_gid=GA1.2.801017220.1551780145; expires=Wed, 06 Mar 2019 11:02:24 CET; path=/; domain=.fistic.com"

here is my code : 

package tests.base;

import org.openqa.selenium.Cookie;
import org.openqa.selenium.WebDriver;

import java.io.*;
import java.util.Set;
import java.util.logging.Level;

import static tests.base.TestBase.logger;

// Cookie Handling in Selenium WebDriver
// code inspiré de cette page : https://www.testingexcellence.com/webdriver-how-to-restore-cookies-in-new-browser-window/
public class CookieManagement {
 private static String cookiePrefix = "./target/Cookie";

 /**
 * Sauvegarde les cookies dans le fichier "/myfm-selenium/MyScmFramework/Cookies.ser"
 */
 public static void saveCookies(WebDriver driver, String cookieFileSuffixe) {

 Set<Cookie> allcookies = driver.manage().getCookies();
 logger.log(Level.FINE, "sauvegarde cookies : \n" + allcookies.toString());

 try {
 String cookieFile = cookiePrefix + cookieFileSuffixe.replaceAll("\\W", "-") + ".ser"; // on remplace les caractères spéciaux pour pouvoir créer le fichie

 File file = new File(cookieFile);
 file.delete();
 file.createNewFile();

 FileOutputStream fileOut = new FileOutputStream(cookieFile);
 ObjectOutputStream out = new ObjectOutputStream(fileOut);
 out.writeObject(allcookies);
 out.close();
 fileOut.close();
 logger.log(Level.FINE, "cookie saved in " + cookieFile);
 } catch (IOException i) {
 logger.log(Level.SEVERE, "Exception saveCookies");
 i.printStackTrace();
 }
 }

 @SuppressWarnings("unchecked")
 // le cast est sensible car on ne sais pas ce qu'il y a dans le fichier, mais pas moyen de se passer d'un cast en cookie
 // d'un autre coté, la seule méthode qui écrit dans ce fichier est saveCookies(), donc ça devrait bien se passer
 public static Set<Cookie> deserializeCookieFromFile(String cookieFile) throws IOException, ClassNotFoundException {
 Set<Cookie> allcookies;
 FileInputStream fis = new FileInputStream(cookieFile);
 ObjectInputStream ois = new ObjectInputStream(fis);
 allcookies = (Set<Cookie>) ois.readObject();
 ois.close();
 fis.close();

 return allcookies;
 }

 /**
 * récupère les cookies dans le fichier "/myfm-selenium/MyScmFramework/Cookies.ser"
 */
 public static boolean restoreCookies(WebDriver driver, String cookieFileSuffixe) {
 boolean statut=false;
 String cookieFile = cookiePrefix + cookieFileSuffixe.replaceAll("\\W", "-") + ".ser";

 Set<Cookie> allcookies;
 File f = new File(cookieFile);
 if (!f.exists()) {
 return false;
 }

 try {
 allcookies = deserializeCookieFromFile(cookieFile);
 }catch(IOException ioe){
 logger.log(Level.SEVERE, "Exception restoreCookies");
 ioe.printStackTrace();
 return false;
 }catch(ClassNotFoundException c){
 logger.log(Level.SEVERE, "Exception restoreCookies");
 logger.log(Level.SEVERE, "Class not found");
 c.printStackTrace();
 return false;
 }

 logger.fine("restauration cookies");
 logger.log(Level.FINE, "detail cookies : " + allcookies.toString());
 try {
 for (Cookie ck : allcookies) {
 // ck.getExpiry().
 //org.openqa.selenium.Cookie.
 driver.manage().addCookie(ck); // This will add the stored cookie to your current session
 statut = true;
 }
 } catch (Exception e) {
 logger.log(Level.INFO, "Exception levée lors de la restauration d'un cookie" + e.toString());
 }
 return statut;
 }
}

[Mark Collin]

Your error is complaining that:

Invalid 'expires' attribute: Tue, 05 Mar 2024 05:14:22 -0000

Whereas the cookies you have shown above have expires attributes that look like:

expires=Thu, 04 Mar 2021 11:02:24 CET;

I think that the way you are writing cookies out to file is not working as expected and is breaking the expires data.

I would suggest having a look at these for inspiration:

https://github.com/mwinteringham/selenium-to-restassured/blob/master/src/main/java/uk/co/mwtestconsultancy/CookieAdapter.java

https://github.com/Ardesco/Powder-Monkey/blob/master/src/main/java/com/lazerycode/selenium/filedownloader/FileDownloader.java#L150

[stan...@fmlogistic.com]

Hello Mark, 

sorry for time to answer but I was becaming crazy as code moficiation had no effect at all. 

Finally, I discovered that this warning was in fact thrown by another peace of code  !

so Faulty code was : 

HttpClientBuilder httpClientBuilder = HttpClientBuilder.create();
HttpHost proxy = new HttpHost(testEnv.getProxyHost(), Integer.parseInt(testEnv.getProxyPort()), "http");
httpClientBuilder.setProxy(proxy);
HttpClient closeableClient = httpClientBuilder.build();
com.julienvey.trello.impl.http.ApacheHttpClient httpClient = new com.julienvey.trello.impl.http.ApacheHttpClient(closeableClient);

Trello trelloApi;
trelloApi = new TrelloImpl(trelloKey, trelloAccessToken, httpClient);

List<Board> boards = trelloApi.getMemberBoards(trelloUsername);

and to solve my WARNING problem, I replaced 

HttpClient closeableClient = httpClientBuilder.build();

by

HttpClient closeableClient = httpClientBuilder
        .setDefaultRequestConfig(RequestConfig.custom()
                .setCookieSpec(CookieSpecs.STANDARD).build())
        .build();

This tip was found in this post :  https://stackoverflow.com/questions/36473478/fixing-httpclient-warning-invalid-expires-attribute-using-fluent-api

Thank you for your help, and sorry for wrong initial cause !