How do I get cookies from an HttpResponse?

[Adam Mackler]

Maybe I'm missing something obvious, but I'm not seeing cookies in the headers of HttpResponses I'm receiving from servers.

I'm using spray can 1.3.1.  I send my request:

IO(Http) ! request

I get my response:

    case HttpResponse(status,entity,headers,protocol) =>

Then I look at the headers, and I see all the headers except Set-Cookie.  I'm watching the response using tcpdump, and I see Set-Cookie headers on the wire.  I've tried this with at least five different servers, and I never see anything cookie-related in the headers of the HttpResponse instance.  Some Set-Cookie headers look like they might not be standard-complaint but some look fine, for example:

Set-Cookie: JSESSIONID=0000fcgRLTTw0qUhpTNWQXmUIs:11q0esoik; Path=/       

Set-Cookie: PHPSESSID=dprac8ihv8gtb2v72s8rt5213; path=/

Is there anything wrong with either of these?

I'm looking for the cookies like this:

    case HttpResponse(status,entity,headers,protocol) =>

      println(s"headers: $headers")

I'm not seeing any warnings, and I think I have the spray log level set to "ALL" (but actually I'm not seeing any log output so maybe I've got something misconfigured there).  I've searched this group, and I've searched the spray documentation.  I've looked at the source code for CookieHeaders.scala and HttpParser.scala but I'm not familiar with parboiled enough to understand how responses are parsed.  I see there's a cookies method for an HttpRequest, but not for an HttpResponse.  I apologize if I'm asking a question that I should be able to answer for myself.  Can anyone give me a clue what I'm doing wrong?  Thank you.

--

Adam Mackler

[Mathias Doenitz]

Adam,

your question is definitely a valid one, so don’t worry about asking!

Here is a transcript of a quick test I just ran within the spray repository:

[info] Loading global plugins from ...
[info] Loading project definition from ...
[info] Set current project to root (in build file:/...)
root > p spray-can
[info] Set current project to spray-can (in build file:/...)
spray-can > console
[info] Starting scala interpreter...
[info]
Welcome to Scala version 2.10.3 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_51).
Type in expressions to have them evaluated.
Type :help for more information.

scala> :paste
// Entering paste mode (ctrl-D to finish)

import akka.pattern.ask
import spray.http._
import scala.concurrent.duration._
implicit val timeout: akka.util.Timeout = 5.seconds
implicit val system = akka.actor.ActorSystem()
import system.dispatcher
akka.io.IO(spray.can.Http).ask(HttpRequest(HttpMethods.HEAD,Uri("http://www.google.com/"))).onComplete(println)

// Exiting paste mode, now interpreting.

import akka.pattern.ask
import spray.http._
import scala.concurrent.duration._
timeout: akka.util.Timeout = Timeout(5 seconds)
system: akka.actor.ActorSystem = akka://default
import system.dispatcher

scala> Success(HttpResponse(200 OK,Empty,List(Transfer-Encoding: chunked, Alternate-Protocol: 80:quic, X-Frame-Options: SAMEORIGIN, X-XSS-Protection: 1; mode=block, Server: gws, P3P: CP="This is not a P3P policy! See http://www.google.com/support/accounts/bin/answer.py?hl=en&answer=151657 for more info.", Set-Cookie: NID=67=gmsD7sp4SHfxAdSstWd43wJePu2VFfrB370ZyRFHBVeJkre-gQHaufJbOwA92yiGbryopU5tolmQvd-1pseRz_Y3TqyjA-EUcDl_qFRyNp87ANOl5kPENQW9J; Expires=Sat, 04 Oct 2014 07:47:49 GMT; Path=/; HttpOnly; domain=.google.com, Set-Cookie: PREF=ID=b33313a2110c825b:FF=0:TM=139597669:LM=1396597669:S=bxuhnvpZTbW8J7; Expires=Sun, 03 Apr 2016 07:47:49 GMT; Path=/; domain=.google.com, Content-Type: text/html; charset=ISO-8859-1, Cache-Control: private, max-age=0, Expires: -1, Date: Fri, 04 Apr 2014 07:47:49 GMT),HTTP/1.1))


scala>


So, as you can see, spray delivers the `Set-Cookie` headers from google.com as expected.

> I'm not seeing any warnings, and I think I have the spray log level set to “ALL"

There is no `ALL` log config.
You should set `akka.loglevel` to `DEBUG` (and configure your logging backend (e.g. logback) to also show DEBUG level messages).

Cheers,
Mathias

---
mat...@spray.io
http://spray.io

> --
> You received this message because you are subscribed to the Google Groups "spray.io User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to spray-user+...@googlegroups.com.
> Visit this group at http://groups.google.com/group/spray-user.
> To view this discussion on the web visit https://groups.google.com/d/msgid/spray-user/995611cf-1273-4c9f-b36f-7786289f2ac6%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Adam Mackler]

Nevermind, I found what was happening.  I'm embarrassed to explain, but since you were kind enough to provide me with the helpful information, I suppose it's my penalty.

My requests were going through a proxy that I myself had configured several months ago to strip cookie headers.  And yes, I did just figure this seconds ago and was about to delete my OP.

Thanks again for the response, Mathias.  I'll take care of the face-palms for both of us.

--

Adam Mackler 

[Mathias Doenitz]

No worries at all, Adam!
Glad you found the problem!

Cheers,
Mathias

---
mat...@spray.io
http://spray.io

> --
> You received this message because you are subscribed to the Google Groups "spray.io User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to spray-user+...@googlegroups.com.
> Visit this group at http://groups.google.com/group/spray-user.

> To view this discussion on the web visit https://groups.google.com/d/msgid/spray-user/f1686481-86ae-48bf-9077-dfc0395cc7bb%40googlegroups.com.

[Adam Mackler]

I do have a followup question: one thing I started wondering about before I found the true problem is whether spray will ever automatically add cookies to outgoing requests.  In this case I'm using the request-level API, but I'm wondering if there is ever a situation using the Host- or Connection-level APIs when spray would automatically read a Set-Cookie header from a response, and then apply its value to subsequent Cookie: headers in requests sent through the same host- or connection-Actor, in simulation of typical browser behavior.  Might that ever happen?

--

Adam Mackler

[André]

Hi Adam,

please see https://github.com/spray/spray/issues/772. Is that what you're looking for?

Cheers

André

[Adam Mackler]

On Monday, April 7, 2014 3:41:05 AM UTC-4, André wrote:

please see https://github.com/spray/spray/issues/772. Is that what you're looking for?

Yes.  This point made by jrudolph is particularly enlightening (to me);

• cookies are logically not bound to a connection, so IMO the implementation should be at a higher level, i.e. not implemented as a pipeline stage but probably on the HttpClientSettingsGroup level. Even for a first cut the implementation on one local connection would bring much confusion because the higher levels may redistribute subsequent requests to different connections.

That's a good point that I had missed, that cookies are not an attribute of a connection, and making connection-centric assumptions about what to do with cookies is best decided by the client actively, not the library automatically.  That's my reading anyway.

Thank you for the helpful reference, André.

--

Adam Mackler