After hard testing about why cookies doesn't work properly in ASProxy,
I could figure out a bug in CookieContainer class.
This class acts like database for cookies to send and receive them
from HttpWebRequest and HttpWebResponse.
The problem is about how CookieContainer handles domain. There are two
ways, one for Set-Cookie2 and one for Set-Cookie that CookieContainer
work. These refers to RFC 2965 and RFC 2109.
The main problem is the CookieContainer doesn't send desired cookies
to the specifed domain:
According to RFC 2109 the domain ".site.com" and "site.com" matches
host "http://site.com". But my test shows the CookieContainer class
doesn't act as expected.
I could find some workaround and hack which their links are blow, they
can fix this partially but they cause another issues. Such as extra
matching cookies which may cause some security issues (Gmail detects
this and that's why it doesn't work with ASProxy).
And now here is my suggestions to overcome this bug:
1- Hacking CookieContainer by using reflection.
2- Creating a new Cookie Database like class to store cookies, which
returns a CookieContainer to send it to WebRequest and WebResponse.
3- You tell!
Note: I have created to class which one of them is excluded from
project but both of them works. They are used to work with cookies and
implements CookieManager.
One is called "CookieManager_Old" and one called "CookieManager_New".
You can have a look at them. But non of them work properly that.
Please check that to see how cookies is implemented in ASProxy.
I'm not good at web program, but I'm wondering if it is possible to
write a new CookieContainer like this:
class NewCookieContainer : CookieContainer
{
public new CookieConnection GetCookies(uri)
{
do some special checks
if(the uri will cause bug)
do my code like use reflection;
else
return base.GetCookies(uri);
}
}
I hope that would work and thank you for your great work.
Samson
On Jul 19, 2:52 am, SalarSoft <salarsoftwa...@gmail.com> wrote:
> After hard testing about why cookies doesn't work properly in ASProxy,
> I could figure out a bug in CookieContainer class.
> This class acts like database for cookies to send and receive them
> from HttpWebRequest and HttpWebResponse.
> The problem is about how CookieContainer handles domain. There are two
> ways, one for Set-Cookie2 and one for Set-Cookie that CookieContainer
> work. These refers to RFC 2965 and RFC 2109.
> The main problem is the CookieContainer doesn't send desired cookies
> to the specifed domain:
> According to RFC 2109 the domain ".site.com" and "site.com" matches
> host "http://site.com". But my test shows the CookieContainer class
> doesn't act as expected.
> I could find some workaround and hack which their links are blow, they
> can fix this partially but they cause another issues. Such as extra
> matching cookies which may cause some security issues (Gmail detects
> this and that's why it doesn't work with ASProxy).
> And now here is my suggestions to overcome this bug:
> 1- Hacking CookieContainer by using reflection.
> 2- Creating a new Cookie Database like class to store cookies, which
> returns a CookieContainer to send it to WebRequest and WebResponse.
> 3- You tell!
> Note: I have created to class which one of them is excluded from
> project but both of them works. They are used to work with cookies and
> implements CookieManager.
> One is called "CookieManager_Old" and one called "CookieManager_New".
> You can have a look at them. But non of them work properly that.
> Please check that to see how cookies is implemented in ASProxy.
> I'm not good at web program, but I'm wondering if it is possible to
> write a new CookieContainer like this:
> class NewCookieContainer : CookieContainer
> {
> public new CookieConnection GetCookies(uri)
> {
> do some special checks
> if(the uri will cause bug)
> do my code like use reflection;
> else
> return base.GetCookies(uri);
> }}
> I hope that would work and thank you for your great work.
> Samson
> On Jul 19, 2:52 am, SalarSoft <salarsoftwa...@gmail.com> wrote:
> > After hard testing about why cookies doesn't work properly in ASProxy,
> > I could figure out a bug in CookieContainer class.
> > This class acts like database for cookies to send and receive them
> > from HttpWebRequest and HttpWebResponse.
> > The problem is about how CookieContainer handles domain. There are two
> > ways, one for Set-Cookie2 and one for Set-Cookie that CookieContainer
> > work. These refers to RFC 2965 and RFC 2109.
> > The main problem is the CookieContainer doesn't send desired cookies
> > to the specifed domain:
> > According to RFC 2109 the domain ".site.com" and "site.com" matches
> > host "http://site.com". But my test shows the CookieContainer class
> > doesn't act as expected.
> > I could find some workaround and hack which their links are blow, they
> > can fix this partially but they cause another issues. Such as extra
> > matching cookies which may cause some security issues (Gmail detects
> > this and that's why it doesn't work with ASProxy).
> > And now here is my suggestions to overcome this bug:
> > 1- Hacking CookieContainer by using reflection.
> > 2- Creating a new Cookie Database like class to store cookies, which
> > returns a CookieContainer to send it to WebRequest and WebResponse.
> > 3- You tell!
> > Note: I have created to class which one of them is excluded from
> > project but both of them works. They are used to work with cookies and
> > implements CookieManager.
> > One is called "CookieManager_Old" and one called "CookieManager_New".
> > You can have a look at them. But non of them work properly that.
> > Please check that to see how cookies is implemented in ASProxy.
> That's a tricky code, seems there is no other way to overcome this
> bug.
> Thanks, I'll take a look at it soon.
> On Sep 5, 6:53 pm, Samson <walkin...@gmail.com> wrote:
> > I'm not good at web program, but I'm wondering if it is possible to
> > write a new CookieContainer like this:
> > class NewCookieContainer : CookieContainer
> > {
> > public new CookieConnection GetCookies(uri)
> > {
> > do some special checks
> > if(the uri will cause bug)
> > do my code like use reflection;
> > else
> > return base.GetCookies(uri);
> > }}
> > I hope that would work and thank you for your great work.
> > Samson
> > On Jul 19, 2:52 am, SalarSoft <salarsoftwa...@gmail.com> wrote:
> > > After hard testing about why cookies doesn't work properly in ASProxy,
> > > I could figure out a bug in CookieContainer class.
> > > This class acts like database for cookies to send and receive them
> > > from HttpWebRequest and HttpWebResponse.
> > > The problem is about how CookieContainer handles domain. There are two
> > > ways, one for Set-Cookie2 and one for Set-Cookie that CookieContainer
> > > work. These refers to RFC 2965 and RFC 2109.
> > > The main problem is the CookieContainer doesn't send desired cookies
> > > to the specifed domain:
> > > According to RFC 2109 the domain ".site.com" and "site.com" matches
> > > host "http://site.com". But my test shows the CookieContainer class
> > > doesn't act as expected.
> > > I could find some workaround and hack which their links are blow, they
> > > can fix this partially but they cause another issues. Such as extra
> > > matching cookies which may cause some security issues (Gmail detects
> > > this and that's why it doesn't work with ASProxy).
> > > And now here is my suggestions to overcome this bug:
> > > 1- Hacking CookieContainer by using reflection.
> > > 2- Creating a new Cookie Database like class to store cookies, which
> > > returns a CookieContainer to send it to WebRequest and WebResponse.
> > > 3- You tell!
> > > Note: I have created to class which one of them is excluded from
> > > project but both of them works. They are used to work with cookies and
> > > implements CookieManager.
> > > One is called "CookieManager_Old" and one called "CookieManager_New".
> > > You can have a look at them. But non of them work properly that.
> > > Please check that to see how cookies is implemented in ASProxy.
Replying on your email, I already studying this CookieContainer, CC
since I read this issue here last 2 weeks.
Also I have read your feedback to Microsoft, also your discussions.
Due to simply answer your feedback fixed in future release, rather
than trying to help you, I disappointed because 4.0 can't be use
widely in the recent days. Then I want to 'hack' the CookieContainer
and inspect what was goes wrong with it.
After some googling, I get this code and modify it a little bit.
public CookieCollection GetAllCookies(CookieContainer cc)
{
CookieCollection lstCookies = new CookieCollection();
Hashtable table = (Hashtable)cc.GetType().InvokeMember
("m_domainTable", System.Reflection.BindingFlags.NonPublic |
System.Reflection.BindingFlags.GetField |
System.Reflection.BindingFlags.Instance, null, cc, new object[] { });
foreach (object pathList in table.Values)
{
SortedList lstCookieCol = (SortedList)pathList.GetType
().InvokeMember("m_list", System.Reflection.BindingFlags.NonPublic |
System.Reflection.BindingFlags.GetField |
System.Reflection.BindingFlags.Instance, null, pathList, new object[]
{ });
foreach (CookieCollection colCookies in
lstCookieCol.Values)
foreach (Cookie c in colCookies)
{
lstCookies.Add(c);
}
}
return lstCookies;
}
This code can retrieve all stored cookie in any domain and path.
Debugging the table, lstCookieCol, colCookies and c, I have better
understanding about how CC store the cookies.
Doing some reflection to the CC also help me.
Its 'grouped' by domain (Hashtable table) and path (SortedList
lstCookieCol). The issue occur when using .Add(Cookie) method.
it different than .Add(Uri, Cookie) in the table key which is domain
name.
.Add(Cookie) - BUG here. simply use domain from the cookie as table
key. So domain.com, .domain.com and sub.domain.com is three different
key.
.Add(Uri, Cookie) - it will make sure the domain have dot in the
beginning. So domain.com and .domain.com use one .domain.com key.
In short, .Add(Uri, Cookie) is doing well but .Add(Cookie) is not.
Your CookieManager_New.cs is using the right one.
.SetCookie(Uri, cookieHeader) is using internal .Add()
Then I found that .GetCookies(Uri) is another BUG. It can't retrieve
cookie from "current sub domain start with dot and parent domain not
start with dot".
So since all domain key start with dot, GetCookies method can't
retrieve current sub domain. So in order to solve it domain with dot
and no dot should be in the table key as you fix it.
It work. Cookie ".domain.com" can be retrieve for "http://domain.com".
Also cookie "sub.domain.com" and ".domain.com" can be retrieve for
"http://sub.domain.com".
Yes I meant it you have solve the CookieContainer issue. I use the new
CookieManager_New.cs I download from ASProxy 5.2. I think it is your
latest one (during you discuss this cookie issue). However your latest
svn just adding BugFix_AddDotCookieDomain after
ApplyRequestToCookieContainer in AddCookiesToRequest function. Its
corrent, because httpWebRequest.CookieContainer is just change. You
need to call BugFix_AddDotCookieDomain each time CookieContainer added
a new cookie or before retrieve any cookie.
Ok, CookieContainer issue fixed. #2 is quite complicated, we discuss
later. Now #1 should be fixed. Cookie is not acting well in ASProxy.
It receive the correct cookie from web response but it send back not
the original one. It was cause cookies grouped into single www.domain.com_ASPX,
yes you can see the right cookie in the browser. but later cookies is
not ungroup to be send the original one.
I checked http traffic between ASProxy and web server by using
Fiddler2. Fiddler shows what cookie sent to the web server is not the
original one. It is www.domain.com_ASPX cookie.
Please use this Fiddler. It help so much. Think that
ApplyRequestToCookieContainer in the AddCookiesToRequest. You can see
that ApplyRequestToCookieContainer is simply get all *_ASPX cookie and
add it to the container to be sent to web server. You need to extract
the cookies inside each *_ASPX as you group it in GetCookieHeader by
using comma seperator.
I am pretty sure that I not confuse the way this CookieManager does. I
am not change the whole behavior but just add lines of code (quite
much) in a function, add functions, remove a few existing codes, and
some other move the location. I still not change the behavior and all
other classes is still remain unchange.
I have the new codes modified but it just modified for #2 and now lots
of codes there. I will give you later, still in modification. The
existing behavior still the same but just small thing I change like
cookie grouped into domain.com_ASPX to make all cookies in the same
domain including all it sub domain stored in the same CookieContainer.
We can discuss and proceed to fix this if you want.
your CookieManager class still not doing right
On Thu, Oct 8, 2009 at 3:40 AM, Salar <salarsoftwa...@gmail.com>
wrote:
> Hi,
> You have an approved post in asproxy group before, so approval is not required anymore.
> I have created this test, which shows CookieContainer does not behave as expected.
> http://stackoverflow.com/questions/1047669/cookiecontainer-bug > After applying BugFix_AddDotCookieDomain the issue behavior changes!!. The new issue is about more cookies than expected for a domain.
> How can you find #1 issue? it is tested very much and all original cookies are sending to the back-end site. I think you are confused with client-user cookie which in embeded in xxx_ASPX cookies.
> I wonder how you fix this. maybe you've changed the whole behavior?!
> The #2 is main problem which not all cookies is send to the back-end site, because of CookieContainer.
> Also google is changing the coookie in scripts which is not known very much, but still it is encoded by __CookieGet/__CookieSet functions. And yes here is another unknown issue.
> Have you read this page:
> https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx... > The bug is accepted and fixed in .NET 4. I'm waiting to see what changes they have made to correct the CookieContainer behaviour, so I can simulate it for .NET 2, this can be a stable solution.
> And I'll be happy to see what changes you've made in the original code. Attach the code if you want.
> Thanks for the attention.
> Regards
> Salar.Kh
> On Wed, Oct 7, 2009 at 9:34 PM, --:| LaNN |:-- <nalz...@gmail.com> wrote:
>> Hi,
>> Sorry I am confused Reply or Reply to author in this google groups. Its not friendly enough.
>> Last week I post about CookieContainer here, quite long post, but the msg seems to miss when I send
>> because it not show like normally something like message sent.
>> It just Your post was successfull but my pust is not shown,
>> or it not show something like the post pending approve by administrator.
>> However I just want to share that I see the bugz here...
>> Last time I notice the CookieContainer have an issue. I learn how it works and what makes the bug.
>> Lastly I found that your BugFix_AddDotCookieDomain() enough to solve the CookieContainer bug. so it almost solved.
>> At the same time I can see the CookieManager is not doing the right.
>> In short I notice there is 2 issue in the CookieManager:
>> #1 The domain.com_ASPX cookie is simply send to web req rather than their every single original cookie. Critical
>> #2 Each cookie should be visible to client script. So it can access and modify at document.cookie. Optional. not sure if google modify cookie in script.
>> but if #2 is not fixed, web that require access and modify cookie is not compatible.
>> I am using Firebug, FireCookie, HttpFox and Fiddler2.
>> I am sure this are the cause of Google login. I notice since google tell me to turn on cookie. Thats why I try to figure it out back now.
>> I try to fix since I know it is not the CookieContainer issue but CookieManager not implemented correctly.
>> I fix the #1. I have no luck to login. After partially fix the #2, I found that my #1 fix is not proper. After I fix the #1 again, I be able to login to google service.
>> Just test in web search, one step before gmail.
>> After 2 weeks since now I can 70% login to gmail . Something like miss one more request response.
>> On Tue, Oct 6, 2009 at 11:25 AM, Salar <salarsoftwa...@gmail.com> wrote:
>>> Hi,
>>> No i didn't got any mail or post from you!
>>> On Mon, Oct 5, 2009 at 6:12 PM, CallMeLaNN <nalz...@gmail.com> wrote:
>>>> Hi,
>>>> Did you receive my post before, in last few days?
>>>> I tell about founding the cookie issue solution,
>>>> it is related to google login.
>>>> CallMeLaNN
>>>> On Sep 6, 2:29 am, SalarSoft <salarsoftwa...@gmail.com> wrote:
>>>> > That's a tricky code, seems there is no other way to overcome this
>>>> > bug.
>>>> > Thanks, I'll take a look at it soon.
>>>> > On Sep 5, 6:53 pm, Samson <walkin...@gmail.com> wrote:
>>>> > > I'm not good at web program, but I'm wondering if it is possible to
>>>> > > write
> Replying on your email, I already studying this CookieContainer, CC
> since I read this issue here last 2 weeks.
> Also I have read your feedback to Microsoft, also your discussions.
> Due to simply answer your feedback fixed in future release, rather
> than trying to help you, I disappointed because 4.0 can't be use
> widely in the recent days. Then I want to 'hack' the CookieContainer
> and inspect what was goes wrong with it.
> After some googling, I get this code and modify it a little bit.
> This code can retrieve all stored cookie in any domain and path.
> Debugging the table, lstCookieCol, colCookies and c, I have better
> understanding about how CC store the cookies.
> Doing some reflection to the CC also help me.
> Its 'grouped' by domain (Hashtable table) and path (SortedList
> lstCookieCol). The issue occur when using .Add(Cookie) method.
> it different than .Add(Uri, Cookie) in the table key which is domain
> name.
> .Add(Cookie) - BUG here. simply use domain from the cookie as table
> key. So domain.com, .domain.com and sub.domain.com is three different
> key.
> .Add(Uri, Cookie) - it will make sure the domain have dot in the
> beginning. So domain.com and .domain.com use one .domain.com key.
> In short, .Add(Uri, Cookie) is doing well but .Add(Cookie) is not.
> Your CookieManager_New.cs is using the right one.
> .SetCookie(Uri, cookieHeader) is using internal .Add()
> Then I found that .GetCookies(Uri) is another BUG. It can't retrieve
> cookie from "current sub domain start with dot and parent domain not
> start with dot".
> So since all domain key start with dot, GetCookies method can't
> retrieve current sub domain. So in order to solve it domain with dot
> and no dot should be in the table key as you fix it.
> It work. Cookie ".domain.com" can be retrieve for "http://domain.com".
> Also cookie "sub.domain.com" and ".domain.com" can be retrieve for
> "http://sub.domain.com".
> Yes I meant it you have solve the CookieContainer issue. I use the new
> CookieManager_New.cs I download from ASProxy 5.2. I think it is your
> latest one (during you discuss this cookie issue). However your latest
> svn just adding BugFix_AddDotCookieDomain after
> ApplyRequestToCookieContainer in AddCookiesToRequest function. Its
> corrent, because httpWebRequest.CookieContainer is just change. You
> need to call BugFix_AddDotCookieDomain each time CookieContainer added
> a new cookie or before retrieve any cookie.
> Ok, CookieContainer issue fixed. #2 is quite complicated, we discuss
> later. Now #1 should be fixed. Cookie is not acting well in ASProxy.
> It receive the correct cookie from web response but it send back not
> the original one. It was cause cookies grouped into singlewww.domain.com_ASPX,
> yes you can see the right cookie in the browser. but later cookies is
> not ungroup to be send the original one.
> I checked http traffic between ASProxy and web server by using
> Fiddler2. Fiddler shows what cookie sent to the web server is not the
> original one. It iswww.domain.com_ASPXcookie.
> Please use this Fiddler. It help so much. Think that
> ApplyRequestToCookieContainer in the AddCookiesToRequest. You can see
> that ApplyRequestToCookieContainer is simply get all *_ASPX cookie and
> add it to the container to be sent to web server. You need to extract
> the cookies inside each *_ASPX as you group it in GetCookieHeader by
> using comma seperator.
> I am pretty sure that I not confuse the way this CookieManager does. I
> am not change the whole behavior but just add lines of code (quite
> much) in a function, add functions, remove a few existing codes, and
> some other move the location. I still not change the behavior and all
> other classes is still remain unchange.
> I have the new codes modified but it just modified for #2 and now lots
> of codes there. I will give you later, still in modification. The
> existing behavior still the same but just small thing I change like
> cookie grouped into domain.com_ASPX to make all cookies in the same
> domain including all it sub domain stored in the same CookieContainer.
> We can discuss and proceed to fix this if you want.
> your CookieManager class still not doing right
> On Thu, Oct 8, 2009 at 3:40 AM, Salar <salarsoftwa...@gmail.com>
> wrote:
> > Hi,
> > You have an approved post in asproxy group before, so approval is not required anymore.
> > I have created this test, which shows CookieContainer does not behave as expected.
> >http://stackoverflow.com/questions/1047669/cookiecontainer-bug > > After applying BugFix_AddDotCookieDomain the issue behavior changes!!. The new issue is about more cookies than expected for a domain.
> > How can you find #1 issue? it is tested very much and all original cookies are sending to the back-end site. I think you are confused with client-user cookie which in embeded in xxx_ASPX cookies.
> > I wonder how you fix this. maybe you've changed the whole behavior?!
> > The #2 is main problem which not all cookies is send to the back-end site, because of CookieContainer.
> > Also google is changing the coookie in scripts which is not known very much, but still it is encoded by __CookieGet/__CookieSet functions. And yes here is another unknown issue.
> > Have you read this page:
> >https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx...
> > The bug is accepted and fixed in .NET 4. I'm waiting to see what changes they have made to correct the CookieContainer behaviour, so I can simulate it for .NET 2, this can be a stable solution.
> > And, there is a CookieManager_New.cs which is completely works with CookieContainer and uses its methods to store/restore cookies. But due CookieContainer but it is completely uselss. Have a look from svn:
> >http://asproxy.svn.sourceforge.net/viewvc/asproxy/SalarSoft.ASProxy/B...
> > And I'll be happy to see what changes you've made in the original code. Attach the code if you want.
> > Thanks for the attention.
> > Regards
> > Salar.Kh
> > On Wed, Oct 7, 2009 at 9:34 PM, --:| LaNN |:-- <nalz...@gmail.com> wrote:
> >> Hi,
> >> Sorry I am confused Reply or Reply to author in this google groups. Its not friendly enough.
> >> Last week I post about CookieContainer here, quite long post, but the msg seems to miss when I send
> >> because it not show like normally something like message sent.
> >> It just Your post was successfull but my pust is not shown,
> >> or it not show something like the post pending approve by administrator.
> >> However I just want to share that I see the bugz here...
> >> Last time I notice the CookieContainer have an issue. I learn how it works and what makes the bug.
> >> Lastly I found that your BugFix_AddDotCookieDomain() enough to solve the CookieContainer bug. so it almost solved.
> >> At the same time I can see the CookieManager is not doing the right.
> >> In short I notice there is 2 issue in the CookieManager:
> >> #1 The domain.com_ASPX cookie is simply send to web req rather than their every single original cookie. Critical
> >> #2 Each cookie should be visible to client script. So it can access and modify at document.cookie. Optional. not sure if google modify cookie in script.
> >> but if #2 is not fixed, web that require access and modify cookie is not compatible.
> >> I am using Firebug, FireCookie, HttpFox and Fiddler2.
> >> I am sure this are the cause of Google login. I notice since google tell me to turn on cookie. Thats why I try to figure it out back now.
> >> I try to fix since I know it is not the CookieContainer issue but CookieManager not implemented correctly.
> >> I fix the #1. I have no luck to login. After partially fix the #2, I found that my #1 fix is not proper. After I fix the #1 again, I be able to login to google service.
> >> Just test in web search, one step before gmail.
> >> After 2 weeks since now I can 70% login to gmail . Something like miss one more request response.
> >> On Tue, Oct 6, 2009 at 11:25 AM, Salar <salarsoftwa...@gmail.com> wrote:
> >>> Hi,
> >>> No i didn't got any mail or post from you!
> >>> On Mon, Oct 5, 2009 at 6:12 PM, CallMeLaNN <nalz...@gmail.com> wrote:
> >>>> Hi,
> >>>> Did you receive my post before, in last few days?
> >>>> I tell about founding the cookie issue solution,
> >>>> it is related to
Ok, i had a look on the code and the bugfix.
I remember that I didn't use this CookieManager_New.cs as default
CookieManager because it doesn't fix the issue, in fact the issue
changes face.
(the default cookieManager which delivers with ASProxy is
CookieManager_Old.cs, you can change these from App_Data/
EngineProviders.xml and the CookieManager_New.cs file is excluded form
project.)
The method BugFix_AddDotCookieDomain you've mentioned doesn't work
well.
Consider that we have these three cookies:
Cookie#1: Test1=val; domain=sub.site.com; path=/
Cookie#2: Test2=val; domain=.site.com; path=/
Cookie#3: Test3=val; domain=site.com; path=/
without bugfix we won't get any cookie for http://site.com url, but
with the bugfix this issue fixes.
But there will be other issue, for http://sub.site.com we will get one
extra cookie which is not expected. the extra cookie is Cookie#3. It
means a cookie from "site.com" should not send to "sub.site.com", but
still ".site.com" can be send.
Why! because in the BugFix_AddDotCookieDomain we hack this
m_domainTable table and we have two sign of every cookie "beginning
with dot"/"no dot in the beginning". That causes the issue and that's
why i call for help! ;) :P
I'm still looking at the codes and will post later.
On Oct 8, 2:14 pm, CallMeLaNN <nalz...@gmail.com> wrote:
> Replying on your email, I already studying this CookieContainer, CC
> since I read this issue here last 2 weeks.
> Also I have read your feedback to Microsoft, also your discussions.
> Due to simply answer your feedback fixed in future release, rather
> than trying to help you, I disappointed because 4.0 can't be use
> widely in the recent days. Then I want to 'hack' the CookieContainer
> and inspect what was goes wrong with it.
> After some googling, I get this code and modify it a little bit.
> This code can retrieve all stored cookie in any domain and path.
> Debugging the table, lstCookieCol, colCookies and c, I have better
> understanding about how CC store the cookies.
> Doing some reflection to the CC also help me.
> Its 'grouped' by domain (Hashtable table) and path (SortedList
> lstCookieCol). The issue occur when using .Add(Cookie) method.
> it different than .Add(Uri, Cookie) in the table key which is domain
> name.
> .Add(Cookie) - BUG here. simply use domain from the cookie as table
> key. So domain.com, .domain.com and sub.domain.com is three different
> key.
> .Add(Uri, Cookie) - it will make sure the domain have dot in the
> beginning. So domain.com and .domain.com use one .domain.com key.
> In short, .Add(Uri, Cookie) is doing well but .Add(Cookie) is not.
> Your CookieManager_New.cs is using the right one.
> .SetCookie(Uri, cookieHeader) is using internal .Add()
> Then I found that .GetCookies(Uri) is another BUG. It can't retrieve
> cookie from "current sub domain start with dot and parent domain not
> start with dot".
> So since all domain key start with dot, GetCookies method can't
> retrieve current sub domain. So in order to solve it domain with dot
> and no dot should be in the table key as you fix it.
> It work. Cookie ".domain.com" can be retrieve for "http://domain.com".
> Also cookie "sub.domain.com" and ".domain.com" can be retrieve for
> "http://sub.domain.com".
> Yes I meant it you have solve the CookieContainer issue. I use the new
> CookieManager_New.cs I download from ASProxy 5.2. I think it is your
> latest one (during you discuss this cookie issue). However your latest
> svn just adding BugFix_AddDotCookieDomain after
> ApplyRequestToCookieContainer in AddCookiesToRequest function. Its
> corrent, because httpWebRequest.CookieContainer is just change. You
> need to call BugFix_AddDotCookieDomain each time CookieContainer added
> a new cookie or before retrieve any cookie.
> Ok, CookieContainer issue fixed. #2 is quite complicated, we discuss
> later. Now #1 should be fixed. Cookie is not acting well in ASProxy.
> It receive the correct cookie from web response but it send back not
> the original one. It was cause cookies grouped into singlewww.domain.com_ASPX,
> yes you can see the right cookie in the browser. but later cookies is
> not ungroup to be send the original one.
> I checked http traffic between ASProxy and web server by using
> Fiddler2. Fiddler shows what cookie sent to the web server is not the
> original one. It iswww.domain.com_ASPXcookie.
> Please use this Fiddler. It help so much. Think that
> ApplyRequestToCookieContainer in the AddCookiesToRequest. You can see
> that ApplyRequestToCookieContainer is simply get all *_ASPX cookie and
> add it to the container to be sent to web server. You need to extract
> the cookies inside each *_ASPX as you group it in GetCookieHeader by
> using comma seperator.
> I am pretty sure that I not confuse the way this CookieManager does. I
> am not change the whole behavior but just add lines of code (quite
> much) in a function, add functions, remove a few existing codes, and
> some other move the location. I still not change the behavior and all
> other classes is still remain unchange.
> I have the new codes modified but it just modified for #2 and now lots
> of codes there. I will give you later, still in modification. The
> existing behavior still the same but just small thing I change like
> cookie grouped into domain.com_ASPX to make all cookies in the same
> domain including all it sub domain stored in the same CookieContainer.
> We can discuss and proceed to fix this if you want.
> your CookieManager class still not doing right
> On Thu, Oct 8, 2009 at 3:40 AM, Salar <salarsoftwa...@gmail.com>
> wrote:
> > Hi,
> > You have an approved post in asproxy group before, so approval is not required anymore.
> > I have created this test, which shows CookieContainer does not behave as expected.
> >http://stackoverflow.com/questions/1047669/cookiecontainer-bug > > After applying BugFix_AddDotCookieDomain the issue behavior changes!!. The new issue is about more cookies than expected for a domain.
> > How can you find #1 issue? it is tested very much and all original cookies are sending to the back-end site. I think you are confused with client-user cookie which in embeded in xxx_ASPX cookies.
> > I wonder how you fix this. maybe you've changed the whole behavior?!
> > The #2 is main problem which not all cookies is send to the back-end site, because of CookieContainer.
> > Also google is changing the coookie in scripts which is not known very much, but still it is encoded by __CookieGet/__CookieSet functions. And yes here is another unknown issue.
> > Have you read this page:
> >https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx...
> > The bug is accepted and fixed in .NET 4. I'm waiting to see what changes they have made to correct the CookieContainer behaviour, so I can simulate it for .NET 2, this can be a stable solution.
> > And, there is a CookieManager_New.cs which is completely works with CookieContainer and uses its methods to store/restore cookies. But due CookieContainer but it is completely uselss. Have a look from svn:
> >http://asproxy.svn.sourceforge.net/viewvc/asproxy/SalarSoft.ASProxy/B...
> > And I'll be happy to see what changes you've made in the original code. Attach the code if you want.
> > Thanks for the attention.
> > Regards
> > Salar.Kh
> > On Wed, Oct 7, 2009 at 9:34 PM, --:| LaNN |:-- <nalz...@gmail.com> wrote:
> >> Hi,
> >> Sorry I am confused Reply or Reply to author in this google groups. Its not friendly enough.
> >> Last week I post about CookieContainer here, quite long post, but the msg seems to miss when I send
> >> because it not show like normally something like message sent.
> >> It just Your post was successfull but my pust is not shown,
> >> or it not show something like the post pending approve by administrator.
> >> However I just want to share that I see the bugz here...
> >> Last time I notice the CookieContainer have an issue. I learn how it works and what makes the bug.
> >> Lastly I found that your BugFix_AddDotCookieDomain() enough to solve the CookieContainer bug. so it almost solved.
> >> At the same time I can see the CookieManager is not doing the right.
> >> In short I notice there is 2 issue in the CookieManager:
> >> #1 The domain.com_ASPX cookie is simply send to web req rather than their every single original cookie. Critical
> >> #2 Each cookie should be visible to client script. So it can access and
I can't receive your posts from this group. Maybe I need to
Unsubscribe and Join again.
I will look into the given link.
I notice the _Old and _New but just tought you prefer in _New, since
the _New cookie management algorithm seems good.
Which one you want to use?
Ok, I understood your cookie bug on the dot.
Now I know that non-dot cookie can't be visible on the child
subdomain.
I will look again on how to hack the CC.
It is security issue.
If only this issue, any web site using cookies should be working.
(because cookie not missing but more than expected)
Please look into #1 we discuss before since that the cause cookie not
working well in any sites. Fiddler would helps.
Good news, now I can open gmail inbox, at least HTML version but still
have small cookie and javascript issue.
On Oct 9, 10:52 pm, SalarSoft <salarsoftwa...@gmail.com> wrote:
The extra cookie is not a big security issue since it is in the same
domain.
However it still a security issue and need to be solved.
If you want to take care the security issue properly, I want to add
you more security issue :)
Note that browser (FireFox javascript) are allowed to set cookie to
parent sub domain.
That means page in http://sub.domain.com.sg/... allowed set cookie
with domain ".domain.com.sg".
I assume that the standard (RFC may be) also allowed it. (I trust
FireFox)
So consider that what about the page set cookie with domain ".com.sg"?
In CookieContainer its possible because it assume "com.*" is a domain
name. It will only know "*.com" is a TLD since it was last dot.
Surely the last one is TLD but there is a TLD have two level (two
dot).
Even, there are so many TLD around the world that is not standard.
Eg: "anydomain.co.uk" - co used instead of com, "permalink.uk" is an
exception that there is no "co" or "com" used and much more.
It is difficult to determine whether the dot is one or two, which one
is domain name and which is TLD.
I have build RegEx before but not all domain can be parsed correctly.
The practice way is using list.
There is a Public Suffix List maintained by Mozilla. It is a list of
TLDs available and up to date.
It is being used by FireFox, Google Crome and Opera to manage this
cookie problem and much more.
I have code that can split the sub domain, domain and TLD accurately
based on the Public Suffix List.
We can implement later after the extra cookie issue solved.
On Oct 10, 10:21 pm, CallMeLaNN <nalz...@gmail.com> wrote:
> I can't receive your posts from this group. Maybe I need to
> Unsubscribe and Join again.
> I will look into the given link.
> I notice the _Old and _New but just tought you prefer in _New, since
> the _New cookie management algorithm seems good.
> Which one you want to use?
> Ok, I understood your cookie bug on the dot.
> Now I know that non-dot cookie can't be visible on the child
> subdomain.
> I will look again on how to hack the CC.
> It is security issue.
> If only this issue, any web site using cookies should be working.
> (because cookie not missing but more than expected)
> Please look into #1 we discuss before since that the cause cookie not
> working well in any sites. Fiddler would helps.
> Good news, now I can open gmail inbox, at least HTML version but still
> have small cookie and javascript issue.
> On Oct 9, 10:52 pm, SalarSoft <salarsoftwa...@gmail.com> wrote:
Cookie c1 = new Cookie("c1", ".asd.qwe.zxc.com.my", "/",
".asd.qwe.zxc.com.my");
Cookie c2 = new Cookie("c2", "asd.qwe.zxc.com.my", "/",
"asd.qwe.zxc.com.my");
Cookie c3 = new Cookie("c3", ".qwe.zxc.com.my", "/",
".qwe.zxc.com.my");
Cookie c4 = new Cookie("c4", "qwe.zxc.com.my", "/",
"qwe.zxc.com.my");
Cookie c5 = new Cookie("c5", ".zxc.com.my", "/",
".zxc.com.my");
Cookie c6 = new Cookie("c6", "zxc.com.my", "/",
"zxc.com.my");
Cookie c7 = new Cookie("c7", ".com.my", "/", ".com.my");
Cookie c8 = new Cookie("c8", "com.my", "/", "com.my");
Uri u = new Uri("Http://asd.qwe.zxc.com.my");
It is a debug output result from Unit Test:
---------------------------------------------------------------------
Testing Add(Uri, Cookie) method (including uri):
CookieContainer Domain keys:
.com.my Cookies Count: 2
.qwe.zxc.com.my Cookies Count: 2
.zxc.com.my Cookies Count: 2
.asd.qwe.zxc.com.my Cookies Count: 2
Actual cookies contain in the CookieContainer:
c7=.com.my Domain key: .com.my
c8=com.my Domain key: .com.my
c3=.qwe.zxc.com.my Domain key: .qwe.zxc.com.my
c4=qwe.zxc.com.my Domain key: .qwe.zxc.com.my
c5=.zxc.com.my Domain key: .zxc.com.my
c6=zxc.com.my Domain key: .zxc.com.my
c1=.asd.qwe.zxc.com.my Domain key: .asd.qwe.zxc.com.my
c2=asd.qwe.zxc.com.my Domain key: .asd.qwe.zxc.com.my
Cookies retrieved by GetCookies(http://asd.qwe.zxc.com.my/) Method:
c3=.qwe.zxc.com.my
c4=qwe.zxc.com.my
c5=.zxc.com.my
c6=zxc.com.my
c7=.com.my
c8=com.my
Testing Add(Cookie) method (excluding uri):
CookieContainer Domain keys:
qwe.zxc.com.my Cookies Count: 1
.zxc.com.my Cookies Count: 1
.com.my Cookies Count: 1
com.my Cookies Count: 1
zxc.com.my Cookies Count: 1
asd.qwe.zxc.com.my Cookies Count: 1
.asd.qwe.zxc.com.my Cookies Count: 1
.qwe.zxc.com.my Cookies Count: 1
Actual cookies contain in the CookieContainer:
c4=qwe.zxc.com.my Domain key: qwe.zxc.com.my
c5=.zxc.com.my Domain key: .zxc.com.my
c7=.com.my Domain key: .com.my
c8=com.my Domain key: com.my
c6=zxc.com.my Domain key: zxc.com.my
c2=asd.qwe.zxc.com.my Domain key: asd.qwe.zxc.com.my
c1=.asd.qwe.zxc.com.my Domain key: .asd.qwe.zxc.com.my
c3=.qwe.zxc.com.my Domain key: .qwe.zxc.com.my
Cookies retrieved by GetCookies(http://asd.qwe.zxc.com.my/) Method:
c2=asd.qwe.zxc.com.my
c3=.qwe.zxc.com.my
c5=.zxc.com.my
c7=.com.my
BugFix:
CookieContainer Domain keys:
qwe.zxc.com.my Cookies Count: 2
.zxc.com.my Cookies Count: 1
.com.my Cookies Count: 1
com.my Cookies Count: 2
zxc.com.my Cookies Count: 2
asd.qwe.zxc.com.my Cookies Count: 2
.asd.qwe.zxc.com.my Cookies Count: 1
.qwe.zxc.com.my Cookies Count: 1
Your test results shows oddly everything is ok! but still I think it
is not.
I still get the same extra cookie, can you send me your Unit test
code?
Did you try with GetCookies("http://xxxxxx.com.my/") ?
It should return only c7, and my tests shows it will return c8 too.
Test GetCookies("http://yyyy.my/") too?
And about TLD, I don't think we should care about it. It is on site
designer to take care of its cookies health and where they go. Why a
webmaster should generate a cookie which won't work!? And because
ASProxy is a proxy between user and a website, extra effort is not
needed to control the back-end site behavior, in my opinion of course.
Maybe implementing that will be needed in order to make it a reliable
for other services than a simple proxy, in future; maybe!
In the result shown the c4, c6 and c8 is not returned, I think that
what you expected.
Sorry I forgot to tell you that the Unit Test is already updated. The
bugfix I just add few lines of codes.
I will send the test in few hours later, since I out now.
Hope that was you expected.
On Oct 11, 11:38 pm, SalarSoft <salarsoftwa...@gmail.com> wrote:
> Your test results shows oddly everything is ok! but still I think it
> is not.
> I still get the same extra cookie, can you send me your Unit test
> code?
> Did you try with GetCookies("http://xxxxxx.com.my/") ?
> It should return only c7, and my tests shows it will return c8 too.
> Test GetCookies("http://yyyy.my/") too?
> And about TLD, I don't think we should care about it. It is on site
> designer to take care of its cookies health and where they go. Why a
> webmaster should generate a cookie which won't work!? And because
> ASProxy is a proxy between user and a website, extra effort is not
> needed to control the back-end site behavior, in my opinion of course.
> Maybe implementing that will be needed in order to make it a reliable
> for other services than a simple proxy, in future; maybe!
Uhha, there we go, the problem is with Add(uri, cookie) method!! without
using it and simply using the Add(cookie) method the returned cookies are
all expected!
Look at the unexpected result is _AddTest(true)
I don't have visual studio test edition so I've changed your test class to
an console application:
watch the results in console!
includeUri=false is what we expect.
On Wed, Oct 14, 2009 at 5:12 PM, --:| LaNN |:-- <nalz...@gmail.com> wrote:
Using Add(cookie) will never modify domain key as with Add(uri,
cookie) will append the dot.
Thats why Add(cookie) doing well by persisting the domain name as in
their cookies.
The BugFix modified to only copy cookies in dot domain key into non-
dot domain key.
This is the trick to solve the GetCookies() function issue.
This BugFix copy cookies algorithm I used was follow the Add()
function (I disassemble the function).
Ok, just change Add(uri, cookie) to Add(cookie) and use the new
BugFix.
Then you can test the CookieManager whether it function properly.
Yeah I did that in the original code and just making some code clean
up. Wait until I release the changes to SVN.
It is time to work on javascript cookie encoder/decoders!
Another problem :(
The BugFix_AddDotCookieDomain does not work with Mono! The problem is
because there is no "m_domainTable" and others.. !!
I don't know if we have the same issue with Mono or not!
On Oct 15, 3:51 pm, CallMeLaNN <nalz...@gmail.com> wrote:
> Using Add(cookie) will never modify domain key as with Add(uri,
> cookie) will append the dot.
> Thats why Add(cookie) doing well by persisting the domain name as in
> their cookies.
> The BugFix modified to only copy cookies in dot domain key into non-
> dot domain key.
> This is the trick to solve the GetCookies() function issue.
> This BugFix copy cookies algorithm I used was follow the Add()
> function (I disassemble the function).
> Ok, just change Add(uri, cookie) to Add(cookie) and use the new
> BugFix.
> Then you can test the CookieManager whether it function properly.
Mono has the same issue, oh my god, they've copied the bugs too! here is the result of tests:
this is without bugfix, cause it doesn't run in Mono. Cookies retrieved by GetCookies(http://333.222.111.org/) Method: c7=.111.org c5=.222.111.org c4=333.222.111.org
the c3 is missing.
The good point is the results are same for both Add(uri, cookie) and Add(cookie)
Ok,
I have updated the CookieManager class with new bugfixes. But still
there is an issue remaining. The SetCookies method.
I've used SetCookies because asproxy tactic requires it.
http://asproxy.svn.sourceforge.net/viewvc/asproxy/SalarSoft.ASProxy/B... We have to solutions, first is to hack SetCookie , second use another
tactic from the beginning.
On Oct 15, 3:51 pm, CallMeLaNN <nalz...@gmail.com> wrote:
From the source, it use SetCookies only once. SetCookies used to
accept cookie header (cookie string).
Simply don't use SetCookie because it will use the internal Add() that
Add(Uri, Cookie) use.
However it doesn't mean to do from begining.
We can parse the cookie header by extract each key value pair, create
a new cookie assign properties into it and Add(Cookie).
One more thing, I see the ToServerString in CallCookieToServerString
that is generate cookie header (inverse of above), I think not good
enough. Because last time I debug, it only take the expires value. The
other like path, domain, port, httponly, etc not take into account. So
better we take all property and generate the key value pair by our
own. Then it will be parsed later like above.
I will try send you the simplified code of the function on the
CookieManager I have modify last time.
Replying into your previous post,
I am not sure about Mono,
but wondering if it have reflection that can hack
and can disassemble like in Microsoft System... assembly.
I just see the RestoreCookiesFromResponse() already changed.
Harder for me to read in the SVN.
How can I download overall project from the latest code?
On Fri, Oct 16, 2009 at 10:34 PM, SalarSoft <salarsoftwa...@gmail.com> wrote:
> Ok, i'm waiting for your codes, you can just attach to email,
Very good, i'll apply your changes to the original code, but is this working? I mean did you test it with all situations. I'm not sure about changing Domain property of Cookie, because it modifies internal m_domainKey and m_domain_implicit variables which may cause some unexpected behaviors.
I didnt test all solution, I just copy and paste from my working CookieManager. It same with yours until you change to compatible with Mono and I found RestoreCookiesFromResponse() modified too many. I think it more easier for me to check in and out for you to review and retest.
I will try to download the latest svn and see in Visual Studio for better understanding and debugging.
On Sat, Oct 17, 2009 at 1:01 AM, Salar <salarsoftwa...@gmail.com> wrote: > Very good, i'll apply your changes to the original code, but is this > working? I mean did you test it with all situations. > I'm not sure about changing Domain property of Cookie, because it modifies > internal m_domainKey and m_domain_implicit variables which may cause some > unexpected behaviors.
|
