Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

URLREAD and websites that use cookies

6 views
Skip to first unread message

Jeffrey Johnson

unread,
Oct 17, 2004, 5:14:53 PM10/17/04
to

Hello all!

I have been trying to use the URLREAD function to get the contents of a
website that uses cookies for login authentication (specifically,
yahoo.com). For instance, the command

x = urlread('My Yahoo URL','post',{'mylogin';'mypassword'});

works, in that it receives the contents of a page, but the problem is that
the page is not the one intended, but rather a page which tells me I'm not
authorized to view the page I requested. Apparently, posting my login and
password is not sufficient to overcome the 'lack' of a proper cookie.

I don't know much about the whole cookie process, but I was wondering if
anybody had ever concocted a workaround (if it's even possible) for
using URLREAD or another function to get the content of pages that require
cookies to view.

Thanks,
Jeffrey Johnson

--
Replace the rent-a-car with 'avis' to reply

Michael Robbins

unread,
Oct 17, 2004, 8:21:34 PM10/17/04
to

I posted a thread about URLREAD and secure sockets. It includes some
JAVA code that releves that inability.

Jeffrey Johnson

unread,
Oct 18, 2004, 2:10:47 AM10/18/04
to
30 spammers agree that Michael Robbins wrote:

>> I have been trying to use the URLREAD function to get the contents
>> of a website that uses cookies for login authentication (specifically,
>> yahoo.com). For instance, the command

<snip>

>I posted a thread about URLREAD and secure sockets. It includes some
>JAVA code that releves that inability.

Thanks! I think I have found the thread you are referring to. The
instructions in that thread are as follows:

in urlread, just add after line 67
urlConnection = url.openConnection;
>>> urlConnection.setRequestProperty('Authorization','Basic
xxxxxxxxxxxxxxxxxx');

where xxxxxxxxx is a base64 version of the string login:password
I used http://www.opinionatedgeek.com/DotNet/Tools/Base64Encode/
to obtain a base64 version of my login:password

If you want to automatize the process, this string can be extract
from your url using
url = java.net.URL('http://aa:b...@www.c.com/')
url.getUserInfo
then there must be a java function to do the conversion or you can
use Peter Acklam code to do so:
http://home.online.no/~pjacklam/matlab/software/util
/datautil/base64encode.m


Unfortunately, I have now run into a second problem. I have used the code
by Peter Acklam above (although I have verified it gives the same output
as the base64encoding website) by adding the following two lines to
urlread.m after line 67:

base64login = base64encode('mylogin:mypassword');
urlConnection.setRequestProperty('Authorization',['Basic ' base64login]);

Matlab gives me the following error on the second line:

??? Java exception occurred:
java.lang.IllegalArgumentException: Illegal character(s) in message
header value: Basic xxxxxxxxxxxxxxxxx

at
sun.net.www.protocol.http.HttpURLConnection.checkMessageHeader(HttpURLConnection.java:144)
at
sun.net.www.protocol.http.HttpURLConnection.setRequestProperty(HttpURLConnection.java:904).


Basically, it seems to think there is an illegal character in
urlConnection.setRequestProperty. I have tried changing 'Basic ' to
'Basic' (omitting the space) and also 'Basic:' to see if that was the
problem, but it is not. I do not know what form the header value is
expected to have. I don't know precisely where I've gone wrong here.

Any help would be greatly appreciated.

0 new messages