Login via php code on an external page

Andrea

unread,

Mar 1, 2007, 8:23:32 AM3/1/07

to

Hi there!
I have to do a php script that daily surf a webpage (I do it with
crontab) and collect some data.
The proble is that to access that webpage I have to do a login. I have
the login data, but do it via browser....ok, I'm not stupid, but do
that via php script...well I'm not able to do it.
The script is running on the server side, so it's useless to do the
login on my browser prior to run the script.
The page where I have to login has this code in it:

<tr>
<td colspan="2" class="login_box">Username
<input name="logon" type="text" class="login_form"
size="15" maxlength="50">
</td>
</tr>
<tr>
<td valign="bottom" class="login_box">Password
<input name="password" class="login_form"
type="password" size="15" maxlength="20" onKeyPress="return
submitenter(this,event)"/>
</td>
<td valign="bottom">

<input name="Invia" type="image" src="/images/
xxxxxxxxx/newhome/login_box_frecce.gif" width="26" height="17">
</td>
</tr>
<tr>
<td colspan="2">
<a href="http://www.xxxxxxxx.xxx/sasa/site/ffds/?
epi_menuItemID=a2dc1cab4db4465fb70c3072e02041ca&beanID=98976176&viewID=richiestaPass&epi-
content=GENERIC" class="link_login_box">password dimenticata?</a>
<a href="http://www.xxxxxxxx.xxx/sasa/site/ffds/?
epi_menuItemID=61c946e879c833d632294560c01041ca"><img src="/images/
xxxxxxxxx/newhome/login_box_registrazione_it.gif" width="129"
height="19" border="0" /></a>
</td>

</tr>
</table>
</td>
<td width="7"><img src="/images/xxxxxxxxx/newhome/
login_box_dx.gif" width="7" height="128" /></td>
</tr>
</table>
</form>

How can I do it???
Thanks,
Andrea

Jerry Stuckle

unread,

Mar 1, 2007, 9:04:38 AM3/1/07

to

Check out CURL.

And have you asked the owner if it's OK to do this? Many websites don't
like people to scrape their pages without permission.

--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstu...@attglobal.net
==================

Andrea

unread,

Mar 1, 2007, 11:03:55 AM3/1/07

to

> Check out CURL.

I've checked out CURL...but without any success.
How do I use it to pass the login info???

>
> And have you asked the owner if it's OK to do this? Many websites don't
> like people to scrape their pages without permission.

Actually it's my website, I'm scanning it to collect and check some
data from another webserver (for monitoring).

Andrea

unread,

Mar 1, 2007, 11:56:00 AM3/1/07

to

I'm doing some test with this script:

$url = "http://xxx.yyyyy.com/test2.php";
$vars = "utente=ut&pass=pw";

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookie.txt');
curl_setopt($ch, CURLOPT_COOKIEFILE, 'cookie.txt');
// if ($method == 'POST') {
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $vars);
// }
$data = curl_exec($ch);
curl_close($ch);

echo $data;

and I'm getting this error:

HTTP/1.1 403 Forbidden Date: Thu, 01 Mar 2007 16:31:54 GMT Server:
Apache Content-Length: 286 Connection: close Content-Type: text/html;
charset=iso-8859-1
Forbidden

You don't have permission to access /test2.php on this server.

The page test2.php has a simple echo for the 2 vars:

echo $_POST['utente'];
echo "<BR>";
echo $_POST['pass'];

What's wrong???

Jerry Stuckle

unread,

Mar 1, 2007, 2:24:32 PM3/1/07

to

Offhand I would say you have the file requiring authorization in your
httpd.conf and/or .htaccess files.

Can you access the page through your browser?

Andrea

unread,

Mar 2, 2007, 6:20:30 AM3/2/07

to

> Offhand I would say you have the file requiring authorization in your
> httpd.conf and/or .htaccess files.

No, There's no authorization required and

>
> Can you access the page through your browser?

yes I can access that page through the browser with no problem.... :(

NC

unread,

Mar 2, 2007, 11:01:16 AM3/2/07

to

On Mar 1, 5:23 am, "Andrea" <andr...@tin.it> wrote:
>
> I have to do a php script that daily surf a webpage (I do it with
> crontab) and collect some data.
> The proble is that to access that webpage I have to do a login.

...

> How can I do it???

Before you do anything else, you need to understand what HTTP headers
are and how they work. Basically, here's what usually happens during
form-based authentication:

1. The authentication script receives a POST request including, among
other things, a login name and a password. It checks the login
name
and password against a database and, if check is successful, sends
to the client one or more cookies signifying the success.
2. The client receives the cookies and starts requesting protected
pages, including those cookies with every request.

You can emulate both of these actions either with cURL or by using
fsockopen(). First, you need to access the authentication page and
get the cookies, then, you can request the actual data by sending
out a GET request including those cookies. Be aware that sometimes
the authentication is a two-step process; the authentication script
checks credentials and, if they are OK, redirects the client to
another script that sets the cookies.

If you have Firefox, you can actually see all the readers being
exchanged between client and server using Live HTTP Headers plugin:

http://livehttpheaders.mozdev.org/

Cheers,
NC

Jerry Stuckle

unread,

Mar 2, 2007, 4:03:10 PM3/2/07

to

Well, the 403 comes from one of two things - either you're trying to
access something you're not allowed to access (i.e. a directory listing
when it is not allowed), or you must be authenticated to access the page.

Not knowing the exact page you're trying to access makes it impossible
to do some testing from this end.

If you're running Firefox (which I highly recommend for web
development), get the Live HTTP Headers extension. Look in it to see if
you have something like:

HTTP/1.x 401 Authorization Required

If you do, the page is requiring authorization.

I'm really wondering if this is the case because of the code you have:

And remember - once you've signed onto the site, your browser will send
your userid/password any time it asks for it (assuming HTTP
authentication, of course).