access moodle using wget

[nitish chandra]

How can I access moodle using wget? I tried this but it gives an error "ERROR 403: Forbidden". Even using wget with 'robots=off' gives the same error. Doesn't moodle allow wget to download pages? Is there any workaround for this?

[Manish Goregaokar]

You're not logged in. Login is based on session cookies, which don't copy over from your browser to wget. 

You can login to the page using `wget` with the `--save--cookies`, `--keep-session-cookies`, and `--post-data` options to get past the login page, and then `--load-cookies` in your next calls.

Or use curl with the -c option (http://stackoverflow.com/questions/15319770/bash-curl-dont-close-connection-in-between-requests-when-run-sequentially).

Or use a python library like `requests` or `mechanize`.

-Manish

-Manish Goregaokar

On Thu, Feb 27, 2014 at 9:35 PM, nitish chandra <nitishcha...@gmail.com> wrote:

How can I access moodle using wget? I tried this but it gives an error "ERROR 403: Forbidden". Even using wget with 'robots=off' gives the same error. Doesn't moodle allow wget to download pages? Is there any workaround for this?

--
--
The website for the club is http://wncc-iitb.org/
To post to this group, send email to wncc...@googlegroups.com
---
You received this message because you are subscribed to the Google Groups "Web and Coding Club IIT Bombay" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wncc_iitb+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[nitish chandra]

That's what I have tried. I used all the three attributes `--save-cookies`, `--keep-session-cookies`, `--post-data` with the URL http://moodle.iitb.ac.in/login/index.php. This gives 403:FORBIDDEN error.

[Saket Choudhary]

On 28 February 2014 14:07, nitish chandra <nitishcha...@gmail.com> wrote:
> That's what I have tried. I used all the three attributes `--save-cookies`,
> `--keep-session-cookies`, `--post-data` with the URL
> http://moodle.iitb.ac.in/login/index.php. This gives 403:FORBIDDEN error.
>
>

$ export http_proxy=''

before you run wget

I would personally not use wget to achieve what you want to. A lot of
people in insti have hacked around Moodle, so you should be able to
locate the relevant code. I believe there was an ITSP project
revolving around this too.
For e.g see [1]

[1] https://github.com/saketkc/iitb-library-sms-interface/blob/master/moodle.py

[Dilawar Singh]

Try this with wget

--user-agent="Mozilla/5.0 (Windows NT 5.2; rv:2.0.1) Gecko/20100101 Firefox/4.0.1"

Targeted site will think that you are using browser. Sometimes this is needed
for many site are configured to respond to browsers only. For iitb-moodle, it is
needed even if you use python-mechanize. [See here how to use machanize with
moodle](https://github.com/dilawar/Scripts/blob/master/moodle.py)

best,
Dilawar

[Saket Choudhary]

On 1 March 2014 01:14, Dilawar Singh <dilawar....@gmail.com> wrote:
>


Dilawar,

Did you recently change your scripts which you use to post here?

I seem to be getting your emails as a noname.txt attachments.

[Dilawar Singh]

[nitish chandra]

This worked with another option `-robots=off`. Thanks

[nitish chandra]

I am doing this just out of interest to use wget to access these pages. That's it.

[Manish Goregaokar]

There is also a no_proxy environment variable that wget accepts, it works with *.iitb.ac.in, IIRC

-Manish Goregaokar