How to get a site from Internet Archive

[Kotitihaere]

Hi,

I am trying to get a backup of my crashed site from the internet
archive
( http://replay.waybackmachine.org/20100727170119/http://www.maori.org.nz/papa_panui/)

I use the above URL but it only downloads the first page and is not
following any of the links.

I tried to run the .wget -wr ia as a command but get invalid --execute
command

I have everything set at default - I have not changed any of the
advanced settings - will do that when I can get it to download move
that just the first page!

What do I have to do to get it to follow the links and download from
there?

Any help will be most appreciated - thank you for your time :D

[Khomsan Ph.]

Hi Kotitihaere,

You can use recursive and level option in advanced options.
Just go to advanced options.
And tick on recursive and maybe also level and set number of levels.

There are other option like convert-links,
you can see all available options on wget manual.
http://www.gnu.org/software/wget/manual/wget.html#Recursive-Retrieval-Options

Best Regards,
Khomsan

On Mar 22, 8:44 am, Kotitihaere <tataih...@maori.org.nz> wrote:
> Hi,
>
> I am trying to get a backup of my crashed site from the internet
> archive

> (http://replay.waybackmachine.org/20100727170119/http://www.maori.org....)

[Khomsan Ph.]

You may also need to untick or uncheck the resuming option.
The resuming option is "continue" and "time stamping".

On Mar 22, 2:41 pm, "Khomsan Ph." <khomsan...@gmail.com> wrote:
> Hi Kotitihaere,
>
> You can use recursive and level option in advanced options.
> Just go to advanced options.
> And tick on recursive and maybe also level and set number of levels.
>
> There are other option like convert-links,

> you can see all available options on wget manual.http://www.gnu.org/software/wget/manual/wget.html#Recursive-Retrieval...

[Kotitihaere]

thank you both for your suggestions

In advanced I have nothing set in
Basic Startup
Logging and Input file
download
https
ftp

In http I have director-prefix set for where to download the files

In Recursive Retrieval I have recursive ticked and level number set to
10

In Recursive Accept/Reject I have a reject list and no parent

it is still only downloading the first page and the robots.txt

If I untick In Recursive Retrieval the recursive I only get the
index.html

I am thinking I need to add in the -wr ia command on the command line
then put in the URL without the waybackmachine part - can someone tell
me how to use the --execute=command syntax


Any help is much appreciated :D

[Khomsan Ph.]

Hello Kotitihaere,

I have try downloading from the url that you provide.
It seem that the robot.txt is blocking wget from crawling.
Because wget respect robot.txt, but good news, it's configurable.
You can turn off robot support of wget by using the execute command.

1. In basic startup options, there is execute command
2. Tick on execute command, and put in the following... robots=off

Best Regards,
Khomsan

[Kotitihaere]

thank you for that - I have tried it and still not had any luck *deep
sigh*

[Khomsan Ph.]

Umm... it's weird, because i already trying it and it works.
I just remove --continue and --timestamping.
And add --recursive and --execute robots=off

[Kotitihaere]

Oh you rock! :D as soon as I set the recursive it worked :D

Thank you SOOO much for all your help - it is soo appreciated :D

[Khomsan Ph.]

No problems.

[daniel king]

late reply (three years i know) but I used http://waybackdownloads.com they downloaded the full site with correct linking. and it only took them about 2 hours