Help with curl statement?
DaveC
This is the format I've tried so far:
user$ curl -a --retry 100 [http URL]
When the remote server doesn't respond, Terminal simply quits with that
message. Apparently retry doesn't work in this context.
How do I get curl to retry when the remote server doesn't respond?
Thanks,
Dave
Tom Stiller
<0001HW.C700D907...@news.eternal-september.org>,
DaveC <inv...@invalid.net> wrote:
Is the http URL of the form 'http://apple.com' ("'" included)?
--
Tom Stiller
PGP fingerprint = 5108 DDB2 9761 EDE5 E7E3 7BDA 71ED 6496 99C0 C7CF
DaveC
No, the url is used without single (or any) quote marks.
Terminal didn't error immediately with a syntax error which leads me to
believe that it's OK without quotes. Only after several minutes did Terminal
return an error after timing out.
Thanks.
Tom Stiller
<0001HW.C7010F52...@news.eternal-september.org>,
DaveC <inv...@invalid.net> wrote:
Well, curl works fine for me with the URL enclosed in "'" characters,
but maybe your system is different.
David Empson
> Someone else wrote:
>> DaveC <inv...@invalid.net> wrote:
>> > Trying to download a file from a http server using curl.
>> >
>> > This is the format I've tried so far:
>> >
>> > user$ curl -a --retry 100 [http URL]
>> >
>> > When the remote server doesn't respond, Terminal simply quits with
>> > that message. Apparently retry doesn't work in this context.
>> >
>> > How do I get curl to retry when the remote server doesn't respond?
> >
>
> No, the url is used without single (or any) quote marks.
If the URL contained any special characters as far as the shell is
concerned, it wouldn't work without quoting. The obvious example is a
question mark, which is interpreted by the shell as a filename globbing
character (single wildcard), but there are a few others which would also
cause problems.
Simple URLs just containing letters, digits, period, colon and slash
won't cause problems.
> Terminal didn't error immediately with a syntax error which leads me to
> believe that it's OK without quotes.
Try doing an echo with the same URL and see if it prints the URL without
modification.
> Only after several minutes did Terminal return an error after timing out.
That may be expected behaviour if the server is repeatedly returning an
error (timeout, or HTTP 5xx error response). If you specfiy --retry 100
then curl will retry after 1, 2, 4, 8, 16, 32, 64, 128, 256, 512
seconds, and then every ten minutes until there are no retries left.
You might also be running into a DNS problem - that could produce
different symptoms from an HTTP server problem.
What does CURL display if you try it without the --retry option?
What do you see if you try to 'dig' the domain name portion of the URL?
I'm not sure what you are trying to achieve with the -a option: it
specifies append mode when uploading.
--
David Empson
dem...@actrix.gen.nz
Barry Margolin
> Trying to download a file from a http server using curl.
>
> This is the format I've tried so far:
>
> user$ curl -a --retry 100 [http URL]
>
> When the remote server doesn't respond, Terminal simply quits with that
> message. Apparently retry doesn't work in this context.
The man page says it retries after transient errors. I guess a down
server isn't considered a transient error.
>
> How do I get curl to retry when the remote server doesn't respond?
This will retry on any error:
until curl -a <URL>
sleep 1
done
--
Barry Margolin, bar...@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
*** PLEASE don't copy me on replies, I'll read them in the group ***
Calum
> Well, curl works fine for me with the URL enclosed in "'" characters,
> but maybe your system is different.
The OP didn't say curl wasn't working, though, just that the --retry
option didn't appear to do anything.
Calum
> The man page says it retries after transient errors. I guess a down
> server isn't considered a transient error.
The man page specifies what constitutes a transient error: "either: a
timeout, an FTP 5xx response code or an HTTP 5xx response code."
Have to say I'd have thought a down server would generate a timeout, but
maybe a timeout is only caused by a stalled transfer, rather than one
that couldn't be started.
Barry Margolin
Calum <com....@scottishwildcat.nospam> wrote:
Yeah, it's a little surprising to me, too. But I guess when it gets an
error during connect() it doesn't actually check the error code, to see
if it was ETIMEDOUT versus ECONNREFUSED. It probably only makes the
distinction once it successfully connects, and then has to do the HTTP
protocol machinery itself.