Hi TCLer's,
I wanted to create a simple web downloader. In general a complex task
but I found a file which cannot be downloaded completely.
$ tclsh test_download.tcl
Loading file outfile1.txt from
http://someonewhocares.org/hosts/hosts
using wget
Loading file outfile2.txt from
http://someonewhocares.org/hosts/hosts
via httpcopy
.......
Date: Fri, 23 Mar 2018 21:23:21 GMT
Server: Apache/2.2.31 (Unix) mod_ssl/2.2.31
OpenSSL/1.0.1e-fips
content-disposition: attachment: filename=hosts
cache-control: public, max-age=86400
Last-Modified: Thu, 22 Mar 2018 08:13:42 GMT
Vary: Accept-Encoding
Content-Encoding: gzip
Connection: close
Transfer-Encoding: chunked
Content-Type: text/plain
HTTP file copy size is 342964, wget filesize is 416015
The used script is the httpcopy example from the 8.6 manpage where I
added a command line option and a wget-call (which must be installed for
the test) to compare the wget result with the tcl downloaded version.
Only the hard-coded url (if you don't provide an url on the command
line) failed until now, all other tests worked.
Does anyone have an idea why this case fails?
Is it a bug or am I doing somenthing wrong?
Another file works as expected:
tclsh test_download.tcl
http://www.tech-edv.co.at/TCMS/downloads/tkfplot-0.35.tgz
Loading file outfile1.txt from
http://www.tech-edv.co.at/TCMS/downloads/tkfplot-0.35.tgz using wget
Loading file outfile2.txt from
http://www.tech-edv.co.at/TCMS/downloads/tkfplot-0.35.tgz via httpcopy
...
Date: Fri, 23 Mar 2018 21:33:41 GMT
Server: Apache
Last-Modified: Tue, 16 Jan 2018 16:14:19 GMT
ETag: "2638-562e705e1bd39"
Accept-Ranges: bytes
Content-Length: 9784
Content-Type: application/x-gzip
Age: 0
Connection: close
HTTP file copy size is 9784, wget filesize is 9784
Here's the complete script: test_download.tcl
package require http
proc httpcopy { url file {chunk 4096} } {
set out [open $file w]
set token [::http::geturl $url -channel $out \
-progress httpCopyProgress -blocksize $chunk]
close $out
# This ends the line started by httpCopyProgress
puts stderr ""
upvar #0 $token state
set max 0
foreach {name value} $state(meta) {
if {[string length $name] > $max} {
set max [string length $name]
}
if {[regexp -nocase ^location$ $name]} {
# Handle URL redirects
puts stderr "Location:$value"
return [httpcopy [string trim $value] $file $chunk]
}
}
incr max
foreach {name value} $state(meta) {
puts [format "%-*s %s" $max $name: $value]
}
return $token
}
proc httpCopyProgress {args} {
puts -nonewline stderr .
flush stderr
}
#
# === Here starts my additional testing code ===
#
if {[llength $argv]} {
set url [lindex $argv 0]
} else {
set url "
http://someonewhocares.org/hosts/hosts"
}
set org "outfile1.txt"
set out "outfile2.txt"
puts "Loading file $org from $url using wget"
catch {exec wget $url -O $org}
puts "Loading file $out from $url via httpcopy"
httpcopy $url $out
puts "HTTP file copy size is [file size $out], wget filesize is [file
size $org]"
--
Gerhard Reithofer - Techn. EDV Reithofer -
http://www.tech-edv.co.at