curl 'https://i.jpillora.com/chisel!' vs open it in the browser.

[hongy...@gmail.com]

Chisel's website describes the following installation method [1]:

See the latest release or download and install it now with curl https://i.jpillora.com/chisel! | bash

[1] https://github.com/jpillora/chisel#binaries

But I tried to provide the above URL to curl and open it in Firefox. To my surprise, it gave different results:

Testing with curl:

##########################
$ curl https://i.jpillora.com/chisel
#!/bin/bash
TMP_DIR="/tmp/tmpinstalldir"
function cleanup {
echo rm -rf $TMP_DIR > /dev/null
}
function fail {
cleanup
msg=$1
echo "============"
echo "Error: $msg" 1>&2
exit 1
}
function install {
#settings
USER="jpillora"
PROG="chisel"
MOVE="false"
RELEASE="v1.7.6"
INSECURE="false"
OUT_DIR="$(pwd)"
GH="https://github.com"
#bash check
[ ! "$BASH_VERSION" ] && fail "Please use bash instead"
[ ! -d $OUT_DIR ] && fail "output directory missing: $OUT_DIR"
#dependency check, assume we are a standard POISX machine
which find > /dev/null || fail "find not installed"
which xargs > /dev/null || fail "xargs not installed"
which sort > /dev/null || fail "sort not installed"
which tail > /dev/null || fail "tail not installed"
which cut > /dev/null || fail "cut not installed"
which du > /dev/null || fail "du not installed"
GET=""
if which curl > /dev/null; then
GET="curl"
if [[ $INSECURE = "true" ]]; then GET="$GET --insecure"; fi
GET="$GET --fail -# -L"
elif which wget > /dev/null; then
GET="wget"
if [[ $INSECURE = "true" ]]; then GET="$GET --no-check-certificate"; fi
GET="$GET -qO-"
else
fail "neither wget/curl are installed"
fi
#find OS #TODO BSDs and other posixs
case `uname -s` in
Darwin) OS="darwin";;
Linux) OS="linux";;
*) fail "unknown os: $(uname -s)";;
esac
#find ARCH
if uname -m | grep 64 > /dev/null; then
ARCH="amd64"
elif uname -m | grep arm > /dev/null; then
ARCH="arm" #TODO armv6/v7
elif uname -m | grep 386 > /dev/null; then
ARCH="386"
else
fail "unknown arch: $(uname -m)"
fi
#choose from asset list
URL=""
FTYPE=""
case "${OS}_${ARCH}" in
"darwin_amd64")
URL="https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_darwin_amd64.gz"
FTYPE=".gz"
;;
"darwin_arm")
URL="https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_darwin_arm64.gz"
FTYPE=".gz"
;;
"linux_386")
URL="https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_linux_386.gz"
FTYPE=".gz"
;;
"linux_amd64")
URL="https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_linux_amd64.gz"
FTYPE=".gz"
;;
"linux_arm")
URL="https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_linux_arm64.gz"
FTYPE=".gz"
;;
*) fail "No asset for platform ${OS}-${ARCH}";;
esac
#got URL! download it...
echo -n "Downloading $USER/$PROG $RELEASE"

echo "....."

#enter tempdir
mkdir -p $TMP_DIR
cd $TMP_DIR
if [[ $FTYPE = ".gz" ]]; then
which gzip > /dev/null || fail "gzip is not installed"
#gzipped binary
NAME="${PROG}_${OS}_${ARCH}.gz"
GZURL="$GH/releases/download/$RELEASE/$NAME"
#gz download!
bash -c "$GET $URL" | gzip -d - > $PROG || fail "download failed"
elif [[ $FTYPE = ".tar.gz" ]] || [[ $FTYPE = ".tgz" ]]; then
#check if archiver progs installed
which tar > /dev/null || fail "tar is not installed"
which gzip > /dev/null || fail "gzip is not installed"
bash -c "$GET $URL" | tar zxf - || fail "download failed"
elif [[ $FTYPE = ".zip" ]]; then
which unzip > /dev/null || fail "unzip is not installed"
bash -c "$GET $URL" > tmp.zip || fail "download failed"
unzip -o -qq tmp.zip || fail "unzip failed"
rm tmp.zip || fail "cleanup failed"
elif [[ $FTYPE = "" ]]; then
bash -c "$GET $URL" > "chisel_${OS}_${ARCH}" || fail "download failed"
else
fail "unknown file type: $FTYPE"
fi
#search subtree largest file (bin)
TMP_BIN=$(find . -type f | xargs du | sort -n | tail -n 1 | cut -f 2)
if [ ! -f "$TMP_BIN" ]; then
fail "could not find find binary (largest file)"
fi
#ensure its larger than 1MB
if [[ $(du -m $TMP_BIN | cut -f1) -lt 1 ]]; then
fail "no binary found ($TMP_BIN is not larger than 1MB)"
fi
#move into PATH or cwd
chmod +x $TMP_BIN || fail "chmod +x failed"

mv $TMP_BIN $OUT_DIR/$PROG || fail "mv failed" #FINAL STEP!
echo "Downloaded to $OUT_DIR/$PROG"
#done
cleanup
}
install
##########################

Open it in Firefox:

########################
repository: https://github.com/jpillora/chisel
user: jpillora
program: chisel
release: v1.7.6
release assets:
[#01] https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_darwin_amd64.gz
[#02] https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_darwin_arm64.gz
[#03] https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_linux_386.gz
[#04] https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_linux_amd64.gz
[#05] https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_linux_arm64.gz
move-into-path: true

to see shell script, visit:
i.jpillora.com/chisel!?type=script

for more information on this server, visit:
github.com/jpillora/installer
########################

OTOH, I also tried with the following url, `https://i.jpillora.com/chisel', i.e., with the last `!' character stripped, and it still gives the same results.

Any hints for the above phenomenon?

Regards,
HZ

[Russell Marks]

"hongy...@gmail.com" <hongy...@gmail.com> wrote:

[longer lines wrapped]

> Chisel's website describes the following installation method [1]:
>
> See the latest release or download and install it now with
> curl https://i.jpillora.com/chisel! | bash
>
> [1] https://github.com/jpillora/chisel#binaries
>
> But I tried to provide the above URL to curl and open it in Firefox.
> To my surprise, it gave different results:
>
> Testing with curl:
>
> ##########################
> $ curl https://i.jpillora.com/chisel
> #!/bin/bash
> TMP_DIR="/tmp/tmpinstalldir"

[...]

> ##########################
>
> Open it in Firefox:
>
> ########################
> repository: https://github.com/jpillora/chisel
> user: jpillora
> program: chisel
> release: v1.7.6

[...]

> ########################
>
> OTOH, I also tried with the following url,
> `https://i.jpillora.com/chisel', i.e., with the last `!' character
> stripped, and it still gives the same results.
>
> Any hints for the above phenomenon?

The differing content is probably based on the User-Agent header,
which the web server will have received before serving the page. A
quick test seems to support this - using wget returns something
similar to the curl version, but "wget -U ''" (which doesn't send a
User-Agent at all) returns the version you got in Firefox. That
version is also returned for "busybox wget" and "lynx -source",
downloading options that presumably weren't accounted for.

-Rus.

[hongy...@gmail.com]

Thank you for pointing this out. The corresponding curl option is "curl -A ''", as shown below:

werner@X10DAi-00:~$ wget -qU '' https://i.jpillora.com/chisel! -O-

repository: https://github.com/jpillora/chisel
user: jpillora
program: chisel
release: v1.7.6

release assets:
[#01] https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_darwin_amd64.gz
[#02] https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_darwin_arm64.gz
[#03] https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_linux_386.gz
[#04] https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_linux_amd64.gz
[#05] https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_linux_arm64.gz
move-into-path: true

to see shell script, visit:
i.jpillora.com/chisel!?type=script

for more information on this server, visit:
github.com/jpillora/installer

werner@X10DAi-00:~$ curl -sA '' https://i.jpillora.com/chisel!

repository: https://github.com/jpillora/chisel
user: jpillora
program: chisel
release: v1.7.6

release assets:
[#01] https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_darwin_amd64.gz
[#02] https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_darwin_arm64.gz
[#03] https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_linux_386.gz
[#04] https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_linux_amd64.gz
[#05] https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_linux_arm64.gz
move-into-path: true

to see shell script, visit:
i.jpillora.com/chisel!?type=script

for more information on this server, visit:
github.com/jpillora/installer

As I've pointed earlier, use the following URLs are equivalent:

https://i.jpillora.com/chisel!
https://i.jpillora.com/chisel