[PATCH] Dockerfile: Carry oe-git-proxy locally
30 views
Skip to first unread message
Jan Kiszka
Aug 30, 2021, 3:56:44 PM8/30/21
to kas-devel, Lisicki, Raphael (MO MM R&D SYS SEC), Henning Schild
From: Jan Kiszka <jan.k...@siemens.com>
This imports revision aa9b9dc9a94f224aab7c22e666f1ac946d9e7408 from
https://git.yoctoproject.org/git/poky to avoid fetching it - and having
to add the missing content validation to prevent supply-chain attacks.
Reported-by: Raphael Lisicki <raphael...@siemens.com>
Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---
Dockerfile | 3 +-
contrib/oe-git-proxy | 187 +++++++++++++++++++++++++++++++++++++++++++
2 files changed, 188 insertions(+), 2 deletions(-)
create mode 100755 contrib/oe-git-proxy
diff --git a/Dockerfile b/Dockerfile
index e8c8dc7..251e09b 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -26,8 +26,7 @@ RUN apt-get install --no-install-recommends -y \
apt-get clean && \
rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
-RUN wget -nv -O /usr/bin/oe-git-proxy "http://git.yoctoproject.org/cgit/cgit.cgi/poky/plain/scripts/oe-git-proxy" && \
- chmod +x /usr/bin/oe-git-proxy
+COPY contrib/oe-git-proxy /usr/local/bin/
ENV GIT_PROXY_COMMAND="oe-git-proxy" \
NO_PROXY="*"
diff --git a/contrib/oe-git-proxy b/contrib/oe-git-proxy
new file mode 100755
index 0000000..aa9b9dc
--- /dev/null
+++ b/contrib/oe-git-proxy
@@ -0,0 +1,187 @@
+#!/bin/bash
+
+# oe-git-proxy is a simple tool to be via GIT_PROXY_COMMAND. It uses socat
+# to make SOCKS5 or HTTPS proxy connections.
+# It uses ALL_PROXY or all_proxy or http_proxy to determine the proxy server,
+# protocol, and port.
+# It uses NO_PROXY to skip using the proxy for a comma delimited list of
+# hosts, host globs (*.example.com), IPs, or CIDR masks (192.168.1.0/24). It
+# is known to work with both bash and dash shells.
+#
+# Example ALL_PROXY values:
+# ALL_PROXY=socks://socks.example.com:1080
+# ALL_PROXY=https://proxy.example.com:8080
+#
+# Copyright (c) 2013, Intel Corporation.
+#
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# AUTHORS
+# Darren Hart <dvh...@linux.intel.com>
+
+# disable pathname expansion, NO_PROXY fields could start with "*" or be it
+set -f
+
+if [ $# -lt 2 -o "$1" = '--help' -o "$1" = '-h' ] ; then
+ echo 'oe-git-proxy: error: the following arguments are required: host port'
+ echo 'Usage: oe-git-proxy host port'
+ echo ''
+ echo 'OpenEmbedded git-proxy - a simple tool to be used via GIT_PROXY_COMMAND.'
+ echo 'It uses socat to make SOCKS or HTTPS proxy connections.'
+ echo 'It uses ALL_PROXY to determine the proxy server, protocol, and port.'
+ echo 'It uses NO_PROXY to skip using the proxy for a comma delimited list'
+ echo 'of hosts, host globs (*.example.com), IPs, or CIDR masks (192.168.1.0/24).'
+ echo 'It is known to work with both bash and dash shells.runs native tools'
+ echo ''
+ echo 'arguments:'
+ echo ' host proxy host to use'
+ echo ' port proxy port to use'
+ echo ''
+ echo 'options:'
+ echo ' -h, --help show this help message and exit'
+ echo ''
+ exit 2
+fi
+
+# Locate the netcat binary
+if [ -z "$SOCAT" ]; then
+ SOCAT=$(which socat 2>/dev/null)
+ if [ $? -ne 0 ]; then
+ echo "ERROR: socat binary not in PATH" 1>&2
+ exit 1
+ fi
+fi
+METHOD=""
+
+# Test for a valid IPV4 quad with optional bitmask
+valid_ipv4() {
+ echo $1 | egrep -q "^([1-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])(\.([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])){3}(/(3[0-2]|[1-2]?[0-9]))?$"
+ return $?
+}
+
+# Convert an IPV4 address into a 32bit integer
+ipv4_val() {
+ IP="$1"
+ SHIFT=24
+ VAL=0
+ for B in $( echo "$IP" | tr '.' ' ' ); do
+ VAL=$(($VAL+$(($B<<$SHIFT))))
+ SHIFT=$(($SHIFT-8))
+ done
+ echo "$VAL"
+}
+
+# Determine if two IPs are equivalent, or if the CIDR contains the IP
+match_ipv4() {
+ CIDR=$1
+ IP=$2
+
+ if [ -z "${IP%%$CIDR}" ]; then
+ return 0
+ fi
+
+ # Determine the mask bitlength
+ BITS=${CIDR##*/}
+ [ "$BITS" != "$CIDR" ] || BITS=32
+ if [ -z "$BITS" ]; then
+ return 1
+ fi
+
+ IPVAL=$(ipv4_val $IP)
+ IP2VAL=$(ipv4_val ${CIDR%%/*})
+
+ # OR in the unmasked bits
+ for i in $(seq 0 $((32-$BITS))); do
+ IP2VAL=$(($IP2VAL|$((1<<$i))))
+ IPVAL=$(($IPVAL|$((1<<$i))))
+ done
+
+ if [ $IPVAL -eq $IP2VAL ]; then
+ return 0
+ fi
+ return 1
+}
+
+# Test to see if GLOB matches HOST
+match_host() {
+ HOST=$1
+ GLOB=$2
+
+ if [ -z "${HOST%%*$GLOB}" ]; then
+ return 0
+ fi
+
+ # Match by netmask
+ if valid_ipv4 $GLOB; then
+ for HOST_IP in $(getent ahostsv4 $HOST | grep ' STREAM ' | cut -d ' ' -f 1) ; do
+ if valid_ipv4 $HOST_IP; then
+ match_ipv4 $GLOB $HOST_IP
+ if [ $? -eq 0 ]; then
+ return 0
+ fi
+ fi
+ done
+ fi
+
+ return 1
+}
+
+# If no proxy is set or needed, just connect directly
+METHOD="TCP:$1:$2"
+
+[ -z "${ALL_PROXY}" ] && ALL_PROXY=$all_proxy
+[ -z "${ALL_PROXY}" ] && ALL_PROXY=$http_proxy
+
+if [ -z "$ALL_PROXY" ]; then
+ exec $SOCAT STDIO $METHOD
+fi
+
+# Connect directly to hosts in NO_PROXY
+for H in $( echo "$NO_PROXY" | tr ',' ' ' ); do
+ if match_host $1 $H; then
+ exec $SOCAT STDIO $METHOD
+ fi
+done
+
+# Proxy is necessary, determine protocol, server, and port
+# extract protocol
+PROTO=${ALL_PROXY%://*}
+# strip protocol:// from string
+ALL_PROXY=${ALL_PROXY#*://}
+# extract host & port parts:
+# 1) drop username/password
+PROXY=${ALL_PROXY##*@}
+# 2) remove optional trailing /?
+PROXY=${PROXY%%/*}
+# 3) extract optional port
+PORT=${PROXY##*:}
+if [ "$PORT" = "$PROXY" ]; then
+ PORT=""
+fi
+# 4) remove port
+PROXY=${PROXY%%:*}
+
+# extract username & password
+PROXYAUTH="${ALL_PROXY%@*}"
+[ "$PROXYAUTH" = "$ALL_PROXY" ] && PROXYAUTH=
+[ -n "${PROXYAUTH}" ] && PROXYAUTH=",proxyauth=${PROXYAUTH}"
+
+if [ "$PROTO" = "socks" ] || [ "$PROTO" = "socks4a" ]; then
+ if [ -z "$PORT" ]; then
+ PORT="1080"
+ fi
+ METHOD="SOCKS4A:$PROXY:$1:$2,socksport=$PORT"
+elif [ "$PROTO" = "socks4" ]; then
+ if [ -z "$PORT" ]; then
+ PORT="1080"
+ fi
+ METHOD="SOCKS4:$PROXY:$1:$2,socksport=$PORT"
+else
+ # Assume PROXY (http, https, etc)
+ if [ -z "$PORT" ]; then
+ PORT="8080"
+ fi
+ METHOD="PROXY:$PROXY:$1:$2,proxyport=${PORT}${PROXYAUTH}"
+fi
+
+exec $SOCAT STDIO "$METHOD"
--
2.31.1
This imports revision aa9b9dc9a94f224aab7c22e666f1ac946d9e7408 from
https://git.yoctoproject.org/git/poky to avoid fetching it - and having
to add the missing content validation to prevent supply-chain attacks.
Reported-by: Raphael Lisicki <raphael...@siemens.com>
Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
---
Dockerfile | 3 +-
contrib/oe-git-proxy | 187 +++++++++++++++++++++++++++++++++++++++++++
2 files changed, 188 insertions(+), 2 deletions(-)
create mode 100755 contrib/oe-git-proxy
diff --git a/Dockerfile b/Dockerfile
index e8c8dc7..251e09b 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -26,8 +26,7 @@ RUN apt-get install --no-install-recommends -y \
apt-get clean && \
rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
-RUN wget -nv -O /usr/bin/oe-git-proxy "http://git.yoctoproject.org/cgit/cgit.cgi/poky/plain/scripts/oe-git-proxy" && \
- chmod +x /usr/bin/oe-git-proxy
+COPY contrib/oe-git-proxy /usr/local/bin/
ENV GIT_PROXY_COMMAND="oe-git-proxy" \
NO_PROXY="*"
diff --git a/contrib/oe-git-proxy b/contrib/oe-git-proxy
new file mode 100755
index 0000000..aa9b9dc
--- /dev/null
+++ b/contrib/oe-git-proxy
@@ -0,0 +1,187 @@
+#!/bin/bash
+
+# oe-git-proxy is a simple tool to be via GIT_PROXY_COMMAND. It uses socat
+# to make SOCKS5 or HTTPS proxy connections.
+# It uses ALL_PROXY or all_proxy or http_proxy to determine the proxy server,
+# protocol, and port.
+# It uses NO_PROXY to skip using the proxy for a comma delimited list of
+# hosts, host globs (*.example.com), IPs, or CIDR masks (192.168.1.0/24). It
+# is known to work with both bash and dash shells.
+#
+# Example ALL_PROXY values:
+# ALL_PROXY=socks://socks.example.com:1080
+# ALL_PROXY=https://proxy.example.com:8080
+#
+# Copyright (c) 2013, Intel Corporation.
+#
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# AUTHORS
+# Darren Hart <dvh...@linux.intel.com>
+
+# disable pathname expansion, NO_PROXY fields could start with "*" or be it
+set -f
+
+if [ $# -lt 2 -o "$1" = '--help' -o "$1" = '-h' ] ; then
+ echo 'oe-git-proxy: error: the following arguments are required: host port'
+ echo 'Usage: oe-git-proxy host port'
+ echo ''
+ echo 'OpenEmbedded git-proxy - a simple tool to be used via GIT_PROXY_COMMAND.'
+ echo 'It uses socat to make SOCKS or HTTPS proxy connections.'
+ echo 'It uses ALL_PROXY to determine the proxy server, protocol, and port.'
+ echo 'It uses NO_PROXY to skip using the proxy for a comma delimited list'
+ echo 'of hosts, host globs (*.example.com), IPs, or CIDR masks (192.168.1.0/24).'
+ echo 'It is known to work with both bash and dash shells.runs native tools'
+ echo ''
+ echo 'arguments:'
+ echo ' host proxy host to use'
+ echo ' port proxy port to use'
+ echo ''
+ echo 'options:'
+ echo ' -h, --help show this help message and exit'
+ echo ''
+ exit 2
+fi
+
+# Locate the netcat binary
+if [ -z "$SOCAT" ]; then
+ SOCAT=$(which socat 2>/dev/null)
+ if [ $? -ne 0 ]; then
+ echo "ERROR: socat binary not in PATH" 1>&2
+ exit 1
+ fi
+fi
+METHOD=""
+
+# Test for a valid IPV4 quad with optional bitmask
+valid_ipv4() {
+ echo $1 | egrep -q "^([1-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])(\.([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])){3}(/(3[0-2]|[1-2]?[0-9]))?$"
+ return $?
+}
+
+# Convert an IPV4 address into a 32bit integer
+ipv4_val() {
+ IP="$1"
+ SHIFT=24
+ VAL=0
+ for B in $( echo "$IP" | tr '.' ' ' ); do
+ VAL=$(($VAL+$(($B<<$SHIFT))))
+ SHIFT=$(($SHIFT-8))
+ done
+ echo "$VAL"
+}
+
+# Determine if two IPs are equivalent, or if the CIDR contains the IP
+match_ipv4() {
+ CIDR=$1
+ IP=$2
+
+ if [ -z "${IP%%$CIDR}" ]; then
+ return 0
+ fi
+
+ # Determine the mask bitlength
+ BITS=${CIDR##*/}
+ [ "$BITS" != "$CIDR" ] || BITS=32
+ if [ -z "$BITS" ]; then
+ return 1
+ fi
+
+ IPVAL=$(ipv4_val $IP)
+ IP2VAL=$(ipv4_val ${CIDR%%/*})
+
+ # OR in the unmasked bits
+ for i in $(seq 0 $((32-$BITS))); do
+ IP2VAL=$(($IP2VAL|$((1<<$i))))
+ IPVAL=$(($IPVAL|$((1<<$i))))
+ done
+
+ if [ $IPVAL -eq $IP2VAL ]; then
+ return 0
+ fi
+ return 1
+}
+
+# Test to see if GLOB matches HOST
+match_host() {
+ HOST=$1
+ GLOB=$2
+
+ if [ -z "${HOST%%*$GLOB}" ]; then
+ return 0
+ fi
+
+ # Match by netmask
+ if valid_ipv4 $GLOB; then
+ for HOST_IP in $(getent ahostsv4 $HOST | grep ' STREAM ' | cut -d ' ' -f 1) ; do
+ if valid_ipv4 $HOST_IP; then
+ match_ipv4 $GLOB $HOST_IP
+ if [ $? -eq 0 ]; then
+ return 0
+ fi
+ fi
+ done
+ fi
+
+ return 1
+}
+
+# If no proxy is set or needed, just connect directly
+METHOD="TCP:$1:$2"
+
+[ -z "${ALL_PROXY}" ] && ALL_PROXY=$all_proxy
+[ -z "${ALL_PROXY}" ] && ALL_PROXY=$http_proxy
+
+if [ -z "$ALL_PROXY" ]; then
+ exec $SOCAT STDIO $METHOD
+fi
+
+# Connect directly to hosts in NO_PROXY
+for H in $( echo "$NO_PROXY" | tr ',' ' ' ); do
+ if match_host $1 $H; then
+ exec $SOCAT STDIO $METHOD
+ fi
+done
+
+# Proxy is necessary, determine protocol, server, and port
+# extract protocol
+PROTO=${ALL_PROXY%://*}
+# strip protocol:// from string
+ALL_PROXY=${ALL_PROXY#*://}
+# extract host & port parts:
+# 1) drop username/password
+PROXY=${ALL_PROXY##*@}
+# 2) remove optional trailing /?
+PROXY=${PROXY%%/*}
+# 3) extract optional port
+PORT=${PROXY##*:}
+if [ "$PORT" = "$PROXY" ]; then
+ PORT=""
+fi
+# 4) remove port
+PROXY=${PROXY%%:*}
+
+# extract username & password
+PROXYAUTH="${ALL_PROXY%@*}"
+[ "$PROXYAUTH" = "$ALL_PROXY" ] && PROXYAUTH=
+[ -n "${PROXYAUTH}" ] && PROXYAUTH=",proxyauth=${PROXYAUTH}"
+
+if [ "$PROTO" = "socks" ] || [ "$PROTO" = "socks4a" ]; then
+ if [ -z "$PORT" ]; then
+ PORT="1080"
+ fi
+ METHOD="SOCKS4A:$PROXY:$1:$2,socksport=$PORT"
+elif [ "$PROTO" = "socks4" ]; then
+ if [ -z "$PORT" ]; then
+ PORT="1080"
+ fi
+ METHOD="SOCKS4:$PROXY:$1:$2,socksport=$PORT"
+else
+ # Assume PROXY (http, https, etc)
+ if [ -z "$PORT" ]; then
+ PORT="8080"
+ fi
+ METHOD="PROXY:$PROXY:$1:$2,proxyport=${PORT}${PROXYAUTH}"
+fi
+
+exec $SOCAT STDIO "$METHOD"
--
2.31.1
Henning Schild
Oct 25, 2021, 6:24:29 AM10/25/21
to Jan Kiszka, kas-devel, Lisicki, Raphael (MO MM R&D SYS SEC)
Am Mon, 30 Aug 2021 21:56:42 +0200
schrieb Jan Kiszka <jan.k...@siemens.com>:
> From: Jan Kiszka <jan.k...@siemens.com>
>
> This imports revision aa9b9dc9a94f224aab7c22e666f1ac946d9e7408 from
> https://git.yoctoproject.org/git/poky to avoid fetching it - and
> having to add the missing content validation to prevent supply-chain
> attacks.
>
> Reported-by: Raphael Lisicki <raphael...@siemens.com>
> Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
> ---
> Dockerfile | 3 +-
> contrib/oe-git-proxy | 187
> +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 188
> insertions(+), 2 deletions(-) create mode 100755 contrib/oe-git-proxy
>
> diff --git a/Dockerfile b/Dockerfile
> index e8c8dc7..251e09b 100644
> --- a/Dockerfile
> +++ b/Dockerfile
> @@ -26,8 +26,7 @@ RUN apt-get install --no-install-recommends -y \
> apt-get clean && \
> rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
>
> -RUN wget -nv -O /usr/bin/oe-git-proxy
> "http://git.yoctoproject.org/cgit/cgit.cgi/poky/plain/scripts/oe-git-proxy"
> && \
> - chmod +x /usr/bin/oe-git-proxy
> +COPY contrib/oe-git-proxy /usr/local/bin/
Just updated to kas 2.6.1 in a layer depending on that. The new
location is not in the PATH for some reason.
in ssh-dir i have "config"
ProxyCommand /usr/local/bin/oe-git-proxy %h %p
StrictHostKeyChecking no
UserKnownHostsFile /dev/null
That absolute path is for some reason now needed.
The PATH in the container is fine. I guess the local might get lost
when doing kas or bitbake.
And i found a way to relaxed file mode of 777 on
/usr/local/bin/oe-git-proxy
Still trying to get my head around it, but maybe we should just place
it in /usr/bin/ as we used to.
Henning
schrieb Jan Kiszka <jan.k...@siemens.com>:
> From: Jan Kiszka <jan.k...@siemens.com>
>
> This imports revision aa9b9dc9a94f224aab7c22e666f1ac946d9e7408 from
> https://git.yoctoproject.org/git/poky to avoid fetching it - and
> having to add the missing content validation to prevent supply-chain
> attacks.
>
> Reported-by: Raphael Lisicki <raphael...@siemens.com>
> Signed-off-by: Jan Kiszka <jan.k...@siemens.com>
> ---
> Dockerfile | 3 +-
> contrib/oe-git-proxy | 187
> +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 188
> insertions(+), 2 deletions(-) create mode 100755 contrib/oe-git-proxy
>
> diff --git a/Dockerfile b/Dockerfile
> index e8c8dc7..251e09b 100644
> --- a/Dockerfile
> +++ b/Dockerfile
> @@ -26,8 +26,7 @@ RUN apt-get install --no-install-recommends -y \
> apt-get clean && \
> rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
>
> -RUN wget -nv -O /usr/bin/oe-git-proxy
> "http://git.yoctoproject.org/cgit/cgit.cgi/poky/plain/scripts/oe-git-proxy"
> && \
> - chmod +x /usr/bin/oe-git-proxy
> +COPY contrib/oe-git-proxy /usr/local/bin/
location is not in the PATH for some reason.
in ssh-dir i have "config"
ProxyCommand /usr/local/bin/oe-git-proxy %h %p
StrictHostKeyChecking no
UserKnownHostsFile /dev/null
That absolute path is for some reason now needed.
The PATH in the container is fine. I guess the local might get lost
when doing kas or bitbake.
And i found a way to relaxed file mode of 777 on
/usr/local/bin/oe-git-proxy
Still trying to get my head around it, but maybe we should just place
it in /usr/bin/ as we used to.
Henning
Henning Schild
Oct 25, 2021, 6:27:59 AM10/25/21
to Jan Kiszka, kas-devel, Lisicki, Raphael (MO MM R&D SYS SEC)
Am Mon, 25 Oct 2021 12:24:25 +0200
schrieb Henning Schild <henning...@siemens.com>:
i guess kas
kas/libkas.py
242: env['PATH'] = '/usr/sbin:/usr/bin:/sbin:/bin'
Henning
schrieb Henning Schild <henning...@siemens.com>:
kas/libkas.py
242: env['PATH'] = '/usr/sbin:/usr/bin:/sbin:/bin'
Henning
Jan Kiszka
Oct 25, 2021, 6:44:32 AM10/25/21
to Henning Schild, kas-devel, Lisicki, Raphael (MO MM R&D SYS SEC)
GIT_PROXY_COMMAND in our Dockerfile. Or is your case not covered by that
var?
Jan
Henning Schild
Oct 25, 2021, 6:49:37 AM10/25/21
to Jan Kiszka, kas-devel, Lisicki, Raphael (MO MM R&D SYS SEC)
Am Mon, 25 Oct 2021 12:44:30 +0200
schrieb Jan Kiszka <jan.k...@web.de>:
In my case it was controlled with .ssh/config where i needed to turn
ProxyCommand oe-git-proxy %h %p
into
ProxyCommand /usr/local/bin/oe-git-proxy %h %p
I did send a patch to move the file back to where it was. Even when
updating the env variable to be absolute, other places - like my ssh
config - might break. So the move was an interface breaking change.
Henning
> Jan
>
schrieb Jan Kiszka <jan.k...@web.de>:
ProxyCommand oe-git-proxy %h %p
into
ProxyCommand /usr/local/bin/oe-git-proxy %h %p
I did send a patch to move the file back to where it was. Even when
updating the env variable to be absolute, other places - like my ssh
config - might break. So the move was an interface breaking change.
Henning
> Jan
>
Reply all
Reply to author
Forward
0 new messages