Upper limit on argument length used by xargs.

[Hongyi Zhao]

Hi all,

I run the following command in order to know the upper limit on argument
length when using xargs on my Debian box:

---------------------------
$ xargs --show-limits --no-run-if-empty < /dev/null
Your environment variables take up 4910 bytes
POSIX upper limit on argument length (this system): 2090194
POSIX smallest allowable upper limit on argument length (all systems):
4096
Maximum length of command we could actually use: 2085284
Size of command buffer we are actually using: 131072
---------------------------

Based on the above output, what's the real smallest allowable upper limit
on argument length for my system? I still cann't figure it out.

Furthermore, I do the following testing on my box:

$ ls ./ovpn/* | wc -l
9050
$ ls ./ovpn/* | xargs -P0 -r -s 4196 awk 'BEGINFILE{ a =0 }
/^<[/]key>/ { a ++ }
ENDFILE{ if(a == 1) print FILENAME }' |
xargs -P0 -r stat --printf="%Y %n\n" | grep 'cannot stat'
stat: cannot stat ‘./ovpn/vpngate_122.135.86.1_tcp_995.ovp./ovpn/
vpngate_174.4.184.152_tcp_1788.ovpn_CA’: No such file or directory
stat: cannot stat ‘n_JP’: No such file or directory
stat: cannot stat ‘./ovpn/vpngate_188.55.144.83_tcp_1342.o./ovpn/
vpngate_181.162.135.132_tcp_1368.ovpn_CL’: No such file or directory
[snipped]

As you can see, the ``-s 4196'' will trigger error; while ``-s 4096''
will run smoothly:

$ ls ./ovpn/* | xargs -P0 -r -s 4096 awk 'BEGINFILE{ a =0 }
/^<[/]key>/{ a ++ }
ENDFILE{ if(a == 1) print FILENAME }' |
xargs -P0 -r stat --printf="%Y %n\n" | grep 'cannot stat'
$

Any hints on these issues?

Regards
--
.: Hongyi Zhao [ hongyi.zhao AT gmail.com ] Free as in Freedom :.

[Hongyi Zhao]

On Mon, 05 Dec 2016 14:01:39 +0000, Hongyi Zhao wrote:

> As you can see, the ``-s 4196'' will trigger error; while ``-s 4096''
> will run smoothly:
>
> $ ls ./ovpn/* | xargs -P0 -r -s 4096 awk 'BEGINFILE{ a =0 }
> /^<[/]key>/{ a ++ }
> ENDFILE{ if(a == 1) print FILENAME }' |
> xargs -P0 -r stat --printf="%Y %n\n" | grep 'cannot stat'
> $

I do some further testings, it shows that the smallest allowable upper
limit on argument length is about 3330 bytes on my box. The methods and
steps are as follows:

[1] Using the ``-n1'' of xargs to obtain a standard reference file:

$ ls ./ovpn/* | xargs -P0 -r -n1 awk 'BEGINFILE{ a =0 }

/^<[/]key>/{ a ++ }
ENDFILE{ if(a == 1) print FILENAME }' |

xargs -P0 -r -n1 stat --printf="%Y %n\n" | sort -k1n > 111

[2] Using different values for ``-s'' of xargs and compare the result
with the result obtained by ``-n1'' of xargs:

$ ls ./ovpn/* |
xargs -P0 -r -s 3340 awk 'BEGINFILE{ a =0 }

/^<[/]key>/{ a ++ }
ENDFILE{ if(a == 1) print FILENAME }' |

xargs -P0 -r -s 3340 stat --printf="%Y %n\n" | sort -k1n > 222

$ diff 111 222
0a1,2
> vpn_ID
> 95.ovpn_US
236d237
< 1480727540 ./ovpn/vpngate_5.53.124.169_tcp_995.ovpn_RU
6952d6952
< 1480941107 ./ovpn/vpngate_68.82.90.209_tcp_1509.ovpn_US
8588c8588
< 1480946476 ./ovpn/vpngate_36.72.168.3_tcp_1577.ovpn_ID
---
> 1480946476 ./ovpn/vpngate_36.72.168.3_tcp_1577.o1480727540 ./ovpn/
vpngate_5.53.124.169_tcp_995.ovpn_RU
8993c8993
< 1480946669 ./ovpn/vpngate_68.82.119.70_tcp_995.ovpn_US
---
> 1480946669 ./ovpn/vpngate_68.82.119.70_tcp_91480941107 ./ovpn/
vpngate_68.82.90.209_tcp_1509.ovpn_US


$ ls ./ovpn/* |
xargs -P0 -r -s 3330 awk 'BEGINFILE{ a =0 }

/^<[/]key>/{ a ++ }
ENDFILE{ if(a == 1) print FILENAME }' |

xargs -P0 -r -s 3330 stat --printf="%Y %n\n" | sort -k1n > 222

$ diff 111 222
$

Why does this happen? Any hints?