Handling "new line" and "space" characters in bash
Maxim Vexler
To sum up the tips:
By Oded Arbel:
a. Use a subshell to avoid mistakenly over riding your shell variables.
b. Use "$(echo)" as portable(?) newline variable scripting style.
By Ehud Karni:
a. Pipeing into bash subshell can be accepted inside the shell with read.
b. using a "while read VAR1 VAR2 VAR3..." is a convenient method to
accepting stdin data.
c. awk has system() !!
By Amos Shapira:
a. General work around is to construct the whole command as text, then
use either piping to sh or bash buildin "expr".
By Valery Reznic:
a. set -- "space delimited word list" can be used as a quick method
for assigning value to number variables ($1..$9). [question: Really?
this does not seem to work for me].
b. bash while loop can get stdin from file IO redirection.
Following is the original post order, as seen on linux-il
---------------------
Maxim Vexler <hq4...@gmail.com> Tue, Nov 14, 2006 at 11:56 AM
To: IGLU Mailing list <linu...@iglu.org.il>
Hi list, any bash gurus in the house ?
I'm having the most annoying issue with bash, one related to space
delimited variables.
I'd like to get a list in the form of :
<<<
user1 password1
user2 password2
>>>
Instead I'm getting:
<<<
user1
password1
user2
password2
>>>
Here's an example:
san-svn:/var/lib/svn# cat passwd.fake
[users]
user1 = password1
user2 = password2
san-svn:/var/lib/svn#
I'd like to automate this the import from this file into something like
san-svn:# htpasswd -b passwd.real user1 password1
For this I've tried this voodoo:
san-svn:/var/lib/svn# for pair in `awk '/^[^[].+[^\n]$/ {print $1,
$3}' passwd.fake`; do htpasswd -b passwd.true "$pair"; done
This does not work for the following reason:
san-svn:/var/lib/svn# for pair in `awk '/^[^[].+[^\n]$/ {print $1,
$3}' passwd.fake`; do echo "$pair"; done
user1
password1
user2
password2
I've tried the following workarounds, that didn't worked:
san-svn:/var/lib/svn# IFS='\n' for pair in `awk '/^[^[].+[^\n]$/
{print $1, $3}' passwd`; do echo $pair; done
-bash: syntax error near unexpected token `do'
san-svn:/var/lib/svn# IFS='!\n' for pair in `awk '/^[^[].+[^\n]$/
{print $1, $3}' passwd`;! do echo $pair;! done
-bash: syntax error near unexpected token `do'
san-svn:/var/lib/svn# for pair in `awk '/^[^[].+[^\n]$/ {print $1,
$3}' passwd.fake`; do xargs "$pair" | echo; done
san-svn:/var/lib/svn# for pair in `awk '/^[^[].+[^\n]$/ {print $1,
$3}' passwd.fake`; do xargs "$pair" | echo -; done
-
I did found the following work around :
<<<
san-svn:/var/lib/svn# for pair in `awk '/^[^[].+[^\n]$/ {print
$1"_"$3}' passwd.fake`; do echo "$pair" | tr _ ' ' | cat; done
user1 password1
user2 password2
>>>
But it's broken because "_" can be a valid character in a password /
usernmae name and besides - I'd to find a smarter solution.
Any help / pokes to right direction would be highly appreciated.
Thank you,
Maxim.
-------------
Oded Arbel <oded-...@typo.co.il> Tue, Nov 14, 2006 at 12:30 PM
To: Maxim Vexler <hq4...@gmail.com>
Cc: IGLU Mailing list <linu...@iglu.org.il>
On Tue, 2006-11-14 at 12:05 +0200, Maxim Vexler wrote:
> On 11/14/06, Maxim Vexler <hq4...@gmail.com> wrote:
> san-svn:/var/lib/svn# for pair in `awk '/^[^[].+[^\n]$/ {print $1,$3}'
> passwd.fake`; do echo "$pair" | xargs echo ; done
> user1
> password1
> user2
> password2
I think you are approaching this the wrong way, and you should use $IFS
(bash record separator characters) for this purpose. Compare this:
for pair in `awk '/^[^[].+[^\n]$/ {print $1,$3}' passwd.fake`; do
echo "$pair"; done
versus
(IFS="$(echo)"; \
for pair in `awk '/^[^[].+[^\n]$/ {print $1,$3}' passwd.fake`; do
echo "$pair"; done)
In the second example, I force the record separator to be only the new
line character (the output from 'echo'. I can probably use \n, but I
wanted to play it safe). Do mind the wrapping of the second form in
parenthesis, otherwise you clobber your global IFS, which is something
you want to avoid.
--
Oded
-------------
Ehud Karni <eh...@unix.mvs.co.il> Tue, Nov 14, 2006 at 12:34 PM
Reply-To: eh...@unix.mvs.co.il
To: hq4...@gmail.com
Cc: linu...@iglu.org.il
On Tue, 14 Nov 2006 11:56:18 Maxim Vexler wrote:
>
> I'm having the most annoying issue with bash, one related to space
> delimited variables.
[snip]
>
> Here's an example:
>
> san-svn:/var/lib/svn# cat passwd.fake
> [users]
> user1 = password1
> user2 = password2
>
[snip]
>
> I'd like to automate this the import from this file into something like
> san-svn:# htpasswd -b passwd.real user1 password1
Your using the awk/bash/xarg combination in a wrong way.
You can do in bash ALONE very simply like this:
PWF="your-fake-pass-file"
cat $PWF | (
while read USR EQ PAS REST
do
if [ "$EQ" = "=" ] ; then
## echo "$USR $PAS" # just to get pairs
htpasswd -b passwd.real $USR $PAS # what you really want
fi
done )
Or, you can do by awk ALONE like this:
PWF="your-fake-pass-file"
awk '/^[^[].+[^\n]$/ { system( \
"htpasswd -b passwd.real " $1 " " $3 ) }' $PWF
Ehud.
--
Ehud Karni Tel: +972-3-7966-561 /"\
Mivtach - Simon Fax: +972-3-7966-667 \ / ASCII Ribbon Campaign
Insurance agencies (USA) voice mail and X Against HTML Mail
http://www.mvs.co.il FAX: 1-815-5509341 / \
GnuPG: 98EA398D <http://www.keyserver.net/> Better Safe Than Sorry
--------------
Amos Shapira <amos.s...@gmail.com> Tue, Nov 14, 2006 at 12:29 PM
To: IGLU Mailing list <linu...@iglu.org.il>
On 14/11/06, Maxim Vexler <hq4...@gmail.com> wrote:
Hi list, any bash gurus in the house ?
I'm having the most annoying issue with bash, one related to space
delimited variables.
I'd like to get a list in the form of :
<<<
user1 password1
user2 password2
>>>
Instead I'm getting:
<<<
user1
password1
user2
password2
>>>
Here's an example:
san-svn:/var/lib/svn# cat passwd.fake
[users]
user1 = password1
user2 = password2
san-svn:/var/lib/svn#
I'd like to automate this the import from this file into something like
san-svn:# htpasswd -b passwd.real user1 password1
Two ways I could think of almost stright away:
1.
# awk '/^[^[].+[^\n]$/ {print "htpasswd -b passwd.real", $1, $3 }'
passwd.fake | sh
2.
# awk '/^[^[].+[^\n]$/ {print $1, $3 }' passwd.fake | xargs -n 2
htpasswd -b passwd.real
Maybe it's cheating but it should do the job... :)
Thanks for the quiz,
--Amos
-----------------
Valery Reznic <valery...@yahoo.com> Tue, Nov 14, 2006 at 1:21 PM
To: Maxim Vexler <hq4...@gmail.com>
What about something like following:
while read line; do
case "x$line" in
x)
# empty line, do nothing
;;
x[ | x])
# you don't like brackets, do nothing too
;;
*)
# Everything else
set -- $line
# Now $1 == user, $3 == passwd (if any)
# do whatever you like with them
;;
esac
done < passwd.fake
Valery
---------------------
Oron Peled <or...@actcom.co.il> Tue, Nov 14, 2006 at 11:29 PM
To: Maxim Vexler <hq4...@gmail.com>
Cc: IGLU Mailing list <linu...@iglu.org.il>
On Tuesday, 14 בNovember 2006 22:31, Maxim Vexler wrote:
> Thanks to everyone for the help, all solution worked.
> To sum up the tips:
Hey, what's the rush? I didn't have my take yet ;-)
Let's do it in simple one liner:
sed -e N -e 's/\n/ = /' passwd.fake
Cheers,
--
Oron Peled Voice/Fax: +972-4-8228492
or...@actcom.co.il http://www.actcom.co.il/~oron
ICQ UIN: 16527398
"Beware of bugs in the above code;
I have only proved it correct, not tried it."
-- Donald E. Knuth
----------------
--
Cheers,
Maxim Vexler
"Free as in Freedom" - Do u GNU ?