Help: How can I parse this properties file?
yuanyun.ken
Recently I need read in a proerties file,
its format is key=value, and it uses \ to escape.
for example:
expression means: key value
a=b=c a b=c
a\=b=c a=b c
a\\=b=c a\ b=c
a\\\=b=c a\=b c
How can I parse this file?
I think it should use regex.
but my knowledage in Regular expression is poor.
any help is greatly appreciated.
Jürgen Exner
>Recently I need read in a proerties file,
>its format is key=value, and it uses \ to escape.
>for example:
>expression means: key value
>a=b=c a b=c
>a\=b=c a=b c
>a\\=b=c a\ b=c
>a\\\=b=c a\=b c
>
>How can I parse this file?
>I think it should use regex.
>but my knowledage in Regular expression is poor.
No need for complex REs. Just have the tokenizer walk through the file
character by character and when it finds a backslash then immeditately
read another character and return that literal character as the next
character in the token, no matter if it is a normal character, another
backslash, or an equal sign.
jue
Tad J McClellan
> Hi, dear all perl users:
> Recently I need read in a proerties file,
> its format is key=value, and it uses \ to escape.
> for example:
> expression means: key value
> a=b=c a b=c
> a\=b=c a=b c
> a\\=b=c a\ b=c
> a\\\=b=c a\=b c
>
> How can I parse this file?
----------------------------
#!/usr/bin/perl
use warnings;
use strict;
foreach my $line ( <DATA> ) {
chomp $line;
$line =~ s/\\\\/&backslash;/g; # translate literal backslashes
my($key, $value) = split /(?<!\\)=/, $line, 2; # use negative look-behind
$key =~ tr/\\//d; # eliminate backslashes used for escaping
$key =~ s/&backslash;/\\/g; # put the literal backslashes back in
printf "%-10s %-10s\n", $key, $value;
}
__DATA__
a=b=c
a\=b=c
a\\=b=c
a\\\=b=c
----------------------------
--
Tad McClellan
email: perl -le "print scalar reverse qq/moc.noitatibaher\100cmdat/"
Ted Zlatanov
TJM> #!/usr/bin/perl
TJM> use warnings;
TJM> use strict;
TJM> foreach my $line ( <DATA> ) {
TJM> chomp $line;
TJM> $line =~ s/\\\\/&backslash;/g; # translate literal backslashes
TJM> my($key, $value) = split /(?<!\\)=/, $line, 2; # use negative look-behind
TJM> $key =~ tr/\\//d; # eliminate backslashes used for escaping
TJM> $key =~ s/&backslash;/\\/g; # put the literal backslashes back in
TJM> printf "%-10s %-10s\n", $key, $value;
TJM> }
TJM> __DATA__
TJM> a=b=c
TJM> a\=b=c
TJM> a\\=b=c
TJM> a\\\=b=c
TJM> ----------------------------
I was thinking of a similar solution, but adding 256 (or some other
large number) to each escaped character (in case there's a '&backslash;'
in the data). As long as it's valid Unicode and the original data
doesn't contain Unicode characters it should be a clean translation.
Ted
Michele Dondi
wrote:
>TJM> $line =~ s/\\\\/&backslash;/g; # translate literal backslashes
>
>TJM> my($key, $value) = split /(?<!\\)=/, $line, 2; # use negative look-behind
>
>TJM> $key =~ tr/\\//d; # eliminate backslashes used for escaping
>
>TJM> $key =~ s/&backslash;/\\/g; # put the literal backslashes back in
>
>TJM> printf "%-10s %-10s\n", $key, $value;
>TJM> }
>
>TJM> __DATA__
>TJM> a=b=c
>TJM> a\=b=c
>TJM> a\\=b=c
>TJM> a\\\=b=c
>TJM> ----------------------------
>
>I was thinking of a similar solution, but adding 256 (or some other
>large number) to each escaped character (in case there's a '&backslash;'
>in the data). As long as it's valid Unicode and the original data
>doesn't contain Unicode characters it should be a clean translation.
I like to be sure thus instead of adding "some other large number" I
actually *find* something that *can't* be there:
my $delim = "&". (sort @delims = $line =~ /&(\0+);/)[-1] . "\0;";
$line =~ s/\\\\/$delim;/g; # translate literal backslashes
# ...
Michele
--
{$_=pack'B8'x25,unpack'A8'x32,$a^=sub{pop^pop}->(map substr
(($a||=join'',map--$|x$_,(unpack'w',unpack'u','G^<R<Y]*YB='
.'KYU;*EVH[.FHF2W+#"\Z*5TI/ER<Z`S(G.DZZ9OX0Z')=~/./g)x2,$_,
256),7,249);s/[^\w,]/ /g;$ \=/^J/?$/:"\r";print,redo}#JAPH,
Michele Dondi
<bik....@tiscalinet.it> wrote:
>I like to be sure thus instead of adding "some other large number" I
>actually *find* something that *can't* be there:
>
> my $delim = "&". (sort @delims = $line =~ /&(\0+);/)[-1] . "\0;";
> $line =~ s/\\\\/$delim;/g; # translate literal backslashes
Sorry! That's what you get out of posting such in a hurry; I meant:
my $delim = "&". (sort "", $line =~ /&(\0+);/g)[-1] . "\0;";
cartercc
> I was thinking of a similar solution, but adding 256 (or some other
> large number) to each escaped character (in case there's a '&backslash;'
> in the data). As long as it's valid Unicode and the original data
> doesn't contain Unicode characters it should be a clean translation.
I find absolutely nothing wrong with Tad's solution. The fact that it
^might^ be a little more verbose than necessary I regard as a mark in
its favor, not a mark against.
I might consider the string '\' rather than '&backslash;' but
that's a simple quibble.
Now, what about a one liner?
CC
s...@netherlands.com
Well, according to your template there, the valid
equal sign separating Key from Value is the first
non-escaped equal sign.
So yes there is a regular expression to do that.
You have data in the escaped form. It can be split up
into key and value using those rules in a regexp, then
unescape the key/val pair.
Below is probably no different than the other suggestions
in terms of how the split occurs using a regex.
Where does this sequence take you? Do you expect the value
side to be escaped?
These appear possible as well, is this something that will be
encountered?
a\\\\=b=c a\\ b=c
a\\\=b\\=c a\=b\ c
a\\=b\\=c a\ b\=c
Like the other possiblities, here is one more. Its hard to see how
you would get a simple one-step solution though. Maybe..
Good luck.
sln
# ** Original
# expression means: key value
# a=b=c a b=c
# a\=b=c a=b c
# a\\=b=c a\ b=c
# a\\\=b=c a\=b c
# ** Output
# a=b=c a b=c
# a\=b=c a=b c
# a\\=b=c a\ b=c
# a\\\=b=c a\=b c
# a\\\\=b=c a\\ b=c
# a\\\=b\\=c a\=b\ c
# a\\=b\\=c a\ b\=c
use strict;
use warnings;
my @property = ();
foreach my $line ( <DATA> ) {
chomp $line;
push @property, $line if (length $line);
}
print "\nexpression means:\tkey\tvalue\n";
for (@property)
{
if (/^((?:(?:\\.)*?|.*?)+)=(.*)$/)
{
# unescape built in sequences
my ($key, $val) = ($1,$2);
$key =~ s/\\(.)/$1/g;
$val =~ s/\\(.)/$1/g;
printf "%-20s\t%s\t%s\n", $_, $key, $val;
}
}
__DATA__
a=b=c
a\=b=c
a\\=b=c
a\\\=b=c
a\\\\=b=c
a\\\=b\\=c
a\\=b\\=c
s...@netherlands.com
[snip]
You could minimalize it as well.
foreach ( <DATA> ) {
chomp;
Ted Zlatanov
MD> On Wed, 05 Nov 2008 11:28:24 -0600, Ted Zlatanov <t...@lifelogs.com>
MD> wrote:
TJM> $line =~ s/\\\\/&backslash;/g; # translate literal backslashes
>>
TJM> my($key, $value) = split /(?<!\\)=/, $line, 2; # use negative look-behind
>>
TJM> $key =~ tr/\\//d; # eliminate backslashes used for escaping
>>
TJM> $key =~ s/&backslash;/\\/g; # put the literal backslashes back in
>>
TJM> printf "%-10s %-10s\n", $key, $value;
TJM> }
>>
TJM> __DATA__
TJM> a=b=c
TJM> a\=b=c
TJM> a\\=b=c
TJM> a\\\=b=c
TJM> ----------------------------
>>
>> I was thinking of a similar solution, but adding 256 (or some other
>> large number) to each escaped character (in case there's a '&backslash;'
>> in the data). As long as it's valid Unicode and the original data
>> doesn't contain Unicode characters it should be a clean translation.
MD> I like to be sure thus instead of adding "some other large number" I
MD> actually *find* something that *can't* be there:
MD> my $delim = "&". (sort @delims = $line =~ /&(\0+);/)[-1] . "\0;";
MD> $line =~ s/\\\\/$delim;/g; # translate literal backslashes
MD> # ...
Surely there's a CPAN module to do this... Or 10...
From the docs of Encode::Escape, there's also String::Escape,
Unicode::Escape, TeX::Encode, HTML::Mason::Escape,
Template::Plugin::XML::Escape, URI::Escape.
Ted
yuanyun.ken
but sorry for my poor understanding on regex, and having to trouble
you again,
here I have another little problem:
if the content ends with a real single backslash, I need read in the
next line.
How to use regex to do this?
for example:
line ends with match
\ yes
\\ no
\\\ yes
\\\\ no
Thanks for any help again.
Tad J McClellan
-------------------
#!/usr/bin/perl
use warnings;
use strict;
foreach my $line ( <DATA> ) {
chomp $line;
if ( $line =~ /(\\+)$/ and length($1) % 2 )
{ print "yes\n" }
else
{ print "no\n" }
}
__DATA__
\
\\
\\\
\\\\
Jürgen Exner
>Thanks for all the reply. and this problem has been solved.
>but sorry for my poor understanding on regex, and having to trouble
>you again,
>here I have another little problem:
>if the content ends with a real single backslash, I need read in the
>next line.
>
>How to use regex to do this?
You don't. When the tokenizer discovers a backslash it just reads
another character and if that character is an EOL then just continue
processing the next line instead of reporting an end-of-token.
jue
s...@netherlands.com
I assume this pertains to the rules set out on the properties
in the original problem statement.
Tad's solution to check then end for 'odd' number of '\' works best
for a line continuation.
Be very cautious!! If you are trying to find a way to fix random
line splits when this file was generated, there is absolutely
NO solution available to you at all !!!
The reason is you already have escaping rules in place
The line split must be intelligently constucted in that only
an odd number of '\' at the end will determine line continuation.
And at the same time be used in the general escaping rules after
it is joined.
You can't just add a '\' where you would like to split the line then
remove it later without counting the existing escapes at the end.
Either way it takes intelligence to construct the file given the
existing escaping rules you laid out for yourself.
Notice the places where the split occurs in DATA below..
Even if you had an intelligent generator that splits the
line on a '\', it could still split on an even boundry.
Or say it adds a complement to make the split odd, still,
even then, the original can not be guaranteed to reassemble
because this conflicts with the original escape logic..
There is no solution then!
sln
use strict;
use warnings;
# ** Original
# expression means: key value
# a=b=c a b=c
# a\=b=c a=b c
# a\\=b=c a\ b=c
# a\\\=b=c a\=b c
# ** Output
# a=b=c a b=c
# a\=b=c a=b c
# a\\=b=c a\ b=c
# a\\\=b=c a\=b c
# a\\\\=b=c a\\ b=c
# a\\\=b\\=c a\=b\ c
# a\\=b\\=c a\ b\=c
# a=b=c a b=c
# a\=b=c a=b c
# a\\=b=c a\ b=c
# a\\\=b=c a\=b c
# a\\\\=b=c a\\ b=c
# a\\\=b\\=c a\=b\ c
# a\\=b\\=c a\ b\=c
my $buf = '';
print "\nexpression means:\tkey\tvalue\n";
foreach ( <DATA> ) {
chomp;
$_ = $buf . $_;
if ( /(\\+)$/ and length($1) % 2 ) {
# wouldn't want to do this -> s/\\$//;
$buf .= $_; # cat this line to buffer
next; # read next line
}
if (/^((?:(?:\\.)*?|.*?)+)=(.*)$/) {
# unescape built in sequences
my ($key, $val) = ($1,$2);
$key =~ s/\\(.)/$1/g;
$val =~ s/\\(.)/$1/g;
printf "%-20s\t%s\t%s\n", $_, $key, $val;
}
$buf = '';
}
__DATA__
# no line splits
a=b=c
a\=b=c
a\\=b=c
a\\\=b=c
a\\\\=b=c
a\\\=b\\=c
a\\=b\\=c
# ok line splits
a=b=c
a\
=b=c
a\
\=b=c
a\\\
=b=c
a\\\
\=b=c
a\\\=b\
\=c
a\\=b\
\=c
#some good/bad line splits
s...@netherlands.com
# =b=c b=c
# a\\\=b=c a\=b c
# =b=c b=c
# a\\\=b\\=c a\=b\ c
# a\\=b\\ a\ b\
# =c
Dr.Ruud
> if ( $line =~ /(\\+)$/ and length($1) % 2 )
> { print "yes\n" }
> else
> { print "no\n" }
> }
/(?<!\\)(?:\\\\)*\\$/
--
Affijn, Ruud
"Gewoon is een tijger."
Tad J McClellan
> Tad J McClellan schreef:
>
>> if ( $line =~ /(\\+)$/ and length($1) % 2 )
>> { print "yes\n" }
>> else
>> { print "no\n" }
>> }
>
> /(?<!\\)(?:\\\\)*\\$/
Which one do you want to figure out after not having
seen this program for six months?
Dr.Ruud
> Dr.Ruud:
>> Tad J McClellan:
>>> if ( $line =~ /(\\+)$/ and length($1) % 2 )
>>> { print "yes\n" }
>>> else
>>> { print "no\n" }
>>> }
>>
>> /(?<!\\)(?:\\\\)*\\$/
>
> Which one do you want to figure out after not having
> seen this program for six months?
Probably this one:
# even number of trailing slashes
print /(?<!\\)(?:\\\\)*$/ ? "no" : "yes";
s...@netherlands.com
^^^^^^^^^^^
$buf = $_; # asign to buffer
# see what happens whey you don't test
Michele Dondi
wrote:
>TJM> $line =~ s/\\\\/&backslash;/g; # translate literal backslashes
[cut]
>TJM> __DATA__
>TJM> a=b=c
>TJM> a\=b=c
>TJM> a\\=b=c
>TJM> a\\\=b=c
>TJM> ----------------------------
>>>
>>> I was thinking of a similar solution, but adding 256 (or some other
>>> large number) to each escaped character (in case there's a '&backslash;'
>>> in the data). As long as it's valid Unicode and the original data
>>> doesn't contain Unicode characters it should be a clean translation.
>
>MD> I like to be sure thus instead of adding "some other large number" I
>MD> actually *find* something that *can't* be there:
>
>MD> my $delim = "&". (sort @delims = $line =~ /&(\0+);/)[-1] . "\0;";
>MD> $line =~ s/\\\\/$delim;/g; # translate literal backslashes
>MD> # ...
>
>Surely there's a CPAN module to do this... Or 10...
>
>From the docs of Encode::Escape, there's also String::Escape,
>Unicode::Escape, TeX::Encode, HTML::Mason::Escape,
>Template::Plugin::XML::Escape, URI::Escape.
There could be even more: each of them would make maximum sense in the
context it would be actually designed for, non of which seems to be
the case for the OP's requirement. That TJM chose (reasonably!) a
"markup" resemblink that appropriate for one of those *::Escape
thingies is a whole another matter. More precisely, it could get
ridiculously improbable, but if those modules don't "find something
that's surely NOT there" then one could actually concoct up an example
data file that would make them fail on it. And I was illustrating the
general concept and technique of "finding something that's surely NOT
there" which I (i) still support now (ii) FOR ONCE don't necessarily
advocate the use of an existing module for, given that it can be
expressed in an excessively big number of ways with a simple statement
too. (Here I chose sort() because the overhead is minimum, but it may
well have been reduction, and so on...)
David Combs
Tad J McClellan <ta...@seesig.invalid> wrote:
>> Hi, dear all perl users:
>> Recently I need read in a proerties file,
>> its format is key=value, and it uses \ to escape.
>> for example:
>> expression means: key value
>> a=b=c a b=c
>> a\=b=c a=b c
>> a\\=b=c a\ b=c
>> a\\\=b=c a\=b c
>>
>> How can I parse this file?
>
>
>----------------------------
>#!/usr/bin/perl
>use warnings;
>use strict;
>
This is a nice example to add to an e.g. PP4, pod, etc. But first
maybe annotate it a bit more:
>
>foreach my $line ( <DATA> ) {
> chomp $line;
> $line =~ s/\\\\/&backslash;/g; # translate literal backslashes
# explain how that converts SINGLE blackslash-chars.
# That is, please explain how the parsing of perl works,
# so the learner can see WHEN it will internally
# convert "//" to "/"
That's converting SINGLE "\"' backslashes to "&backslash"s
> my($key, $value) = split /(?<!\\)=/, $line, 2; # use negative look-behind
Maybe add WHY you use the neg look-behind.
And just what kind of changes will that make to the file?
> $key =~ tr/\\//d; # eliminate backslashes used for escaping
>
> $key =~ s/&backslash;/\\/g; # put the literal backslashes back in
>
> printf "%-10s %-10s\n", $key, $value;
>}
>
>__DATA__
>a=b=c
>a\=b=c
>a\\=b=c
>a\\\=b=c
>----------------------------
>
>
>--
>Tad McClellan
>email: perl -le "print scalar reverse qq/moc.noitatibaher\100cmdat/"
Thanks!
David
David Combs
yuanyun.ken <yuany...@gmail.com> wrote:
>Thanks for all the reply. and this problem has been solved.
>but sorry for my poor understanding on regex, and having to trouble
You want to learn about perl's regexps? You gotta buy a book,
THIS book:
"Mastering Regular Expressions", 2nd edition,
by Jeffrey Friedl, O'Reilly. (num pages: 460)
Just about everyone here will agree that this one book
is the unix|linux|perl "Regex BIBLE", and that no other
book even comes close!
Try it -- you'll like it.
David
Tim Greer
> "Mastering Regular Expressions", 2nd edition,
> by Jeffrey Friedl, O'Reilly. (num pages: 460)
Probably, the third edition is a better suggestion? (542 pages, and
updated)
--
Tim Greer, CEO/Founder/CTO, BurlyHost.com, Inc.
Shared Hosting, Reseller Hosting, Dedicated & Semi-Dedicated servers
and Custom Hosting. 24/7 support, 30 day guarantee, secure servers.
Industry's most experienced staff! -- Web Hosting With Muscle!
Tad J McClellan
> In article <slrngh3kfe...@tadmc30.sbcglobal.net>,
> Tad J McClellan <ta...@seesig.invalid> wrote:
>>yuanyun.ken <yuany...@gmail.com> wrote:
>>> Hi, dear all perl users:
>>> Recently I need read in a proerties file,
>>> its format is key=value, and it uses \ to escape.
>>> for example:
>>> expression means: key value
>>> a=b=c a b=c
>>> a\=b=c a=b c
>>> a\\=b=c a\ b=c
>>> a\\\=b=c a\=b c
>>>
>>> How can I parse this file?
>>
>>
>>----------------------------
>>#!/usr/bin/perl
>>use warnings;
>>use strict;
>>
>
> This is a nice example to add to an e.g. PP4, pod, etc.
What is PP4?
> But first
> maybe annotate it a bit more:
>
>>
>>foreach my $line ( <DATA> ) {
>> chomp $line;
>
>> $line =~ s/\\\\/&backslash;/g; # translate literal backslashes
>
> # explain how that converts SINGLE blackslash-chars.
That is impossible to explain...
... because that does NOT convert single backslach characters!
> That's converting SINGLE "\"' backslashes to "&backslash"s
No it isn't.
It is converting DOUBLE backslashes to "&backslash;"s
>> my($key, $value) = split /(?<!\\)=/, $line, 2; # use negative look-behind
>
> Maybe add WHY you use the neg look-behind.
Because at this point any backslashes that remain in $line are "meta".
If an equal sign is preceded by a backslash, then this is not the
equal sign that we want to split on (because it is escaped).
> And just what kind of changes will that make to the file?
None.
My code does not change ANY file...
>> $key =~ tr/\\//d; # eliminate backslashes used for escaping
>>
>> $key =~ s/&backslash;/\\/g; # put the literal backslashes back in
>>
>> printf "%-10s %-10s\n", $key, $value;
>>}
>>
>>__DATA__
>>a=b=c
>>a\=b=c
>>a\\=b=c
>>a\\\=b=c
>>----------------------------
>>
>>
>>--
>>Tad McClellan
>>email: perl -le "print scalar reverse qq/moc.noitatibaher\100cmdat/"
It is poor manners to quote .sigs. Please don't do that.
David Combs
Tim Greer <t...@burlyhost.com> wrote:
>David Combs wrote:
>
>> "Mastering Regular Expressions", 2nd edition,
>> by Jeffrey Friedl, O'Reilly. (num pages: 460)
>
>Probably, the third edition is a better suggestion? (542 pages, and
>updated)
Thank you! I had no idea there was one.
I'll get it -- but briefly, roughly, what did he add in
those 80 new pages?
Thanks!
David
Tim Greer
I actually have no idea. The 3rd might not be better than the 2nd
edition, but the newer versions of books like this are almost always
better (usually also contain errata/fixes and more up to date
examples).