Regex to remove non printable characters
Larry
I'd like to remove all characters with ascii values > 127 from a
string...that's to say i'd like to remove non printable chars...
is the following fine?
my $input =~ s/[^ -~]+//g;
thanks ever so much!
Glenn Jackman
> Hi peeps,
>
> I'd like to remove all characters with ascii values > 127 from a
> string...that's to say i'd like to remove non printable chars...
You might want:
$string =~ s/\P{IsPrint}//g;
See perldoc perlre
--
Glenn Jackman
"You can only be young once. But you can always be immature." -- Dave Barry
Jürgen Exner
> I'd like to remove all characters with ascii values > 127 from a
ASCII is a 7 bit encoding system where sometimes the eights bit is used as
parity bit. There are no ASCII characters > 127, therefore your request
doesn't make sense.
>string...that's to say i'd like to remove non printable chars...
In case you are not talking about ASCII but about e.g Windows-1252 or
ISO-Latin-x or any of the dozen other code pages that share the lower 128
characters with ASCII then please be advised that the vast majority of
those characters > 127 _ARE_ printable, at least in your typical commonly
used code pages.
The non-printable characters can be found in the lower part from 0x00 to
0x1F, no matter if ASCII or Windows-1252 or ISO-Latin-x or many, many
others.
Therefore your request makes even less sense. Maybe you want to clarify
first what you are talking about?
>is the following fine?
>
>my $input =~ s/[^ -~]+//g;
That will remove pretty much all the lower case English letters and a few
special characters. Wonder what they have to do with non-printable or
non-ASCII.
jue
John W. Krahn
>
> I'd like to remove all characters with ascii values > 127 from a
> string
$input =~ s/[^[:ascii:]]+//g;
>...that's to say i'd like to remove non printable chars...
$input =~ s/[^[:print:]]+//g;
> is the following fine?
>
> my $input =~ s/[^ -~]+//g;
my() creates a new variable with no contents so there is nothing for the
substitution operator to remove.
$ perl -wle'my $input =~ s/[^ -~]+//g;'
Use of uninitialized value in substitution (s///) at -e line 1.
John
--
Perl isn't a toolbox, but a small machine shop where you
can special-order certain sorts of tools at low cost and
in short order. -- Larry Wall
Larry
"John W. Krahn" <du...@example.com> wrote:
> $input =~ s/[^[:ascii:]]+//g;
>
>
> >...that's to say i'd like to remove non printable chars...
>
> $input =~ s/[^[:print:]]+//g;
is this fine?
$input =~ tr/\x80-\xFF//d;
Dr.Ruud
> John W. Krahn:
> [remove non printable chars]
> is this fine?
> $input =~ tr/\x80-\xFF//d;
No. How about chr(0x00)..chr(0x1F)?
And characters > "\x{FF}"?
--
Affijn, Ruud
"Gewoon is een tijger."
Jürgen Exner
Depends what you are looking for (you still didn't clarify).
It will remove non-ASCII character in the typical 8-bit encodings.
It will _NOT_ remove non-printable characters.
Maybe you should make up your mind and let us know _which_ of these two
you are actually trying to do.
jue
Petr Vileta
my $input =~ s/[\x00-\x09\x0B\x0C\x0E-\x1F\x80-\xFF]//g;
--
Petr Vileta, Czech republic
(My server rejects all messages from Yahoo and Hotmail. Send me your
mail from another non-spammer site please.)
Please reply to <petr AT practisoft DOT cz>
John W. Krahn
Your subject line says you want a regex. The tr/// operator doesn't use regular expressions.
Jürgen Exner
>Larry <dontme...@got.it> wrote:
>> is this fine?
>>
>> $input =~ tr/\x80-\xFF//d;
>
>Your subject line says you want a regex. The tr/// operator doesn't use regular expressions.
Good point. However, if you are splitting hairs, then let's be accurate:
Regular expressions match a string but they never remove anything as
requested by the OP. Therefore taking literally the OPs question is
non-sensical in the first place.
And he still didn't tell us if he wanted to remove non-ASCII or
non-printable, two very different categories which have no relationship with
each other whatsoever.
jue
Larry
J?rgen Exner <jurg...@hotmail.com> wrote:
> And he still didn't tell us if he wanted to remove non-ASCII or
> non-printable, two very different categories which have no relationship with
> each other whatsoever.
I have yet to understand the differences...in the meanwhile I think I'll
settle for the following:
tr/\x80-\xFF//d;
thanks
Jürgen Exner
>In article <p57tm3dqffe88o8gc...@4ax.com>,
> J?rgen Exner <jurg...@hotmail.com> wrote:
>
>> And he still didn't tell us if he wanted to remove non-ASCII or
>> non-printable, two very different categories which have no relationship with
>> each other whatsoever.
>
>I have yet to understand the differences..
Well, there is no communallity at all. It's two totally different things,
like colour and texture. A specific object can be green and smooth or green
and rough or blue and rough or blue and smooth or whatever combination you
can imagine.
Non-printable characters are characters that don't have a glyph assigned to
them and therefore cannot be printed. Another word for them is control
characters and they include e.g. line feed, carriage return, delete,
backspace, end-of-transmission, header start, etc., etc.
In ASCII and most other modern code pages the non-printable characters are
in the range 0x00 to 0x1F and 0x7F.
Non-ASCII characters on the other hand are characters that are not included
in the 7-bit ASCII encoding at all like e.g. symbols, graphics, and what
some people refer to as 'extended' characters like German umlauts, French
and Spanish accented characters, scandinavian extended characters, but also
Greek, Cyrillic, Arabic,Chinese, ... characters. Basically anything you can
imagine that is not typically used in the English language or that's not on
a US typewriter.
That's not surprising because as the name suggests ASCII is an _AMERICAN_
Standard Code for Information Interchange and Lyndon B. Johnson surely
didn't care about the rest of the world when he mandated its use back in
1968.
For e.g. ISO-Latin-1 those non-ASCII characters would be
Ax NBSP ¡ ¢ £ ¤ ¥ ¦ § ¨ ©
ª « ¬ SHY ® ¯
Bx ° ± ² ³ ´ µ ¶ · ¸ ¹
º » ¼ ½ ¾ ¿
Cx À Á Â Ã Ä Å Æ Ç È É
Ê Ë Ì Í Î Ï
Dx Ð Ñ Ò Ó Ô Õ Ö × Ø Ù
Ú Û Ü Ý Þ ß
Ex à á â ã ä å æ ç è é
ê ë ì í î ï
Fx ð ñ ò ó ô õ ö ÷ ø ù
ú û ü ý þ ÿ
However almost all non-ASCII characters do have a glyph and obviously they
can be printed very well(*), just see the list above.
Or do you really think I would just omit the second letter of my first name
'Jürgen' when printing it?
*1: You could argue if the NBSP and and in particular SHY are printable or
not because they have an additional semantic on top of their (blank resp.
dash) glyphs.
*2: There are exceptions in the code pages for more exotic languages
(Arabic, Thai, Tamil, ...) , where some characters my not have a glyph
assigned but instead they alter the appearence and/or the meaning of
preceeding or following characters.
jue
Larry
Jürgen Exner <jurg...@hotmail.com> wrote:
> Well, there is no communallity at all. It's two totally different things,
> like colour and texture. A specific object can be green and smooth or green
> and rough or blue and rough or blue and smooth or whatever combination you
> can imagine.
ok...to me those are ascii printable chars:
#!/usr/bin/perl
use strict;
use warnings;
for my $k (33 .. 126)
{
print "$k => " . chr($k) . "\n";
}
plus chr(10) and chr(13)
Jürgen Exner
>ok...to me those are ascii printable chars:
>
>#!/usr/bin/perl
>
>use strict;
>use warnings;
>
>for my $k (33 .. 126)
>{
> print "$k => " . chr($k) . "\n";
>}
Agreed, those characters are the intersection of the set of printable
characters and the set of ASCII characters, except that commonly the space
character 0x20 is considered a printable character, too. It just has a blank
glyph.
>plus chr(10) and chr(13)
This however conflicts with customary understanding. From "perldoc perlre"
on POSIX character classes:
print
Any alphanumeric or punctuation (special) character or space.
While on the other hand
cntrl
Any control character. Usually characters that don't produce output
as such but instead control the terminal somehow: for example
newline and backspace are control characters. All characters with
ord() less than 32 are most often classified as control characters
(assuming ASCII, the ISO Latin character sets, and Unicode).
It appears LF and CR are control characters, not printable characters. After
all why should LF be a printable character but its cousin FF not?
jue
Larry
J?rgen Exner <jurg...@hotmail.com> wrote:
> Agreed, those characters are the intersection of the set of printable
> characters and the set of ASCII characters, except that commonly the space
> character 0x20 is considered a printable character, too. It just has a blank
> glyph.
by the way, I'd like to get rid of 0x00 also! The thing is that I'm
coding a _strip bad chars_ sub and I would like to keep only 0x20 0x13
0x10 and those ranging from 0x21 to 0x7E
is that doable?
thanks
Larry
Larry <dontme...@got.it> wrote:
> 0x20 0x13
> 0x10 and those ranging from 0x21 to 0x7E
I'm hopeless at hex values...let's say:
chr(10)
chr(13)
chr(32) to chr(126)
thanks
Larry
Larry <dontme...@got.it> wrote:
> I'm hopeless at hex values...let's say:
>
> chr(10)
> chr(13)
> chr(32) to chr(126)
>
> thanks
well, for the moment I'll go along with keeping those ranging from 0x20
to 0x7E ... so that I don't have to chomp and all...
Jürgen Exner
> J?rgen Exner <jurg...@hotmail.com> wrote:
> The thing is that I'm
>coding a _strip bad chars_ sub and I would like to keep only 0x20 0x13
>0x10 and those ranging from 0x21 to 0x7E
Thank you for calling me a person with a bad char.
*PLONK*
jue
Jürgen Exner
Larry
J?rgen Exner <jurg...@hotmail.com> wrote:
> What a concept!
> I am giving up.
please don't! it's xmas time after all...
i need this to get values (commands) from CGI->param and need to get rid
of those chars
Larry
"Petr Vileta" <sto...@practisoft.cz> wrote:
> my $input =~ s/[\x00-\x09\x0B\x0C\x0E-\x1F\x80-\xFF]//g;
thank you so much ... btw, what is chr (127) ??
I think I'll make it this way:
$input =~ s/[\x00-\x09\x0B\x0C\x0E-\x1F\x7F-\xFF]//g;
thanks
comp.llang.perl.moderated
> Larry <dontmewit...@got.it> wrote:
> > J?rgen Exner <jurge...@hotmail.com> wrote:
> > The thing is that I'm
> >coding a _strip bad chars_ sub and I would like to keep only 0x20 0x13
> >0x10 and those ranging from 0x21 to 0x7E
>
> Thank you for calling me a person with a bad char.
>
> *PLONK*
>
smiley after that wonderful flash of wit....
Of course, maybe you were sitting in a bad
"char" :)
--
Charles DeRykus