Re: Rejecting Certain Non-ASCII Characters

[Lew Pitcher]

On Friday 19 April 2013 12:16, in comp.lang.php, ILi...@Privacy.bogus
wrote:

>
> I have a problem with people entering a slashed zero vs a standard
> ASCII zero into HTML forms intended to store data in a MySQL database.
>
> Is there a simple way in PHP to restrict input to the ASCII Character
> set, specifically hex 0x20 - 0x7E ?

http://www.w3.org/TR/html401/interact/forms.html#adef-accept-charset

[snip]

--
Lew Pitcher
"In Skills, We Trust"

[Daniel Pitts]

On 4/19/13 9:16 AM, Jim Higgins wrote:
>
> I have a problem with people entering a slashed zero vs a standard
> ASCII zero into HTML forms intended to store data in a MySQL database.
>
> Is there a simple way in PHP to restrict input to the ASCII Character

> set, specifically hex 0x20 - 0x7E ? Or a simple way to detect
> characters outside this range before committing them to the database?
>
> My mind says it should be easy to detect and that I should be able to
> do it myself, but for some reason I'm drawing a huge blank.
>
> Thank you.
>

Are they literally putting "\0", or do you mean it is sending the byte 0
(where "0" is the byte 48)?

If they are putting "\0", then this indicates you are not escaping the
string appropriately. Don't filter it, escape it.

If it is the byte 0, then that is a very unusual thing for them to submit.

[Jerry Stuckle]

On 4/19/2013 12:16 PM, Jim Higgins wrote:
>
> I have a problem with people entering a slashed zero vs a standard
> ASCII zero into HTML forms intended to store data in a MySQL database.
>
> Is there a simple way in PHP to restrict input to the ASCII Character
> set, specifically hex 0x20 - 0x7E ? Or a simple way to detect
> characters outside this range before committing them to the database?
>
> My mind says it should be easy to detect and that I should be able to
> do it myself, but for some reason I'm drawing a huge blank.
>
> Thank you.
>

Short of parsing the string some way, no.

If it's a short string, you could loop through it character by
character. Longer strings might benefit from a regex.

But be aware just because YOU are working with an ASCII charset, not
everyone is.

--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstu...@attglobal.net
==================

[Christoph Becker]

Jim Higgins wrote:
> I have a problem with people entering a slashed zero vs a standard
> ASCII zero into HTML forms intended to store data in a MySQL database.

Is it really a slashed zero (U+0030 U+0338) they're entering, or do they
enter some similar looking character such as the Danish Ø? In the
former case you can simply replace the slashed zero with a standard
zero. Assuming UTF-8 encoding:

$input = str_replace('\xCC\xB8', '', $input);

> Is there a simple way in PHP to restrict input to the ASCII Character
> set, specifically hex 0x20 - 0x7E ? Or a simple way to detect
> characters outside this range before committing them to the database?

If you're dealing with a numeric column, you may consider checking for
is_numeric().

--
Christoph M. Becker

[Arno Welzel]

Jim Higgins, 2013-04-19 18:16:

[...]

> Is there a simple way in PHP to restrict input to the ASCII Character
> set, specifically hex 0x20 - 0x7E ? Or a simple way to detect
> characters outside this range before committing them to the database?

Maybe ctype_print() may help:

<http://www.php.net/manual/en/function.ctype-print.php>



--
Arno Welzel
http://arnowelzel.de
http://de-rec-fahrrad.de

[Thomas 'PointedEars' Lahn]

Christoph Becker wrote:

> Jim Higgins wrote:
>> I have a problem with people entering a slashed zero vs a standard
>> ASCII zero into HTML forms intended to store data in a MySQL database.
>
> Is it really a slashed zero (U+0030 U+0338) they're entering,

Could also be U+0030 U+337 or any other (allowed composition) of the at
least 65536 Unicode characters that look(s) similar. For example, it could
be U+2205 EMPTY SET.

> or do they enter some similar looking character such as the Danish Ø?

That character, U+00D8 LATIN CAPITAL LETTER O WITH STROKE *and* its
lowercase counterpart, U+00F8 LATIN SMALL LETTER O WITH STROKE, are present
at least in Danish, Norwegian, and Faroese; They are also used in the
International Phonetic Alphabet (IPA).

> In the former case you can simply replace the slashed zero with a standard
> zero. Assuming UTF-8 encoding:
>
> $input = str_replace('\xCC\xB8', '', $input);

You do not consider that, ambiguity aside, even in UTF-8 there are *several*
ways to produce Unicode characters. See: Unicode Normalization Forms.

>> Is there a simple way in PHP to restrict input to the ASCII Character
>> set, specifically hex 0x20 - 0x7E ? Or a simple way to detect
>> characters outside this range before committing them to the database?
>
> If you're dealing with a numeric column, you may consider checking for
> is_numeric().

There are also regular expressions. Testing against '/[^\0x00-\x7F]/', and
rejecting anything that matches, appears to be the best approach here. If
characters actually should be converted, iconv() should be used instead of a
hardcoded conversion.

However, it is better in the long term to convert the MySQL database (or the
relevant table and column) to utf8_general_ci, and upgrade MySQL if
necessary (character sets and collations were not supported before MySQL 5;
the current stable version is 5.6).

<http://php.net/pcre>
<http://dev.mysql.com/>


PointedEars
--
Prototype.js was written by people who don't know javascript for people
who don't know javascript. People who don't know javascript are not
the best source of advice on designing systems that use javascript.
-- Richard Cornford, cljs, <f806at$ail$1$8300...@news.demon.co.uk>

[Christoph Becker]

Thomas 'PointedEars' Lahn wrote:
> Christoph Becker wrote:
>
>> Jim Higgins wrote:
>>> I have a problem with people entering a slashed zero vs a standard
>>> ASCII zero into HTML forms intended to store data in a MySQL database.
>>
>> Is it really a slashed zero (U+0030 U+0338) they're entering,
>
> Could also be U+0030 U+337 or any other (allowed composition) of the at
> least 65536 Unicode characters that look(s) similar. For example, it could
> be U+2205 EMPTY SET.

Which I was referring to in a abbreviated form in the rest of the
sentence. :)

>> or do they enter some similar looking character such as the Danish Ø?
>
> That character, U+00D8 LATIN CAPITAL LETTER O WITH STROKE *and* its
> lowercase counterpart, U+00F8 LATIN SMALL LETTER O WITH STROKE, are present
> at least in Danish, Norwegian, and Faroese; They are also used in the
> International Phonetic Alphabet (IPA).
>
>> In the former case you can simply replace the slashed zero with a standard
>> zero. Assuming UTF-8 encoding:
>>
>> $input = str_replace('\xCC\xB8', '', $input);
>
> You do not consider that, ambiguity aside, even in UTF-8 there are *several*
> ways to produce Unicode characters. See: Unicode Normalization Forms.

If I'm not mistaken here, this str_replace() will work for NFD and NFC,
but indeed I had not considered NFKD and NFKC. Thanks for pointing this
out.

>>> Is there a simple way in PHP to restrict input to the ASCII Character
>>> set, specifically hex 0x20 - 0x7E ? Or a simple way to detect
>>> characters outside this range before committing them to the database?
>>
>> If you're dealing with a numeric column, you may consider checking for
>> is_numeric().
>
> There are also regular expressions. Testing against '/[^\0x00-\x7F]/', and
> rejecting anything that matches, appears to be the best approach here.

For a numeric column? Why not at least trim down the possibilites of
setting illegal values for the SQL statement?

> If
> characters actually should be converted, iconv() should be used instead of a
> hardcoded conversion.
>
> However, it is better in the long term to convert the MySQL database (or the
> relevant table and column) to utf8_general_ci, and upgrade MySQL if
> necessary (character sets and collations were not supported before MySQL 5;
> the current stable version is 5.6).
>
> <http://php.net/pcre>
> <http://dev.mysql.com/>

--
Christoph M. Becker

[Jerry Stuckle]

On 4/19/2013 5:46 PM, Jim Higgins wrote:

> On Fri, 19 Apr 2013 12:48:14 -0400, in <kkrsb6$lec$1...@dont-email.me>,
> Jerry Stuckle <jstu...@attglobal.net> wrote:
>
>> On 4/19/2013 12:16 PM, Jim Higgins wrote:
>>>
>>> I have a problem with people entering a slashed zero vs a standard
>>> ASCII zero into HTML forms intended to store data in a MySQL database.
>>>
>>> Is there a simple way in PHP to restrict input to the ASCII Character
>>> set, specifically hex 0x20 - 0x7E ? Or a simple way to detect
>>> characters outside this range before committing them to the database?
>>>
>>> My mind says it should be easy to detect and that I should be able to
>>> do it myself, but for some reason I'm drawing a huge blank.
>>>
>>> Thank you.
>>>
>>
>> Short of parsing the string some way, no.
>>
>> If it's a short string, you could loop through it character by
>> character. Longer strings might benefit from a regex.
>
>

> The strings are very short, 1 - 7 characters.

>
>> But be aware just because YOU are working with an ASCII charset, not
>> everyone is.
>

> I'm not looking for worldwide compatibility. My primary concern is
> with the annoying affectation a few users have of using a slashed zero
> in place of a just plain zero. None are using a non-ASCII character
> set otherwise.
>

I understand - but YOU have to be aware that people ARE using different
charsets, and handle it according to what YOUR needs are.

[Christoph Becker]

Jim Higgins wrote:
> On Fri, 19 Apr 2013 19:38:02 +0200, in
> <kkrvda$od5$1...@speranza.aioe.org>, Christoph Becker <cmbec...@gmx.de>

> wrote:
>
>> Jim Higgins wrote:
>>> I have a problem with people entering a slashed zero vs a standard
>>> ASCII zero into HTML forms intended to store data in a MySQL database.
>>
>> Is it really a slashed zero (U+0030 U+0338) they're entering, or do they
>> enter some similar looking character such as the Danish Ø? In the
>> former case you can simply replace the slashed zero with a standard
>> zero. Assuming UTF-8 encoding:
>>
>> $input = str_replace('\xCC\xB8', '', $input);
>
>

> It's usually 0x41 0x7E, but sometimes 0xD8.

0xD8 is Ø in ISO-8859-1 for example; I do not know which character
encoding represents the same or a similar character as 0x41 0x7E.
Anyway, ISTM you're missing to enforce a particular character encoding
for your document (see <http://www.w3.org/TR/html4/charset.html> for
HTML 4.01 documents).

> Rather than do
> replacement I'd just like to detect and give them an error message
> telling them to use their keyboard zero vs Alt Key plus numeric pad to
> create special characters.

>
>
>>> Is there a simple way in PHP to restrict input to the ASCII Character
>>> set, specifically hex 0x20 - 0x7E ? Or a simple way to detect
>>> characters outside this range before committing them to the database?
>>
>> If you're dealing with a numeric column, you may consider checking for
>> is_numeric().
>
>

> Thank you. That will solve half the problem - the case where data is
> numeric only. The other half of the issue is string fields containing
> mixed alpha/numeric.

Then you should have a look at the answers of Arno
(<news:5171A6C4...@arnowelzel.de>) and Thomas
(<news:62120436....@PointedEars.de>), who recommend using
ctype_print() resp. the regular expression '/[^\0x00-\x7F]/'.

--
Christoph M. Becker

[Thomas 'PointedEars' Lahn]

Christoph Becker wrote:

> Thomas 'PointedEars' Lahn wrote:
>> Christoph Becker wrote:
>>> Jim Higgins wrote:
>>>> Is there a simple way in PHP to restrict input to the ASCII Character
>>>> set, specifically hex 0x20 - 0x7E ? Or a simple way to detect
>>>> characters outside this range before committing them to the database?
>>>
>>> If you're dealing with a numeric column, you may consider checking for
>>> is_numeric().
>>
>> There are also regular expressions. Testing against '/[^\0x00-\x7F]/',
>> and rejecting anything that matches, appears to be the best approach
>> here.
>
> For a numeric column? Why not at least trim down the possibilites of

^^^^^^^^^^^^^^^^

> setting illegal values for the SQL statement?

A numeric column is *your* idea. The questions I have answered were the
ones that were *actually* asked:

| Is there a simple way in PHP to restrict input to the ASCII Character
| set, specifically hex 0x20 - 0x7E ? Or a simple way to detect
| characters outside this range before committing them to the database?

The character at US-ASCII code point 0x7E is _not_ a digit in any number
base.


PointedEars
--
Danny Goodman's books are out of date and teach practices that are
positively harmful for cross-browser scripting.
-- Richard Cornford, cljs, <cife6q$253$1$8300...@news.demon.co.uk> (2004)

[Thomas 'PointedEars' Lahn]

Jim Higgins wrote:

> Jerry Stuckle <jstu...@attglobal.net> wrote:
>> But be aware just because YOU are working with an ASCII charset, not
>> everyone is.

ACK.

> I'm not looking for worldwide compatibility.

On the World Wide Web, most certainly you should. US-ASCII is simply
obsolete there, because it hinders more than it helps. The default
character encoding for HTTP/1.0 and later is ISO-8859-1 (which AFAIK is
PHP's default, too), and using UTF-8 or UTF-16 becomes increasingly common
both on the Web and in current operating systems (see e. g. the percent
encoding in RFC 3986 how to handle that). Most recently, IDNs have been
introduced.

PHP is fully equipped to deal with that (through functions), even though it
has no native Unicode string support.

> My primary concern is with the annoying affectation a few users have of
> using a slashed zero in place of a just plain zero. None are using a non

> ASCII character set otherwise.

One problem with helping you with your problem is that the term “slashed
zero” actually is _not_ well-defined.


PointedEars
--
var bugRiddenCrashPronePieceOfJunk = (
navigator.userAgent.indexOf('MSIE 5') != -1
&& navigator.userAgent.indexOf('Mac') != -1
) // Plone, register_function.js:16

[Jerry Stuckle]

On 4/19/2013 6:26 PM, Christoph Becker wrote:
> Jim Higgins wrote:
>> On Fri, 19 Apr 2013 19:38:02 +0200, in
>> <kkrvda$od5$1...@speranza.aioe.org>, Christoph Becker <cmbec...@gmx.de>
>> wrote:
>>
>>> Jim Higgins wrote:
>>>> I have a problem with people entering a slashed zero vs a standard
>>>> ASCII zero into HTML forms intended to store data in a MySQL database.
>>>
>>> Is it really a slashed zero (U+0030 U+0338) they're entering, or do they
>>> enter some similar looking character such as the Danish Ø? In the
>>> former case you can simply replace the slashed zero with a standard
>>> zero. Assuming UTF-8 encoding:
>>>
>>> $input = str_replace('\xCC\xB8', '', $input);
>>
>>
>> It's usually 0x41 0x7E, but sometimes 0xD8.
>
> 0xD8 is Ø in ISO-8859-1 for example; I do not know which character
> encoding represents the same or a similar character as 0x41 0x7E.
> Anyway, ISTM you're missing to enforce a particular character encoding
> for your document (see <http://www.w3.org/TR/html4/charset.html> for
> HTML 4.01 documents).
>

This is a recommendation only. The browser is free to ignore it. There
is no way to force a browser to do anything in HTML.

>> Rather than do
>> replacement I'd just like to detect and give them an error message
>> telling them to use their keyboard zero vs Alt Key plus numeric pad to
>> create special characters.
>>
>>
>>>> Is there a simple way in PHP to restrict input to the ASCII Character
>>>> set, specifically hex 0x20 - 0x7E ? Or a simple way to detect
>>>> characters outside this range before committing them to the database?
>>>
>>> If you're dealing with a numeric column, you may consider checking for
>>> is_numeric().
>>
>>
>> Thank you. That will solve half the problem - the case where data is
>> numeric only. The other half of the issue is string fields containing
>> mixed alpha/numeric.
>
> Then you should have a look at the answers of Arno
> (<news:5171A6C4...@arnowelzel.de>) and Thomas
> (<news:62120436....@PointedEars.de>), who recommend using
> ctype_print() resp. the regular expression '/[^\0x00-\x7F]/'.
>


--

[Christoph Becker]

Jerry Stuckle wrote:
> On 4/19/2013 6:26 PM, Christoph Becker wrote:
>> Jim Higgins wrote:
>>> On Fri, 19 Apr 2013 19:38:02 +0200, in
>>> <kkrvda$od5$1...@speranza.aioe.org>, Christoph Becker <cmbec...@gmx.de>
>>> wrote:
>>>
>>>> Jim Higgins wrote:
>>>>> I have a problem with people entering a slashed zero vs a standard
>>>>> ASCII zero into HTML forms intended to store data in a MySQL database.
>>>>
>>>> Is it really a slashed zero (U+0030 U+0338) they're entering, or do
>>>> they
>>>> enter some similar looking character such as the Danish Ø? In the
>>>> former case you can simply replace the slashed zero with a standard
>>>> zero. Assuming UTF-8 encoding:
>>>>
>>>> $input = str_replace('\xCC\xB8', '', $input);
>>>
>>>
>>> It's usually 0x41 0x7E, but sometimes 0xD8.
>>
>> 0xD8 is Ø in ISO-8859-1 for example; I do not know which character
>> encoding represents the same or a similar character as 0x41 0x7E.
>> Anyway, ISTM you're missing to enforce a particular character encoding
>> for your document (see <http://www.w3.org/TR/html4/charset.html> for
>> HTML 4.01 documents).
>>
>
> This is a recommendation only. The browser is free to ignore it. There
> is no way to force a browser to do anything in HTML.

The mentioned W3C recommendation also elaborates on the "charset"
parameter of the "Content-Type" header, which should be respected by all
user agents conforming to RFC 2616, if they have requested the URI with
a suitable "Accept-Charset" header. Otherwise the PHP script may
respond with "406 Not acceptable" (and a body explaining the requirements).

--
Christoph M. Becker

[Jerry Stuckle]

SHOULD BE RESPECTED is the key phrase here.

All HTML is recommendations - including the charset. Not all browsers
follow all recommendations - or follow them the same way.

It does not guarantee you will not get non-ASCII characters, especially
if the user is using a non-ASCII charset.

And PHP will not respond with a 406 unless the user sends a 406. PHP
has no idea what charset is set in the outgoing header.

[Christoph Becker]

A HTTP response header has nothing to do with HTML as you know.

> Not all browsers
> follow all recommendations - or follow them the same way.
>
> It does not guarantee you will not get non-ASCII characters, especially
> if the user is using a non-ASCII charset.

I didn't want to suggest to send

Content-Type: text/html; charset=ASCII

I merely noticed, that a developer should be aware of charset issues and
do the best he can to /minimize/ unexpected or even arbitrary behavior.
Letting the user agent /guess/ how the response is encoded
(respectively rely on the webserver's default), and how it should encode
the follow-up request is surely not the best solution.

> And PHP will not respond with a 406 unless the user sends a 406. PHP
> has no idea what charset is set in the outgoing header.

Of course. Therefore I've written "the PHP script ...". The word
"script" was intended to imply, that the script author is responsible
for doing this.

--
Christoph M. Becker

[M. Strobel]

Am 19.04.2013 18:16, schrieb Jim Higgins:
>
> I have a problem with people entering a slashed zero vs a standard
> ASCII zero into HTML forms intended to store data in a MySQL database.
>

> Is there a simple way in PHP to restrict input to the ASCII Character
> set, specifically hex 0x20 - 0x7E ? Or a simple way to detect
> characters outside this range before committing them to the database?
>

> My mind says it should be easy to detect and that I should be able to
> do it myself, but for some reason I'm drawing a huge blank.
>

I skipped the long discussion, it looks like having little to do with your question.

Input data has to be filtered, IN ANY CASE, plus properly escaped upon insertion into
the database.

For number input, do something like

$val = sscanf($_POST[$key], '%d')

for signed numbers, or

preg_match('#^[0-9]+$#', $_POST[$key])

for ASCII numbers. Other helpful functions are cast to integer, ctype_digit(), ...

/Str.

[Thomas 'PointedEars' Lahn]

Christoph Becker wrote:

> Thomas 'PointedEars' Lahn wrote:
>> Christoph Becker wrote:
>>> In the former case you can simply replace the slashed zero with a
>>> standard zero. Assuming UTF-8 encoding:
>>>
>>> $input = str_replace('\xCC\xB8', '', $input);
>>
>> You do not consider that, ambiguity aside, even in UTF-8 there are
>> *several*
>> ways to produce Unicode characters. See: Unicode Normalization Forms.
>
> If I'm not mistaken here, this str_replace() will work for NFD and NFC,
> but indeed I had not considered NFKD and NFKC. Thanks for pointing this
> out.

[I admit that Unicode Normalization Forms are tricky. Even I did not know
the full range of their complexities before your posting, so thank you for
making me look into it more thoroughly.]

AIUI, your approach will work reliably *only* for Normalization Form D
(NFD), which is the result of “Canonical *D*ecomposition” [1].
Normalization Form C (NFC) is the result of “Canonical Decomposition,
followed by Canonical *C*omposition“ (ibid.) That is, if there is a
character in Unicode that is a composition of the used glyphs, then there
will be no code sequence for a combination character for you to remove.

[NFKD and NFKC are entirely different animals: The NFKD of U+1E9B (“ẛ”)
U+0323 is *U+0073* (“s”) U+0323 U+0307; its NFKC is *U+1E69* (“ṩ”).]

For example, if the original character was U+212B ANGSTROM SIGN (“Å”), then
with NFC there will be only a code sequence for U+00C5 LATIN CAPITAL LETTER
A WITH RING ABOVE (“Å”), and you will not find a code sequence for U+030A
COMBINING RING ABOVE that you could remove in order to get the US-ASCII-
compliant U+0041 LATIN CAPITAL LETTER A (“A”).


However, apparently iconv(), which I suggested, cannot help here either:

$ php -r 'echo iconv("UTF-8", "US-ASCII", "-\xE2\x84\xAB-") . "\n";'
PHP Notice: iconv(): Detected an illegal character in input string in
Command line code on line 1
PHP Stack trace:
PHP 1. {main}() Command line code:0
PHP 2. iconv() Command line code:1
-

$ php -r 'echo iconv("UTF-8", "US-ASCII//TRANSLIT", "-\xE2\x84\xAB-") .
"\n";'
-?-

$ php -r 'echo iconv("UTF-8", "US-ASCII//IGNORE", "-\xE2\x84\xAB-") . "\n";'
PHP Notice: iconv(): Detected an illegal character in input string in
Command line code on line 1
PHP Stack trace:
PHP 1. {main}() Command line code:0
PHP 2. iconv() Command line code:1
--

(Expected: -A-)

It might be possible using recode_string(), but I have not found out yet
how. My PHP was not compiled “--with-recode” and I can only get recode(1)
to print “"A” for “Ä”, which is not helpful here.


PointedEars
___________
[1] <http://www.unicode.org/reports/tr15/tr15-37.html>
--
Anyone who slaps a 'this page is best viewed with Browser X' label on
a Web page appears to be yearning for the bad old days, before the Web,
when you had very little chance of reading a document written on another
computer, another word processor, or another network. -- Tim Berners-Lee

[Thomas 'PointedEars' Lahn]

Nobody said anything about numbers before, though, and with your From header
field value you are *still* violating the Acceptable Use Policy of your
Usenet service provider, uni-berlin.de.

[M. Strobel]

Am 20.04.2013 14:15, schrieb Thomas 'PointedEars' Lahn:
> M. Strobel wrote:
>
>> Am 19.04.2013 18:16, schrieb Jim Higgins:
>>> I have a problem with people entering a slashed zero vs a standard
>>> ASCII zero into HTML forms intended to store data in a MySQL database.
>>>
>>> Is there a simple way in PHP to restrict input to the ASCII Character
>>> set, specifically hex 0x20 - 0x7E ? Or a simple way to detect
>>> characters outside this range before committing them to the database?
>>>
>>> My mind says it should be easy to detect and that I should be able to
>>> do it myself, but for some reason I'm drawing a huge blank.
>>
>> I skipped the long discussion, it looks like having little to do with your
>> question.
>>
>> Input data has to be filtered, IN ANY CASE, plus properly escaped upon
>> insertion into the database.
>>
>> For number input, do something like
>>
>> $val = sscanf($_POST[$key], '%d')
>>
>> for signed numbers, or
>>
>> preg_match('#^[0-9]+$#', $_POST[$key])
>>
>> for ASCII numbers. Other helpful functions are cast to integer,
>> ctype_digit(), ...
>
> Nobody said anything about numbers before, though, and with your From header
> field value you are *still* violating the Acceptable Use Policy of your
> Usenet service provider, uni-berlin.de.
>
>
> PointedEars
>

My code can be easily adapted. And then:

Wall Street Journal online

http://online.wsj.com/article/SB10001424052748704805204575594423931135084.html#

or Reuters

http://www.reuters.com/article/2011/03/17/us-eu-internet-privacy-idUSTRE72G48Z20110317

[Thomas 'PointedEars' Lahn]

M. Strobel wrote:

> Am 20.04.2013 14:15, schrieb Thomas 'PointedEars' Lahn:
>> M. Strobel wrote:
>>> For number input, do something like
>>>
>>> $val = sscanf($_POST[$key], '%d')
>>>
>>> for signed numbers, or
>>>
>>> preg_match('#^[0-9]+$#', $_POST[$key])
>>>
>>> for ASCII numbers. Other helpful functions are cast to integer,
>>> ctype_digit(), ...
>>
>> Nobody said anything about numbers before, though, and with your From
>> header field value you are *still* violating the Acceptable Use Policy of
>> your Usenet service provider, uni-berlin.de.
>

> My code can be easily adapted.

Your code is a repetition of what was posted before (by me), and does not
relate to the question at all, which was how to *reject* strings with
certain characters.

> And then:
> [tl;dr]

You should tell that to your service provider after they canceled your
account, so that they will at least have a good laugh.

[Arno Welzel]

M. Strobel, 2013-04-20 14:26:

[...]

> My code can be easily adapted. And then:
>
> Wall Street Journal online
>
> http://online.wsj.com/article/SB10001424052748704805204575594423931135084.html#
>
> or Reuters
>
> http://www.reuters.com/article/2011/03/17/us-eu-internet-privacy-idUSTRE72G48Z20110317

If you don't want to use a valid address, then use

<inv...@invalid.invalid>

Also see <http://www.individual.de/faq.php#5.3>

Did you get it now?

[Jerry Stuckle]

On 4/20/2013 8:41 AM, Arno Welzel wrote:
> M. Strobel, 2013-04-20 14:26:
>
> [...]
>> My code can be easily adapted. And then:
>>
>> Wall Street Journal online
>>
>> http://online.wsj.com/article/SB10001424052748704805204575594423931135084.html#
>>
>> or Reuters
>>
>> http://www.reuters.com/article/2011/03/17/us-eu-internet-privacy-idUSTRE72G48Z20110317
>
> If you don't want to use a valid address, then use
>
> <inv...@invalid.invalid>
>
> Also see <http://www.individual.de/faq.php#5.3>
>
> Did you get it now?
>
>

What does this have to do with PHP, troll?

[Arno Welzel]

Jerry Stuckle, 2013-04-20 14:43:

> On 4/20/2013 8:41 AM, Arno Welzel wrote:
>> M. Strobel, 2013-04-20 14:26:
>>
>> [...]
>>> My code can be easily adapted. And then:
>>>
>>> Wall Street Journal online
>>>
>>> http://online.wsj.com/article/SB10001424052748704805204575594423931135084.html#
>>>
>>> or Reuters
>>>
>>> http://www.reuters.com/article/2011/03/17/us-eu-internet-privacy-idUSTRE72G48Z20110317
>>
>> If you don't want to use a valid address, then use
>>
>> <inv...@invalid.invalid>
>>
>> Also see <http://www.individual.de/faq.php#5.3>
>>
>> Did you get it now?
>>
>>
>
> What does this have to do with PHP, troll?

You forgot to ask this M. Strobel and Thomas Lahn as well...

[Jerry Stuckle]

Answer the question, troll. What does this have to do with PHP?

[The Natural Philosopher]

give him some raw fish and he will go back under his bridge.



--
Ineptocracy

(in-ep-toc’-ra-cy) – a system of government where the least capable to lead are elected by the least capable of producing, and where the members of society least likely to sustain themselves or succeed, are rewarded with goods and services paid for by the confiscated wealth of a diminishing number of producers.

[M. Strobel]

Am 20.04.2013 14:41, schrieb Arno Welzel:
> M. Strobel, 2013-04-20 14:26:
>
> [...]
>> My code can be easily adapted. And then:
>>
>> Wall Street Journal online
>>
>> http://online.wsj.com/article/SB10001424052748704805204575594423931135084.html#
>>
>> or Reuters
>>
>> http://www.reuters.com/article/2011/03/17/us-eu-internet-privacy-idUSTRE72G48Z20110317
>
> If you don't want to use a valid address, then use
>
> <inv...@invalid.invalid>
>
> Also see <http://www.individual.de/faq.php#5.3>
>
> Did you get it now?
>
>

Get what? This domain exists:

str@suse122-intel:~/Desktop> dig nowhere.dee @208.67.222.222

; <<>> DiG 9.9.2-P2 <<>> nowhere.dee @208.67.222.222
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 17975
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 16384
;; QUESTION SECTION:
;nowhere.dee. IN A

;; ANSWER SECTION:
nowhere.dee. 0 IN A 67.215.65.132

;; Query time: 68 msec
;; SERVER: 208.67.222.222#53(208.67.222.222)
;; WHEN: Sat Apr 20 17:42:03 2013
;; MSG SIZE rcvd: 56

/Str.

[Thomas 'PointedEars' Lahn]

Thomas 'PointedEars' Lahn wrote:

> However, apparently iconv(), which I suggested, cannot help here either:
>
> $ php -r 'echo iconv("UTF-8", "US-ASCII", "-\xE2\x84\xAB-") . "\n";'
> PHP Notice: iconv(): Detected an illegal character in input string in
> Command line code on line 1
> PHP Stack trace:
> PHP 1. {main}() Command line code:0
> PHP 2. iconv() Command line code:1
> -
>
> $ php -r 'echo iconv("UTF-8", "US-ASCII//TRANSLIT", "-\xE2\x84\xAB-") .
> "\n";'
> -?-
>
> $ php -r 'echo iconv("UTF-8", "US-ASCII//IGNORE", "-\xE2\x84\xAB-") .
> "\n";'
> PHP Notice: iconv(): Detected an illegal character in input string in
> Command line code on line 1
> PHP Stack trace:
> PHP 1. {main}() Command line code:0
> PHP 2. iconv() Command line code:1
> --
>
> (Expected: -A-)
>
> It might be possible using recode_string(), but I have not found out yet
> how. My PHP was not compiled “--with-recode” and I can only get recode(1)
> to print “"A” for “Ä”, which is not helpful here.

On the other hand:

$ php -r 'echo iconv("UTF-8", "US-ASCII//TRANSLIT", \
"-Ȧ-Århus-\x30\xCC\xB8-") . "\n";'
-A-AArhus-0-

So while iconv() does not do Unicode normalization, it can be helpful here
(you can replace “AArhus” unambiguously with the correct alternative
spelling for the city, “Aarhus”, for example).


PointedEars
--
> If you get a bunch of authors […] that state the same "best practices"
> in any programming language, then you can bet who is wrong or right...
Not with javascript. Nonsense propagates like wildfire in this field.
-- Richard Cornford, comp.lang.javascript, 2011-11-14

[Thomas 'PointedEars' Lahn]

It figures that you would post the non-authoritative answer from your
misconfigured nameserver to support your anti-social behavior. That does
not make your From header field value technically valid or socially
acceptable, though, because an *address* there has to have at least a FQDN,
which nowhere.dee is no where near to. (There is no registry for the dee
TLD to begin with. Not *yet*, that is.)

Perhaps someone should inform your service provider to *make* you heed the
rules.

[M. Strobel]

Haha, gotcha, you did not understand what happens.

This was one of the spamming name servers, they give an address for anything... Never
heard of it? Time to learn.

/Str.

[Thomas 'PointedEars' Lahn]

No, I am not in need of their services.

> Time to learn.

Why should I learn about how to commit network abuse best? I for one obey
the forms.

Instead, it is about time for *you* to learn what an address header is
before someone makes you to.


X-Post & F'up2 news.admin.net-abuse.usenet

[Arno Welzel]

M. Strobel, 2013-04-20 17:43:

> Am 20.04.2013 14:41, schrieb Arno Welzel:
>> M. Strobel, 2013-04-20 14:26:
>>
>> [...]
>>> My code can be easily adapted. And then:
>>>
>>> Wall Street Journal online
>>>
>>> http://online.wsj.com/article/SB10001424052748704805204575594423931135084.html#
>>>
>>> or Reuters
>>>
>>> http://www.reuters.com/article/2011/03/17/us-eu-internet-privacy-idUSTRE72G48Z20110317
>>
>> If you don't want to use a valid address, then use
>>
>> <inv...@invalid.invalid>
>>
>> Also see <http://www.individual.de/faq.php#5.3>
>>
>> Did you get it now?
>>
>>
> Get what? This domain exists:

No it does not:

<sorry_no_...@nowhere.dee>: Host or domain name not found. Name
service
error for name=nowhere.dee type=AAAA: Host not found

But even if it would - i doubt, that <sorry_no_...@nowhere.dee> is
a valid address or that you own the domain nowhere.dee..

In <http://www.individual.de/faq.php#5.3> is very clear about this topic
- either use a *valid* address or <inv...@invalid.invalid>.

[Arno Welzel]

Jerry Stuckle, 2013-04-20 15:04:

> On 4/20/2013 8:55 AM, Arno Welzel wrote:
>> Jerry Stuckle, 2013-04-20 14:43:
>>
>>> On 4/20/2013 8:41 AM, Arno Welzel wrote:
>>>> M. Strobel, 2013-04-20 14:26:
>>>>
>>>> [...]
>>>>> My code can be easily adapted. And then:
>>>>>
>>>>> Wall Street Journal online
>>>>>
>>>>> http://online.wsj.com/article/SB10001424052748704805204575594423931135084.html#
>>>>>
>>>>> or Reuters
>>>>>
>>>>> http://www.reuters.com/article/2011/03/17/us-eu-internet-privacy-idUSTRE72G48Z20110317
>>>>
>>>> If you don't want to use a valid address, then use
>>>>
>>>> <inv...@invalid.invalid>
>>>>
>>>> Also see <http://www.individual.de/faq.php#5.3>
>>>>
>>>> Did you get it now?
>>>>
>>>>
>>>
>>> What does this have to do with PHP, troll?
>>
>> You forgot to ask this M. Strobel and Thomas Lahn as well...
>>
>>
>
> Answer the question, troll. What does this have to do with PHP?

<http://www.individual.de/faq.php#5.3> is an example for the use of PHP
(PHP 5.3.3 to be precise) ;-)

[Denis McMahon]

On Sat, 20 Apr 2013 17:43:13 +0200, M. Strobel wrote:

> Get what? This domain (nowhere.dee) exists ...

<?php

$hosts[] = "localhost";
$hosts[] = "nowhere.dee";
$hosts[] = "invalid.invalid";
$hosts[] = "example.com";
foreach ( $hosts as $host ) {
$ip = gethostbyname( $host );
if ( $ip == $host )
echo "Error - {$host} - host not found\n";
else
echo "{$host} has ip address {$ip}\n";
}

gives output:

localhost has ip address 127.0.0.1
Error - nowhere.dee - host not found
Error - invalid.invalid - host not found
example.com has ip address 192.0.43.10

What you have defined locally is irrelevant. Internet domains do not
exist unless their tld is registered with the internet GTLD root servers,
and there is a dns query path to the relevant domain nameserver from the
internet GTLD root servers.

If you must obscure your email address, I suggest you use
em...@example.com instead. Using <something>@invalid.invalid is just as
bad as using any other made up domain name. (see <http://www.iana.org/
domains/special> )

postmaster@localhost is of course another possibility, and should be
valid everywhere.

--
Denis McMahon, denismf...@gmail.com

[Jerry Stuckle]

ROFLMAO, troll. Did you even read it? (Unlike you, I do understand
German, although I'm a bit rusty since it has been over 40 years...)

You can't even answer the question. How like a troll!

[Jerry Stuckle]

On 4/20/2013 10:07 AM, The Natural Philosopher wrote:
> On 20/04/13 13:55, Arno Welzel wrote:
>> Jerry Stuckle, 2013-04-20 14:43:
>>
>>> On 4/20/2013 8:41 AM, Arno Welzel wrote:
>>>> M. Strobel, 2013-04-20 14:26:
>>>>
>>>> [...]
>>>>> My code can be easily adapted. And then:
>>>>>
>>>>> Wall Street Journal online
>>>>>
>>>>> http://online.wsj.com/article/SB10001424052748704805204575594423931135084.html#
>>>>>
>>>>>
>>>>> or Reuters
>>>>>
>>>>> http://www.reuters.com/article/2011/03/17/us-eu-internet-privacy-idUSTRE72G48Z20110317
>>>>>
>>>> If you don't want to use a valid address, then use
>>>>
>>>> <inv...@invalid.invalid>
>>>>
>>>> Also see <http://www.individual.de/faq.php#5.3>
>>>>
>>>> Did you get it now?
>>>>
>>>>
>>> What does this have to do with PHP, troll?
>> You forgot to ask this M. Strobel and Thomas Lahn as well...
>>
> give him some raw fish and he will go back under his bridge.
>
>
>

Ah, yes, the troll who uses a 'nym because he's scared to death that
someone will find out he's really an out-of-work ditch digger and not
the programmer or EE he claims to be strikes again!

Keep it up, TNP. The more you do, the more people who are laughing at
you. After all, nobody takes you seriously.

[Arno Welzel]

Jerry Stuckle, 2013-04-21 02:03:

> On 4/20/2013 3:29 PM, Arno Welzel wrote:
>> Jerry Stuckle, 2013-04-20 15:04:
>>
>>> On 4/20/2013 8:55 AM, Arno Welzel wrote:
>>>> Jerry Stuckle, 2013-04-20 14:43:
>>>>
>>>>> On 4/20/2013 8:41 AM, Arno Welzel wrote:
>>>>>> M. Strobel, 2013-04-20 14:26:
>>>>>>
>>>>>> [...]
>>>>>>> My code can be easily adapted. And then:
>>>>>>>
>>>>>>> Wall Street Journal online
>>>>>>>
>>>>>>> http://online.wsj.com/article/SB10001424052748704805204575594423931135084.html#
>>>>>>>
>>>>>>> or Reuters
>>>>>>>
>>>>>>> http://www.reuters.com/article/2011/03/17/us-eu-internet-privacy-idUSTRE72G48Z20110317
>>>>>>
>>>>>> If you don't want to use a valid address, then use
>>>>>>
>>>>>> <inv...@invalid.invalid>
>>>>>>
>>>>>> Also see <http://www.individual.de/faq.php#5.3>
>>>>>>
>>>>>> Did you get it now?
>>>>>>
>>>>>>
>>>>>
>>>>> What does this have to do with PHP, troll?
>>>>
>>>> You forgot to ask this M. Strobel and Thomas Lahn as well...
>>>>
>>>>
>>>
>>> Answer the question, troll. What does this have to do with PHP?
>>
>>
>> <http://www.individual.de/faq.php#5.3> is an example for the use of PHP
>> (PHP 5.3.3 to be precise) ;-)
>>
>>
>
> ROFLMAO, troll. Did you even read it? (Unlike you, I do understand
> German, although I'm a bit rusty since it has been over 40 years...)
>
> You can't even answer the question. How like a troll!

But you even don't understand a joke... but i'm not surprised.

[Arno Welzel]

Arno Welzel, 2013-04-21 06:51:

PS: To help you a, if you still don't get it:

GET /faq.php HTTP/1.1
Host: www.individual.de

...

HTTP/1.1 200 OK
Date: Sun, 21 Apr 2013 04:56:22 GMT
Server: Apache/2.2.16 (Debian)
X-Powered-By: PHP/5.3.3-7+squeeze15

[Jerry Stuckle]

Answer the question, troll!

[Jerry Stuckle]

Answer the question, troll!

[Jerry Stuckle]

On 4/21/2013 10:09 AM, Tim Streater wrote:
> In article <kl0mrd$98a$2...@dont-email.me>,

> Christ on a bicycle, Jerry. Is everyone a troll in your book now? Get a
> grip, why don't you.
>

Just the trolls, Timmy.

[Christoph Becker]

Thomas 'PointedEars' Lahn wrote:
> A numeric column is *your* idea. The questions I have answered were the
> ones that were *actually* asked:

>
> | Is there a simple way in PHP to restrict input to the ASCII Character
> | set, specifically hex 0x20 - 0x7E ? Or a simple way to detect
> | characters outside this range before committing them to the database?

The OP had reported that users sometimes enter a slashed zero instead of
a "normal" zero. In a column of type "char" or "varchar" this might not
matter too much, so I thought the problem might be related to numeric
columns instead. Therefore, I had written:

| If you're dealing with a numeric column, [...]
^^

--
Christoph M. Becker

[Thomas 'PointedEars' Lahn]

And again, I was _not_ referring to that. I wrote:

>>> Testing against '/[^\0x00-\x7F]/', and rejecting anything that matches,
>>> appears to be the best approach here.
^^^^^^^^^^^^^^^^^^^^^^^


PointedEars
--
When all you know is jQuery, every problem looks $(olvable).

[Thomas 'PointedEars' Lahn]

Denis McMahon wrote:

> What you have defined locally is irrelevant.

ACK.

> Internet domains do not exist unless their tld is registered with the
> internet GTLD root servers, and there is a dns query path to the relevant
> domain nameserver from the internet GTLD root servers.

Some exist even without being registered that way, though:

> If you must obscure your email address, I suggest you use
> em...@example.com instead.

That would be a violation of RFC 5536 by everyday (mis)use of the
“example.com” SLD. That domain, and the “example” TLD, are for *examples*,
not address munging. See RFC 2606.

> Using <something>@invalid.invalid is just as
> bad as using any other made up domain name. (see <http://www.iana.org/
> domains/special> )

Some NetNews providers argue that it is technically correct, as because of

,-<http://tools.ietf.org/html/rfc2606>
|
| ".invalid" is intended for use in online construction of domain
| names that are sure to be invalid and which it is obvious at a
| glance are invalid.

there cannot be a mailbox there.

However, the TLDs defined in RFC 2606 are not for address munging; they are
“for Testing, & Documentation Examples”. The example SLDs in defined in RFC
2606 are not for address munging either; they “can be used as examples”
only. And the current (2009) NetNews RFC does not allow them:

,-<http://tools.ietf.org/html/rfc5536#section-3.1.2>
|
| 3.1.2. From
|
| The From header field is the same as that specified in Section 3.6.2
| of [RFC5322], with the added restrictions detailed above in
| Section 2.2.
|
| from = "From:" SP mailbox-list CRLF

That is, added *restrictions*, _not_ added freedoms.

Because using “.invalid” (or another of those domains) in a *pseudo*-address
(RFC 5322, since at least RFC 2822: “a mailbox *receives* mail”) prevents
e-mail communication in a non-obvious way (especially to non-English
speakers), whereas e-mail communication *is* important to keep Network news
clean of one-to-one communication (to clarify misunderstandings before they
evolve into flamewars; to give technical or other advice that nobody in the
group should know or care about; to arrange for personal meetings; to offer
jobs because of obvious technical expertise; etc.), it is still considered
anti-social behavior and is subject to killfile rules.

Use it for postings that you want to be read (by knowledgable people), *at
your own risk*; consider that not being read (by knowledgable people) on
NetNews is equivalent to not participate in NetNews at all.

> postmaster@localhost is of course another possibility, and should be
> valid everywhere.

Nonsense. Because it specifies a different mailbox everywhere, *if at all*,
it is _not_ Valid. “localhost” is _not_ a *Fully Qualified* *Domain Name*.
It is the *hostname* for 127.0.0.1 by default. Unless you are running an
MTA on the computer you are reading NetNews with, there cannot be such a
mailbox. And then it will be *your* mailbox you would send e-mail to, not
the author of the posting.

Use that ”address” to annoy people to no end by spamming themselves (worst
case scenario is a bounce days later, or an unexpected log file entry) when
they are trying to give you friendly off-group advice (or in any of the
aforementioned scenarios).


See also: <http://www.interhack.net/pubs/munging-harmful/>

The paper may be old, but its arguments are still valid. Address munging in
any way is *helping* spammers, and *hurting* the NetNews community,
including and firstly oneself. IOW, address munging is one of the most
stupid things you can do.

X-Post & F'up2 news.admin.net-abuse.usenet

PointedEars
--

Use any version of Microsoft Frontpage to create your site.
(This won't prevent people from viewing your source, but no one
will want to steal it.)
-- from <http://www.vortex-webdesign.com/help/hidesource.htm> (404-comp.)