[PSR-11] Characters allowed in service IDs

[Matthieu Napoli]

Splitting of the main thread, quoting Larry:

> The spec should specify what legal characters are for an entry 

identifier, and what if any reserved characters there are.  It should 
also specify minimum supported key length and character sets, for 
completeness.  I recommend borrowing PSR-6's language here, which I 
believe addresses this area well.  Whether it uses the same reserved 
character list or another one I don't feel strongly about. I would not 
consider this a Review-breaking change. 

Why should we specify that? This is explicitly a "non-goal" of PSR-11: https://github.com/php-fig/fig-standards/blob/master/proposed/container-meta.md#32-non-goals

[David Négrier]

I agree with Matthieu.

Specifying what legal characters are supported definitely belongs to another PSR (the one where we put things into the container). I'll propose a PR in container-interop/service-provider to define precisely the allowed identifiers.

Unlike PSR-6, there is no "set" in this PSR, therefore no need to standardize the acceptable identifiers. For PSR-11, if the identifier passed is "🍕" (the pizza slice emoji) and the container does not support emoji identifiers (what a shame! :) ), the container should return a NotFoundException.

David.

[Larry Garfield]

I disagree for exactly that reason. :-)

If the goal is interoperability, then containers need to have at least a common baseline of what they should allow.  It's essentially an additional layer of type checking beyond just "string".  That doesn't mean it has to specify a dot-delimited format, or require/disallow /, or whatever.  Just the legal world of opaque strings.

I believe PSR-6's definition allows any UTF-8 character, so that would include pizza emoji. :-)  However, if I'm trying to connect two containers (for delegation), and one uses a pizza emoji and the other only allows ASCII characters, then they're not actually compatible.  (Or, god forbid, one tries to use Windows-1252 for some ungodly reason.)  Or perhaps one is limited to only 5 character strings for some reason (stored in a database column)?  Then passing in a longer string wouldn't work.

PSR-6's requirement was simply "at leas a 64 character UTF-8 string, and these chars are reserved".  If you don't want to reserve the same/any characters that's fine, but at least a character encoding and minimum legal length should be specified.

--Larry Garfield

--
You received this message because you are subscribed to the Google Groups "PHP Framework Interoperability Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to php-fig+u...@googlegroups.com.
To post to this group, send email to php...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/php-fig/5fa24b40-1b48-4c0e-907b-3fc88cc965dc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Pedro Cordeiro]

>  However, if I'm trying to connect two containers (for delegation), and one uses a pizza emoji and the other only allows ASCII characters, then they're not actually compatible.

The latter should break when setting the entries, not when fetching them. Restricting what entries can be fetched doesn't make much sense. Restricting what entries can be configured do. What if I add a pizza-emoji service, does it make any sense disallowing fetching that service?

I agree with validating this when we are discussing setting entries. For now, I'd consider this out of scope.

[Matthew Weier O'Phinney]

On Thu, Nov 3, 2016 at 9:41 AM, Larry Garfield <la...@garfieldtech.com> wrote:

I disagree for exactly that reason. :-)

If the goal is interoperability, then containers need to have at least a common baseline of what they should allow.  It's essentially an additional layer of type checking beyond just "string".  That doesn't mean it has to specify a dot-delimited format, or require/disallow /, or whatever.  Just the legal world of opaque strings.

I disagree. We didn't specify beyond the string type in PSR-7 for any of the various properties that represented lookup tables (attributes, server params, cookies, query params, etc); doing so is essentially unnecessarily verbose, when "non-empty string" essentially implies it's up to the user to determine what keys are valid and what are not, with the baseline assumption that *any* can be used.

I've played with a fair number of containers at this point, and, honestly, I cannot think of *any* that had restrictions on characters. In a worst case scenario, the lookup will fail. For containers with sets of reserved characters, they can normalize before lookup and/or injection.

Swapping containers is usually not something done lightly, and I would expect that if an implementation has a more restrictive set of characters allowed, that would be a consideration in the choice when switching.

 

I believe PSR-6's definition allows any UTF-8 character, so that would include pizza emoji. :-)  However, if I'm trying to connect two containers (for delegation), and one uses a pizza emoji and the other only allows ASCII characters, then they're not actually compatible.  (Or, god forbid, one tries to use Windows-1252 for some ungodly reason.)  Or perhaps one is limited to only 5 character strings for some reason (stored in a database column)?  Then passing in a longer string wouldn't work.

PSR-6's requirement was simply "at leas a 64 character UTF-8 string, and these chars are reserved".  If you don't want to reserve the same/any characters that's fine, but at least a character encoding and minimum legal length should be specified.

--Larry Garfield

On 11/03/2016 03:47 AM, David Négrier wrote:

I agree with Matthieu.

Specifying what legal characters are supported definitely belongs to another PSR (the one where we put things into the container). I'll propose a PR in container-interop/service-provider to define precisely the allowed identifiers.

Unlike PSR-6, there is no "set" in this PSR, therefore no need to standardize the acceptable identifiers. For PSR-11, if the identifier passed is "🍕" (the pizza slice emoji) and the container does not support emoji identifiers (what a shame! :) ), the container should return a NotFoundException.

David.



Le mercredi 2 novembre 2016 22:49:33 UTC+1, Matthieu Napoli a écrit :

Splitting of the main thread, quoting Larry:

> The spec should specify what legal characters are for an entry 

identifier, and what if any reserved characters there are.  It should 
also specify minimum supported key length and character sets, for 
completeness.  I recommend borrowing PSR-6's language here, which I 
believe addresses this area well.  Whether it uses the same reserved 
character list or another one I don't feel strongly about. I would not 
consider this a Review-breaking change. 

Why should we specify that? This is explicitly a "non-goal" of PSR-11: https://github.com/php-fig/fig-standards/blob/master/proposed/container-meta.md#32-non-goals

--
You received this message because you are subscribed to the Google Groups "PHP Framework Interoperability Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to php-fig+unsubscribe@googlegroups.com.

To post to this group, send email to php...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/php-fig/5fa24b40-1b48-4c0e-907b-3fc88cc965dc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "PHP Framework Interoperability Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to php-fig+unsubscribe@googlegroups.com.

To post to this group, send email to php...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/php-fig/074d736b-f5b4-db27-765a-814de60387c7%40garfieldtech.com.

For more options, visit https://groups.google.com/d/optout.

--

Matthew Weier O'Phinney
mweiero...@gmail.com
https://mwop.net/

[Gary Hockin]

This, as in the shared containers thread, is surely an implementation detail rather than part of the standard? It would be a shame to limit use of the standard by restricting which character sets are acceptable based on nominal research by people in the western world (I mean this in terms of character sets rather than social or political).

G

To unsubscribe from this group and stop receiving emails from it, send an email to php-fig+u...@googlegroups.com.

To post to this group, send email to php...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/php-fig/5fa24b40-1b48-4c0e-907b-3fc88cc965dc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "PHP Framework Interoperability Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to php-fig+u...@googlegroups.com.

To post to this group, send email to php...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/php-fig/074d736b-f5b4-db27-765a-814de60387c7%40garfieldtech.com.

For more options, visit https://groups.google.com/d/optout.

[Larry Garfield]

To reiterate what I said before: I am mostly suggesting a *lower bound*, not *upper bound*.  As PSR-6 states:

" A string of at least one character that uniquely identifies a cached item. Implementing libraries MUST support keys consisting of the characters A-Z, a-z, 0-9, _, and . in any order in UTF-8 encoding and a length of up to 64 characters. Implementing libraries MAY support additional characters and encodings or longer lengths, but must support at least that minimum."

And then it also has 8 reserved characters.  With the exception of the reserved punctuation marks (which as I said are highly debatable for PSR-11), it does not limit at all the keys that can be used; it's exactly the opposite.

Vis, I am in no way suggesting the spec should say "UTF-32 characters are illegal."  I am saying the spec should say "if all you support is basic ASCII, that's not enough."

I admit to being quite confused as to how this is a controversial suggestion, and why people keep misinterpreting it as suggesting to heavily restrict legal key values.

--Larry Garfield

To view this discussion on the web visit https://groups.google.com/d/msgid/php-fig/471a6868-f78a-40f7-a16b-212d64f6eaef%40googlegroups.com.

[Matthieu Napoli]

Hi Larry, you are right, let's move forward: what sentence do you suggest we should add to the spec?

Matthieu 

You received this message because you are subscribed to a topic in the Google Groups "PHP Framework Interoperability Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/php-fig/I2CPPp0Su-Y/unsubscribe.
To unsubscribe from this group and all its topics, send an email to php-fig+u...@googlegroups.com.

To post to this group, send email to php...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/php-fig/747f1318-fcad-54c5-15db-c056f500bbfa%40garfieldtech.com.

[Gary Hockin]

I hear you Larry, but I still disagree and say this is an implementation detail.

G

[Larry Garfield]

I filed a PR here with my recommendation:

https://github.com/php-fig/fig-standards/pull/837

Thanks.

--Larry Garfield

[Matthieu Napoli]

I filed a PR here with my recommendation:

https://github.com/php-fig/fig-standards/pull/837

Thanks.

--Larry Garfield

Thanks, I feel it's easier to discuss that now:

>  An entry identifier is a string of at least one character that uniquely identifies an item within a container.

Sounds good to me, an empty string would be, IMO, problematic. And it wouldn't surprise me that many containers would throw an exception with an empty identifier (I even wonder if PHP-DI does it :p).

Is "uniquely" an issue? With aliases (for example) 2 IDs can identify the same entry.

> Implementing libraries MUST support identifiers consisting of the characters A-Z, a-z, 0-9, _, and . in any order in UTF-8 encoding and a length of up to 64 characters.

That list of characters doesn't include \ which is used by all autowiring containers (where the ID is the class name). This is IMO an example of a problem with enumerating all allowed characters: we are bound to miss some. Same for "-" which is currently not allowed. What if someone wants to name a service with an emoji? (for whatever reasons).

And I understand you said you wanted to place a "lower bound", but if some containers don't support "\" then it's a big issue with interoperability.

Another problem I see is the max length: why such a maximum? It doesn't seem very high either, though of course in most cases it would be fine.

I would suggest something like that instead:

> [An entry identifier is a string of at least one character that uniquely identifies an item within a container.] It can consist of any characters in UTF-8 encoding.

>  Implementing libraries MAY support additional characters and encodings or longer lengths, but must support at least that minimum.

IMO we can get rid of this sentence because this is exactly the Liskov Substitution Principle and we removed the sentence about extra optional parameters for "get()" for the same reason.

> An entry identifier is an opaque string, so callers SHOULD NOT assume that the structure of the string caries any semantic meaning.

After thinking about it I've switched from "I don't see the point" to "it's a good idea": some containers use for example "." as a separator/way to namespace entries, it's good to clear up any potential confusion here: it's just a string, conventions and structure can come later but they are out of scope of PSR-11.

Matthieu

[John Flatness]

Let me preface this comment by saying that I understand fully that this proposal is setting a floor on allowable characters, not a ceiling. That being said, I still don't understand the purpose behind specifying it as proposed:

"MUST support identifiers consisting of the characters A-Z, a-z, 0-9, _, and . in any order in UTF-8 encoding and a length of up to 64 characters"

Larry's explanation behind this change was to mandate: "if all you support is basic ASCII, that's not enough." However, this proposed wording still allows just that, as all the defined required characters are in ASCII and their UTF-8 encoding is the same as their ASCII encoding. In other words, even with the additional wording here, I can still write a conforming container that only supports ASCII identifiers. Unless I was writing a _very_ restrictive container as to characters (that for example only allows uppercase Latin letters) the only effective change here is the requirement to allow at least 64-character-long identifiers.

Even that length limit is a little ambiguous, I think. If I restrict to just the minimal set of required characters, I can limit the string length to 64 bytes and always safely fulfill the length requirement. However, if I allow all of UTF-8, am I then required to use multibyte string length comparison so that 64 _characters_ is still the limit? I suppose I can instead decide to increase the limit to 256 bytes and still be in compliance.

In sum, my objection is this: while it sets some restrictions on what containers can do, the proposed limitation really does too little (for example, it doesn't solve the "pizza emoji" problem). If I as a user actually want the compatibility as now guaranteed by the standard, I'm limited to using only the quite small minimal required set of characters. Additionally, the very act of setting any minimum at all makes it more likely that future implementations align with that minimum, simply because it's an easy way to make a decision. If anything, I think this wording codifies the idea of an ASCII-only container rather than providing any meaningful restriction.

I think it would be wise to either significantly raise the "floor" on allowed characters if one must be specified, or alternatively jettison the limitation entirely. The idea of a length requirement is fine, but it seems to me that it would be simpler to define it in terms of bytes rather than characters, given the allowance for multiple character sets.

-John Flatness

[Larry Garfield]

On 11/13/2016 03:16 AM, Matthieu Napoli wrote:

I filed a PR here with my recommendation:

https://github.com/php-fig/fig-standards/pull/837

Thanks.

--Larry Garfield

Thanks, I feel it's easier to discuss that now:

>  An entry identifier is a string of at least one character that uniquely identifies an item within a container.

Sounds good to me, an empty string would be, IMO, problematic. And it wouldn't surprise me that many containers would throw an exception with an empty identifier (I even wonder if PHP-DI does it :p).

Is "uniquely" an issue? With aliases (for example) 2 IDs can identify the same entry.

I read uniquely to mean the same key string always refers to the same service, not that a service is represented by only one string.  Viz, "foo" and "bar" can both give me the database object, but "foo" won't sometimes give me a DB object and other times a 3rd party authentication key string.

> Implementing libraries MUST support identifiers consisting of the characters A-Z, a-z, 0-9, _, and . in any order in UTF-8 encoding and a length of up to 64 characters.

That list of characters doesn't include \ which is used by all autowiring containers (where the ID is the class name). This is IMO an example of a problem with enumerating all allowed characters: we are bound to miss some. Same for "-" which is currently not allowed. What if someone wants to name a service with an emoji? (for whatever reasons).

And I understand you said you wanted to place a "lower bound", but if some containers don't support "\" then it's a big issue with interoperability.

Adding more characters to the minimum list is fine by me.  I started with the basic identifier set used by PSR-6 and others but expanding it is fine.

If we want to expand it as far as "any legal UTF-8 character" as John Flatness suggests that's an option, although I'm slightly skeptical there.  While we can talk about the hypothetical pizza emoji service, I would in all honesty immediately reject any patch to a project of mine that tried to do so.  I can't say I've ever seen a service name myself that was more than the "Standard identifier set" (as above).

The more practical question would be the use of non-latin languages.  I've never worked in a codebase that was written in Hebrew, Arabic, Cyrillic, or Kanji, and I'm not sure there are any significant projects that do so even though the language would, technically, support it.  (That may be simply my own ignorance talking; I really have no idea if such projects are a thing.) 

Another problem I see is the max length: why such a maximum? It doesn't seem very high either, though of course in most cases it would be fine.

Maximum?  It's not a maximum.  I have no expectation that any implementation would realistically support a PHP_MAX_INT number of characters in an identifier; if for no other reason than memory limits or constraints in an underlying file system.  An implementation backed by a datastore that requires specifying a length (such as SQL) is going to have to set a max length of something.  (To be fair an SQL-backed container is an unlikely implementation, but given how generic the spec currently reads I would argue it's more realistic than a pizza emoji service.)  64 is suggested as the minimum size an implementation must support.  It could have a cap of 128, 512, or 16 million characters if it wants, but 12 would not be legal.

I have zero objection to bumping that number higher.  64 is simply what I borrowed from PSR-6 as a starting point. 

I would suggest something like that instead:

> [An entry identifier is a string of at least one character that uniquely identifies an item within a container.] It can consist of any characters in UTF-8 encoding.

>  Implementing libraries MAY support additional characters and encodings or longer lengths, but must support at least that minimum.

IMO we can get rid of this sentence because this is exactly the Liskov Substitution Principle and we removed the sentence about extra optional parameters for "get()" for the same reason.

I disagree, in the interest of clarity.  Especially given how easily people mistook my intent for setting maximums on possible identifiers, not minimums, explicitly saying "more than this is completely OK, really!" makes sense here.  In the case of get(), it's arguably a bad but not forbidden idea to add proprietary parameters.  In the case of the identifier, supporting more variety than the bare minimum is a very good thing.

> An entry identifier is an opaque string, so callers SHOULD NOT assume that the structure of the string caries any semantic meaning.

After thinking about it I've switched from "I don't see the point" to "it's a good idea": some containers use for example "." as a separator/way to namespace entries, it's good to clear up any potential confusion here: it's just a string, conventions and structure can come later but they are out of scope of PSR-11.

That's the reason PSR-6 reserved some characters; on anticipation of, in the future, using some of them for more advanced functionality like namespacing.  I could see an argument either way for PSR-11's case, either reserving characters for that or deliberately not reserving characters but noting that there is no spec-defined meaning.  The question becomes IF a follow-up spec defines a meaning for the {} characters (for instance), and an existing implementation has already decided to make it mean something, then what?

Not a clear-cut answer, but a worthwhile question to chew on.

--Larry Garfield

[GeeH]

I'm worried by this suggested change. We tried doing something similar in Zend ServiceManager 2.0 and had to wait until 3.0 to be able to fix the mistake. I don't understand why any legal string in PHP can't be a container key. Limiting the range of characters in a legal key is an implementation detail. I realise I'm repeating myself (again), but it's important to make the point that this is a mistake we had to live with for the entire duration of ZF2 until we could introduce a breaking change in ZF3.

G

[GeeH]

Sorry, reading that back (which I should have done before posting) I realise I worded that incredibly badly. Let me rephrase.

> Implementing libraries MUST support identifiers consisting of the characters A-Z, a-z, 0-9, _, and . in any order in UTF-8 encoding and a length of up to 64 characters.

This is surely an implementation detail. If my container wants to disallow "." in valid service names, it should absolutely be able to do so. This would limit the interoperability in that I wouldn't be able to drop this container into my project without any changes if I used "." in my key names, but this is entirely fair. Interoperability should be about minimum effort, not no effort. I've been able to change containers very easily using the existing container-interop which places no such boundaries on key names, as with any change I had to choose my new container wisely in order to ensure compatibility. I'm worried because PSR-11 will be attempting to replace an already widely adopted standard, and any changes from that standard should be thought about very carefully. At the moment it would be trivial to replace container-interop with PSR-11, indeed, it may be possible to use both interfaces, but these kind of changes that (admittedly in my opinion) don't bring much to the table scare the hell out of me with regards to adoption.

G

[David Négrier]

Hi,

I kind of agree with @GeeH here.

> Implementing libraries MUST support identifiers consisting of the characters A-Z, a-z, 0-9, _, and . in any order in UTF-8 encoding and a length of up to 64 characters.

If I read this sentence correctly, this means that one should be able to set in a container any possible identifier from the proposed set of characters.

Now, let's consider the container below:

class ConfigurationContainer implements ContainerInterface
{
    public function get($id)
    {
        $value = getenv($id);
        if ($value === false) {
            throw new NotFoundException("Entry ".$id." not found.");
        }
        return $value;
    }
    
    public function has($id)
    {
        return getenv($id) !== false;
    }
}

This container simply exposes the environment variables as entries. You might say: "hey, this is not even a real 'container' ". And you are right. Still, this piece of code can be very valuable if we use it along another container (using the delegate dependency feature for instance).

Of course, since this container only exposes environment variables, and since environment variables cannot contain ".", it does not fulfill the criteria to be an eligible PSR-11 container (which makes me sad because in my opinion, this is a legit use case).

At this point, I realize this might certainly be the right moment to speak about the "delegate lookup feature" of PSR-11. I'll try to open a separate thread regarding the "delegate lookup" feature in the next few days (busy week!). Thinking about it might shed a different light on this particular thread.

++

David.

--

You received this message because you are subscribed to a topic in the Google Groups "PHP Framework Interoperability Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/php-fig/I2CPPp0Su-Y/unsubscribe.
To unsubscribe from this group and all its topics, send an email to php-fig+u...@googlegroups.com.
To post to this group, send email to php...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/php-fig/8f1f9936-a771-4a13-b560-42cefaf295f8%40googlegroups.com.

[Alessandro Pellizzari]

On 14/11/2016 17:50, David Négrier wrote:

>> Implementing libraries MUST support identifiers consisting of the
> characters A-Z, a-z, 0-9, _, and . in any order in UTF-8 encoding and a
> length of up to 64 characters.
>
> If I read this sentence correctly, this means that one should be able to

> *set* in a container any possible identifier from the proposed set of
> characters.

Before replying, I'd like to say that I agree that containers should
allow any UTF8 character in keys, of any length.

I simply see no reason to limit them.

If a container uses MongoDB (which doesn't allow `.`) as a backend, for
example, it will be the container's responsibility to encode/decode the
keys.

> Of course, since this container only exposes environment variables, and
> since environment variables cannot contain ".", it does not fulfill the
> criteria to be an eligible PSR-11 container (which makes me sad because
> in my opinion, this is a legit use case).

It's the opposite, IMHO. The container supports them (there is no `if`
that checks that). If you decide to put only ASCII keys in there it's
not the container's fault, so it's PSR-11 compatible.

Or maybe I misunderstood your sentence. :)

Bye.

[GeeH]

After discussion with Larry last night I suggested an edit that states that all containers should support a valid PHP string in UTF-8 format. I made that suggestion on the pull request.

G

[David Négrier]

Hey Gary,

It makes sense to me. I like what we are coming to.
Thanks to both of you for discussing it. I wish I could have been with you at PHP[World] :)

++
David.

[Anthony Ferrara]

All,

On Wed, Nov 16, 2016 at 9:37 AM, GeeH <ga...@hock.in> wrote:

After discussion with Larry last night I suggested an edit that states that all containers should support a valid PHP string in UTF-8 format. I made that suggestion on the pull request.

My suggestion would be not to specify "UTF-8" character encoding. 

There are a few reasons for this, but to put it quickly:

1. There are plenty of other character sets in common use around the world. Why limit it?

2. The notion of variance suggests that implementers of the interface can accept more than UTF-8. That's awesome. However, those that depend cannot without losing the benefit of the interface. This effectively prevents anyone from sing the interface unless they use UTF-8 in the first place.

 

In the eyes of interoperability and standard, unless there's a strong and compelling reason to add "UTF-8" to the spec, why specify a character set at all? What benefit is there in artificially limiting the available strings?

Why not simply say "support all valid PHP strings" which would by definition be a super-set of UTF-8, and indeed be any representable character no matter the encoding.

My suggestion anyway.

Anthony

[Daniel Plainview]

> In the eyes of interoperability and standard, unless there's a strong and compelling reason to add "UTF-8" to the spec, why specify a character set at all? What benefit is there in artificially limiting the available strings?

Quick example: you want to log every service names that were called more than X time per request, let's say. When you know that it is UTF-8 string, you can easily in MySQL with utf8 charset, but when you don't know... You are in position like "argh, do I need to store it like binary string just because someone allowed everything?". 

Nobody likes vague contracts (expecting lawyers, maybe). Limitations help make things precise.

[Larry Garfield]

Per the open meeting at php[world] I have updated the PR again to remove mention of the character encoding.

--Larry Garfield

--

You received this message because you are subscribed to the Google Groups "PHP Framework Interoperability Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to php-fig+u...@googlegroups.com.

To post to this group, send email to php...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/php-fig/8f9916d0-1ff4-4c6a-993a-54e90139eceb%40googlegroups.com.