hub.verify - clarification on repeated keywords/parameters?

[Pádraic Brady]

Hi all,

Just wondering if we could clarify the construction of the hub.verify parameter in the spec. There are two terms used in Section 6.1 "keyword" and "parameter". My own reading has me interpreting the sentence "This parameter may be repeated to indicate multiple supported modes." as meaning something like:

hub.verify=sync&hub.verify=async

Whereas the final sentence refers to "Where repeated keywords are used, their order indicates the subscriber's order of preference", which seems to suggest:

hub.verify=sync,async

I went with the first initially since it seemed to be saying the parameter could be repeated, but I gather Superfeedr (haven't checked the reference hub tests yet) rely on the keywords existing within a single parameter (comma delimited).

I'm confused myself as to which is correct. Note that the first does get treated differently by http clients. Some will compact the first automatically into the second example (so they end up the same anyway), others will simply drop one of the parameters instead of compacting them.

 

Pádraic Brady

http://blog.astrumfutura.com
http://www.survivethedeepend.com
OpenID Europe Foundation Irish Representative

[Brett Slatkin]

On Mon, Oct 19, 2009 at 11:55 AM, Pádraic Brady <padrai...@yahoo.com> wrote:
> Hi all,
>
> Just wondering if we could clarify the construction of the hub.verify
> parameter in the spec. There are two terms used in Section 6.1 "keyword" and
> "parameter". My own reading has me interpreting the sentence "This parameter
> may be repeated to indicate multiple supported modes." as meaning something
> like:
>
> hub.verify=sync&hub.verify=async
>
> Whereas the final sentence refers to "Where repeated keywords are used,
> their order indicates the subscriber's order of preference", which seems to
> suggest:
>
> hub.verify=sync,async
>
> I went with the first initially since it seemed to be saying the parameter
> could be repeated, but I gather Superfeedr (haven't checked the reference
> hub tests yet) rely on the keywords existing within a single parameter
> (comma delimited).
>
> I'm confused myself as to which is correct. Note that the first does get
> treated differently by http clients. Some will compact the first
> automatically into the second example (so they end up the same anyway),
> others will simply drop one of the parameters instead of compacting them.

Initially it was #2. Then we changed it to #1 in the 0.2 spec.
Python's CGI module will give you back a set of key/list pairs, with
the values listed in the order they appeared in the string.
Query-string encoding does enough variable munging that we shouldn't
have to add yet another level of parsing on top of that, which is why
we're going with #1. How hard is it to make this work right in PHP for
clients?

[Julien Genestoux]

Huh... The http params parser that we use (in ruby) messes up with
that...
:/

I'll see I'd there is any way around.
--
Julien Genestoux
julien.g...@gmail.com
+1 (415) 254-7340
http://www.ouvre-boite.com

Envoyé depuis mon iPhone

[Brett Slatkin]

On Mon, Oct 19, 2009 at 12:21 PM, Julien Genestoux
<julien.g...@gmail.com> wrote:
>
> Huh... The http params parser that we use (in ruby) messes up with that...
> :/
>
> I'll see I'd there is any way around.

Yeah? Let's see how hard this is to get right from both the client
(subscriber) and server (hub) perspectives. If it's easy for a client
and hard for a Hub, that's fine. The subscriber is the important part.

[Pádraic Brady]

It's not difficult at all in PHP - it means the parameters should be parsed outside of the PHP mechanism (which is fudgy as hell anyway) but it's just a few simple lines of code. It's necessary in any case to ensure compliance with RFC3986.

Mainly, since it was failing in Superfeedr I wasn't sure if it was a bug on my side or Julien's or something in between. It looks now like a 0.1 to 0.2 migration issue for Superfeedr - so I can keep the original behaviour in my code.

Really can't wait for all Hubs to finish migrating to 0.2 ;).

Thanks for the clarification.

Paddy

 

Pádraic Brady

http://blog.astrumfutura.com
http://www.survivethedeepend.com
OpenID Europe Foundation Irish Representative

---

From: Brett Slatkin <bsla...@gmail.com>
To: pubsub...@googlegroups.com
Sent: Mon, October 19, 2009 5:12:38 PM
Subject: [pubsubhubbub] Re: hub.verify - clarification on repeated keywords/parameters?

[Pádraic Brady]

It's easy or negligible either way for a Subscriber in most cases. The need to ensure correct URL encoding means most of them will be bypassing POST parameter setting, and setting the raw POST body themselves. Most http clients allow raw POST body setting out of a similar necessity.

 

Pádraic Brady

http://blog.astrumfutura.com
http://www.survivethedeepend.com
OpenID Europe Foundation Irish Representative

---

From: Brett Slatkin <bsla...@gmail.com>
To: pubsub...@googlegroups.com

Sent: Mon, October 19, 2009 5:24:08 PM

Subject: [pubsubhubbub] Re: hub.verify - clarification on repeated keywords/parameters?

[Jay Rossiter]

On 10/19/2009 9:12 AM, Brett Slatkin wrote:

Python's CGI module will give you back a set of key/list pairs, with
the values listed in the order they appeared in the string.
Query-string encoding does enough variable munging that we shouldn't
have to add yet another level of parsing on top of that, which is why
we're going with #1. How hard is it to make this work right in PHP for
clients?
  

    The issue with PHP is their inane refusal to natively support repeated parameters.   Repeated parameters must be passed in an array format.

    hub.verify[]=sync&hub.verify[]=async

    If passed in a normally specified way (hub.verify=&hub.verify=) .. the latter parameters simply overwrite the former.  Variable stomping.

    You have to fread "php://input" (the raw POST stream) and parse it into variables on your own.

--

Jay Rossiter | Software Engineer/System Administrator
Pioneering RSS Advertising Solutions

jros...@pheedo.com | Phone: 503.896.6187 | Fax: 503.235.2216
Website: www.pheedo.com | RSS: www.pheedo.info/index.xml

[Pádraic Brady]

PHP has a few "clever" and "helpful" features. For example, a posted hub.verify is accessed as $_POST['hub_verify']. Besides Jay's observation, it also munges the period character into an underscore during parsing. I believe it does the same thing for a few other characters. So yes, self implemented parsing is almost a given for protocols like Pubsubhubbub. Normal web apps design around all this by force of habit - it's surprising how many PHP developers aren't even aware it's a problem...

I really love PHP, but it can try even my patience sometimes ;).

 

Pádraic Brady

http://blog.astrumfutura.com
http://www.survivethedeepend.com
OpenID Europe Foundation Irish Representative

---

From: Jay Rossiter <jros...@pheedo.com>
To: "pubsub...@googlegroups.com" <pubsub...@googlegroups.com>
Sent: Mon, October 19, 2009 8:47:19 PM

Subject: [pubsubhubbub] Re: hub.verify - clarification on repeated keywords/parameters?

[Jay Rossiter]

    Most of these quirks are holdovers from the days when register_globals was commonly used.  Since ., -, and many other non-word chars aren't valid in variable names, they were all converted to _.

    The "must use arrays in POST keys" is simply a result of Rasmus being bullheaded.  (http://bugs.php.net/bug.php?id=16195)

    "No, until such a time that a browser exists where [] doesn't work, I see no reason to change anything.  Standards mean very little on the web. Most of the web is completely non-standard as it is."

[Julien Genestoux]

Ok,

So I confirm that the HTTP params parser used by rack in ruby doesn't deal well with 2 paramswith the same name. (at least not like expected).

So, I did a small middleware to fix that, and superfeedr's hub should now deal with both ways pretty well :)

Here is the fix for anyone else using ruby :http://gist.github.com/213940

Paddy, I think you can test again... sorry for all that trouble!

Ju

--
Julien Genestoux,

http://twitter.com/julien51
http://superfeedr.com

+1 (415) 254 7340
+33 (0)9 70 44 76 29