WordPress.com PuSH support and PuSHPress WordPress.org Plugin
Joseph Scott
http://en.blog.wordpress.com/2010/03/03/rub-a-dub-dub-in-the-pubsubhubbub/
And for WordPress.org users can do the same thing with the PuSHPress
plugin - http://wordpress.org/extend/plugins/pushpress/
I've written up a few more details at
http://josephscott.org/archives/2010/03/pushpress-a-pubsubhubbub-plugin-for-wordpress/
--
Joseph Scott
jos...@josephscott.org
http://josephscott.org/
Julien Genestoux
Brett Slatkin
On Wed, Mar 3, 2010 at 8:55 AM, Joseph Scott <jos...@josephscott.org> wrote:
John Panzer
William Ymesei
Bob Wyman
bob wyman
Jay Rossiter
Is there a way for Wordpress users to disable this? Since Wordpress has an internal hub, I'm seeing one potential side-effect. Some sites (e.g. techcrunch) use both Wordpress and FeedBurner.
Since FeedBurner already uses the appspot hub, there are now two hub references in the feed. Depending on which one you subscribe to you're going to get completely different content. One which will come directly from Wordpress without any of FB's analytics added (since I can only assume that WP is handling feedcontent internally, and not HTTP-retrieving its own content), and one from Appspot, which has been modified by FeedBurner.
This kind of issue is only going to get progressively worse as more and more services implement PuSH.
On 3/3/2010 8:55 AM, Joseph Scott wrote:
This morning I turned on PuSH support for all WordPress.com blogs - http://en.blog.wordpress.com/2010/03/03/rub-a-dub-dub-in-the-pubsubhubbub/ And for WordPress.org users can do the same thing with the PuSHPress plugin - http://wordpress.org/extend/plugins/pushpress/ I've written up a few more details at http://josephscott.org/archives/2010/03/pushpress-a-pubsubhubbub-plugin-for-wordpress/
Jay Rossiter | Software Engineer/System Administrator
Pioneering RSS Advertising Solutions
jros...@pheedo.com | Phone: 503.896.6187 | Fax: 503.235.2216
Website: www.pheedo.com | RSS: www.pheedo.info/index.xml
Brett Slatkin
On Thu, Mar 4, 2010 at 11:33 AM, Jay Rossiter <jros...@pheedo.com> wrote:
>
> Is there a way for Wordpress users to disable this? Since Wordpress has an internal hub, I'm seeing one potential side-effect. Some sites (e.g. techcrunch) use both Wordpress and FeedBurner.
>
> Since FeedBurner already uses the appspot hub, there are now two hub references in the feed. Depending on which one you subscribe to you're going to get completely different content. One which will come directly from Wordpress without any of FB's analytics added (since I can only assume that WP is handling feedcontent internally, and not HTTP-retrieving its own content), and one from Appspot, which has been modified by FeedBurner.
>
> This kind of issue is only going to get progressively worse as more and more services implement PuSH.
I'd say this is mainly a problem with FeedBurner, not WordPress.
Feed proxy services should do it like this: 1) subscribe to the source
feed with URL X, 2) publish updates using the proxy's hub (or a
user-configured hub if desired) on URL Y, 3) have blog services
advertise URL Y as their feed URL for subscribers. Then subscribers
will only pick up the proxied feed and there will be no conflict.
In the case of FeedBurner, I know the guys on that team are actively
working to fix this problem. Does that solution make sense to you,
Jay?
-Brett
Joseph Scott
Is there a way for Wordpress users to disable this? Since Wordpress has an internal hub, I'm seeing one potential side-effect. Some sites (e.g. techcrunch) use both Wordpress and FeedBurner.
Since FeedBurner already uses the appspot hub, there are now two hub references in the feed. Depending on which one you subscribe to you're going to get completely different content. One which will come directly from Wordpress without any of FB's analytics added (since I can only assume that WP is handling feedcontent internally, and not HTTP-retrieving its own content), and one from Appspot, which has been modified by FeedBurner.
Jay Rossiter
Wordpress publishes feed with WP hub
Feedburner pulls feed from WP, modifies feed, removes WP hub, adds Appspot hub with new 'self' href.
Users see only one option to subscribe to, which has the content containing all modifications.
To make updates fastest, FB (etc.) should implement hub chaining so that updates aren't poll-based, but this also has problems if a third-party hub is inserted by the publisher (e.g. Superfeedr), because they're likely polling content post modification. It leads to a chain-loop.
Brett Slatkin
>
> Since FeedBurner already uses the appspot hub, there are now two hub references in the feed. Depending on which one you subscribe to you're going to get completely different content. One which will come directly from Wordpress without any of FB's analytics added (since I can only assume that WP is handling feedcontent internally, and not HTTP-retrieving its own content), and one from Appspot, which has been modified by FeedBurner.
>
> This kind of issue is only going to get progressively worse as more and more services implement PuSH.
>
> I'd say this is mainly a problem with FeedBurner, not WordPress.
>
> Feed proxy services should do it like this: 1) subscribe to the source
> feed with URL X, 2) publish updates using the proxy's hub (or a
> user-configured hub if desired) on URL Y, 3) have blog services
> advertise URL Y as their feed URL for subscribers. Then subscribers
> will only pick up the proxied feed and there will be no conflict.
>
> In the case of FeedBurner, I know the guys on that team are actively
> working to fix this problem. Does that solution make sense to you,
> Jay?
>
> We've discussed this internally several times as well. My stance is that any service which modifies and republishes the feed should be scrubbing any hub references from previous sources. I think that's in agreement with what you said -
>
> Wordpress publishes feed with WP hub
> Feedburner pulls feed from WP, modifies feed, removes WP hub, adds Appspot hub with new 'self' href.
> Users see only one option to subscribe to, which has the content containing all modifications.
>
> To make updates fastest, FB (etc.) should implement hub chaining so that updates aren't poll-based, but this also has problems if a third-party hub is inserted by the publisher (e.g. Superfeedr), because they're likely polling content post modification. It leads to a chain-loop.
I agree with you that in practice it's easiest if the proxy strips out
any source-feed hubs. This is especially true in a multi-datacenter
scenario where proxied feeds may not be consistent in all geographies
(thus only the overriding hub could have a complete picture of the
feed).
However, I don't like how this reduces the control of the publisher
over how they syndicate their content. It'd be nice if the originating
hub could be used for syndicating the proxied content as well. To do
that with PuSHPress, you'd have to configure the plug-in to also allow
for the proxied URL to be pushed through that hub, right?
What do you all think?
-Brett
Jay Rossiter
I agree with you that in practice it's easiest if the proxy strips out any source-feed hubs. This is especially true in a multi-datacenter scenario where proxied feeds may not be consistent in all geographies (thus only the overriding hub could have a complete picture of the feed). However, I don't like how this reduces the control of the publisher over how they syndicate their content. It'd be nice if the originating hub could be used for syndicating the proxied content as well. To do that with PuSHPress, you'd have to configure the plug-in to also allow for the proxied URL to be pushed through that hub, right? What do you all think?
Brett Slatkin
>
> On 3/4/2010 12:14 PM, Brett Slatkin wrote:
>
> I agree with you that in practice it's easiest if the proxy strips out
> any source-feed hubs. This is especially true in a multi-datacenter
> scenario where proxied feeds may not be consistent in all geographies
> (thus only the overriding hub could have a complete picture of the
> feed).
>
> However, I don't like how this reduces the control of the publisher
> over how they syndicate their content. It'd be nice if the originating
> hub could be used for syndicating the proxied content as well. To do
> that with PuSHPress, you'd have to configure the plug-in to also allow
> for the proxied URL to be pushed through that hub, right?
>
> What do you all think?
>
> In theory, I agree that it would be nice, but I don't think it's practical in most cases. Since WP (as an example) is using a simple internal hub, they don't currently have to worry about http polling, or hub chaining. For them to support publishing of the proxied URL they need to add polling support at the least, and it slows down the 'near realtime' goal of PuSH unless the proxy is also doing backwards publish notification.
The proxy would notify the originating hub that the proxied content is
available. The originating hub would fetch the proxied feed and then
deliver to all subscribers. This requires the WP hub to know the
proxied feed is authorized and to be sure to post the source feed and
proxied feeds on different URLs.
I don't think it would slow anything down, but the dance is probably
more complex than average users are going to want to configure.
Jay Rossiter
Matthew Terenzio
I remember it being suggested that the order they were listed might indicate the priority. Might that work here?
Am I correct in my understanding that this is only an issue with hosted solutions that provide a hub and allow their users to use a third part proxy like Feedburner?
I mean, the users of feedburner must get SOME traffic on their native feeds in the polling world. They've lived with that. Why can't they live with this?
I haven't thought that through much, just asking.
Jay Rossiter
The spec allows for multiple hubs, but all of the hubs should be distributing the same content, and all of them should respond to the published self link.
In the case of WP, when FB adds their hub information, they also change the self link. If a user were to work in a top-down priority for hub subscription, they hit an error. WP is the first hub listed in the feed, but it refuses subscriptions for the Feedburner self-link. (As expected, since it's a private hub.)
The subscriber thus has to be willing and able to move on to the next hub in the list and attempt to subscribe there.
The problem isn't that a user may be getting content directly from the source. If that was the case, the WP hub would work perfectly because the content hasn't been modified by an outside source and can subscribe to the WP-self link. The issue at hand is really how proxy services should handle feeds that are already hub-enabled when they make their modifications, because it adds two possible issues; the first is what I described - that the first hub won't allow subscriptions because it's not authoritative for that URL, and the second is that it DOES allow subscriptions, but it provides the wrong content.
The second issue is the worst of the two if the client subscribes to both hubs, because they will both provide different sets of content, which will cause duplication in the client's dataset.
Joseph Scott
> I agree with you that in practice it's easiest if the proxy strips out
> any source-feed hubs. This is especially true in a multi-datacenter
> scenario where proxied feeds may not be consistent in all geographies
> (thus only the overriding hub could have a complete picture of the
> feed).
>
> However, I don't like how this reduces the control of the publisher
> over how they syndicate their content. It'd be nice if the originating
> hub could be used for syndicating the proxied content as well. To do
> that with PuSHPress, you'd have to configure the plug-in to also allow
> for the proxied URL to be pushed through that hub, right?
I think in the general picture of feed proxies people expect them to
take care of distributing the feed, which in this case includes pings
as well. Chaining pings out from most authoritative (PuSHPress for
example) to less authoritative (feed proxy) until it gets to the end
feed consumer (Google Reader, Bloglines, etc.) makes the most sense.
This allows for feed proxies to alter the ping data with their
tracking/ads. The act of using a feed proxy implies that you are
willing to give up some level of control over how your feed is
distributed, this includes pings.
Now if I were a feed proxy I go one step further. When I find a hub
already listed in the original feed I'd let the user know and ask them
what they want me to do:
- Should the feed proxy chain the pings, replacing the hub line with
it's own (explain the pros/cons)
- Or leave the originating hub (explain pros/cons)
Brett Slatkin
I think we're all in agreement. Hooray. =)
James Holderness
> This morning I turned on PuSH support for all WordPress.com blogs -http://en.blog.wordpress.com/2010/03/03/rub-a-dub-dub-in-the-pubsubhu...
>
> And for WordPress.org users can do the same thing with the PuSHPress
> plugin -http://wordpress.org/extend/plugins/pushpress/
I just looked at the code for the PuSHPress plugin (don't know whether
that is the same code that Wordpress.com uses) and it looks to me like
it does no verification when receiving an unsubscribe request. That
seems like a fairly serious flaw to me. Or have I misunderstood the
way the code works?
Joseph Scott
class-pushpress.php file.
On Sun, Mar 7, 2010 at 4:42 AM, James Holderness <j4j...@gmail.com> wrote:
> I just looked at the code for the PuSHPress plugin (don't know whether
> that is the same code that Wordpress.com uses) and it looks to me like
> it does no verification when receiving an unsubscribe request. That
> seems like a fairly serious flaw to me. Or have I misunderstood the
> way the code works?
--
James Holderness
> Take a look at the verify_request method of the PuSHPress class in the
> class-pushpress.php file.
Do you mean this one?
http://plugins.trac.wordpress.org/browser/pushpress/trunk/class-pushpress.php?rev=212708#L221
I was asking about *unsbuscribe* requests, and that just seems to be
verifying *subscribe* requests. The hub.mode parameter is always set
to "subscribe" and the function isn't called when unsubscribing.
Joseph Scott
auto pilot after that. I'll take a look.
On Mon, Mar 8, 2010 at 2:57 PM, James Holderness <j4j...@gmail.com> wrote:
>
> Do you mean this one?
> http://plugins.trac.wordpress.org/browser/pushpress/trunk/class-pushpress.php?rev=212708#L221
>
> I was asking about *unsbuscribe* requests, and that just seems to be
> verifying *subscribe* requests. The hub.mode parameter is always set
> to "subscribe" and the function isn't called when unsubscribing.
--
Joseph Scott
verify_request for unsubscribe requests as well (and removes the hard
coded subscribe mode from the verification process).
Thanks for pointing this out. I should go back and re-read the spec
again, apparently things didn't sink in well enough the first few
dozen times :-(
On Mon, Mar 8, 2010 at 2:57 PM, James Holderness <j4j...@gmail.com> wrote:
> Do you mean this one?
> http://plugins.trac.wordpress.org/browser/pushpress/trunk/class-pushpress.php?rev=212708#L221
>
> I was asking about *unsbuscribe* requests, and that just seems to be
> verifying *subscribe* requests. The hub.mode parameter is always set
> to "subscribe" and the function isn't called when unsubscribing.
--
Brett Slatkin
> Confirmed, I've just committed version 0.1.5 of PuSHPress that calls
> verify_request for unsubscribe requests as well (and removes the hard
> coded subscribe mode from the verification process).
>
> Thanks for pointing this out. I should go back and re-read the spec
> again, apparently things didn't sink in well enough the first few
> dozen times :-(
No worries! Even the best reading of specs has its issues. This is why
we have a hub test suite:
http://code.google.com/p/pubsubhubbub/wiki/HubTestSuite
And also why it's good your code was open source, so we can all review it!
-Brett
Jay Rossiter
On a similar note - I was just noticing that the Wordpress lease_seconds returned during verification indicate that subscriptions won't be checked for ten years. Sounds a little bit long... PuSH may have gotten to 1.0, and then fallen off into obscurity before the first subscription is ever revalidated. :p
Joseph Scott
On Mon, Mar 8, 2010 at 4:45 PM, Jay Rossiter <jros...@pheedo.com> wrote:
> On a similar note - I was just noticing that the Wordpress lease_seconds
> returned during verification indicate that subscriptions won't be checked
> for ten years. Sounds a little bit long... PuSH may have gotten to 1.0, and
> then fallen off into obscurity before the first subscription is ever
> revalidated. :p
--
Tim Drijvers
however i'm not allowed to subscribe to these feeds. The hub keeps
returning an error 400.
Joseph Scott
supports the RSS2 and ATOM feed for a posts from a blog, so /tag/* and
/category/* aren't included right now.
--
Jay Rossiter
As we discussed off-list a week or two ago, this also applies in other situations, such as author feeds.
/author/<name>/feed/
Also, any parameters passed in a feed request are appended to the self link (this mostly makes sense, except that pushpress won't allow subscriptions):
Retrieving: /feed/?noredirect=1
Returns a feed with a self link of: /feed/?noredirect=1
While retrieving a normal feed from /feed/, I've also seen self links specified as (these simply don't make sense):
/
/wp-admin/post.php
/wp-cron.php?doing_wp_cron
/xmlrpc.php
On 3/24/2010 8:32 AM, Joseph Scott wrote:
I'll take a look at that. Currently the PuSHPress plugin only supports the RSS2 and ATOM feed for a posts from a blog, so /tag/* and /category/* aren't included right now. On Wed, Mar 24, 2010 at 6:58 AM, Tim Drijvers <t...@wise-guys.nl> wrote:The /tag/* and /category/* feeds on *.wordpress.com also list a hub, however i'm not allowed to subscribe to these feeds. The hub keeps returning an error 400.