reliability and Hardware requirements for pubsubhubbub

[Hady elsahar]

Hello all ,

i'm trying to setup a HUB for RSS feed for WikiData[1] updates , it's kinda a fast updating feed there might be 10 new updates per minute in busy times

so i'm asking about 

1. the hardware requirements , is the protocol needy considering the number of subscribers wont be large but High updates rate ?
   
2. in the protocol itself what is the rate that the HUB queries the publisher for new updates ? 
3.  RSS feed allows only to check limited number of last updates , the last 10 , could the PubSubHubbub miss something if the update rate is too fast ? 

thanks 

regards

1- http://www.wikidata.org/

2- http://www.wikidata.org/w/index.php?title=Special:RecentChanges&feed=atom

[Julien Genestoux]

Hady,

I am not familiar with the PubSubHubbub for AppEngine code base, but do you absolutely require to run your own hub? Most publishers actually decide to offload that.

Also, the hub should not query the publisher for updates! That's the whole point of pubsubhubbub: the publisher MUST ping the hub. If you do that the risk of missing anything is reduced drastically!

Thanks,

--
 
---
You received this message because you are subscribed to the Google Groups "Pubsubhubbub" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pubsubhubbub...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Hady elsahar]

i'll use it to build our temporary Hub that will work for a couple of months most probably until the publisher supports the PubSubHubbub itself 

i saw in the algorithm that the Hub would support Querying the publisher if the publisher doesn't support pushing 

what do you mean by missing anything ? , u mean if i implemented it that way it wouldn't be reliable ? how much efficiency should i expect for example ? 

thanks 

--
 
---
You received this message because you are subscribed to a topic in the Google Groups "Pubsubhubbub" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/pubsubhubbub/NcgvDIzWysU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to pubsubhubbub...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

-------------------------------------------------

Hady El-Sahar

Research Assistant 

Center of Informatics Sciences | Nile University

email : hadye...@gmail.com

Phone : +2-01220887311 

http://hadyelsahar.me/

 

[Julien Genestoux]

Hadi,

I'm not sure that the AppEngine hub as polling in it, but if it does it's probably not a great solution of high frequency feeds.

I have no idea what you should expect, but generally, PubSubHubbub is not *black magic*. If you don't use it in the way it's supposed to work, it's unlikely that you'll get the results you're expecting!

good luck.

[Roman]

Hello Hady,

Do I understand it correctly that you want to build a hub that will contain a single feed?

If so, it shouldn't be hard to build. The content acquisition part will probably be the easiest to build: just poll the feed every 5 seconds or so. This way you shouldn't miss anything and won't place any noticeable load neither on the hub nor on wikidata. Handling subscription requests will require a bit more work but it's also not terribly complicated.

The protocol itself isn't needy in terms of resource usage. Every new update will have to be distributed to all subscribers, so the resources needed to operate the hub are O(S * U) where S is the number of subscribers and U is the update rate of the feed.

Roman.

On Mon, Jun 24, 2013 at 3:59 AM, Hady elsahar <hadye...@gmail.com> wrote: