Content received lags behind last successful fetch and last ping

[WarrenG]

Hello,

I am a publisher and have an RSS feed that is on a test site. We use the https://pubsubhubbub.appspot.com as the publisher hub.
This gets pinged when a person updates a post.

When I use the https://pubsubhubbub.appspot.com/publisher to debug I can see that the Last successful fetch and the Last Ping are from when we updated a post. So far so good.
However the Last item retrieved is from 30mins before.

I have even tried replacing the feed entirely with a completely different feed with different content from the original. In this case the Last item retrieved still remains the same as it was 30 mins ago even if I update another post.

It looks like the feed was not fetched at all after the last ping.

We have added the following to the feeds... and the ping looks to be set up correctly so I cannot see what is wrong.

    <!-- PubSubHubbub Discovery -->
    <link rel="hub"  href="http://pubsubhubbub.appspot.com" xmlns="http://www.w3.org/2005/Atom" />
    <link rel="self" href="http://www.website/rss.xml" xmlns="http://www.w3.org/2005/Atom" />
    <!-- End Of PubSubHubbub Discovery -->


Any help would be much appreciated.

Thanks

[Julien Genestoux]

Warren,

May I ask if you see the same beavior when using another hub, like http://pubsubhubbub.superfeedr.com/ ?

Thanks,

--

Got a blog? Make following it simple: https://www.subtome.com/

Julien Genestoux,
http://twitter.com/julien51

+1 (415) 830 6574
+33 (0)9 70 44 76 29

--
 
---
You received this message because you are subscribed to the Google Groups "Pubsubhubbub" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pubsubhubbub...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[WarrenG]

Hi Julien,

We tried with Superfeedr but could not find a debugging tool that was similar to the google one.

I could not see a way to find out what the latest items were that superfeedr had received.

When we were looking at Superfeedr, we subscribed to the feed via Feedly but the content was still not being updated.
(http://blog.superfeedr.com/state-of-pubsubhubbub/)

Many thanks

[Roman]

Hi,

This is a reporting problem. The "Last successful fetch" and "Last ping" are updated in real time but the "Last item retrieved" may be updated with a delay. If you give it a few minutes, you'll see the expected item.

To summarize, the fetching and distribution works correctly but the diagnostics data is lagging behind. I opened a bug to fix this.

Roman.

On Mon, Aug 19, 2013 at 7:19 PM, WarrenG <mrwarr...@googlemail.com> wrote:

--

[WarrenG]

Hi Roman,

Thank you for your reply.

I do not think this is limited to a reporting problem because I also cannot see the updated items appearing in the feed readers (feedly and http://theoldreader.com/ - which both support pubsubhub).

Warren

[Roman]

Could you provide a link to the specific topic (or two) experiencing this problem? I'll take a look what's happening to them.

Roman.

[WarrenG]

Hi Roman,

Please find a link to the topic https://pubsubhubbub.appspot.com/topic-details?hub.url=http://stage.cityam.com/feed/liveblog/rss.xml

Thanks

[Roman]

Here's what happens.

1. At some point blog-post-24 was the last post in http://stage.cityam.com/feed/liveblog/rss.xml.

2. blog-post-26 was added to http://stage.cityam.com/feed/liveblog/rss.xml.

3. http://stage.cityam.com/feed/liveblog/rss.xml got pinged in the hub.

4. The hub fetched http://stage.cityam.com/feed/liveblog/rss.xml.

5. The hub got the previous content without blog-post-26!

I pinged this feed right now and the hub now shows blog-post-26 as the last retreived item.

These problems often happen due to caches or distributed serving systems. Is it possible that stage.cityam.com may serve stale feed content after sending a ping to the hub?

Roman.

[WarrenG]

Hi Roman,

I have just updated the http://stage.cityam.com/feed/liveblog/rss.xml with another post (Blog Post 27) which I can see in the feed. However it is not updated in the last item still.

The xml feed is passed from Varnish and drupal so there is no cache on it from what I can tell in the headers.

[Roman]

Aha, this is indeed weird. Kudos for being persistent, Warren!

I'm adding more logging to the hub to pinpoint the problem. I'll get back to you tomorrow and ask to publish another post. Appreciate your help!

Roman.

[WarrenG]

That's great thanks Roman, please let me know tomorrow and I will update with another post.

Warren

[Julien Genestoux]

Warren, what us the feed URL?

Thanks

[WarrenG]

Hi Julien,

The feed is http://stage.cityam.com/feed/liveblog/rss.xml

Thanks

[WarrenG]

Hey Roman,

Let me know how you got on and if you need me to add another blog post.

Warren

On Tuesday, 20 August 2013 15:27:15 UTC+1, Roman Perepelitsa wrote:

[Roman]

Hi Warren,

I wasn't able to push my changes in logging to production due to an ongoing release that I can't clash with. If I'm not asking for too much, would you be still available tomorrow?

Roman.

[WarrenG]

Hi Roman,

No worries - let me know tomorrow and I will do whatever I can.

Warren

[WarrenG]

Hi Roman,

Any luck today?

On Wednesday, 21 August 2013 15:52:10 UTC+1, Roman Perepelitsa wrote:

[Roman]

The push has *just* finished and the extra logging I added appears to be working. Please go ahead, add a post, verify that it's included in the feed when you download it and then publish the feed. I hope to have enough info then to figure out where the hub is dropping the ball.

Roman.

[WarrenG]

Hi Roman,

I have published another couple of blog posts and still see no update. Please let me know if you would like me to do anything else / any findings you see...

Thanks

[Roman]

[bcc:pubsub...@googlegroups.com]

I believe the hub didn't receive a ping after you added blog-post-28. I just pinged it and "unfortunately" blog-post-28 got handled correctly by the hub. Could you add another post and ping afterwards? Thanks!

Roman.

[Isosceles]

Don't mean to hijack this thread but I was seeing similar behavior I reported in another thread last week. In the end, the same feed worked when I moved both the feed and callback to another server.  I just assumed the problem was with the first server not allowing for the posts to go through to my callback URL, although I was always getting successful "Last Ping" indicators on the debug page.  Later on I tried to post random data to the callback URL from outside of network and it worked every time.  So despite I got it working from another server, the puzzle is still there for me as to why the pattern was so odd with the first one. Oh, also forgot to mention, it would actually allow for one update a day to go through and then nothing. I would try first thing in the morning, it would work, then nothing all day. Then next morning it would work one time, and nothing again until the next day.

[WarrenG]

Hi Roman I've added another post and pinged the hub again... could you take another look?

Many thanks

On Thursday, 22 August 2013 16:32:31 UTC+1, Roman Perepelitsa wrote:

[bcc:pubs...@googlegroups.com]

[Roman]

Thanks!

Here's what I see in the logs: Parse Error [mismatched tag].

The feed ends with "</item></rss>" while it should be "</item></channel></rss>". The hub currently drops feeds that can't be parsed. I opened a bug to report parse errors in the diagnostics page.

HTH,

Roman.

[WarrenG]

Hi Roman,

I've added the closing channel tag but the content is still not updating with the latest post. I've just published another post and cannot see it in the content received, nor in feedly...

[Roman]

Extensive logging is paying off!

I think I have fixed the problem by disabling the cache that I didn't know existed. Could you try publishing again?

Roman.

[WarrenG]

Hi Roman,

Just published another post but it's still not being picked up... sorry

Warren

[WarrenG]

Hi Roman,

Did you have any luck?

Many thanks
Warren

[Roman]

Sorry, I should've send an update earlier.

I still see in the logs that the hub received an old version of the feed when it fetched it immediately after the ping. I'm building a self-contained test to localize the issue.

Meanwhile, can you try removing the "Cache-Control: public, max-age=1800" header when serving http://stage.cityam.com/feed/liveblog/rss.xml?

Roman.

[WarrenG]

Hi Roman,

I've removed the cache control and this appears to be working now. Thank you for all of your help!!

I will now look into the subscription side.

Warren

[Sandra Perera]

helpfully.....http://www.thepushhub.com   have an RSS feed