Incrementally syncing states?

[Mariano Kamp]

Hi Mihai.

I have a question about syncing states. I got a couple of bug reports about syncing the starred state of articles not working anymore, but I believe this worked before.

When I try to get articles that have recently been unstarred I use a query like this:

$ curl --header "Authorization: GoogleLogin auth=AUTH"  "https://www.google.com/reader/api/0/stream/items/ids?s=user/-/state/com.google/starred&s=deleted/user/-/state/com.google/starred&n=10000&ot=1287000000&client=newsrob"

 

However I get an empty list even though I just unstarred an article:

<object>

  <list name="itemRefs" />

</object>

To find articles that have been recently starred this still works however:

$ curl --header "Authorization: GoogleLogin auth=AUTH"  "https://www.google.com/reader/api/0/stream/items/ids?s=user/-/state/com.google/starred&s=deleted/user/-/state/com.google/starred&n=10000&ot=1287000000&client=newsrob"

<object>

  <list name="itemRefs">

    <object>

      <number name="id">SOME_ID</number>

      <list name="directStreamIds">

        <string>

        user/SOME_USER/state/com.google/starred</string>

      </list>

      <number name="timestampUsec">SOME_TIMESTAMP</number>

    </object>

  </list>

</object>

Any idea? I found this conversation where you hinted that you're investigating removing syncing/incremental updates: http://groups.google.com/group/fougrapi/browse_thread/thread/a4c2dff84d1047e1

Has this happened already? Are you dropping support for like, shared, broadcast-friends also?

Are there any plans for a replacement? Maybe even with support for read states?

Is it really more efficient (load / bandwidth) for you to get the 10,000 latest starred, 10,000 latest liked, 10,000 latest broadcast-friends etc? It sure isn't for a syncing client.

Can you share your general view on syncing clients and their future with respect to the Google Reader API? Are they relevant to your footprint/bottom line at all?

Cheers,

Mariano

[Mariano Kamp]

Let's assume the feature has been silently dropped, no replacement in sight, no official stance 3rd party apps and future directions.

Based on that, is it possible to make the approach to gather all 10,000 read, liked, starred, broadcast-friends, shared etc. more efficient? At least in terms of bandwidth?

Could a checksum be added to the result and when I pass in the checksum the response will be emtpy?

I don't think this will help much with the read state, but for the other states this could be a real help in terms of saved bandwidth.

[Mihai Parparita]

Note that the only thing that was dropped was the deleted stream
support, the regular ones should still work.

Though not as good as a checksum, there is a count endpoint:

https://www.google.com/reader/api/0/stream/items/count?s=user/-/state/com.google/starred&n=10000

You could use that to see if the total number of starred, shared, etc.
items has changed, and only request the full set of IDs if it has.

Mihai

[Nick Bradbury]

Hi Mihai,

I wasn't aware of that endpoint - I'm sure I could use it to cut
bandwidth, too. Does it accept any other parameters besides "s" and
"n"?

Thanks,
Nick

On Oct 26, 1:16 pm, Mihai Parparita <mih...@google.com> wrote:
> Note that the only thing that was dropped was the deleted stream
> support, the regular ones should still work.
>
> Though not as good as a checksum, there is a count endpoint:
>

> https://www.google.com/reader/api/0/stream/items/count?s=user/-/state...

[Mariano Kamp]

Hi Mihai.

The deleted/changed items work in tandem. If I don't get which ones are removed I would need to go for all ids anyway if the sync should be accurate.

The approach you mentioned may not be accurate too, if one new article was starred and a new "processed" one is not starred anymore. This might be at least confusion to users, because the behavior is non-obvious and hence a support issue also. It would not be a problem if it produces false indicators of a changed state, as it would just mean more work occasionally, but a false indicator of nothing to work will put a syncing client out-of-sync.

Maybe it is something that the network infrastructure can do using an ETag (http://en.wikipedia.org/wiki/HTTP_ETag)? It would fit the bill nicely as the url wouldn't change, because an &ot parameter isn't supported anyway for those cases and so the url remains fixed. 

Of course I don't know nothing about the Google HTTP infrastructure, but in general the url/ETag scheme could be (maybe already is?) supported totally without changing backend logic.

[Mihai Parparita]

On Wed, Oct 27, 2010 at 8:45 AM, Nick Bradbury <nick.b...@gmail.com> wrote:
> I wasn't aware of that endpoint - I'm sure I could use it to cut
> bandwidth, too.  Does it accept any other parameters besides "s" and
> "n"?

No, that's it.

> Maybe it is something that the network infrastructure can do using an ETag
> (http://en.wikipedia.org/wiki/HTTP_ETag)? It would fit the bill nicely as
> the url wouldn't change, because an &ot parameter isn't supported anyway for
> those cases and so the url remains fixed.
> Of course I don't know nothing about the Google HTTP infrastructure, but in
> general the url/ETag scheme could be (maybe already is?) supported totally
> without changing backend logic.

Yes, ETag and If-Modified-Since support could be added, but I don't
think we have the engineering resources to promis that anytime soon
(I'm no longer on Reader, so I can't say for sure).

Mihai

[Mariano Kamp]

> Maybe it is something that the network infrastructure can do using an ETag
> (http://en.wikipedia.org/wiki/HTTP_ETag)? It would fit the bill nicely as
> the url wouldn't change, because an &ot parameter isn't supported anyway for
> those cases and so the url remains fixed.
> Of course I don't know nothing about the Google HTTP infrastructure, but in
> general the url/ETag scheme could be (maybe already is?) supported totally
> without changing backend logic.

Yes, ETag and If-Modified-Since support could be added, but I don't
think we have the engineering resources to promis that anytime soon
(I'm no longer on Reader, so I can't say for sure).

Sounds great.

Do the other guys left on the team read here? Could we get the deleted stream back until the ETag implementation? ;)

Not sure if appropriate ... but if so ... congrats on your new job ;)

Cheers,

Mariano

[Nick Bradbury]

Didn't realize you'd moved on - hopefully congratulations are in
order!

How many people are left on the Reader team now?

Nick

[Mihai Parparita]

On Wed, Oct 27, 2010 at 2:31 PM, Nick Bradbury <nick.b...@gmail.com> wrote:
> Didn't realize you'd moved on - hopefully congratulations are in
> order!

Yep, I'm on Chrome now:
http://blog.persistent.info/2010/08/moving-down-stack.html

> How many people are left on the Reader team now?

Arif Siddiquee is on the Reader team, I believe he reads this group.

Mihai

[Nick Bradbury]

Not sure how I missed that blog post of yours - if only I had a tool
that could make sure I see relevant articles in my feeds! :)

Moving to Chrome sounds like a great step for you, so congrats
definitely are in order. Thanks for all the help you've provided
here, and for pushing RSS ahead with your work on Reader through the
years.

Nick

On Oct 27, 5:34 pm, Mihai Parparita <mih...@google.com> wrote:

[Mariano Kamp]

Yes, absolutely.

Thank you very much.