Incremental sync of read state.
72 views
Skip to first unread message
Milo
Sep 24, 2009, 10:27:11 AM9/24/09
to Friends of the Unofficial Google Reader API
Mihai suggested using the following API to determine which items are
read in an incremental sync:
http://www.google.com/reader/api/0/stream/items/ids?s=user/-/state/com.google/reading-list&n=10000&xt=user/-/state/com.google/read
Is this really the only way? The response could potentially be very
large (for me right now it’s ~120KB), and I suppose there’s no
possibility of using the “ot” parameter with this because it’s a
stream of the reading list, albeit with read items excluded. Am I
missing something?
Mihai also suggested comparing the response from the following API to
infer the “read state locked” bit of items:
http://www.google.com/reader/api/0/stream/items/ids?s=user/-/state/com.google/read&n=10000&output=json
I suppose the “ot” parameter can be used with this, so it won’t be so
terribly expensive, but is it necessary? What exactly is this “read
state locked” metadata?
Many thanks,
Milo
read in an incremental sync:
http://www.google.com/reader/api/0/stream/items/ids?s=user/-/state/com.google/reading-list&n=10000&xt=user/-/state/com.google/read
Is this really the only way? The response could potentially be very
large (for me right now it’s ~120KB), and I suppose there’s no
possibility of using the “ot” parameter with this because it’s a
stream of the reading list, albeit with read items excluded. Am I
missing something?
Mihai also suggested comparing the response from the following API to
infer the “read state locked” bit of items:
http://www.google.com/reader/api/0/stream/items/ids?s=user/-/state/com.google/read&n=10000&output=json
I suppose the “ot” parameter can be used with this, so it won’t be so
terribly expensive, but is it necessary? What exactly is this “read
state locked” metadata?
Many thanks,
Milo
Mariano Kamp
Sep 27, 2009, 3:55:51 PM9/27/09
to foug...@googlegroups.com
large (for me right now it’s ~120KB)
It compresses well ;) But yeah, that would be the only syncing part that isn't incremental. It would be great if the incremental read state could be accessed the same way as the other states.
Also it seems to run a couple of seconds. I say "seems", because it's a feel for now and I haven't measured it yet.
FWIW in my case I ask for n=500, but don't have 500 unread articles on the stream.
... and I suppose there’s no
possibility of using the “ot” parameter with this because it’s a
stream of the reading list, albeit with read items excluded.
Mihai mentioned that passing in ot is possible, but I don't know how that would work.
... What exactly is this “read
state locked” metadata?
No clue, sorry.
Btw. I meanwhile use the incremental syncing and it really rocks. Huuuuuuge difference! Thanks Mihai.
Still testing and will release it sometime next week.
Cheers,
Mariano
Mariano Kamp
Oct 12, 2009, 1:01:40 PM10/12/09
to foug...@googlegroups.com
Hi guys,
the incremental read state is not working for me.
I either don't understand it properly or it is broken. What is your experience?
My understanding was, that I use "http://www.google.com/reader/atom/user/-/state/com.google/reading-list" to get my reading list, "http://www.google.com/reader/api/0/stream/items/ids?s=user/-/state/com.google/starred&s=deleted/user/-/state/com.google/starred&s=user/-/state/com.google/broadcast&s=deleted/user/-/state/com.google/broadcast&s=user/-/state/com.google/broadcast-friends&s=deleted/user/-/state/com.google/broadcast-friends" to get state updates for all states except the read state. Because of the mark-all-read-optimization I need to issue "http://www.google.com/reader/api/0/stream/items/ids?s=user/-/state/com.google/reading-list&xt=user/-/state/com.google/read".
Then I apply the status deltas for the starred, broadcast and broadcast-friends states. For the read state I set all my articles to read and then use ids from the last mentioned request above (stream/items/ids) and set all the referenced articles back to unread.
The underlying assumption is that all the unread articles from my reading list would also exist in the third request ("http://www.google.com/reader/api/0/stream/items/ids?s=user/-/state/com.google/reading-list&xt=user/-/state/com.google/read") and that it's sorted the same way as the reading-list. (*)
To verify run "ruby analyze_incremental_sync.rb <SID GOES HERE>" on a Mac or Linux (they should've Ruby installed, no?) box.
I hope it's not broken and I just need to change my understanding and code ;-) Otherwise I would really hope that this read-state extra functionality gets folded into the delta mechanism for the other states ... which would be great anyway ;-)
Just as a sidenote. Besides this new functionality being inaccurate, it's really fast now. People give me feedback that it has a serious impact on their phones' battery life. Until the above describe issue is resolved I need to turn it off in the next update, but I would really like to turn it back on.
Cheers,
Mariano
(*) Wrong sorting is not the issue, at least not in the example. I ask for 10.000 ids, so no matter how they are sorted I have all unread ids as I just get back 434.
Best viewed with a fixed font width.
Found 378 unread ids in the reading list.
Found 434 unread ids in the response of the ids request.
id r-list ids
----------------!-------!------
27734418c9611a4c true true
d3cec2a3b26617dc true true
1dfb4d448704195d true true
024025978b5e50d2 true false <-- See article xml below
660fdf00a2877f07 true true
fb115bd6d34a8e9f true false
c0b819cd878a1bb1 true true
c53b69a0cba3df31 true true
3a6be7ada9c974e6 true true
ec996d7c1ffe87dc true true
0dbd56a577b364d9 true false
ddaa9e9afd021e25 true true
ba9f2281f3d1fbab true true
4210a56ec3cf6c1b true true
604b3bddc417c6df true true
ab21866324077f15 true true
15e93fa540ef39b7 true true
62b555e36772fcc7 true true
675fcb31fbdde50c true true
66ad25e753382758 true true
d94b7c54ba80f05a true true
9e423ede3408644c true true
9e189a1ff71d1c3c true true
7b7c82da930324f6 true true
946b08d20764bff1 true true
e39e6c377ad3efbd true true
c4e330d8e3b9906c true true
927bd4804e6aa391 true true
9a9fdf1135d456bd true true
5a5aa2a9fe9bae51 true true
eb5ea6253bf166da true true
99a564ebde973b22 true true
ea1c1d8c1d083ceb true true
75f676cb2e92c5dd true true
f532dfdcea9f8aeb true false
85c7a03aafd1738e true true
cb645e9061153752 true true
15d09603685457c4 true true
ea00531b45c81794 true true
6ae328f0f0f19bc5 true true
9e20054f126fbc1d true true
cf9d8e2924446201 true true
e1eaa146247ad7ae true true
c2a687ae1f05c0e5 true true
709d1366e16576f6 true true
58336563e86ade6f true true
ce555458be4830ba true true
e69bc25b46d8a29b true true
3e4a5d13a8e50831 true true
b4021a79d7c09f64 true true
4508c750eef2a141 true true
93ea551940ea0151 true true
49b5f9b3a13f3ffc true true
bfd096bdd59fdf9d true true
f60ac17e5641edcc true false
0b08e0c198648e37 true false
13d4041982315897 true true
5f88100b445affae true true
c984ae11bf933d5a true true
fa30a754fa0d22b7 true false
ee318dbea9c7b012 true true
b81bf42cff015451 true true
be162bfaa69b1eba true true
26d3d17b3bbb8589 true true
9769e75980ae188b true true
e2333e689a79ad60 true true
4cad332147f34fd2 true true
af5913d1bf5ce843 true true
96c4df7c4bdc28a5 true true
4ace885eee5c9155 true true
9cda5dcd0c05b9e0 true true
5f6e68dd63c5bae2 true true
0bf9d260fdfec22f true false
ef9e8b2aa8d9535d true true
14ac8c204e2f1607 true true
47f302afab8003f4 true true
ea59ca680f88b8d9 true true
08cff6b4e86c7f9c true false
56bbc4e5c7f1eae3 true true
6710e27bda470f45 true true
fd451235b428cbf4 true false
17c4e27601ae99c4 true true
4f538a2d1faf84b6 true true
9f3954d791cd83a5 true true
0337898a4dc4bc43 true false
3bc3f09c37b9b115 true true
dd33ba1eaa4a3283 true true
27187d5b08acaf70 true true
9c47fb1bbac6242e true true
c6522d9af6e78bb0 true true
1c02f069a3e37da3 true true
d041baf351ea7bab true true
28dcd4a0fcbe8716 true true
70f924c790ac3a3c true true
bafc31d885e3a832 true true
94d68316daa31224 true true
f5b46328164b7209 true false
fe408211befb9d15 true false
e41203094857f5d9 true true
2122303b97519cc4 true true
ef924ff0e9db18b4 true true
e8455d2080f504f8 true true
2330627cbb47d9bc true true
f83d70ccf8eb7f38 true false
83ca9adeab905709 true true
5765a2d7df03d820 true true
486b04b1abfdd4ad true true
5831d9f203a962c6 true true
4175dfdaf29f9702 true true
7160d9e725f553f5 true true
92fe325c3d184aed true true
9669d7fe9b1ee579 true true
f4badafeb773a42b true false
f4d07056d4bdff93 true false
0d7b1f963e2ef022 true false
2601def10635dd29 true true
6312378503d3d6ec true true
15ed234d463d8e84 true true
4aec1c24fbe404b3 true true
dcffbbd69d0e0978 true true
59db35a4f7bfed25 true true
541a2cc93b7dea73 true true
cffeec495c9b3881 true true
c816295148c6f3b3 true true
0fcf1c30414912b6 true false
7f255384b622e02e true true
92b1eca640a12854 true true
0e95dc13d404546c true false
310cbec6b5486d2b true true
2154182328213b68 true true
e91f135d3e0edb2f true true
cbb47fb77e84e32d true true
13620f10205c03a0 true true
51e052f69f4f3e9a true true
521c30673ddc122b true true
a6de2cde71441263 true true
a27ccbd66b0433e8 true true
41539803113ed797 true true
f4b91916c14028bd true false
f357cee369245b7b true false
665fb39d49791c24 true true
04c9c40b2a9eb15f true false
ef3dbd14556ddeb2 true true
0ca96c2ce886b57b true false
4fe882a3c036da0d true true
f7533c0b3840d6f2 true false
e5655d1bb22ee13e true true
085ce1564c5cb986 true false
fb478d1927b7a824 true false
0c37f30d181c2e11 true false
0c7d5fb0fadbddaf true false
1c6ed04a33573467 true true
00d895a12d3a88f3 true false
1e41cc78c135d105 true true
f6b64c4fceb009e5 true false
c59fd5fa30acac31 true true
d731a441ecf8d728 true true
aa6a4af384da0d22 true true
894e03211168a166 true true
4885e31594f70fe3 true true
2d57515367e98430 true true
d060a95720072348 true true
b73216aa48266891 true true
2fd13284fb51b051 true true
bffd146973e92cde true true
e8e2e5a7496a7ee2 true true
17be1ea2e3d81584 true true
cf34ee2f0e3a87ca true true
f3c307fde4c32c87 true false
a5b5e05ddc5e0dc1 true true
07c5a7ae3da9b70d true false
434849327291b780 true true
ea890959b11702ea true true
19899357e224bbc7 true true
0563f60b00103348 true false
93b279c53de6c611 true true
993b3bbb6d1be80d true true
8ea522ad852fb5b1 true true
1e8f69f6e8d20c89 true true
34b7b460940ab499 true true
9c82999000cd702a true true
661f71779ed6ef48 true true
05384a86868da437 true false
fd63e8a302da1c8e true false
d30b1d89f1eb5a11 true true
7eb24b7bfa65a02b true true
6f07ebf050e65a2f true true
5d8744e76ef52748 true true
b129336b0b6dc712 true true
61d86670aea82559 true true
2c778cd97a066e92 true true
bf96c47d0ea1815c true true
e3f1e8d6df223164 true true
14d4e7b9654a36d3 true true
23de979299131fd4 true true
4b48b9edf03c0471 true true
82efaf2c0743b4c1 true true
38739153cf55ea00 true true
2d8f370aa1d38783 true true
63f09eaba8ce76f7 true true
05ff00e2b61b214e true false
f690ffcd8b5024eb true false
4718b569cbb9d39c true true
c0684737b0d92ea1 true true
1389e9226350072e true true
d3edcc1d2c5add5a true true
d9bcf7cbca3ff9bc true true
13274bf2dc0fe1d1 true true
16c1bd4e3ff0f3e0 true true
c9f16613222cf7c4 true true
b032759d8d48919a true true
e7adf10fbc8bca0f true true
907b77cf4fb23463 true true
988dc29237b9521c true true
b0e7c861c7307e30 true true
a1153f21428af9e3 true true
d46b07005b7bb98b true true
b6b120e8dfa9b818 true true
1e96052da39ca964 true true
0397fc98e7eee06e true false
17f8634898f98a4e true true
5a9a733bac6507de true true
d8cd36431943bdf0 true true
05518a615784b814 true false
8381f7a8e7fd45dc true true
4643828201f57e35 true true
316f30e7ea8b7666 true true
084c9455b757133c true false
7980afe796894c1a true true
002ddf7a8fac1ff3 true false
9afa5984b92d41da true true
f71efc2515d37dbb true false
c6626f47225cb966 true true
fc5b3b971e235bc2 true false
92b3c4d66d2deff0 true true
e682df9760182161 true true
9b8a83d3c207fa1c true true
97630d7182ec9680 true true
59ac94e13e981e6d true true
5e5d745f2ff31077 true true
ed3d3238d9b16e28 true true
66da6c996d05a501 true true
9d2be267d1899b00 true true
c55504ea051c8cdd true true
024fef89cf518bb3 true false
9d1a6a7678498eaf true true
cf277e8677931f63 true true
024920794f4b8b6c true false
cf64f7df76b940ba true true
ce2735716974d192 true true
f0e2979771a5073c true false
9dbd96a00bf16a25 true true
2e50f2346584e8cc true true
efb2a119f36e0c17 true true
8b3c50576243df3c true true
d00652dd7c076ad8 true true
dc44282406f61818 true true
0755fbf2177478c6 true false
be027e27fe324cb7 true true
e93e997da40290f9 true true
7326722d52e899a4 true true
608c378c812f89b7 true true
aaf07e8e4481c0af true true
8cfb1341d368ff5f true true
4534cfe37d388c48 true true
76a52a6f9233d739 true true
d90aa8c6b625a1f3 true true
5c0fb8b483ae4f43 true true
5d67e62b463a21d0 true true
3ca89c5230400625 true true
dabd2733dddc0180 true true
a8da2c3bf43e7ec6 true true
5155c4648893cbe3 true true
9c3f456a2435e511 true true
51b8ef29f9f9e71c true true
1c1d2e80155e13ca true true
962fe9447448f3ff true true
0cfafbe4406f71fa true false
512afc31740ca422 true true
604e1da5b8f6198e true true
fa94d153d9f3ff75 true false
cbc64a583323cb75 true true
eae7c8c9b6f46f21 true true
9f4f612ee34763be true true
5f4c9ddba75677f4 true true
ba10431d13ddb8dd true true
bbb39e72ff417463 true true
2a813c3c7481f428 true true
2d69224ecd210218 true true
60b32c46682305ec true true
752c0db207ec421a true true
dfb2176f5c1bd41a true true
2dc43fac9d1965cf true true
dcc07122c56cbb6d true true
8e5d65485b6a29f2 true true
3765c021dd9deb21 true true
db37ad0e429424dd true true
671190a5076decee true true
f2e4cac46f528fb4 true false
044ffd03f9f40b01 true false
f5cec6b73b344c16 true false
85458043af133025 true true
5e16e513cd1501b3 true true
24cb72f27016e9de true true
52cab2b3ca33c50d true true
d734ccd27682ee8c true true
b4a377dfda51496c true true
7796e6d5bf360a0e true true
e18ee56f6cab0194 true true
f8a0cd6dc574f88d true false
346cb4762b876522 true true
d58696e0d184908a true true
6df571fa3a214eba true true
a1da883bb4e3a948 true true
b0ef194d3d5ef8bb true true
320c72cb7ee2ae1a true true
894a2d9de22117b6 true true
db640add605a0bc8 true true
8b169d1aa0abb052 true true
32b4294db3c4d524 true true
52f9bbc4953d5dfa true true
78c6b7e7e564fcd7 true true
b4ecb0d92d461d75 true true
e8e022a77dd5ada7 true true
e6ed70959bb662cb true true
50c2b5eb8868bd13 true true
52175718f387d6eb true true
4d355bed2c545be9 true true
aacd38ed01363f02 true true
6780a2527b21ee3e true true
2a78cd2d69c02ae8 true true
e5b5646d51559873 true true
35d3c2a531c322d0 true true
077b9049399aaa98 true false
4bba6266f72305f5 true true
16417821ee25de55 true true
1bec09e2e16f7cc3 true true
6bbd18ce17e5986d true true
502e928238067959 true true
3737cc3ecfbe6ca2 true true
9cfc5060745e9aa9 true true
c2f937b2342a6642 true true
68cbc4bda12a4a7e true true
8cdeff126949e1b0 true true
6f7a26f73e295737 true true
684fddbc7c9707f0 true true
6b6ac6a84d363f12 true true
51c387d270d38b7e true true
3f872a524cd61508 true true
f038c72528a1df99 true false
5a1ea105b83ecae5 true true
202829b7e6f699ae true true
96dbb5c2f804d46d true true
17e2048a36c46d46 true true
e0e624993f925a9d true true
0fde1e1be2b0b545 true false
0083929b7686b6e0 true false
58552d0ca5493875 true true
a21f1704e4fca5b0 true true
5ac9751deff0a7ab true true
bfe8935ebfacc60f true true
5cfd89e50355172c true true
7244810888b66e38 true true
27238b710c501291 true true
9d804214f56a61bc true true
f507f669c04bb4e3 true false
003563ab283b7029 true false
b5a4729918d491f5 true true
7fa18c44f9a07af3 true true
bbc3ed7ec12ac42d true true
9cbc1ebd3cbaa535 true true
aca89ca285938416 true true
7f3711df8a43f984 true true
218718fbf8de90de true true
b8702376952c14e6 true true
42b15223031736a5 true true
256174c7038f3def true true
03b45e17b80eaec9 true false
--> This article is in the "reading list" and doesn't not have the "read" label.
<entry gr:crawl-timestamp-msec="1255356559426">
<id gr:original-id="http://coedmagazine.com/?p=106802">
tag:google.com,2005:reader/item/024025978b5e50d2</id>
<category term="user/16051647003118153658/state/com.google/reading-list"
scheme="http://www.google.com/reader/" label="reading-list" />
<category term="user/16051647003118153658/label/problem"
scheme="http://www.google.com/reader/" label="problem" />
<category term="user/16051647003118153658/state/com.google/fresh"
scheme="http://www.google.com/reader/" label="fresh" />
<category term="Humor" />
<category term="Video" />
<category term="funny" />
<category term="humor" />
<category term="sarah-silverman" />
<category term="sell the vatican" />
<category term="the vatican" />
<category term="vatican" />
<title type="html">Sarah Silverman Has an Idea [Video]</title>
<published>2009-10-12T13:41:37Z</published>
<updated>2009-10-12T13:41:37Z</updated>
<link rel="alternate"
href="http://coedmagazine.com/2009/10/12/sarah-silverman-has-an-idea-video/"
type="text/html" />
<media:group>
<media:content url="http://1.gravatar.com/avatar/b2c9f9bb05c361e7b4c5e39def85a27c?s=96&d=http%3A%2F%2F1.gravatar.com%2Favatar%2Fad516503a11cd5ca435acc9bb6523536%3Fs%3D96" />
</media:group>
<media:group>
<media:content url="http://img.youtube.com/vi/3bObItmxAGc/2.jpg" />
</media:group>
<summary xml:base="http://coedmagazine.com/" type="html">I'm
not exactly sure what to make of Sarah Silverman's newest
video. On the one hand, it just kind of seems like the comedian
is trying desperately to hit that perfect note between
outrageous, and just cute enough to make everyone like her, in
order to have her own viral video. Either that, or she really
just wants to sell the Vatican to end world hunger. (Which is
not a bad idea, btw...)<img alt="" border="0"
src="http://stats.wordpress.com/b.gif?host=coedmagazine.com&amp;blog=3508545&amp;post=106802&amp;subd=coedmagazine&amp;ref=&amp;feed=1"></summary>
<author>
<name>Andrew - Hunter College</name>
</author>
<source gr:stream-id="feed/http://coedmagazine.com/feed/">
<id>
tag:google.com,2005:reader/feed/http://coedmagazine.com/feed/</id>
<title type="html">COED Magazine</title>
<link rel="alternate" href="http://coedmagazine.com"
type="text/html" />
</source>
</entry>
the incremental read state is not working for me.
I either don't understand it properly or it is broken. What is your experience?
My understanding was, that I use "http://www.google.com/reader/atom/user/-/state/com.google/reading-list" to get my reading list, "http://www.google.com/reader/api/0/stream/items/ids?s=user/-/state/com.google/starred&s=deleted/user/-/state/com.google/starred&s=user/-/state/com.google/broadcast&s=deleted/user/-/state/com.google/broadcast&s=user/-/state/com.google/broadcast-friends&s=deleted/user/-/state/com.google/broadcast-friends" to get state updates for all states except the read state. Because of the mark-all-read-optimization I need to issue "http://www.google.com/reader/api/0/stream/items/ids?s=user/-/state/com.google/reading-list&xt=user/-/state/com.google/read".
Then I apply the status deltas for the starred, broadcast and broadcast-friends states. For the read state I set all my articles to read and then use ids from the last mentioned request above (stream/items/ids) and set all the referenced articles back to unread.
The underlying assumption is that all the unread articles from my reading list would also exist in the third request ("http://www.google.com/reader/api/0/stream/items/ids?s=user/-/state/com.google/reading-list&xt=user/-/state/com.google/read") and that it's sorted the same way as the reading-list. (*)
To verify run "ruby analyze_incremental_sync.rb <SID GOES HERE>" on a Mac or Linux (they should've Ruby installed, no?) box.
I hope it's not broken and I just need to change my understanding and code ;-) Otherwise I would really hope that this read-state extra functionality gets folded into the delta mechanism for the other states ... which would be great anyway ;-)
Just as a sidenote. Besides this new functionality being inaccurate, it's really fast now. People give me feedback that it has a serious impact on their phones' battery life. Until the above describe issue is resolved I need to turn it off in the next update, but I would really like to turn it back on.
Cheers,
Mariano
(*) Wrong sorting is not the issue, at least not in the example. I ask for 10.000 ids, so no matter how they are sorted I have all unread ids as I just get back 434.
Best viewed with a fixed font width.
Found 378 unread ids in the reading list.
Found 434 unread ids in the response of the ids request.
id r-list ids
----------------!-------!------
27734418c9611a4c true true
d3cec2a3b26617dc true true
1dfb4d448704195d true true
024025978b5e50d2 true false <-- See article xml below
660fdf00a2877f07 true true
fb115bd6d34a8e9f true false
c0b819cd878a1bb1 true true
c53b69a0cba3df31 true true
3a6be7ada9c974e6 true true
ec996d7c1ffe87dc true true
0dbd56a577b364d9 true false
ddaa9e9afd021e25 true true
ba9f2281f3d1fbab true true
4210a56ec3cf6c1b true true
604b3bddc417c6df true true
ab21866324077f15 true true
15e93fa540ef39b7 true true
62b555e36772fcc7 true true
675fcb31fbdde50c true true
66ad25e753382758 true true
d94b7c54ba80f05a true true
9e423ede3408644c true true
9e189a1ff71d1c3c true true
7b7c82da930324f6 true true
946b08d20764bff1 true true
e39e6c377ad3efbd true true
c4e330d8e3b9906c true true
927bd4804e6aa391 true true
9a9fdf1135d456bd true true
5a5aa2a9fe9bae51 true true
eb5ea6253bf166da true true
99a564ebde973b22 true true
ea1c1d8c1d083ceb true true
75f676cb2e92c5dd true true
f532dfdcea9f8aeb true false
85c7a03aafd1738e true true
cb645e9061153752 true true
15d09603685457c4 true true
ea00531b45c81794 true true
6ae328f0f0f19bc5 true true
9e20054f126fbc1d true true
cf9d8e2924446201 true true
e1eaa146247ad7ae true true
c2a687ae1f05c0e5 true true
709d1366e16576f6 true true
58336563e86ade6f true true
ce555458be4830ba true true
e69bc25b46d8a29b true true
3e4a5d13a8e50831 true true
b4021a79d7c09f64 true true
4508c750eef2a141 true true
93ea551940ea0151 true true
49b5f9b3a13f3ffc true true
bfd096bdd59fdf9d true true
f60ac17e5641edcc true false
0b08e0c198648e37 true false
13d4041982315897 true true
5f88100b445affae true true
c984ae11bf933d5a true true
fa30a754fa0d22b7 true false
ee318dbea9c7b012 true true
b81bf42cff015451 true true
be162bfaa69b1eba true true
26d3d17b3bbb8589 true true
9769e75980ae188b true true
e2333e689a79ad60 true true
4cad332147f34fd2 true true
af5913d1bf5ce843 true true
96c4df7c4bdc28a5 true true
4ace885eee5c9155 true true
9cda5dcd0c05b9e0 true true
5f6e68dd63c5bae2 true true
0bf9d260fdfec22f true false
ef9e8b2aa8d9535d true true
14ac8c204e2f1607 true true
47f302afab8003f4 true true
ea59ca680f88b8d9 true true
08cff6b4e86c7f9c true false
56bbc4e5c7f1eae3 true true
6710e27bda470f45 true true
fd451235b428cbf4 true false
17c4e27601ae99c4 true true
4f538a2d1faf84b6 true true
9f3954d791cd83a5 true true
0337898a4dc4bc43 true false
3bc3f09c37b9b115 true true
dd33ba1eaa4a3283 true true
27187d5b08acaf70 true true
9c47fb1bbac6242e true true
c6522d9af6e78bb0 true true
1c02f069a3e37da3 true true
d041baf351ea7bab true true
28dcd4a0fcbe8716 true true
70f924c790ac3a3c true true
bafc31d885e3a832 true true
94d68316daa31224 true true
f5b46328164b7209 true false
fe408211befb9d15 true false
e41203094857f5d9 true true
2122303b97519cc4 true true
ef924ff0e9db18b4 true true
e8455d2080f504f8 true true
2330627cbb47d9bc true true
f83d70ccf8eb7f38 true false
83ca9adeab905709 true true
5765a2d7df03d820 true true
486b04b1abfdd4ad true true
5831d9f203a962c6 true true
4175dfdaf29f9702 true true
7160d9e725f553f5 true true
92fe325c3d184aed true true
9669d7fe9b1ee579 true true
f4badafeb773a42b true false
f4d07056d4bdff93 true false
0d7b1f963e2ef022 true false
2601def10635dd29 true true
6312378503d3d6ec true true
15ed234d463d8e84 true true
4aec1c24fbe404b3 true true
dcffbbd69d0e0978 true true
59db35a4f7bfed25 true true
541a2cc93b7dea73 true true
cffeec495c9b3881 true true
c816295148c6f3b3 true true
0fcf1c30414912b6 true false
7f255384b622e02e true true
92b1eca640a12854 true true
0e95dc13d404546c true false
310cbec6b5486d2b true true
2154182328213b68 true true
e91f135d3e0edb2f true true
cbb47fb77e84e32d true true
13620f10205c03a0 true true
51e052f69f4f3e9a true true
521c30673ddc122b true true
a6de2cde71441263 true true
a27ccbd66b0433e8 true true
41539803113ed797 true true
f4b91916c14028bd true false
f357cee369245b7b true false
665fb39d49791c24 true true
04c9c40b2a9eb15f true false
ef3dbd14556ddeb2 true true
0ca96c2ce886b57b true false
4fe882a3c036da0d true true
f7533c0b3840d6f2 true false
e5655d1bb22ee13e true true
085ce1564c5cb986 true false
fb478d1927b7a824 true false
0c37f30d181c2e11 true false
0c7d5fb0fadbddaf true false
1c6ed04a33573467 true true
00d895a12d3a88f3 true false
1e41cc78c135d105 true true
f6b64c4fceb009e5 true false
c59fd5fa30acac31 true true
d731a441ecf8d728 true true
aa6a4af384da0d22 true true
894e03211168a166 true true
4885e31594f70fe3 true true
2d57515367e98430 true true
d060a95720072348 true true
b73216aa48266891 true true
2fd13284fb51b051 true true
bffd146973e92cde true true
e8e2e5a7496a7ee2 true true
17be1ea2e3d81584 true true
cf34ee2f0e3a87ca true true
f3c307fde4c32c87 true false
a5b5e05ddc5e0dc1 true true
07c5a7ae3da9b70d true false
434849327291b780 true true
ea890959b11702ea true true
19899357e224bbc7 true true
0563f60b00103348 true false
93b279c53de6c611 true true
993b3bbb6d1be80d true true
8ea522ad852fb5b1 true true
1e8f69f6e8d20c89 true true
34b7b460940ab499 true true
9c82999000cd702a true true
661f71779ed6ef48 true true
05384a86868da437 true false
fd63e8a302da1c8e true false
d30b1d89f1eb5a11 true true
7eb24b7bfa65a02b true true
6f07ebf050e65a2f true true
5d8744e76ef52748 true true
b129336b0b6dc712 true true
61d86670aea82559 true true
2c778cd97a066e92 true true
bf96c47d0ea1815c true true
e3f1e8d6df223164 true true
14d4e7b9654a36d3 true true
23de979299131fd4 true true
4b48b9edf03c0471 true true
82efaf2c0743b4c1 true true
38739153cf55ea00 true true
2d8f370aa1d38783 true true
63f09eaba8ce76f7 true true
05ff00e2b61b214e true false
f690ffcd8b5024eb true false
4718b569cbb9d39c true true
c0684737b0d92ea1 true true
1389e9226350072e true true
d3edcc1d2c5add5a true true
d9bcf7cbca3ff9bc true true
13274bf2dc0fe1d1 true true
16c1bd4e3ff0f3e0 true true
c9f16613222cf7c4 true true
b032759d8d48919a true true
e7adf10fbc8bca0f true true
907b77cf4fb23463 true true
988dc29237b9521c true true
b0e7c861c7307e30 true true
a1153f21428af9e3 true true
d46b07005b7bb98b true true
b6b120e8dfa9b818 true true
1e96052da39ca964 true true
0397fc98e7eee06e true false
17f8634898f98a4e true true
5a9a733bac6507de true true
d8cd36431943bdf0 true true
05518a615784b814 true false
8381f7a8e7fd45dc true true
4643828201f57e35 true true
316f30e7ea8b7666 true true
084c9455b757133c true false
7980afe796894c1a true true
002ddf7a8fac1ff3 true false
9afa5984b92d41da true true
f71efc2515d37dbb true false
c6626f47225cb966 true true
fc5b3b971e235bc2 true false
92b3c4d66d2deff0 true true
e682df9760182161 true true
9b8a83d3c207fa1c true true
97630d7182ec9680 true true
59ac94e13e981e6d true true
5e5d745f2ff31077 true true
ed3d3238d9b16e28 true true
66da6c996d05a501 true true
9d2be267d1899b00 true true
c55504ea051c8cdd true true
024fef89cf518bb3 true false
9d1a6a7678498eaf true true
cf277e8677931f63 true true
024920794f4b8b6c true false
cf64f7df76b940ba true true
ce2735716974d192 true true
f0e2979771a5073c true false
9dbd96a00bf16a25 true true
2e50f2346584e8cc true true
efb2a119f36e0c17 true true
8b3c50576243df3c true true
d00652dd7c076ad8 true true
dc44282406f61818 true true
0755fbf2177478c6 true false
be027e27fe324cb7 true true
e93e997da40290f9 true true
7326722d52e899a4 true true
608c378c812f89b7 true true
aaf07e8e4481c0af true true
8cfb1341d368ff5f true true
4534cfe37d388c48 true true
76a52a6f9233d739 true true
d90aa8c6b625a1f3 true true
5c0fb8b483ae4f43 true true
5d67e62b463a21d0 true true
3ca89c5230400625 true true
dabd2733dddc0180 true true
a8da2c3bf43e7ec6 true true
5155c4648893cbe3 true true
9c3f456a2435e511 true true
51b8ef29f9f9e71c true true
1c1d2e80155e13ca true true
962fe9447448f3ff true true
0cfafbe4406f71fa true false
512afc31740ca422 true true
604e1da5b8f6198e true true
fa94d153d9f3ff75 true false
cbc64a583323cb75 true true
eae7c8c9b6f46f21 true true
9f4f612ee34763be true true
5f4c9ddba75677f4 true true
ba10431d13ddb8dd true true
bbb39e72ff417463 true true
2a813c3c7481f428 true true
2d69224ecd210218 true true
60b32c46682305ec true true
752c0db207ec421a true true
dfb2176f5c1bd41a true true
2dc43fac9d1965cf true true
dcc07122c56cbb6d true true
8e5d65485b6a29f2 true true
3765c021dd9deb21 true true
db37ad0e429424dd true true
671190a5076decee true true
f2e4cac46f528fb4 true false
044ffd03f9f40b01 true false
f5cec6b73b344c16 true false
85458043af133025 true true
5e16e513cd1501b3 true true
24cb72f27016e9de true true
52cab2b3ca33c50d true true
d734ccd27682ee8c true true
b4a377dfda51496c true true
7796e6d5bf360a0e true true
e18ee56f6cab0194 true true
f8a0cd6dc574f88d true false
346cb4762b876522 true true
d58696e0d184908a true true
6df571fa3a214eba true true
a1da883bb4e3a948 true true
b0ef194d3d5ef8bb true true
320c72cb7ee2ae1a true true
894a2d9de22117b6 true true
db640add605a0bc8 true true
8b169d1aa0abb052 true true
32b4294db3c4d524 true true
52f9bbc4953d5dfa true true
78c6b7e7e564fcd7 true true
b4ecb0d92d461d75 true true
e8e022a77dd5ada7 true true
e6ed70959bb662cb true true
50c2b5eb8868bd13 true true
52175718f387d6eb true true
4d355bed2c545be9 true true
aacd38ed01363f02 true true
6780a2527b21ee3e true true
2a78cd2d69c02ae8 true true
e5b5646d51559873 true true
35d3c2a531c322d0 true true
077b9049399aaa98 true false
4bba6266f72305f5 true true
16417821ee25de55 true true
1bec09e2e16f7cc3 true true
6bbd18ce17e5986d true true
502e928238067959 true true
3737cc3ecfbe6ca2 true true
9cfc5060745e9aa9 true true
c2f937b2342a6642 true true
68cbc4bda12a4a7e true true
8cdeff126949e1b0 true true
6f7a26f73e295737 true true
684fddbc7c9707f0 true true
6b6ac6a84d363f12 true true
51c387d270d38b7e true true
3f872a524cd61508 true true
f038c72528a1df99 true false
5a1ea105b83ecae5 true true
202829b7e6f699ae true true
96dbb5c2f804d46d true true
17e2048a36c46d46 true true
e0e624993f925a9d true true
0fde1e1be2b0b545 true false
0083929b7686b6e0 true false
58552d0ca5493875 true true
a21f1704e4fca5b0 true true
5ac9751deff0a7ab true true
bfe8935ebfacc60f true true
5cfd89e50355172c true true
7244810888b66e38 true true
27238b710c501291 true true
9d804214f56a61bc true true
f507f669c04bb4e3 true false
003563ab283b7029 true false
b5a4729918d491f5 true true
7fa18c44f9a07af3 true true
bbc3ed7ec12ac42d true true
9cbc1ebd3cbaa535 true true
aca89ca285938416 true true
7f3711df8a43f984 true true
218718fbf8de90de true true
b8702376952c14e6 true true
42b15223031736a5 true true
256174c7038f3def true true
03b45e17b80eaec9 true false
--> This article is in the "reading list" and doesn't not have the "read" label.
<entry gr:crawl-timestamp-msec="1255356559426">
<id gr:original-id="http://coedmagazine.com/?p=106802">
tag:google.com,2005:reader/item/024025978b5e50d2</id>
<category term="user/16051647003118153658/state/com.google/reading-list"
scheme="http://www.google.com/reader/" label="reading-list" />
<category term="user/16051647003118153658/label/problem"
scheme="http://www.google.com/reader/" label="problem" />
<category term="user/16051647003118153658/state/com.google/fresh"
scheme="http://www.google.com/reader/" label="fresh" />
<category term="Humor" />
<category term="Video" />
<category term="funny" />
<category term="humor" />
<category term="sarah-silverman" />
<category term="sell the vatican" />
<category term="the vatican" />
<category term="vatican" />
<title type="html">Sarah Silverman Has an Idea [Video]</title>
<published>2009-10-12T13:41:37Z</published>
<updated>2009-10-12T13:41:37Z</updated>
<link rel="alternate"
href="http://coedmagazine.com/2009/10/12/sarah-silverman-has-an-idea-video/"
type="text/html" />
<media:group>
<media:content url="http://1.gravatar.com/avatar/b2c9f9bb05c361e7b4c5e39def85a27c?s=96&d=http%3A%2F%2F1.gravatar.com%2Favatar%2Fad516503a11cd5ca435acc9bb6523536%3Fs%3D96" />
</media:group>
<media:group>
<media:content url="http://img.youtube.com/vi/3bObItmxAGc/2.jpg" />
</media:group>
<summary xml:base="http://coedmagazine.com/" type="html">I'm
not exactly sure what to make of Sarah Silverman's newest
video. On the one hand, it just kind of seems like the comedian
is trying desperately to hit that perfect note between
outrageous, and just cute enough to make everyone like her, in
order to have her own viral video. Either that, or she really
just wants to sell the Vatican to end world hunger. (Which is
not a bad idea, btw...)<img alt="" border="0"
src="http://stats.wordpress.com/b.gif?host=coedmagazine.com&amp;blog=3508545&amp;post=106802&amp;subd=coedmagazine&amp;ref=&amp;feed=1"></summary>
<author>
<name>Andrew - Hunter College</name>
</author>
<source gr:stream-id="feed/http://coedmagazine.com/feed/">
<id>
tag:google.com,2005:reader/feed/http://coedmagazine.com/feed/</id>
<title type="html">COED Magazine</title>
<link rel="alternate" href="http://coedmagazine.com"
type="text/html" />
</source>
</entry>
Mihai Parparita
Oct 13, 2009, 1:16:50 PM10/13/09
to foug...@googlegroups.com
Hi Mariano,
I think this might be a problem with how you're doing the base 10 <->
conversion. If you look at the IDs that are "missing":
024025978b5e50d2
fb115bd6d34a8e9f
0dbd56a577b364d9
f532dfdcea9f8aeb
f60ac17e5641edcc
0b08e0c198648e37
There's a pattern: either they have a leading 0, or they're bigger
than 2^63. When you go from base 10 to base 16, you need to 0-pad the
hex IDs so that they are always 16 digits wide. You also need to take
into account the fact that base 10 representations are signed while
base 16 ones are unsigned.
Mihai
Mariano Kamp
Oct 13, 2009, 3:32:54 PM10/13/09
to foug...@googlegroups.com
Mihai,
yes, you are right on both counts. Sorry for outsourcing my debugging to you ;-( I wasn't aware of any of the decimal <-> hex conversion things. Never used that before.
The issue was in the extracted Ruby code as well as in the actual Java code.
---
There is one tiny thing left. I got 570 unread articles in my reading-list and therefore asked for the unread ids with n=570. For the last four unread articles in my reading list I didn't get an unread id back. Any idea about that? What is the best number for n?
I then tried with 570*1.4=798, which returned:
<dl>
<dt>n</dt>
<dd>Too many decimal digits</dd>
</dl>
Then I set n manually to 650 and that worked. I got a response and that matched 100% with my unread articles.
Cheers,
Mariano
yes, you are right on both counts. Sorry for outsourcing my debugging to you ;-( I wasn't aware of any of the decimal <-> hex conversion things. Never used that before.
The issue was in the extracted Ruby code as well as in the actual Java code.
---
There is one tiny thing left. I got 570 unread articles in my reading-list and therefore asked for the unread ids with n=570. For the last four unread articles in my reading list I didn't get an unread id back. Any idea about that? What is the best number for n?
I then tried with 570*1.4=798, which returned:
<dl>
<dt>n</dt>
<dd>Too many decimal digits</dd>
</dl>
Then I set n manually to 650 and that worked. I got a response and that matched 100% with my unread articles.
Cheers,
Mariano
Mihai Parparita
Oct 13, 2009, 3:40:28 PM10/13/09
to foug...@googlegroups.com
On Tue, Oct 13, 2009 at 3:32 PM, Mariano Kamp <marian...@gmail.com> wrote:
> There is one tiny thing left. I got 570 unread articles in my reading-list
> and therefore asked for the unread ids with n=570. For the last four unread
> articles in my reading list I didn't get an unread id back. Any idea about
> that? What is the best number for n?
> There is one tiny thing left. I got 570 unread articles in my reading-list
> and therefore asked for the unread ids with n=570. For the last four unread
> articles in my reading list I didn't get an unread id back. Any idea about
> that? What is the best number for n?
Generally, for all /api/0/stream/item/ids you should use n=10000.
There's a known issue when n is set too low whereby untagged items are
not handled correctly if you have more than "n" (e.g. if a user marked
more than 570 articles as read and then kept unread in your case).
> I then tried with 570*1.4=798, which returned:
>
> <dl>
> <dt>n</dt>
> <dd>Too many decimal digits</dd>
> </dl>
Are you sure you passed in 798 exactly, and not something like 798.0?
The parameter only accepts integers.
Mihai
Mariano Kamp
Oct 13, 2009, 3:54:28 PM10/13/09
to foug...@googlegroups.com
Generally, for all /api/0/stream/item/ids you should use n=10000.
There's a known issue when n is set too low whereby untagged items are
not handled correctly if you have more than "n" (e.g. if a user marked
more than 570 articles as read and then kept unread in your case).
Not sure if I understand that.
Ok, but that means a serious amount of extra data, in particular for users that don't have many unread items.
You mentioned in an earlier post that &ot would work. I am not sure how the semantics are then? Should I also pass in the biggest crawl-timestamp-ms I have like for the reading list? To what would I then compare the result?
Any change we can get the read-state like the other states? ;-)
You mentioned in an earlier post that &ot would work. I am not sure how the semantics are then? Should I also pass in the biggest crawl-timestamp-ms I have like for the reading list? To what would I then compare the result?
Any change we can get the read-state like the other states? ;-)
Are you sure you passed in 798 exactly, and not something like 798.0?
The parameter only accepts integers.
Right ;( Now that you mention it must have been 798.0.
Mihai Parparita
Oct 13, 2009, 4:22:21 PM10/13/09
to foug...@googlegroups.com
On Tue, Oct 13, 2009 at 3:54 PM, Mariano Kamp <marian...@gmail.com> wrote:
> Ok, but that means a serious amount of extra data, in particular for users
> that don't have many unread items.
> Ok, but that means a serious amount of extra data, in particular for users
> that don't have many unread items.
Are you getting gzip-ed responses? That should cut down on the
response size significantly.
> You mentioned in an earlier post that &ot would work. I am not sure how the
> semantics are then? Should I also pass in the biggest crawl-timestamp-ms I
> have like for the reading list? To what would I then compare the result?
Unfortunately I don't think you can use &ot= here, since then you
wouldn't know about about old items that got marked as read or unread.
> Any change we can get the read-state like the other states? ;-)
Because of the additional complexity of the mark-all-as-read
high-water mark, I don't think we can simplify this any further.
Mihai
Nick Bradbury
Oct 13, 2009, 4:40:44 PM10/13/09
to Friends of the Unofficial Google Reader API
Mihai, in my case I'm *not* getting gzip-ed responses, even though I
send an "Accept-Encoding: gzip" header.
send an "Accept-Encoding: gzip" header.
Mihai Parparita
Oct 13, 2009, 4:48:02 PM10/13/09
to foug...@googlegroups.com
Ah, I'd forgotten that Google's common code has pretty strict rules
about when it sends gzipped responses, since even if a browser claims
to support it, it might have bugs (older versions of IE). You can
force it to give you gzip by having the string "gzip" somewhere in
your user agent (in addition to the Accept-Encoding header).
about when it sends gzipped responses, since even if a browser claims
to support it, it might have bugs (older versions of IE). You can
force it to give you gzip by having the string "gzip" somewhere in
your user agent (in addition to the Accept-Encoding header).
Mihai
Nick Bradbury
Oct 13, 2009, 5:02:20 PM10/13/09
to Friends of the Unofficial Google Reader API
> You can force it to give you gzip by having the string "gzip" somewhere
> in your user agent (in addition to the Accept-Encoding header).
That's definitely good to know - I'll make that change right away.
> in your user agent (in addition to the Accept-Encoding header).
Mariano Kamp
Oct 13, 2009, 5:13:10 PM10/13/09
to foug...@googlegroups.com
On Tue, Oct 13, 2009 at 3:54 PM, Mariano Kamp <marian...@gmail.com> wrote:Are you getting gzip-ed responses? That should cut down on the
> Ok, but that means a serious amount of extra data, in particular for users
> that don't have many unread items.
response size significantly.
Yes, I do and I believe I've seen 80% reduction, but after reading Nick's response I will double check that.
That would be the case for all API calls then, not only for this one in particular, right?
> You mentioned in an earlier post that &ot would work. I am not sure how theUnfortunately I don't think you can use &ot= here, since then you
> semantics are then? Should I also pass in the biggest crawl-timestamp-ms I
> have like for the reading list? To what would I then compare the result?
wouldn't know about about old items that got marked as read or unread.
Because of the additional complexity of the mark-all-as-read
> Any change we can get the read-state like the other states? ;-)
high-water mark, I don't think we can simplify this any further.
This is unfortunate, but ok, that's still better than not having incremental sync at all.
Maybe at some point in point you will get read of the special treatment of mark-all-all read and bite the O(n) problem, but have reduced complexity then and would allow users to mark an article as unread that was previously victim to mark-all-read.
One can dream.
Thanks for your help.
Nick Bradbury
Oct 13, 2009, 8:18:58 PM10/13/09
to Friends of the Unofficial Google Reader API
> On Oct 13, 4:13 pm, Mariano Kamp <mariano.k...@gmail.com> wrote:
> That [gzip] would be the case for all API calls then, not only for this one
> in particular, right?
Yes, this appears to be the case for all API calls. Once I added
"gzip" to FeedDemon's user agent string, all API responses were
compressed.
> That [gzip] would be the case for all API calls then, not only for this one
> in particular, right?
Yes, this appears to be the case for all API calls. Once I added
"gzip" to FeedDemon's user agent string, all API responses were
compressed.
Reply all
Reply to author
Forward
0 new messages