Number downloaded items and image problems

[Pavlin]

Hi,

I have two questions:
1. Is it possible to have more than 1000 items downoaded?
As it is not possible to sync only one or two folders, I have about
900 new items every day. But there are feeds I want to read every day,
others when I have more free time. Now I need to rush thru reading the
articles ...
As this is an user specified option this will not affect all of the
users - I use 8 GB SDcart, so I can have several thousand items
stored.

2. Every day or two I get a bunch of items with broken images. It will
just show the placehoder for the image and blue square with "?". It
will happen on, say, 20 - 50 items and then next ones will be fine.
When this happens i need to clear cache and redownload all of the
several houndred items again.

Thanks!

[Mariano Kamp]

Hi Pavlin.

Thanks for your feedback/questions.

1. Is it possible to have more than 1000 items downoaded?
As it is not possible to sync only one or two folders,

Yes, this is very annoying and given that the new version of Google listen just dumps its subscriptions in there too, this gets more annoying. 

I already tried to lobby Google to enhance the API to let 3rd party clients filter out feeds/labels, but was turned down. With the advent of the new Google Listen version I'll try to make them listen to me again, but don't hold your breath. In a way we can be lucky that they let use their internal API anyway and for their web interfaces they don't need this functionality.

 

I have about
900 new items every day. But there are feeds I want to read every day,
others when I have more free time. Now I need to rush thru reading the
articles ...

What sync type setting do you use? "unread only"

 

As this is an user specified option this will not affect all of the
users - I use 8 GB SDcart, so I can have several thousand items
stored.

In theory it would be ok to enable more than 1,000 articles, but it will slow down the GUI significantly, in particular on slower devices. I could probably add many more indices to the database, but this would slow down everybody and some operations will remain slow and it will result in many more ill tempered comments about NewsRob.

Feel free to add a suggestion about raising the limit to newsrob.uservoice.com. But I am really not looking forward to that, as it doesn't seem to be a solution, just a patch.

Have you thought about changing your workflow? E.g. starring the articles that you want to read later? 

FWIW I share those articles, it's just a small number in my case, in del.icio.us with a "toread" tag.

2. Every day or two I get a bunch of items with broken images. It will
just show the placehoder for the image and blue square with "?". It
will happen on, say, 20 - 50 items and then next ones will be fine.
When this happens i need to clear cache and redownload all of the
several houndred items again.

So this happens with the "feed" view? Or does it happen with the "mobile web page" download type setting?

Cheers,

Mariano

[Giles Antonio Radford]

On 20 February 2010 17:02, Mariano Kamp <marian...@gmail.com> wrote:

I have about
900 new items every day. But there are feeds I want to read every day,
others when I have more free time. Now I need to rush thru reading the
articles ...

What sync type setting do you use? "unread only"

 

As this is an user specified option this will not affect all of the
users - I use 8 GB SDcart, so I can have several thousand items
stored.

In theory it would be ok to enable more than 1,000 articles, but it will slow down the GUI significantly, in particular on slower devices. I could probably add many more indices to the database, but this would slow down everybody and some operations will remain slow and it will result in many more ill tempered comments about NewsRob.

Feel free to add a suggestion about raising the limit to newsrob.uservoice.com. But I am really not looking forward to that, as it doesn't seem to be a solution, just a patch.

Have you thought about changing your workflow? E.g. starring the articles that you want to read later? 

FWIW I share those articles, it's just a small number in my case, in del.icio.us with a "toread" tag.

I have a similar problem. I have a few really high traffic news feeds, that I tend only to dip into when I've got nothing better to do, things like The Register's news feed, or new articles on the Android Forums, that sort of thing.  The problem is that a few of these feeds can easily go up to about 300 odd posts a day. Personally, I don't read them all, but I'd like to at least have the headers available, even if I then have to download the article content. The big problem is that when you have two or three of these, it affects the ability to read the lesser-trafficked feeds because their articles are normally older than the high-trafficked ones, and occasionally, I lose them off the bottom of the reading queue.

Some solutions I could suggest here, in order to brainstorm:

• Manually or automatically be able to mark certain feeds as "high-traffic", and deal with their articles specially and outside of the normal store count
• Be able to limit the per-feed number of articles to cache, such that on these high-traffic feeds, I only get the freshest stuff, and it doesn't impact the lower-trafficked feeds
• Differentiate the number of cached headers form the number of cached articles. That way, my headers-only feeds don't impact how many articles I can store on the phone.

You may have to deal with them in the download stage, but at least it's possible to selectively choose what to cache.

I suppose, for high-traffic feeds, I'd like to choose backlog in terms of number of hours or days rather than articles. For android forums, I'd only be interested in the past 12 hours or so. For The Register, maybe the past three days or so.

For what it's worth, a lot of these I flick through, and star the ones I want to read fully later. 

Currently, I have to go into my "high traffic" folder and mark all as read, then initiate a new sync to be sure I have everything, and sometimes even then unread articles fall off the bottom of the list. I cannot reproduce how that happens, and have been unable to deduce a pattern there. The point is, I haven't read them, so marking them all as read is a misnomer. it's just I have no interest in them at present.

Hope that helps you visualise the problem a little more,

Moof

[Mariano Kamp]

Giles,

thanks for your contribution. Even if Google changes the API to allow feeds/labels to be excluded this feature, dealing with high-traffic feeds, might be useful. 

In terms of UI performance the headers are the issue here as those are stored in the database. The actual downloaded content of articles are stored on the sd card and are almost never (only during "Clear Cache") accessed without the fully qualified path name or putting it another way the number of articles is for all practical purposes, except aging out old or read articles, irrelevant.

Long story short, a differentiation in headers / articles doesn't help all that much.

Following up on your ideas I would like the following.

• Via "Manage Feed" a feed can be set manually to "High Traffic"
• This leads to more settings to be enabled:
• Article retention: 12/24/36/48/72 hours or forever
• Mark read after retention expired: yes/no (better name?)
• During every sync all expired articles are removed and possibly marked as read
• This retention period is ignored when an article is starred
  This might be a non-obvious feature that many people will not understand then
• Articles in high traffic feeds are ignored during NewsRob download phase, except if they are also starred

So the content is not downloaded during a sync for articles in a high traffic feed. If such an article is then starred though, the content will be downloaded during the next sync.

The solution is not ideal, but much better than the situation is today. A drawback is that the feeds that don't have proper feed content, i.e. you need the full page to make sense of the article, are slower to read as the content is not downloaded then and, of course, you need to have network access.

Also I'd like the solution to be more general in nature. I think a custom retention span would also be beneficial for other feeds too, but in the outline above the retention span & starred/download combo are coupled to the high traffic flag, which is nice in order to understand the feature, but also limiting in its application. Well, maybe one cannot have it all.

From an architectural point of view it is a bit uncool that feed management (automatically marking articles as read after a certain period) is done on the client. Given that I have no influence on Google and what they do, this seems to be the most pragmatic approach.

In parallel you could ask Google to add custom expirations/mark read for feeds to their service.

Last, but also least, this would make my implementation much more complex than it is today. But if enough people vote for it, I would still do that.

--
You received this message because you are subscribed to the Google Groups "NewsRob User Group" group.
To post to this group, send email to new...@googlegroups.com.
To unsubscribe from this group, send email to newsrob+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/newsrob?hl=en.

[Mariano Kamp]

Btw. just to prevent misunderstanding. I just outlined a solution that
I would be able to implement. To make this actually reality it still
needs to be tracked in uservoice and get some votes there.

On Feb 20, 6:31 pm, Mariano Kamp <mariano.k...@gmail.com> wrote:
> Giles,
>
> thanks for your contribution. Even if Google changes the API to allow
> feeds/labels to be excluded this feature, dealing with high-traffic feeds,
> might be useful.
>
> In terms of UI performance the headers are the issue here as those are
> stored in the database. The actual downloaded content of articles are stored
> on the sd card and are almost never (only during "Clear Cache") accessed
> without the fully qualified path name or putting it another way the number
> of articles is for all practical purposes, except aging out old or read
> articles, irrelevant.
> Long story short, a differentiation in headers / articles doesn't help all
> that much.
>
> Following up on your ideas I would like the following.
>

>    - Via "Manage Feed" a feed can be set manually to "High Traffic"
>    - This leads to more settings to be enabled:
>       - Article retention: 12/24/36/48/72 hours or forever
>       - Mark read after retention expired: yes/no (better name?)
>    - During every sync all expired articles are removed and possibly marked
>    as read
>    - This retention period is ignored when an article is starred

>    This might be a non-obvious feature that many people will not understand
>    then

>    - Articles in high traffic feeds are ignored during NewsRob download

>    phase, except if they are also starred
>
> So the content is not downloaded during a sync for articles in a high
> traffic feed. If such an article is then starred though, the content will be
> downloaded during the next sync.
>
> The solution is not ideal, but much better than the situation is today. A
> drawback is that the feeds that don't have proper feed content, i.e. you
> need the full page to make sense of the article, are slower to read as the
> content is not downloaded then and, of course, you need to have network
> access.
>
> Also I'd like the solution to be more general in nature. I think a custom
> retention span would also be beneficial for other feeds too, but in the
> outline above the retention span & starred/download combo are coupled to the
> high traffic flag, which is nice in order to understand the feature, but
> also limiting in its application. Well, maybe one cannot have it all.
>
> From an architectural point of view it is a bit uncool that feed management
> (automatically marking articles as read after a certain period) is done on
> the client. Given that I have no influence on Google and what they do, this
> seems to be the most pragmatic approach.
> In parallel you could ask Google to add custom expirations/mark read for
> feeds to their service.
>
> Last, but also least, this would make my implementation much more complex
> than it is today. But if enough people vote for it, I would still do that.
>

> On Sat, Feb 20, 2010 at 5:36 PM, Giles Antonio Radford <m...@metamoof.net>wrote:

> >    - Manually or automatically be able to mark certain feeds as

> >    "high-traffic", and deal with their articles specially and outside of the
> >    normal store count

> >    - Be able to limit the per-feed number of articles to cache, such that

> >    on these high-traffic feeds, I only get the freshest stuff, and it doesn't
> >    impact the lower-trafficked feeds

> >    - Differentiate the number of cached headers form the number of cached

> >    articles. That way, my headers-only feeds don't impact how many articles I
> >    can store on the phone.
>
> > You may have to deal with them in the download stage, but at least it's
> > possible to selectively choose what to cache.
>
> > I suppose, for high-traffic feeds, I'd like to choose backlog in terms of
> > number of hours or days rather than articles. For android forums, I'd only
> > be interested in the past 12 hours or so. For The Register, maybe the past
> > three days or so.
>
> > For what it's worth, a lot of these I flick through, and star the ones I
> > want to read fully later.
>
> > Currently, I have to go into my "high traffic" folder and mark all as read,
> > then initiate a new sync to be sure I have everything, and sometimes even
> > then unread articles fall off the bottom of the list. I cannot reproduce how
> > that happens, and have been unable to deduce a pattern there. The point is,
> > I haven't read them, so marking them all as read is a misnomer. it's just I
> > have no interest in them at present.
>
> > Hope that helps you visualise the problem a little more,
>
> > Moof
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "NewsRob User Group" group.
> > To post to this group, send email to new...@googlegroups.com.
> > To unsubscribe from this group, send email to

> > newsrob+u...@googlegroups.com<newsrob%2Bunsu...@googlegroups.com >

[Mariano Kamp]

A simplified version of this has been submitted as a suggestion:
http://newsrob.uservoice.com/admin2/forums/35624-general/suggestions/685746-stop-noisy-feeds-drowning-out-quiet-feeds

Short summary: In the future you should be able to specify a maximum
number of articles on an individual feed basis.

> >    - Articles inhightrafficfeeds are ignored during NewsRob download

> >    phase, except if they are also starred
>
> > So the content is not downloaded during a sync for articles in ahigh

> >trafficfeed. If such an article is then starred though, the content will be

> > downloaded during the next sync.
>
> > The solution is not ideal, but much better than the situation is today. A
> > drawback is that the feeds that don't have proper feed content, i.e. you
> > need the full page to make sense of the article, are slower to read as the
> > content is not downloaded then and, of course, you need to have network
> > access.
>
> > Also I'd like the solution to be more general in nature. I think a custom
> > retention span would also be beneficial for other feeds too, but in the
> > outline above the retention span & starred/download combo are coupled to the

> >hightrafficflag, which is nice in order to understand the feature, but

> > > I have a similar problem. I have a few reallyhightrafficnews feeds, that

> > > I tend only to dip into when I've got nothing better to do, things like The
> > > Register's news feed, or new articles on the Android Forums, that sort of
> > > thing.  The problem is that a few of these feeds can easily go up to about
> > > 300 odd posts a day. Personally, I don't read them all, but I'd like to at
> > > least have the headers available, even if I then have to download the
> > > article content. The big problem is that when you have two or three of
> > > these, it affects the ability to read the lesser-trafficked feeds because

> > > their articles are normally older than thehigh-trafficked ones, and

> > > occasionally, I lose them off the bottom of the reading queue.
>
> > > Some solutions I could suggest here, in order to brainstorm:
>
> > >    - Manually or automatically be able to mark certain feeds as
> > >    "high-traffic", and deal with their articles specially and outside of the
> > >    normal store count
> > >    - Be able to limit the per-feed number of articles to cache, such that

> > >    on thesehigh-trafficfeeds, I only get the freshest stuff, and it doesn't

> > >    impact the lower-trafficked feeds
> > >    - Differentiate the number of cached headers form the number of cached
> > >    articles. That way, my headers-only feeds don't impact how many articles I
> > >    can store on the phone.
>
> > > You may have to deal with them in the download stage, but at least it's
> > > possible to selectively choose what to cache.
>

> > > I suppose, forhigh-trafficfeeds, I'd like to choose backlog in terms of

> > > number of hours or days rather than articles. For android forums, I'd only
> > > be interested in the past 12 hours or so. For The Register, maybe the past
> > > three days or so.
>
> > > For what it's worth, a lot of these I flick through, and star the ones I
> > > want to read fully later.
>

> > > Currently, I have to go into my "hightraffic" folder and mark all as read,

> > > then initiate a new sync to be sure I have everything, and sometimes even
> > > then unread articles fall off the bottom of the list. I cannot reproduce how
> > > that happens, and have been unable to deduce a pattern there. The point is,
> > > I haven't read them, so marking them all as read is a misnomer. it's just I
> > > have no interest in them at present.
>
> > > Hope that helps you visualise the problem a little more,
>
> > > Moof
>
> > > --
> > > You received this message because you are subscribed to the Google Groups
> > > "NewsRob User Group" group.
> > > To post to this group, send email to new...@googlegroups.com.
> > > To unsubscribe from this group, send email to
> > > newsrob+u...@googlegroups.com<newsrob%2Bunsu...@googlegroups.com >
> > > .
> > > For more options, visit this group at
> > >http://groups.google.com/group/newsrob?hl=en.

--
You received this message because you are subscribed to the Google Groups "NewsRob User Group" group.
To post to this group, send email to new...@googlegroups.com.

To unsubscribe from this group, send email to newsrob+u...@googlegroups.com.