Collection Limits
Doug Kaye
Max Programs per Collection = 1,000
Max Age of Program in Collection = 1 year
Some of you have more than 1,000 programs in a collection. This change
will cause the oldest programs (according to when they were added to
the collection) to be removed from those collections.
If this is a problem, please let me know ASAP. I'll probably make the
change Sunday night or Monday morning.
BTW, you can now specify the limits on a per-collection basis by
clicking Info link under Edit This Collection in the right column of
the collection's page.
Thanks.
...doug
Doug Kaye, Executive Director
The Conversations Network
A 501(c)(3) Non-Profit
do...@rds.com
v: 415.453.1400
twitter: dougkaye
facebook.com/doug.kaye
Thilo Planz
> I'm about to turn on collection limits with these default values:
> Max Programs per Collection = 1,000
> Max Age of Program in Collection = 1 year
What is the reasoning behind the one-year-limit?
This makes sense for queues, but for other kind of collections
not so much.
If you want to clear out inactive accounts, maybe deleting
collections after the owner has not been seen for a year would be
better.
Also, do these limits apply do external feeds, too?
I am hoping not, because Spokenword is providing a nice archive
of podcast back numbers, even if the original sources do not,
and even if the podcasts have long faded away.
(Of course, the linked programmes will then be not accessible anymore,
but still, all the meta-data is still interesting).
Thilo
Malc Dow
Data storage is not so much a problem these days I thought!
Regards,
Malc
2009/7/12 Doug Kaye <do...@rds.com>:
Doug Kaye
>> I'm about to turn on collection limits with these default values:
>> Max Programs per Collection = 1,000
>> Max Age of Program in Collection = 1 year
>
> What is the reasoning behind the one-year-limit?
It's the default. There's also a 'never' option for the time-based
removal.
> This makes sense for queues, but for other kind of collections
> not so much.
>
> If you want to clear out inactive accounts, maybe deleting
> collections after the owner has not been seen for a year would be
> better.
I'm not sure I agree. Some people might want to create a theme-based
collection and leave it there forever.
> Also, do these limits apply do external feeds, too?
No, this is for collections only.
On Jul 12, 2009, at 3:51 AM, Malc Dow wrote:
> I am curious, why limit at all? And why remove?!
> Data storage is not so much a problem these days I thought!
We have a number of "runaway" collections. They follow feeds or other
collections and grow at a great then are abandoned by the members.
Data storage may be cheap, but database storage isn't. As the size of
the database grows, there an increased cost for backups, maintenance
and performance.
Also, remember that RSS feeds are as large as your collection, so when
you have 1,00 items in a collection, you have 1,000 items in the RSS
feed. We have to generate that feed on demand and your software has to
parse it.
...doug
Doug Kaye
here's more info.
There are two reasons for the limits: (1) Our database resources.
Keeping a program in a collection requires just s few bytes in a
database table, but the problem is the auto-collect features. If you
subscribe to a feed or (even more so) follow another collection, the
number of programs in a collection can grow very rapidly. I think I've
successfully blocked recursive following, but it can still get out of
control. A lot of collections are created then abandoned, but they
continue to grow. (2) Your convenience. Many have asked for some
automatic aging, and when I thought about it, it appeared there were
two kinds of aging desired: time based, and count based.
At the moment (subject to your feedback) the choices are:
by count: 10, 25, 100, 250, 1000
by age: one week, one month, one year, never (keep forever)
Defaults are count=1,000 and one year
So let me know if you think that the choices or the defaults are
inappropriate and why.
Thanks again.
Malc Dow
>
>
> So let me know if you think that the choices or the defaults are
> inappropriate and why.
>
Makes sense to me!
Thanks for the info, and I can see now why 'Collection Limits' are an issue.
Malc
Ken Kennedy
have actual curated collections who don't subscribe to this list. Or
who don't check...I almost didn't see this thread, and I'm sure I have
collections w/ programs more than a year old.
Is the db issue so critical that you couldn't give a little more time,
and perhaps a soft launch of this limitation? Maybe an email generated
to people who will be affected or something? Give them a week to fix?
OTOH, if the db is about to break, I understand, trust me. *grin*
Ken Kennedy
Contact info: http://kenzoid.com/me/contact
Thilo Planz
> So let me know if you think that the choices or the defaults are
> inappropriate and why.
The 1000 items per collection is a reasonable maximum, but I still
think the one-year-limit could be a problem.
For the server, big collections could be an issue, but old collections?
> I'm not sure I agree. Some people might want to create a theme-based
> collection and leave it there forever.
And I therefore think that "keep forever" should be the default behaviour.
Of course, you can set it to forever manually, so this
is not really a problem.
(It could take a few people by surprise, though)
> Also, remember that RSS feeds are as large as your collection, so when
> you have 1,00 items in a collection, you have 1,000 items in the RSS
> feed. We have to generate that feed on demand and your software has to
> parse it.
Paging/trimming on the output side is indeed desirable.
I wonder if there is some standard for paged RSS.
Fifty items at first, and then links to get the rest if the user so chooses.
Thilo
Doug Kaye
hasn't been running that long. :-)
...doug
Doug Kaye
Delete old when >1,000 programs
Delete old when older than 'never'
Should now be implemented sometime after an email announcement to all
SpokenWord.org registered members.
Thanks for the feedback.
Malc Dow
>
> Not likely that anyone has a collection more than a year old. The site
> hasn't been running that long. :-)
>
longer perpective than a year!
http://www.archive.org/web/web.php
... has saved me a couple of times, web based projects from ten years
ago sitting on long gone servers...
Malc
Ken Kennedy
>> hasn't been running that long. :-)
>>
> But one trusts it will, and I think people will be looking at a far
> longer perpective than a year!
more than a year old". I guess it means "more than a year since the
program was added to the collection"?
Doug Kaye
>
>>> Not likely that anyone has a collection more than a year old. The
>>> site
>>> hasn't been running that long. :-)
>>>
>> But one trusts it will, and I think people will be looking at a far
>> longer perpective than a year!
>
> Plus, I misunderstood the measurement. I thought it meant a "program
> more than a year old". I guess it means "more than a year since the
> program was added to the collection"?
Correct.
...doug
Thilo Planz
is it possible to update the language flag of a programme?
Some are marked incorrectly.
Thilo