bouvard
unread,Aug 15, 2009, 9:12:57 PM8/15/09Sign in to reply to author
Sign in to forward
You do not have permission to delete messages in this group
Sign in to report message
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to votersdaily
I've just committed a language-agnostic scheduler to the master
repository. I have also migrated per-scraper configuration to a
separate 'config' file that resides in each scrapers directory. This
should make it theoretically possible to support a variety of
languages for scrapers with a minimum of hassle. However, this raises
a quasi-philosophically point that I would like to address: in
documenting the schema for the database rather than the API for the
Python module I invited individuals to use whatever language they felt
comfortable with, however, I can not support every language and I
won't be able to adequately move forward with things if I have try
to. At the same time, I absolutely do not want to fragment the
codebase. So, my thinking regarding languages is that I will accept
pull requests for scrapers written in any language which meets the
following criteria:
1) Can be executed from the command line,
2) without compilation,
3) and with tools readily available on all platforms.
4) Someone must have done the work of writing an EventScraper-
analogue that has reached feature-parity with the Python version.
I think this will prevent me from getting into a situation where I
can't adequately support the project, but still provide plenty of
freedom to accept contributions in PHP, Ruby, Perl, Python, or
whatever else people write their best code in. I'm just glad people
want to help! :-)
One final caveat: I will, _always_, prefer scrapers be written in
Python. If there are two equally valid implementations of the same
scraper in two different languages, I will commit the Python one to
master, followed in order of preference by Ruby.
Thanks!
Chris