meeting 2009-03-03
Kragen Javier Sitaker
here as if it were true, and someone isn’t sure it’s true, please post
to the list, and I’ll be more careful.
Reverse-engineered agenda:
* continuous deployment
* OCLC spam
* change-of-address FOIA
* QA
* ID generation bug
* the vision thing
* ETL
* handshakes
* proposal
* FEC data
* next tasks
Continuous Deployment
---------------------
Devi pushed several changes out this week, so we are now pushing
changes live several times a week, which should dramatically reduce
integration headaches.
OCLC Spam
---------
Devi found two hundred seven spam signatures on the OCLC petition.
For some reason they had not been sent to Congresspeople. We need a
strategy for spam and will discuss this on-list. Possibilities
discussed included deleting spam accounts, blacklisting spam accounts,
dspam (and similar content-based filtering), captchas, and petition
owner approval of “suspicious” signatures.
Change-of-address FOIA
----------------------
Christopher has been trying to get change-of-address records from the
Postal Service with a Freedom of Information Act Request, so far
without any response.
QA
--
Kragen reported that the site still has lots of problems; Simon
agreed. Kragen promised a fuller report. Devi has fixed a few
crashers and pushed the fixes live.
ID generation bug
-----------------
Devi found, and is fixing, some serious bugs in generating politician
IDs. At present, <http://dev.watchdog.net/p/duncan_hunter> is a
different person from <http://watchdog.net/p/duncan_hunter>, as a
result of one of these bugs.
The Vision Thing
----------------
Christopher has been working on his big vision and planning document
(“a somewhat colossal document”) and talking to other groups. He is
going to post his current draft to [the Wiki]
(http://watchdog.jottit.com/) to facilitate discussion. Apparently
Jottit is a little flaky but almost always works if you hit Reload.
Kragen has thoughts he hasn’t yet written in to the mailing list with.
We need to agree on a direction and it needs to be a direction that
inspires all of us, and hopefully lots of other people as well.
ETL
---
Kragen reported a bit on his reading about “Extract, Transform, and
Load” software. It seems that what we’ve built so far is something
very much like what is called a “data warehouse” in traditional
business data processing. There are concepts in the literature (and
perhaps in ETL software) that can help us out.
Handshakes
----------
Simon and Aaron had a really interesting phone call with an
investigative journalist who works for Sunlight, is excited about the
handshake seeker, and gave them lots of ideas.
Proposal
--------
We are overdue to put together a proposal to the Sunlight Foundation
asking that they fund the next stage of our work; we need to figure
out what to propose. One thought is to center this proposal on the
handshake seeker.
For this proposal and for future proposals, we need a work breakdown
structure of our planned projects with estimates, like the one Aaron
wrote for his [earmark seeker gameplan] [] post.
[earmark seeker gameplan]: http://groups.google.com/group/watchdog-devel/browse_thread/thread/799fdda2b9aa6be0?fwc=1
FEC data
--------
OpenSecrets might be making the FEC efiling data available in a
cleaned-up format.
Next Tasks
----------
Christopher will be—
* working on and sharing his vision document
* in particular writing to the mailing list about data visualization,
* and forging connections with other groups.
Kragen will be—
* fixing a bunch of small things this week,
* posting meeting minutes (today),
* checking out some lobbyist-disclosure parsing code (today),
* posting his thoughts on the vision thing,
* handling FEC filing amendments (Thursday),
* importing FEC expenditures into the database,
* profiling the FEC filing contribution data in preparation for
importing it,
* reading more about approaches to ETL, and
* writing about the state of the site and what he thinks we should be
doing next.
Devi will be—
* fixing the politicians whose IDs got msessed up,
* fixing the `gen_ids.py` problem that messed them up,
* working on draft mode for petitions (which she says is taking too
long),
* dealing with the spam problem, and
* helping Kragen push his small fixes live.
Becky will be—
* writing a list of interesting visualizations we could add to the
site, and
* illustrating the funding proposal.
Aaron will be—
* sending Devi some jottit code explaining a technique called
“spinning”
Simon did not commit to any tasks.