OpenElections update

[Derek Willis]

Hey all,

It has been awhile since we gave all of you an update on the project. This weekend we posted county-level results data from Missouri, and in the past few months we've added Louisiana (parish & precinct) and Virginia (precincts since 2005) as well. Our volunteers continue to help us gather results from Oregon, Kansas, Alabama, Tennessee and Washington state, while we work on ingesting county results from other states. One of our most prolific volunteers, Pete Huang, just wrote about his involvement:

http://peteh.me/2015/11/09/50-documents-converted-many-more-to-go/

As we look toward having data from 18-20 states in time for the 2016 election, here are the next states in the data processing pipeline:

1. South Carolina

2. Illinois

3. Colorado

4. New Mexico

5. Georgia

6. Ohio

For states that use Clarity election software (like SC & KY), we've developed a Python library, Clarify, that provides URLs to data files from that system. An updated version of Clarify will be released this week. If you're looking to plug in, we're currently revamping our volunteer documentation (https://github.com/openelections/docs/issues/29), and also looking for basic election metadata from OK, SD and MA. Don't hesitate to get in touch, or hit us up with any questions.

Thanks,

Derek & Serdar

[Michael Carroll]

Hi Derek,

Newbie questions:

I'm not clear on what you're calling "raw data" because some documentation suggests your raw data has already been cleaned up. The Arizona Secretary of State has election results in both PDF and CSV format. Do you want copies of them, with no processing at all?

I imagine your Clarify library takes CSV files as input, not PDFs, but that's just a guess. Can you clarify? (Couldn't resist that one, sorry.)

Documentation at the AZ SoS office says counties in AZ use Diebold and ES&S. I haven't seen those mentioned here. Is Clarity the only system you have a library for?

I wonder how many systems of election software are used in the US. Is there a list anywhere?

Thanks.

Mike Carroll 

[Derek Willis]

Hi Mike,

Good questions. What we call "raw data" are results that are loaded into our general schema but without the standardizations that are part of our full results spec (https://github.com/openelections/specs/wiki/Results-Data-Spec-Version-2). So, for example, we don't split candidates names and standardize them, or do the same for political parties. That's why we call it raw - it's in our database, but it's not yet standardized to our spec.

For Arizona, we have links to the PDF files for county results, so we don't need them. If results files are posted online, we generally don't need copies of them. But if we need to request them, we post them in state-specific repositories (Kansas is a good example: https://github.com/openelections/openelections-sources-ks).

Clarify only works with Clarity-based results systems, and we wrote it because Clarity has some unusual aspects to it (specifically, URLs to precinct-level data aren't easy to discover). I don't have a comprehensive list of election results software used in the U.S., but most other systems generate either electronic PDFs or text files that we can parse without having to write a specific library like Clarify.

Derek