new data posted

[Susi]

Hi folks,

At long last, I'm happy to announce that we've updated the 2012 campaign finance and PFD data available in our OpenData.  This release includes data through Q2 for most committees and candidates.  The PFD data includes the 2010 filings released last summer, 

The campaign finance data has all the FEC data structure changes that I've mentioned in past posts.  These structure changes only apply to the 2012 campaign finance bulk data, We will be bringing past cycles into line with this new structure as time permits.  The structure changes are listed in our documentation - both the Open Data Guide and the online table definitions, where we list both the 2012 structure and the previous cycle structure.  In summary:

All FECTransID/FECRecNo fields change from 7 to 19 characters

All City fields change from 18 characters to 30 characters

All organization-type fields change from 40 to 50 characters

Indivs.Contrib field changes from 34 to 50 characters

Indivs.FECOccEmp field is removed

Indivs.Occ_EF field is renamed to Occupation

Indivs.Emp_EF field is renamed to Employer

PACs.Amount change from int to float

Pac_Other.FECOccEmp change from 35 to 38 characters

Cands.FirstLastP change from 40 to 50 characters

We are working to provide a more ubiquitous CSV format for the campaign finance data, and hope to have that available for 2012 only later this week.  We will continue to provide and recommend the pipe delimited comma separated files.  

We would love to hear from all of you as to the use(s) you make of the data or any metrics that you could share, with the group or confidentially with CRP.  Funding continues to be a significant challenge for us, and any information you provide would be used to stengthen our pitch with funders.  As always, we would love to hear from anyone interested in joint grant funding opportunities.

If you have ideas, suggestions, compliments -- drop me a line at sal...@crp.org or post to the group. I'd love to hear from you.

Susi

[Matthew Skomarovsky]

Hi Susi, 

Thank you for maintaining this immeasurably valuable resource. Over here at LittleSis.org we've been matching our own entity records to your donation records using your old IDs for a few years, including a bunch of donations from one of your earlier releases of the 2012 cycle (about a year ago). Now I'm trying to import the most recent 2012 donations into our database, but with the new 19-character FECTransIDs I don't know how to determine whether we already have a particular donation record in our system, or whether it's a new one. I don't want to duplicate donation records, one with the old ID and one with the new ID, but I can't just wipe it clean and start over because we've already matched many of the old ones. Do you have any suggestions for how to map the new records to old ones?

Matthew

[Luke Rosiak]

Thank you for the alert and for all the hard work you do. I can't
believe the outsize influence you guys have despite limited resources.

A clarification. When you say 2012 is only through Q2, do you mean
that indivs donations through election day are still not available? Do
you have an estimate for when they will be?

Luke

> --
> You received this message because you are subscribed to the Google Groups
> "OpenSecrets Open Data" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/opensecrets-open-data/-/LVhQqgPGTO0J.
> To post to this group, send email to opensecret...@googlegroups.com.
> To unsubscribe from this group, send email to
> opensecrets-open...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/opensecrets-open-data?hl=en.

[Susi Alger]

Hello Matthew.  There is no good way to map old records to new ones.  You can try matching contrib, recipid, amount, date, type, city, state, zip, but then exclude those turning up duplicate matches and only use those that don't.  I'm afraid that the bad news is that fectransid, even when you have the new 19 character version in your system, will regularly be replaced due to the new system that the FEC has created. 

Susi

--
You received this message because you are subscribed to the Google Groups "OpenSecrets Open Data" group.

To unsubscribe from this group and stop receiving emails from it, send an email to opensecrets-open...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.