Fwd: [OpenCal] Hack session next Saturday (5/16)

[Peter Hall]

This will be the organized weekend activity! If you need more info, ask away (-:

---------- Forwarded message ----------
From: Asha John <ajmobi...@gmail.com>
Date: Wed, May 13, 2015 at 10:06 AM
Subject: Re: [OpenCal] Hack session next Saturday (5/16)
To: Ray Kiddy <r...@ganymede.org>
Cc: Sean Finney <sea...@gmail.com>, Peter Hall <pt....@gmail.com>, Tom Dooner <tomd...@gmail.com>, howard dyckoff <howa...@gmail.com>

Ray, 

Andrew, has already set up AWS for us.

My teammates Sean and Aaron have access. They are on the backend team.

If you can make it out to the next hack session, we'll get you access.

I'll talk to CfA folks and see if we can get something set up so that, we can include remote video collaboration.

That way anyone who can't make it out to the hack session can still be part of the hacksession.

thanks, 

Asha

On Tue, May 12, 2015 at 7:02 PM, howard dyckoff <howa...@gmail.com> wrote:

I don't quite remember the name at AWS, but there are management tools that can provide role-based access to instances and resources.  So we can have sub-domain admins, depending on our AWS structure.

On Tue, May 12, 2015 at 5:10 PM, Ray Kiddy <r...@ganymede.org> wrote:

On Tue, 12 May 2015 16:01:15 -0700
howard dyckoff <howa...@gmail.com> wrote:

> Asha mentioned that CfA was going to host this on AWS.
>
> IF that is so, then we can do PaaS or Iaas, which would mean using
> their DB as a service, RDS.  That might be easier.
>
>

Asha -

If it was decided that we are going to use the AWS services, can I get
in touch with Andrew and let him know that we want the credits?

As I was saying, we do not need any particular e-mail address or domain
to use this. Any gmail account, or any other random account, would work.
We would just have to make sure the proper people knew the password that
would work for that e-mail address so that they could log in to the AWS
console system.

- ray

>
> On Tue, May 12, 2015 at 2:26 PM, Ray Kiddy <r...@ganymede.org> wrote:
>
> > On Sat, 9 May 2015 20:03:40 -0700
> > Tom Dooner <tomd...@gmail.com> wrote:
> >
> > > Hi all,
> > >
> > > It was good seeing many of you this morning at the CFA HQ. We made
> > > some good progress getting on the same page about goals and
> > > desired architecture. For anyone that couldn't make it, you can
> > > relive it vicariously through the minutes
> > > <
> > https://docs.google.com/document/d/16iJ30sORg2hBW9mKNPXsMs8uS4XvmB9OzHD-qGx8_Rg/edit
> > >
> >
> > I was sorry to have missed the meeting.
> >
> > > We believe we have enough of an idea of where we want to go that
> > >
> > > *it's time to start building something together.*
> >
> > Defn!
> >
> > > *Next Saturday (May 16th),* will be the first (of hopefully many)
> > > hack sessions where we start building and designing where we're
> > > going from here. If you can't make it, fear not, there will be
> > > more opportunities. The agenda for this next week is simple:
> > > start working on the backend that will ETL the data from NetFile.
> >
> > I will look forward to being there.
> >
> > > Thanks to Peter and Code for SF, we will be having this hack day
> > > at the CFA HQ from *10am-2pm*.
> > >
> > > To review the architecture we're thinking about building (speak
> > > now or forever hold your peace):
> > >
> > >    - We are building a shared backend across multiple
> > > jurisdictions
> >
> > Are we going to start with the CCDC as a base? If we want to
> > generalize that project, adding modifiers for the particular
> > government entity, the CCDC people may be ok with that. Then their
> > coe would be just a single "CA" case, next to the "SJC", "SFC",
> > "OAK" and "SCT" cases.
> >
> > >       - First we will focus on pulling all data from the NetFile
> > > API <https://netfile.com/connect2>
> >
> > >       - Later we will focus on Cal-Access
> > >       <
> > http://www.sos.ca.gov/campaign-lobbying/cal-access-resources/raw-data-campaign-finance-and-lobbying-activity/
> > >
> >
> > >       - It is important to maintain a copy of the original source
> > > data
> > >    - This will help us audit any transformations we apply to the
> > > data (deduplication, associations, name cleanup, etc.)
> >
> > If you want to scope out the de-duplication task, see this table:
> >
> >
> > http://opencalaccess.org/calaccess_raw_20150509_000246_PDT/calaccess_campaign_browser_identity.sql.gz
> >
> > FYI, the table at
> >
> > http://opencalaccess.org/calaccess_raw_20150509_000246_PDT/calaccess_campaign_browser_identity_attribute.sql.gz
> > contains links from those names to the ID numbers assigned in the
> > DIME project (http://data.stanford.edu/dime). If anyone wants to
> > peruse it. I found matches in the Cal-Access data for 87,778 names
> > in the DIME data.
> >
> > >       - This will also help us perform subsequent updates
> > > incrementally, which will be a helpful performance benefit
> >
> > >    - Python is our *lingua franca* due to the team's general
> > > familiarity with it as well as the many available data processing
> > > tools like dedupe and the CCDC Cal-Access apps.
> >
> > >    - We will be using a relational data store, likely PostgreSQL.
> >
> > Technical merits aside, MySQL probably has more presence in the
> > database world. How strong is the preference here?
> >
> > >    - This will be a new repo under the opencalifornia github repo.
> >
> > > Again, the intent of the hack session is to make some actual
> > > progress towards this. We have a lot of inertia on the
> > > partnerships side
> >
> > Was that "inertia"? Or "momentum"? :-) We do not need more
> > inertia....
> >
> > > (props to John and Asha for your hard work on that), but now it's
> > > time for the programming rubber to meet the road of campaign
> > > finance! (Okay, so that was a weird way to say it, you get the
> > > idea.)
> > >
> > > If anyone has questions, or wants to discuss architectural
> > > details, here is a good place to do that.
> > > Best,
> > > Tom
> >
> > Look forward to doing some code!
> >
> > cheers - ray
> >
> >
> > --
> > OpenCal Discourse http://104.131.98.144/
> > ---
> > You received this message because you are subscribed to the Google
> > Groups "OpenCalifornia" group.
> > To unsubscribe from this group and stop receiving emails from it,
> > send an email to opencal+u...@googlegroups.com.
> > To post to this group, send email to ope...@googlegroups.com.
> > Visit this group at http://groups.google.com/group/opencal.
> > To view this discussion on the web visit
> > https://groups.google.com/d/msgid/opencal/20150512142634.4f037cf1%40ubu01.
> > For more options, visit https://groups.google.com/d/optout.
> >