extending openstates data model?

[Joe Germuska]

Hello.

I'm getting back into some work with the IL open states code, and I'm thinking about some things I could pull out which might be useful. I was wondering what the overall approach to this kind of extension might be.

Since I've been experimenting, I've thought I might want to add references to the statutes impacted by the bills (or at least those which become law) and I've thought that there might be a few new action categories (I'd have to review to recall what they were...)

those are two different kinds of extensions, but in either case... are there any precedents to follow?

Joe

-- 

Joe Germuska

J...@Germuska.com * http://blog.germuska.com * http://twitter.com/JoeGermuska    

"Participation. That's what's gonna save the human race." --Pete Seeger

[James Turk]

Hi Joe,

First let me say again that I really appreciate the work you've done
recently on IL.

The statute idea is one thas has been suggested a few times and if
you're moving on it in IL I'm happy to make it an officially supported
(albeit optional) attribute.

As of now the scraper can collect any additional info (just treat the
Bill object like a python dict), but if it gets a field it doesn't
recognize it appends a + to it in the API.

I'd be happy to talk through a format/name for the new field, but
something like related_code = ['720 ILCS 105', ...] makes sense to me.

As far as additional action_types, those are a bit more complicated as
the validator will reject invalid ones right now. If you can look
over it and let me know which ones you'd like to add I can see if
they'd make sense to add universally or if we might need an extension
mechanism, perhaps something like il:specific_type.

-James

> --
> You received this message because you are subscribed to the Google Groups
> "Open State Project" group.
> To post to this group, send email to fifty-sta...@googlegroups.com.
> To unsubscribe from this group, send email to
> fifty-state-pro...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/fifty-state-project?hl=en.
>

[Joe Germuska]

On Nov 17, 2011, at 3:16 PM, James Turk wrote:
> First let me say again that I really appreciate the work you've done
> recently on IL.

Happy to help! After all, you know the NewsApps origin story and all...

> The statute idea is one thas has been suggested a few times and if
> you're moving on it in IL I'm happy to make it an officially supported
> (albeit optional) attribute.
>
> As of now the scraper can collect any additional info (just treat the
> Bill object like a python dict), but if it gets a field it doesn't
> recognize it appends a + to it in the API.
>
> I'd be happy to talk through a format/name for the new field, but
> something like related_code = ['720 ILCS 105', ...] makes sense to me.

I think that's what I'd do. There are some (omnibus-type) bills with thousands of them, but that's life. Is "statute" one of those words that only some states use? I totally defer to those with more comparative experience. For my purposes, I could have the statute/code and section in most cases, but I figure that the statute is the most valuable part.

> As far as additional action_types, those are a bit more complicated as
> the validator will reject invalid ones right now. If you can look
> over it and let me know which ones you'd like to add I can see if
> they'd make sense to add universally or if we might need an extension
> mechanism, perhaps something like il:specific_type.

So I audited everything which came up tagged 'other', moved a few into currently existing categories, and came up with a couple of dozen more which account for thousands of unclassified actions.

Writ large, they mostly have to do with changes in sponsorship or the life cycle of amendments. I'm not really sure I see the value in tracking that minutely. One other which I can extract from bills which become law is "effective date", which is maybe interesting, perhaps as it's own key even though in IL it turns up in the list of actions.

Joe

--
Joe Germuska
J...@Germuska.com * http://blog.germuska.com * http://twitter.com/JoeGermuska

"Science's job is to map our ignorance." --David Byrne

[James Turk]

I'm not sure about code vs. statute, perhaps someone else on this list
can chime in? I'll also look at a sample of states and see what is
usually provided.

effective_date as a new key is a good idea too, it does tend to show
up in actions but that isn't really the best place for it

I've considered adding something like sponsor:added, sponsor:removed
as those are pretty common across states but I have never been sure of
the utility of them.

-James

[Joe Germuska]

Further on this: my thought was to only list 'related_code' for bills which become law, but that is probably something which should be agreed upon.

I guess there is probably interest in bills which fail to change a code. What I'm concerned about is the possibility that something would be in one version of a bill and not in a later one. I expect to extract them from the full text. I could just use the last version of the bill.

Joe

--
Joe Germuska
J...@Germuska.com * http://blog.germuska.com * http://twitter.com/JoeGermuska

"Participation. That's what's gonna save the human race." --Pete Seeger

[James Turk]

Good point,

I hadn't thought about states where it requires looking at the text
itself to determine what is changed (a handful of states include this
as a field somewhere in their LIS site, I had figured that IL was
one).

We're only starting to do things that analyze the actual text of the
bill, but I imagine down the line this can be part of that process.
Don't worry about that for now, but it may wind up refactored to a
second post-scrape step at some point.

I think the latest version would make sense, though a more complex but
complete option would be to associate the field with versions instead
of top level bill objects.

-James

[twneale]

As far as I know, "statute" is valid lingo for specific sections of
code in every state. It can also be used to refer to uncodified
sections of law as well (for example, the US Statutes at Large).

For the key name in the bill object, it might be desirable to use a
really generic term like 'related_citation'. In the example Joe gave,
720 ILCS 105 refers to the former Abandoned Child Prevention Act (a
sequence of statutes) in the Illinois Compiled Statutes (a code). So
arguably neither "statute" nor "code" fits quite right in this case.
Plus, in at least one jurisdiction (New York), the legislature also
frequently amends previous session laws. For example, they might
extend the sunset date of a particular statute by amending the
original expiration date given in the session law that added the
statute. So to encompass these three types of things (statute, code,
session law), some kind of generic name would be necessary.

New York could also provide some challenges for 'effective_date' too.
Some bills don't have a single effective date, but instead have a
paragraph like "Section (1) of this act shall take effect Jan 1, 2011.
Section (2) of this act shall take effect 30 days after blah blah..."
So to accommodate all this weirdness, there might also need to be an
alternative key for effective_date_text or similar.

Thom

> > J...@Germuska.com *http://blog.germuska.com*http://twitter.com/JoeGermuska

[Gregory Combs]

In Texas, they have Companion Bills/Legislation in the XML bill info, but I don't think they've populated that field yet.

I'm hoping they fill in some more info next session. I think I've convinced them to include the various stages of a given bill in the legislative process. That's one feature that I wish every state provided us.

Greg