I'm getting back into some work with the IL open states code, and I'm thinking about some things I could pull out which might be useful. I was wondering what the overall approach to this kind of extension might be.
Since I've been experimenting, I've thought I might want to add references to the statutes impacted by the bills (or at least those which become law) and I've thought that there might be a few new action categories (I'd have to review to recall what they were...)
those are two different kinds of extensions, but in either case... are there any precedents to follow?
First let me say again that I really appreciate the work you've done recently on IL.
The statute idea is one thas has been suggested a few times and if you're moving on it in IL I'm happy to make it an officially supported (albeit optional) attribute.
As of now the scraper can collect any additional info (just treat the Bill object like a python dict), but if it gets a field it doesn't recognize it appends a + to it in the API.
I'd be happy to talk through a format/name for the new field, but something like related_code = ['720 ILCS 105', ...] makes sense to me.
As far as additional action_types, those are a bit more complicated as the validator will reject invalid ones right now. If you can look over it and let me know which ones you'd like to add I can see if they'd make sense to add universally or if we might need an extension mechanism, perhaps something like il:specific_type.
On Thu, Nov 17, 2011 at 3:58 PM, Joe Germuska <j...@germuska.com> wrote: > Hello. > I'm getting back into some work with the IL open states code, and I'm > thinking about some things I could pull out which might be useful. I was > wondering what the overall approach to this kind of extension might be. > Since I've been experimenting, I've thought I might want to add references > to the statutes impacted by the bills (or at least those which become law) > and I've thought that there might be a few new action categories (I'd have > to review to recall what they were...) > those are two different kinds of extensions, but in either case... are there > any precedents to follow? > Joe > -- > Joe Germuska > J...@Germuska.com * http://blog.germuska.com * > http://twitter.com/JoeGermuska > "Participation. That's what's gonna save the human race." --Pete Seeger
> -- > You received this message because you are subscribed to the Google Groups > "Open State Project" group. > To post to this group, send email to fifty-state-project@googlegroups.com. > To unsubscribe from this group, send email to > fifty-state-project+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/fifty-state-project?hl=en.
> First let me say again that I really appreciate the work you've done > recently on IL.
Happy to help! After all, you know the NewsApps origin story and all...
> The statute idea is one thas has been suggested a few times and if > you're moving on it in IL I'm happy to make it an officially supported > (albeit optional) attribute.
> As of now the scraper can collect any additional info (just treat the > Bill object like a python dict), but if it gets a field it doesn't > recognize it appends a + to it in the API.
> I'd be happy to talk through a format/name for the new field, but > something like related_code = ['720 ILCS 105', ...] makes sense to me.
I think that's what I'd do. There are some (omnibus-type) bills with thousands of them, but that's life. Is "statute" one of those words that only some states use? I totally defer to those with more comparative experience. For my purposes, I could have the statute/code and section in most cases, but I figure that the statute is the most valuable part.
> As far as additional action_types, those are a bit more complicated as > the validator will reject invalid ones right now. If you can look > over it and let me know which ones you'd like to add I can see if > they'd make sense to add universally or if we might need an extension > mechanism, perhaps something like il:specific_type.
So I audited everything which came up tagged 'other', moved a few into currently existing categories, and came up with a couple of dozen more which account for thousands of unclassified actions.
Writ large, they mostly have to do with changes in sponsorship or the life cycle of amendments. I'm not really sure I see the value in tracking that minutely. One other which I can extract from bills which become law is "effective date", which is maybe interesting, perhaps as it's own key even though in IL it turns up in the list of actions.
I'm not sure about code vs. statute, perhaps someone else on this list can chime in? I'll also look at a sample of states and see what is usually provided.
effective_date as a new key is a good idea too, it does tend to show up in actions but that isn't really the best place for it
I've considered adding something like sponsor:added, sponsor:removed as those are pretty common across states but I have never been sure of the utility of them.
On Thu, Nov 17, 2011 at 4:38 PM, Joe Germuska <j...@germuska.com> wrote: > On Nov 17, 2011, at 3:16 PM, James Turk wrote: >> First let me say again that I really appreciate the work you've done >> recently on IL.
> Happy to help! After all, you know the NewsApps origin story and all...
>> The statute idea is one thas has been suggested a few times and if >> you're moving on it in IL I'm happy to make it an officially supported >> (albeit optional) attribute.
>> As of now the scraper can collect any additional info (just treat the >> Bill object like a python dict), but if it gets a field it doesn't >> recognize it appends a + to it in the API.
>> I'd be happy to talk through a format/name for the new field, but >> something like related_code = ['720 ILCS 105', ...] makes sense to me.
> I think that's what I'd do. There are some (omnibus-type) bills with thousands of them, but that's life. Is "statute" one of those words that only some states use? I totally defer to those with more comparative experience. For my purposes, I could have the statute/code and section in most cases, but I figure that the statute is the most valuable part.
>> As far as additional action_types, those are a bit more complicated as >> the validator will reject invalid ones right now. If you can look >> over it and let me know which ones you'd like to add I can see if >> they'd make sense to add universally or if we might need an extension >> mechanism, perhaps something like il:specific_type.
> So I audited everything which came up tagged 'other', moved a few into currently existing categories, and came up with a couple of dozen more which account for thousands of unclassified actions.
> Writ large, they mostly have to do with changes in sponsorship or the life cycle of amendments. I'm not really sure I see the value in tracking that minutely. One other which I can extract from bills which become law is "effective date", which is maybe interesting, perhaps as it's own key even though in IL it turns up in the list of actions.
> "Science's job is to map our ignorance." --David Byrne
> -- > You received this message because you are subscribed to the Google Groups "Open State Project" group. > To post to this group, send email to fifty-state-project@googlegroups.com. > To unsubscribe from this group, send email to fifty-state-project+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/fifty-state-project?hl=en.
Further on this: my thought was to only list 'related_code' for bills which become law, but that is probably something which should be agreed upon.
I guess there is probably interest in bills which fail to change a code. What I'm concerned about is the possibility that something would be in one version of a bill and not in a later one. I expect to extract them from the full text. I could just use the last version of the bill.
I hadn't thought about states where it requires looking at the text itself to determine what is changed (a handful of states include this as a field somewhere in their LIS site, I had figured that IL was one).
We're only starting to do things that analyze the actual text of the bill, but I imagine down the line this can be part of that process. Don't worry about that for now, but it may wind up refactored to a second post-scrape step at some point.
I think the latest version would make sense, though a more complex but complete option would be to associate the field with versions instead of top level bill objects.
On Thu, Nov 17, 2011 at 5:43 PM, Joe Germuska <j...@germuska.com> wrote: > Further on this: my thought was to only list 'related_code' for bills which become law, but that is probably something which should be agreed upon.
> I guess there is probably interest in bills which fail to change a code. What I'm concerned about is the possibility that something would be in one version of a bill and not in a later one. I expect to extract them from the full text. I could just use the last version of the bill.
> "Participation. That's what's gonna save the human race." --Pete Seeger
> -- > You received this message because you are subscribed to the Google Groups "Open State Project" group. > To post to this group, send email to fifty-state-project@googlegroups.com. > To unsubscribe from this group, send email to fifty-state-project+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/fifty-state-project?hl=en.
As far as I know, "statute" is valid lingo for specific sections of code in every state. It can also be used to refer to uncodified sections of law as well (for example, the US Statutes at Large).
For the key name in the bill object, it might be desirable to use a really generic term like 'related_citation'. In the example Joe gave, 720 ILCS 105 refers to the former Abandoned Child Prevention Act (a sequence of statutes) in the Illinois Compiled Statutes (a code). So arguably neither "statute" nor "code" fits quite right in this case. Plus, in at least one jurisdiction (New York), the legislature also frequently amends previous session laws. For example, they might extend the sunset date of a particular statute by amending the original expiration date given in the session law that added the statute. So to encompass these three types of things (statute, code, session law), some kind of generic name would be necessary.
New York could also provide some challenges for 'effective_date' too. Some bills don't have a single effective date, but instead have a paragraph like "Section (1) of this act shall take effect Jan 1, 2011. Section (2) of this act shall take effect 30 days after blah blah..." So to accommodate all this weirdness, there might also need to be an alternative key for effective_date_text or similar.
Thom
On Nov 17, 4:42 pm, James Turk <james.p.t...@gmail.com> wrote:
> I'm not sure about code vs. statute, perhaps someone else on this list > can chime in? I'll also look at a sample of states and see what is > usually provided.
> effective_date as a new key is a good idea too, it does tend to show > up in actions but that isn't really the best place for it
> I've considered adding something like sponsor:added, sponsor:removed > as those are pretty common across states but I have never been sure of > the utility of them.
> -James
> On Thu, Nov 17, 2011 at 4:38 PM, Joe Germuska <j...@germuska.com> wrote: > > On Nov 17, 2011, at 3:16 PM, James Turk wrote: > >> First let me say again that I really appreciate the work you've done > >> recently on IL.
> > Happy to help! After all, you know the NewsApps origin story and all...
> >> The statute idea is one thas has been suggested a few times and if > >> you're moving on it in IL I'm happy to make it an officially supported > >> (albeit optional) attribute.
> >> As of now the scraper can collect any additional info (just treat the > >> Bill object like a python dict), but if it gets a field it doesn't > >> recognize it appends a + to it in the API.
> >> I'd be happy to talk through a format/name for the new field, but > >> something like related_code = ['720 ILCS 105', ...] makes sense to me.
> > I think that's what I'd do. There are some (omnibus-type) bills with thousands of them, but that's life. Is "statute" one of those words that only some states use? I totally defer to those with more comparative experience. For my purposes, I could have the statute/code and section in most cases, but I figure that the statute is the most valuable part.
> >> As far as additional action_types, those are a bit more complicated as > >> the validator will reject invalid ones right now. If you can look > >> over it and let me know which ones you'd like to add I can see if > >> they'd make sense to add universally or if we might need an extension > >> mechanism, perhaps something like il:specific_type.
> > So I audited everything which came up tagged 'other', moved a few into currently existing categories, and came up with a couple of dozen more which account for thousands of unclassified actions.
> > Writ large, they mostly have to do with changes in sponsorship or the life cycle of amendments. I'm not really sure I see the value in tracking that minutely. One other which I can extract from bills which become law is "effective date", which is maybe interesting, perhaps as it's own key even though in IL it turns up in the list of actions.
> > "Science's job is to map our ignorance." --David Byrne
> > -- > > You received this message because you are subscribed to the Google Groups "Open State Project" group. > > To post to this group, send email to fifty-state-project@googlegroups.com. > > To unsubscribe from this group, send email to fifty-state-project+unsubscribe@googlegroups.com. > > For more options, visit this group athttp://groups.google.com/group/fifty-state-project?hl=en.
In Texas, they have Companion Bills/Legislation in the XML bill info, but I don't think they've populated that field yet.
I'm hoping they fill in some more info next session. I think I've convinced them to include the various stages of a given bill in the legislative process. That's one feature that I wish every state provided us.
