CA actions have many errors and omissions

[Mike Hedblom]

Hello, I have been using OpenStates API for about a year now and first let me say a BIG THANK YOU to all who have worked on this.

There several errors in the mappings of CA actions to action-categories. There are also several omissions. I have been looking at ca/actions.py and using its rules to see how they map.  I can basically read the file and make some suggested changes, but other changes are more than I know how to define with code.  Though I do have examples of how the actions should be mapped.

How do I propose the changes?  Thank you. -- Mike

[Andy Lo]

Hi Mike,

Thanks for your interest in Open States!  You may propose the changes however you feel comfortable, including just posting your recommendations here on the Google Group.  If you'd like to become more regularly involved, you can sign up for a Github account (if you don't have one already) and fork the Open States project.  If you're unsure of what to code, you can write some Python comments defining each of the mapping rules you'd like to add and submit the changes as a pull request, which I can then transcribe.

If you have any other questions, please feel free to ask.

[Mike Hedblom]

Hi Andy, I am not comfortable (yet) with working with the source. I will look into it for the future. I'm especially not sure about the regex and other mappings in the /ca/actions.py file. For now, I'll list my suggested changes here.

California has about 255 different Action Descriptions that OpenStates is scrapping and interpreting.  I've encountered several errors in the mappings to Action Types. Some of them due to punctuation. I manually checked these suggestions against some approximately 60,000 records in the Calif Bill History table, and these all appear (to me) to check out.

I also found a few actions that don't map well to the existing Action types, and so I am suggesting some new Action types to better describe them.

These are my corrected mappings:

'Adopted and to','bill:passed,
'Adopted,.','amendment:passed'                # period
'amend\,','amendment:introduced'                # coma
'amended','amendment:introduced
'amended\, adopted','amendment:passed'        # coma
'approved by the Governor','governor:signed,
'be adopted','committee:passed:favorable'
'died','bill:failed'
'do pass','amendment:passed'
'enrolled and filed','bill:passed'
'failed passage','committee:failed'
'from committee','committee:passed'
'introduced','bill:introduced'                # 'introduced' not followed by 'measure'
'introduced measure','other'                # 'introduced' followed by 'measure'
'Passed\.','bill:passed'                        # period
'presented to the Governor','governor:received'
'Read first time','bill:reading:1'
'Read second time','bill:reading:2'
'Read third time','bill:reading:3'
'Read\.','bill:reading:1'                    # period'
'referred to','committee:referred'
'refused adoption','amendment:failed'
'refused passage','bill:failed'
're-refer to','committee:referred'
're-referred to','committee:referred'
'returned to Secretary of Senate','bill:failed'
'tabled','amendment:tabled'
'to Assembly','bill:passed'
'to com\.','committee:referred'                # period
'to Engrossing and Enrolling','bill:passed'
'to Senate','bill:passed'
'to the Assembly','bill:passed'
'to the Senate','bill:passed'
'veto stricken from file','bill:veto_override:failed'
'vetoed','governor:vetoed'
'with author\'s amendments','amendment:passed'    # apostrophe
'with item veto','governor:vetoed:line-item'
'without further action','committee:failed'


These are the corrected mappings for the joint conference committee using the existing Action types. I also included suggested new Action types (concurrence:) to better describe the activity:

'adopted Conference Committee','bill:passed'            # 'bill:passed'
'adopts Conference Committee','bill:passed'                # 'bill:passed'
'amendments concurred in','committee:passed:favorable'    # 'concurrence:passed'
'from Conference Committee','committee:passed:favorable'    # 'concurrence:passed'
'refused to concur','committee:failed'                    # 'concurrence:failed'
'to Conference Committee','committee:referred'            # 'concurrence:referred'


These are two additional suggested Action types to identify when a bill is pulled from committee or from the governor

'returned by the Governor','governor:withdrawn'
'withdrawn from committee','committee:withdrawn'


These are suggestions for new Action type for "in committee" hearings:

'first hearing','committee:hearing:set'
'hearing canceled','committee:hearing:canceled'
'hearing postponed','committee:hearing:postponed'
'hearing:','committee:hearing:set'
'in committee: hearing','committee:hearing:set'
'in committee: set\,','committee:hearing:set'        # coma
'set for hearing','committee:hearing:set'

[Andy Lo]

Hi Mike,

Just getting to this after returning from vacation.  I'll be reviewing these suggestions and seeing about implementation.  Many thanks.

On Monday, May 2, 2016 at 10:31:31 AM UTC-4, Mike Hedblom wrote:

[Mike Hedblom]

Hi Andy,

Welcome back.  I hope you had a good break.

I forked the project and have been making changes locally.  There are some things I don't yet understand.  Maybe I should push my change (is that the right action)? so you can see what I have done.

-- Mike