I recently learned that the House is actively considering a redesign/upgrade of their public vote posting procedures, in terms of the format and functionality of the Clerk's House floor votes area.
I'd like to hear what sort of ideas and priorities everyone has about how one can interact with votes data. This is important from two perspectives, the data perspective and the citizen perspective.
As database managers, what features would be most helpful to see added to public votes data access? I was just told that one helpful aspect would be to have the DTD or schema itself be published or available after it has been decided on, so that one can see the specific way that the data is organized. From a data perspective, what else is most important to include? A standardized reference to an index of bill information (links to THOMAS/LIS)? Standardized representative elements that link to lawmakers' pages? Some sort of change log, or a feed or new votes?
From the perspective of an individual citizen, what would you like to see there? Votes searchable by topic? Votes indexed by lawmaker? Links back to the party position, by whip statement? Indexing to Congressional Record statements (one can dream...)?
What are the most important improvements we'd like to see implemented? (I'd also like to see the Senate move in this direction too.)
John
-- John Wonderlich
Program Director The Sunlight Foundation (202) 742-1520 ext. 234
What I would like to see above all else would be universal access to all the underlying data, via XML for example, as in the case of House roll call votes, or some other sturctured data format (or direct database access). This would allow programmer-citizens like me the means to provide their own views on this data, in whatever way occurs to them. That's the way to truly open up access to this information.
(I'm a little new to THOMAS so maybe there is a way to access more of this information without painful and difficult scraping through HTML pages that I haven't found. And of course, the Senate data is much worse.)
Eric Fredricksen
On Oct 25, 11:30 am, "John Wonderlich" <johnwonderl...@gmail.com> wrote:
> I recently learned that the House is actively considering a redesign/upgrade > of their public vote posting procedures, in terms of the format and > functionality of the Clerk's House floor votes area.
> I'd like to hear what sort of ideas and priorities everyone has about how > one can interact with votes data. This is important from two perspectives, > the data perspective and the citizen perspective.
> As database managers, what features would be most helpful to see added to > public votes data access? I was just told that one helpful aspect would be > to have the DTD or schema itself be published or available after it has been > decided on, so that one can see the specific way that the data is > organized. From a data perspective, what else is most important to > include? A standardized reference to an index of bill information (links to > THOMAS/LIS)? Standardized representative elements that link to lawmakers' > pages? Some sort of change log, or a feed or new votes?
> From the perspective of an individual citizen, what would you like to see > there? Votes searchable by topic? Votes indexed by lawmaker? Links back > to the party position, by whip statement? Indexing to Congressional Record > statements (one can dream...)?
> What are the most important improvements we'd like to see implemented? (I'd > also like to see the Senate move in this direction too.)
> John
> -- > John Wonderlich
> Program Director > The Sunlight Foundation > (202) 742-1520 ext. 234
Eric Fredricksen wrote: > (I'm a little new to THOMAS so maybe there is a way to access more of > this information without painful and difficult scraping through HTML > pages that I haven't found. And of course, the Senate data is much > worse.)
Nope... (Except of course through GovTrack.)
To respond to John's question-- I think the House XML vote files are actually pretty good, and could be a model for the Senate, and committee votes. They refer to related bills and amendments in a reliable way and identify MoCs by their Bioguide ID. It's a pleasure to use that data. A DTD or something would be helpful for some of the fields, but it's really not something I would even say was worth worrying about. Finding new votes is easy to get from screen-scraping the votes pages on the House site, so this isn't even all that important either from my perspective, though an RSS feed for instance would certainly be a useful thing for the public in general.
"Yields falsehood when preceded by its quotation! Yields falsehood when preceded by its quotation!" Achilles to Tortoise (in "Gödel, Escher, Bach" by Douglas Hofstadter)
> Eric Fredricksen wrote: > > (I'm a little new to THOMAS so maybe there is a way to access more of > > this information without painful and difficult scraping through HTML > > pages that I haven't found. And of course, the Senate data is much > > worse.)
> Nope... (Except of course through GovTrack.)
> To respond to John's question-- I think the House XML vote files are > actually pretty good, and could be a model for the Senate, and committee > votes. They refer to related bills and amendments in a reliable way and > identify MoCs by their Bioguide ID. It's a pleasure to use that data. A > DTD or something would be helpful for some of the fields, but it's > really not something I would even say was worth worrying about. Finding > new votes is easy to get from screen-scraping the votes pages on the > House site, so this isn't even all that important either from my > perspective, though an RSS feed for instance would certainly be a useful > thing for the public in general.
> "Yields falsehood when preceded by its quotation! Yields > falsehood when preceded by its quotation!" Achilles to > Tortoise (in "Gödel, Escher, Bach" by Douglas Hofstadter)
Where did I recently read about a review of the Senate's vote presentation? I think the Secretary of the Senate is specifically authorized to publish votes in HTML, but not in XML, and that's the reason that the XML needs to remain an in-house test for now.
I wonder if a switch to XML in the Senate would need to be attached to an appropriations bill, or if a few sentences attached to something else would be sufficient.
john
On 10/26/07, Derek Willis <dwil...@gmail.com> wrote:
> We asked about these files and were told they were tested but would > not be deployed at large, which seems rather a shame.
> Derek
> On Oct 26, 8:25 pm, Josh Tauberer <taube...@govtrack.us> wrote: > > Eric Fredricksen wrote: > > > (I'm a little new to THOMAS so maybe there is a way to access more of > > > this information without painful and difficult scraping through HTML > > > pages that I haven't found. And of course, the Senate data is much > > > worse.)
> > Nope... (Except of course through GovTrack.)
> > To respond to John's question-- I think the House XML vote files are > > actually pretty good, and could be a model for the Senate, and committee > > votes. They refer to related bills and amendments in a reliable way and > > identify MoCs by their Bioguide ID. It's a pleasure to use that data. A > > DTD or something would be helpful for some of the fields, but it's > > really not something I would even say was worth worrying about. Finding > > new votes is easy to get from screen-scraping the votes pages on the > > House site, so this isn't even all that important either from my > > perspective, though an RSS feed for instance would certainly be a useful > > thing for the public in general.
> > "Yields falsehood when preceded by its quotation! Yields > > falsehood when preceded by its quotation!" Achilles to > > Tortoise (in "Gödel, Escher, Bach" by Douglas Hofstadter)
-- John Wonderlich
Program Director The Sunlight Foundation (202) 742-1520 ext. 234
John Wonderlich wrote: > Where did I recently read about a review of the Senate's vote > presentation? I think the Secretary of the Senate is specifically > authorized to publish votes in HTML, but not in XML, and that's the > reason that the XML needs to remain an in-house test for now.
I think that was in the report that I posted a link to, quoting Derek.
> I wonder if a switch to XML in the Senate would need to be attached to > an appropriations bill, or if a few sentences attached to something else > would be sufficient.
I got the impression from some digging around a year ago that it might even only take a decision from Senate Administration (whatever that means). I don't think it's in Senate rules that the Senate website cannot use structured data, for instance.
"Yields falsehood when preceded by its quotation! Yields falsehood when preceded by its quotation!" Achilles to Tortoise (in "Gödel, Escher, Bach" by Douglas Hofstadter)
On Oct 28, 7:47 pm, Josh Tauberer <taube...@govtrack.us> wrote:
> > I wonder if a switch to XML in the Senate would need to be attached to > > an appropriations bill, or if a few sentences attached to something else > > would be sufficient.
> I got the impression from some digging around a year ago that it might > even only take a decision from Senate Administration (whatever that > means). I don't think it's in Senate rules that the Senate website > cannot use structured data, for instance.
Josh is correct. The Secretary of the Senate oversees senate.gov, and the secretary is selected by the Senate majority via its leader (in this case, Sen. Harry Reid of Nevada). Now, I suppose they could put a rule into effect, but Senate rules being mostly procedural in nature, I don't see that happening. Basically, it would take the will of the majority as expressed through the Secretary of the Senate to do this. That also means that a decision to publish structured data could be reversed by another majority.
If someone would like to write up a couple of paragraphs with what the "ask" is (and brief background on why we're asking) I'd be happy to pass the ask on to friends who work for members on the Senate Rules and Administration committee.
On 10/28/07, Derek Willis <dwil...@gmail.com> wrote:
> On Oct 28, 7:47 pm, Josh Tauberer <taube...@govtrack.us> wrote: > > > I wonder if a switch to XML in the Senate would need to be attached to > > > an appropriations bill, or if a few sentences attached to something > else > > > would be sufficient.
> > I got the impression from some digging around a year ago that it might > > even only take a decision from Senate Administration (whatever that > > means). I don't think it's in Senate rules that the Senate website > > cannot use structured data, for instance.
> Josh is correct. The Secretary of the Senate oversees senate.gov, and > the secretary is selected by the Senate majority via its leader (in > this case, Sen. Harry Reid of Nevada). Now, I suppose they could put a > rule into effect, but Senate rules being mostly procedural in nature, > I don't see that happening. Basically, it would take the will of the > majority as expressed through the Secretary of the Senate to do this. > That also means that a decision to publish structured data could be > reversed by another majority.
A few representative votes (only a few from the early congresses) were published out to the active site during some testing periods. I really need to remove them from the site. We are not authorized to publish the XML structured vote information. The Committee on Rules and Administration has authorized us to publish vote tally information in HTML format [not a structured format]. Senators prefer to be the ones to publish their own voting records. As you know, looking at a series of vote results by Senator or by subject does not tell the whole story. Senators have a right to present and comment on their votes to their constituents in the manner they prefer. This issue was reviewed again recently and the policy did not change.81
81 E-mail from Cheri Allen, Senate Webmaster, to Derek Wills, Research Database Editor, The Washington Post (Nov. 16, 2007, 15:42 EST) (on file with author).
I wonder if the reviewing was by the Senate Rules committee, or between the Secretary of the Senate and Senate leadership, or between the Webmaster and the Secretary?
I'm also wondering where the "We are not authorized to publish the XML" authorizing language would be? (perhaps an old legislative branch approps bill?)
John
On 10/30/07, Rob Pierson <piers...@gmail.com> wrote:
> If someone would like to write up a couple of paragraphs with what the > "ask" is (and brief background on why we're asking) I'd be happy to pass > the ask on to friends who work for members on the Senate Rules and > Administration committee.
> On 10/28/07, Derek Willis <dwil...@gmail.com> wrote:
> > On Oct 28, 7:47 pm, Josh Tauberer <taube...@govtrack.us> wrote: > > > > I wonder if a switch to XML in the Senate would need to be attached > > to > > > > an appropriations bill, or if a few sentences attached to something > > else > > > > would be sufficient.
> > > I got the impression from some digging around a year ago that it might > > > even only take a decision from Senate Administration (whatever that > > > means). I don't think it's in Senate rules that the Senate website > > > cannot use structured data, for instance.
> > Josh is correct. The Secretary of the Senate oversees senate.gov, and > > the secretary is selected by the Senate majority via its leader (in > > this case, Sen. Harry Reid of Nevada). Now, I suppose they could put a > > rule into effect, but Senate rules being mostly procedural in nature, > > I don't see that happening. Basically, it would take the will of the > > majority as expressed through the Secretary of the Senate to do this. > > That also means that a decision to publish structured data could be > > reversed by another majority.
> > Derek
-- John Wonderlich
Program Director The Sunlight Foundation (202) 742-1520 ext. 234
