ID's of debates

[Matthew Landauer]

Hiya,

This morning I rolled out a change in the numbering scheme of the debates. You've probably noticed that links to debates look something like this:
http://www.openaustralia.org/debates/?id=2008-06-26.4.2

Basically every speech in every debate has a unique id. The slightly painful thing is that the parser has to generate those id's. The trick is to come up with a numbering scheme which makes it fairly unlikely that a particular speech will have a different id when you make a change to the parser.

The id has the form date.major.minor

The major number is assigned in order from the subsection of the day's Hansard. For instance, on http://parlinfoweb.aph.gov.au/piweb/browse.aspx?path=Chamber%20%3E%20House%20Hansard%20%3E%202008%20%3E%2026%20June%202008 the speeches contained within each link will have the same major number. There are certain subsections that are not currently handled by the parser, such as divisions, but these are still assigned a unique number so that when we support this in future it won't cause any conflicts.

The change I made this morning is to extend that ideas within the subsection. Look at this page:
http://parlinfoweb.aph.gov.au/piweb/TranslateWIPILink.aspx?Folder=HANSARDR&Criteria=DOC_DATE:2008-06-26%3BSEQ_NUM:62%3B

It used to be that every separate speech was given a different minor number. However, this causes problems whenever we discover a problem with the parser where an extra speech would get inserted (where for example an interjection doesn't get properly separated out). The change I made this morning is to give each paragraph a unique number and depending on what the number of the first paragraph in the speech is that is the number that is assigned to the speech.

Obviously changing the numbering sheme is a *bad* thing to do, since it change URLs, and hopefully this won't be something we'll have to do for a while. The comments that had already been made I had to rejig to ensure that they were pointing at the correct speeches. Also, I checked the incoming links with Google Webmaster Tools and checked that there weren't any links going directly to speeches rather than the whole debate.

So, hopefully this all worked. One thing, though, is that a set of email alerts will probably go out this morning even though there aren't actually any new speeches. It seems that this happens when the parser is run on old data. We'll have to live with this for the time being but it would be nice to figure out why that happens.

Oh, yeah, this whole change was motivated by fixing this ticket: http://trac2.assembla.com/openaustralia/ticket/158 - which is fixed now. Phew..

Matthew

[Rob McKinnon]

Hi Matthew,

For comparison, the NZ site has solved the thorny ID issue by no
longer putting ID numbers in the URL (for the most part). Instead the
title is converted to be part of the URL, e.g.

http://theyworkforyou.co.nz/portfolios/statistics/2008/feb/14/australia_nz_migration
http://theyworkforyou.co.nz/general_debate/2008/jul/02

Another difference is that there are no pages for separate speeches.
Instead clearly marked anchor names exist to let people link to a
speech on a debate page. e.g.

http://theyworkforyou.co.nz/portfolios/statistics/2008/feb/14/australia_nz_migration#labour_9
(doh! just noticed the ID problem has crept back in to my anchor names ;)

The URLs work when stripped back to show index pages:

http://theyworkforyou.co.nz/portfolios/statistics
http://theyworkforyou.co.nz/general_debate

cheers,
Rob

[Matthew Landauer]

Thanks for that! The change I just made with the ID's a couple of days ago actually only affected the ID's of the speeches, so in your case the names of the tags.

Just to play devil's advocate here - I guess what you use to name your links really depends on what you think is least likely to change. So, in your case, by using the titles of the debates that will work fine as long as, for instance, a mistake wasn't made with a title on the original Hansard.

By using id's numerically ordered, that is resistant against changes of title but will break if an extra speech is inserted.

Anyway, all that aside, I think your solution works great and makes for nice readable links as well which is always good. I'm going to add a ticket to software.openaustralia.org to see about adding this feature.

All the best,
Matthew

[Rob McKinnon]

On Tue, Jul 8, 2008 at 10:57 PM, Matthew Landauer
<mat...@openaustralia.org> wrote:
> Just to play devil's advocate here - I guess what you use to name your links
> really depends on what you think is least likely to change. So, in your
> case, by using the titles of the debates that will work fine as long as, for
> instance, a mistake wasn't made with a title on the original Hansard.
> By using id's numerically ordered, that is resistant against changes of
> title but will break if an extra speech is inserted.

Haven't found it the case that titles get reported incorrectly on
parliament.nz in practice. My devil's advocate comment on the kiwi
urls are that can become too long - leads to urls getting line wrapped
in emails, and makes twittering more difficult.

> Anyway, all that aside, I think your solution works great and makes for nice
> readable links as well which is always good. I'm going to add a ticket to
> software.openaustralia.org to see about adding this feature.

I agree, the bright side is you can guess what the URL is about by
reading it. Google seems to index the text of the URL amongst other
things, so it might help with search optimization.

The fun happens on the index pages where you get to see things like
who's been over for a visit this year, or what they're asking about
climate change:

http://theyworkforyou.co.nz/visitors/2008
http://theyworkforyou.co.nz/portfolios/climate_change

cheers,
Rob