MyCelium and Parliamentary Video Liscensing

[Jennifer Bell]

A while back, I had an email conversation with Kevin McArthur (on this
group) about what it would take to turn his project, mycelium:
http://mycelium.chanterelle.ca/ into a site like metavid: http://metavid.org,
with searchable transcripts.

He said that while the technical challenges were not so terrible, the
real issue was the crown copyright on the source video from parlvu:
http://parlvu.parl.gc.ca/Parlvu/UpcomingEvents.aspx. Recently, I was
reminded [1] that a group in the UK called the PublicWhip managed to
negotiate a click-use liscense for using the hansard transcript,
enabling them to re-package it as XML. I wonder if a similar deal is
possible with parlvu.

I'll do a bit of research on the public whip liscense this week, and
try sending an email to the parlvu contact email, and see if they
reply. In the meantime, please let me know if you have knowledge/
suggestions/ideas on how to free our parliamentary video.

Jennifer
[1] when re-reading this: http://powerofinformation.wordpress.com/2008/06/19/more-architecture/

[Kevin McArthur]

Hi Jennifer, et al...

Let me know what you find out. I've tried to run the site on the
conservative side of fair dealing, and I know a bunch of MPs are aware
of the site and no one has complained yet. That said, the HoC has pulled
stuff from youtube and appear to defend their copyright when its abused
(see http://www.michaelgeist.ca/content/view/1989/125/ ).

So its clearly a very thin line between what they find acceptable use,
and what they find to be abuse of copyright.

I'm not interested in 'poking the bear' to find out exactly where that
line is, but if you guys want to, I'll be happy to pass along my work
and let you run it.

Let me know,

Kevin

> --~--~---------~--~----~------------~-------~--~----~
> You received this message because you are subscribed to the Google Groups "VisibleGovernment Discuss" group.
> To post to this group, send email to visiblegover...@googlegroups.com
> To unsubscribe from this group, send email to visiblegovernment-...@googlegroups.com
> For more options, visit this group at http://groups.google.com/group/visiblegovernment-discuss?hl=en
> -~----------~----~----~----~------~----~------~--~---
>
>

--

Kevin McArthur

StormTide Digital Studios Inc.
Author of the recently published book, "Pro PHP"
http://www.stormtide.ca

[Andy Kaplan-Myrth]

Hi everyone,

One of the sites that got me interested in the whole idea of building
online tools for government transparency and accessibility was
MySociety's video matching game:
http://www.theyworkforyou.com/video/

So MyCelium is a very interesting project to me, and I'd hate to see
Crown Copyright get in the way of a truly useful and valuable public
interest project like this.

It is of course always best to have an explicit licence to use the works
(both the video and the transcript in this case). So I do think it would
be worth exploring whether such a licence could be negotiated for little
to no cost for a project that is clearly in the public interest.

But I also think such a use of parlvu video would likely be fair dealing
in Canada. The boundaries of fair dealing are hard to identify, so there
is some inherent risk in relying on fair dealing as the grounds on which
you use a work. But the fact remains that there *is* a fair dealing
exemption, and *something* falls within it. If unmodified, freely
available parliamentary video and transcripts, improved by indexing with
hansard transcripts is not fair dealing, then what is? And while it's
true that the HoC has claimed infringement of copyright in the past, in
at least the case of that one parody video on YouTube, the video in that
particular case was much, much farther from the definition of fair
dealing identified in the case CCH v. LSUC:
The hilarious parody: http://www.youtube.com/watch?v=MlD9hu7y8bU
CCH v. LSUC: http://is.gd/hXS1

Watch that video, look at MyCelium and Metavid.org, and then read
through the six factors that the Supreme Court of Canada outlined to
define fair dealing, starting from paragraph 54 of the Court's decision.
I think it is pretty clear how the parody video would differ on each
factor from the supposed metavid.CA. Especially the "purpose",
"character", "nature" and "effect" factors.

We would obviously need a legal opinion before pushing ahead on this
path, but it is hard for me to imagine how a Metavid.CA type of site
would fall outside of what the Court defined as fair dealing.

I am a lawyer, but this email is not that legal advice! However, I think
we could get advice to that effect without too much trouble :-)

On a different but related note, it was suggested to me that a major
expense in some government departments is the purchase of licences to
use other departments' works that are under Crown Copyright. Like,
Environment Canada writes a report, which is protected under Crown
Copyright, and then in order for Industry Canada to use that report in
their work, IC buys a licence from EC. Can anybody shed any light on
these sorts of licensing arrangements?

Cheers,
Andy

Andy Kaplan-Myrth, M.A., LL.B.
Barrister & Solicitor
------------------------------------------------
email: an...@kaplan-myrth.ca
web: http://kaplan-myrth.ca
blog: http://blog.kaplan-myrth.ca
------------------------------------------------

[Jennifer Bell]

As an update on this: I wasn't able to find much of a record of the
liscense that the PublicWhip group is using. I've emailed them my
questions directly.

In the meantime, I came accross this reccomendation in the Power of
Information Task Force's most recent report:

"When the public sector publishes information people should understand
that it is intended for re-use. Action is required to improve
understanding of Crown Copyright, which the Taskforce found to be mis-
understood by creators and re-users of data. Crown Copyright, despite
its historic name, is designed to encourage re-use in the majority of
cases yet the taskforce found little appreciation of this. There were
even suggestions that it was deterring potential re-users. OPSI
should begin a communications campaign to re-present and improve
understanding of the permissive aspects of Crown Copyright along the
lines of creative commons by end June 2009."

Crown Copyright is designed to encourage re-use? Really? Can anyone
comment on whether the UK version has evolved differently than the
Canadian one?

I've been sick with a cold for almost a week, so have been offline for
long stretches.

Jennifer

> >>http://mycelium.chanterelle.ca/into a site like metavid:

> >>http://metavid.org,
> >> with searchable transcripts.
>
> >> He said that while the technical challenges were not so terrible, the
> >> real issue was the crown copyright on the source video from parlvu:
> >>http://parlvu.parl.gc.ca/Parlvu/UpcomingEvents.aspx.  Recently, I was
> >> reminded [1] that a group in the UK called the PublicWhip managed to
> >> negotiate a click-use liscense for using the hansard transcript,
> >> enabling them to re-package it as XML.  I wonder if a similar deal is
> >> possible with parlvu.
>
> >> I'll do a bit of research on the public whip liscense this week, and
> >> try sending an email to the parlvu contact email, and see if they
> >> reply.  In the meantime, please let me know if you have knowledge/
> >> suggestions/ideas on how to free our parliamentary video.
>
> >> Jennifer
> >> [1] when re-reading this:
> >>http://powerofinformation.wordpress.com/2008/06/19/more-architecture/
>
> --
> Andy Kaplan-Myrth, M.A., LL.B.
> Barrister & Solicitor
> ------------------------------------------------

> email: a...@kaplan-myrth.ca

[Andy Kaplan-Myrth]

That's very interesting indeed. In Canada, the gloss that various
governments put on Crown Copyright is that it allows some degree of
control for the accuracy of reproductions of government documents. The
federal government has an official policy permitting reproduction of
legislation and court decisions.

But my understanding from other sources is that of the organizations
that pay for licences under Crown Copyright, the one that by far pays
the most is the government itself. That is, one department actually buys
reproduction rights to a document published under Crown Copyright by
another department.

In fact, a friend of mine heard about Visible Government and asked if
we'd be interested in a project that looked into an accounting of
spending for Crown Copyright licences...

Cheers,
Andy

email: an...@kaplan-myrth.ca

[joe.m...@jmaconsulting.biz]

I found the following paragraph potentially of interest to this group:

Google.org has made over $100 million in investments in three areas:
global health, clean energy and access to information. They include
projects like developing renewable energy that is cheaper than coal and
preventing the spread of disease.

http://www.nytimes.com/2009/02/24/technology/companies/24google.html

Joe

[Kevin McArthur]

Crown Copyright is designed to encourage re-use?  Really?

http://thetyee.ca/News/2009/02/24/FOIWarnings/

Food for thought.

K

Jennifer Bell wrote:

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "VisibleGovernment Discuss" group.
To post to this group, send email to visiblegover...@googlegroups.com

To unsubscribe from this group, send email to visiblegovernment-...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/visiblegovernment-discuss?hl=en
-~----------~----~----~----~------~----~------~--~---

  

[Jennifer Bell]

Thanks for the link to the article, Kevin. I like the example of use
for a picture being denied because it was going to be used in a
negative article. Nice!

I finally found a link for the official site of the UK's 'Click-Use'
liscense. It's apparently something invented by the UK government's
Office of Public Sector Information, a division of the naitional
archives. The site states that the liscense enables re-use of crown-
copyright materials (which seems to imply that UK crown copyright is
not so re-useable, after all).

http://www.opsi.gov.uk/click-use/

Also noteable is the Office of Public Sector Information unlocking
service, which seems to be collecting votes for unlocking public data
sets. Strangely, it doesn't seem that well used.

From Andy:

> In fact, a friend of mine heard about Visible Government and asked if
> we'd be interested in a project that looked into an accounting of
> spending for Crown Copyright licences...

Maybe! Where can I find out more?

Jennifer

[Laura]

Here's an interesting comment about where Crown copyright prevented re-
use of information in Australia. I'm not well educated on this topic,
but it seems worth exploring.

From: http://content.techrepublic.com.com/2346-1035_11-270469-5.html

"The massive forest fire that continues to ravage Australia's mainland
became a struggle for the freedom of information. While geographical
data about the fires was readily available from the Country Fire
Authority, when Google tried to get information on fires that were on
public lands the company hit a dead end.

Victoria's Department of Sustainability and Environment would not
share its data, which was deemed private due to the Crown copyright
provisions, a set of rules that limits government information from
being shared with the public or private companies. Google Australia
instead went with satellite imagery from NASA for use on its special
maps layer that's tracking the spread and control of fires.

This satellite imagery of the quake was provided by NASA when the
Australian government would not provide geodata on where the fires
were."

So, let me get this straight - NASA info = open, A state department
funded with public money = closed. I blame bureaucracy.

> >>>>http://mycelium.chanterelle.ca/intoa site like metavid:

> >>>>http://metavid.org,
> >>>> with searchable transcripts.
>
> >>>> He said that while the technical challenges were not so terrible, the
> >>>> real issue was the crown copyright on the source video from parlvu:
> >>>>http://parlvu.parl.gc.ca/Parlvu/UpcomingEvents.aspx.  Recently, I was
> >>>> reminded [1] that a group in the UK called the PublicWhip managed to
> >>>> negotiate a click-use liscense for using the hansard transcript,
> >>>> enabling them to re-package it as XML.  I wonder if a similar deal is
> >>>> possible with parlvu.
>
> >>>> I'll do a bit of research on the public whip liscense this week, and
> >>>> try sending an email to the parlvu contact email, and see if they
> >>>> reply.  In the meantime, please let me know if you have knowledge/
> >>>> suggestions/ideas on how to free our parliamentary video.
>
> >>>> Jennifer
> >>>> [1] when re-reading this:
> >>>>http://powerofinformation.wordpress.com/2008/06/19/more-architecture/
>
> >> --
> >> Andy Kaplan-Myrth, M.A., LL.B.
> >> Barrister & Solicitor
> >> ------------------------------------------------
> >> email: a...@kaplan-myrth.ca
> >> web:http://kaplan-myrth.ca
> >> blog:http://blog.kaplan-myrth.ca
> >> ------------------------------------------------
>
> > >

> --
>
> Kevin McArthur
>
> StormTide Digital Studios Inc.
> Author of the recently published book, "Pro PHP"http://www.stormtide.ca
>

>  smime.p7s
> 4KViewDownload

[Jennifer Bell]

I had another conversation recently that reminded me of this neat
thread. I never heard back from parlvu [1] when I enquired to their
contact email about the possiblity for a click-use liscense[2], like
the one offered by the Office of the Public Sector in the UK, to free
up content for re-use. (This is the liscense that the Public Whip
uses to re-distribute the Hansard in the UK.)

It was recently suggested to me that the Government of Canada Library
of Parliament would be a good resource and potential ally. It's
quite possible that this department might be very good as a first
contact for talking about redistribution liscenses. If anyone knows
someone in the Library of Parliament, please let me know. Sometimes
personal contacts help get things rolling.

Are there other groups who are actively working on this?

Jennifer

[1] http://parlvu.parl.gc.ca/
[2] http://www.opsi.gov.uk/click-use/

> > >> SoMyCeliumis a very interesting project to me, and I'd hate to see

> > >> Crown Copyright get in the way of a truly useful and valuable public
> > >> interest project like this.
>
> > >> It is of course always best to have an explicit licence to use the works
> > >> (both the video and the transcript in this case). So I do think it would
> > >> be worth exploring whether such a licence could be negotiated for little
> > >> to no cost for a project that is clearly in the public interest.
>
> > >> But I also think such a use of parlvu video would likely be fair dealing
> > >> in Canada. The boundaries of fair dealing are hard to identify, so there
> > >> is some inherent risk in relying on fair dealing as the grounds on which
> > >>   you use a work. But the fact remains that there *is* a fair dealing
> > >> exemption, and *something* falls within it. If unmodified, freely
> > >> available parliamentary video and transcripts, improved by indexing with
> > >> hansard transcripts is not fair dealing, then what is? And while it's
> > >> true that the HoC has claimed infringement of copyright in the past, in
> > >> at least the case of that one parody video on YouTube, the video in that
> > >> particular case was much, much farther from the definition of fair
> > >> dealing identified in the case CCH v. LSUC:
> > >>    The hilarious parody:http://www.youtube.com/watch?v=MlD9hu7y8bU
> > >>    CCH v. LSUC:http://is.gd/hXS1
>

> > >> Watch that video, look atMyCeliumand Metavid.org, and then read

> > >>>>http://mycelium.chanterelle.ca/intoasite like metavid:

[Kevin McArthur]

Jennifer,

Have you tried the Office of the Speaker... might be more helpful than the ParlVU crew.

Kevin

Jennifer Bell wrote:

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "VisibleGovernment Discuss" group.
To post to this group, send email to visiblegover...@googlegroups.com

To unsubscribe from this group, send email to visiblegovernment-...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/visiblegovernment-discuss?hl=en
-~----------~----~----~----~------~----~------~--~---

  

-- 

Kevin McArthur

StormTide Digital Studios Inc.

http://www.stormtide.ca

[Alistair Croll]

I was reading in the Globe and Mail today about a government site that publishes statistics on schools. Teachers' unions (big surprise) are complaining that this is a "ranking site" -- if you've seen Bill Gates' TED presentation, you'll know how resistant teachers are to public scrutiny and transparency.

It seems like this is a case of the government trying to do the right thing, and getting scolded. We should be encouraging this initiative, IMHO, otherwise the teachers will have their way.

www.edu.gov.on.ca/eng/sift

"This is absolutely not about ranking schools. This is about transparency and providing the information to parents," said a spokeswoman for Minister of Education Kathleen Wynne, "Parents have expressed that they want to know more about their schools. What we've done through this site is bring a variety of information together in one place."

Thoughts?

A.

[Andy Kaplan-Myrth]

I had a contact in the Speakers office who I was trying to engage on
this, but our email exchange died. I could try to start that up again
if you want.

--
Andy Kaplan-Myrth, LL.B., M.A.
Barrister & Solicitor
------------------------------------------------
email: an...@kaplan-myrth.ca
web: http://kaplan-myrth.ca
------------------------------------------------
PGP Key ID 0xE9349025
------------------------------------------------

[Jennifer Bell]

Alistair brought up a good point last week (mixed in with this thread
here):

http://groups.google.com/group/visiblegovernment-discuss/t/7b3eb76ee4963374

about a story in the Globe and Mail on how Ontario's education
ministry was criticized for publishing statistics on scools, which
teachers called a 'ranking site'. Controversially, one stat published
was ethnicity of students. I've found a re-print of the story here:

http://sgetfo.ca/2009/04/06/website-for-information-on-schools-criticized/

I agree with Alistair that the province should be commended for being
more transparent, not criticized. While I was initially uneasy about
the ethnicity stats, I think it is relevant. Speaking from personal
experience: last year, I had a great experience in a Montreal French
language class which was racially mixed, but a much worse experience
in the next class where the class was entirely one racial group, and I
was unique. 'Fitting in', or not, has a big impact on your comfort
level, even when you're an adult.

In my view, publishing, rather than hiding, this information is better
because:

- it allows people looking for 'diverse' background schools to choose
what they want
- it allows people who are looking for a school that includes a
particular ethnic group to make that choice (why is that bad?)
- it allows for self-correction on the part of the school or
community if there is an imbalance

Racial controversy aside, I agree that the board of education was
doing, in principle, the right thing. This behavior should be re-
inforced, not punished.

Jennifer

[Alistair Croll]

IMHO the racial stuff, while controversial, was a red herring: Vested
interests don't want transparency. In this case, the government was
actually forced to kill a bunch of the functionality because of the
pressure from a vocal minority.

A.

[joe.m...@jmaconsulting.biz]

This is worth an op-ed. Much of the controversial information is from the
EQAO standardized test results still available online at
https://eqaoweb.eqao.com/pbs/Listing.aspx

[Jennifer Bell]

On the topic of click-use liscensing -- it turns out there are
potential ways in to both the Office of the Speaker and the Library of
Parliamentary for pushing this.

A question that should be investigated before-hand, though:

The group the Public-Whip, in the UK, uses the click-use lisence[1] to
re-distribute scraped information under creative-commons, as described
here:
http://powerofinformation.wordpress.com/2008/06/19/more-architecture/

Is this really allowed by click-use, or is this just what PublicWhip
is doing, and no-one's stopped them? To investigate...

Jennifer
[1] http://www.opsi.gov.uk/click-use/

On Apr 9, 8:37 am, Andy Kaplan-Myrth <a...@kaplan-myrth.ca> wrote:
> I had a contact in the Speakers office who I was trying to engage on
> this, but our email exchange died. I could try to start that up again
> if you want.
>
> --
> Andy Kaplan-Myrth, LL.B., M.A.
> Barrister & Solicitor
> ------------------------------------------------

> email: a...@kaplan-myrth.ca

> >http://mycelium.chanterelle.ca/intoasitelike metavid:

> ...
>
> read more »

[Andy Kaplan-Myrth]

Having taken only a quick look through the Click-Use licensing program
and one of the actual licences, it looks like this kind of re-use and
re-licensing under CC is not allowed, but I'm sure PublicWhip has
looked at this in detail. Should I try to follow up with them and get
back to the list with more information about how they arranged licensing?

It will likely be informative and may give us some ideas, but I'm not
sure how directly relevant it will be to us since we don't have that
Click-Use regime for works under Crown Copyright. And it's not even
clear that parliamentary video is under Crown Copyright, frankly.

But I can gather info from PublicWhip if people think that would be
useful.

Cheers,
Andy

Jennifer Bell wrote:

> On the topic of click-use liscensing -- it turns out there are
> potential ways in to both the Office of the Speaker and the Library of
> Parliamentary for pushing this.
>
> A question that should be investigated before-hand, though:
>
> The group the Public-Whip, in the UK, uses the click-use lisence[1] to
> re-distribute scraped information under creative-commons, as described
> here:
> http://powerofinformation.wordpress.com/2008/06/19/more-architecture/
>
> Is this really allowed by click-use, or is this just what PublicWhip
> is doing, and no-one's stopped them? To investigate...
>
> Jennifer
> [1] http://www.opsi.gov.uk/click-use/
>
> On Apr 9, 8:37 am, Andy Kaplan-Myrth<a...@kaplan-myrth.ca> wrote:
>> I had a contact in the Speakers office who I was trying to engage on
>> this, but our email exchange died. I could try to start that up again
>> if you want.
>>
>> --
>> Andy Kaplan-Myrth, LL.B., M.A.

>> Barrister& Solicitor

>>> Barrister& Solicitor

>>> ------------------------------------------------
>>> email: a...@kaplan-myrth.ca
>>> web:http://kaplan-myrth.ca
>>> blog:http://blog.kaplan-myrth.ca
>>> ------------------------------------------------
>>> --
>>> Kevin McArthur
>>> StormTide Digital Studios Inc.
>>> Author of the recently published book, "Pro PHP"http://www.stormtide.ca
>>> smime.p7s
>>> 4KViewDownload
>>> --
>>> Kevin McArthur
>>> StormTide Digital Studios
>> ...
>>
>> read more »
> >
>

--
Andy Kaplan-Myrth, M.A., LL.B.

Social Media and Law
------------------------------------------------
email: an...@kaplan-myrth.ca

[Joe Murray]

I believe there are also minor differences in copyright law between UK and
Canada.

[Jennifer Bell]

To keep this going: I've emailed the Public Whip before, with no
response... but perhaps if you try, Andy, with a targeted question on
the liscense, they'll answer.

I've also posted a question as a comment to the POITF blog post which
set this all off, which is being moderated.
http://powerofinformation.wordpress.com/2008/06/19/more-architecture/

Really, I got excited about this because it seemed like an example of
a cut-and-dried way of redistribuing gov data without Crown Copyright
headaches. If this isn't it, then maybe a new one can be invented.

On that note, some open-data'ers in Ottawa are gearing up for a
meeting with the Library of Parliament, which is cool.

JB

On Apr 17, 11:38 am, "Joe Murray" <joe.mur...@jmaconsulting.biz>
wrote:

> > >>> thread.  I never heard back fromparlvu[1] when I enquired to their

> > >>> But I also think such a use ofparlvuvideo would likely be fair

> ...
>
> read more »

[Andy Kaplan-Myrth]

I could try to meet with the Public Whip about it. Could you send me
contact info you've used off-list?

I met the stimuluswatch gang the other day at the social media
breakfast in Ottawa and in a few minutes speaking with them I
mentioned dealing with Crown Copyright for their purposes. I'm hoping
to go to their next meeting so I can keep plugging away at that issue.

--
Andy Kaplan-Myrth, M.A., LL.B.

[Kevin McArthur]

So, according to http://www.michaelgeist.ca/content/view/3953/135/ this morning.... looks like we've got the right permissions for Mycelium finally. The direct link is http://www2.parl.gc.ca/HousePublications/Publication.aspx?DocId=3712105&Language=E&Mode=1&Parl=40&Ses=2
So, with that in place, I have a few questions.

1) Are there any features for mycelium that need to be worked on/added. I've been reluctant in the past to do much for fear of triggering the type of copyright complaint that this other group did. That said... looks like the permissions ended up being more than I would have expected, and are well in-line with the type of video clip service I've been trying to run.
2) I have an archive of recorded question period videos going back 10-16-2007. Is there any interest on publication of these archives? Should they be open for excerpting?
3) Can anyone suggest how to best add metadata as a cross-reference search for video content. I'm thinking searches from hansard and finding associated days videos... but, I don't see any good/easy way to really get a good timecode.

Ideas?

K

-- 

Kevin McArthur

StormTide Digital Studios Inc.
http://www.stormtide.ca

[Andy Kaplan-Myrth]

Mysociety.org in the UK crowdsourced the task of matching up hansard
with video using a Flash game, with legendary success. I visit their
site from time to time to try it out, but it's so successful that all
their timestamping is always complete!

http://www.theyworkforyou.com/video/

Can we get the code they built that on and set up something similar?
Is Canada's population large enough to support the task? Or would
others outside Canada help?

Cheers,
Andy

--

[Jennifer Bell]

That's wild. The committee report is here:

http://www2.parl.gc.ca/HousePublications/Publication.aspx?DocId=3712105&Language=E&Mode=1&Parl=40&Ses=2

"Reproduction of the proceedings of the House of Commons and its
Committees, in whole or in part and in any medium, is hereby permitted
provided that the reproduction is accurate and is not presented as
official. This permission does not extend to reproduction,
distribution or use for commercial purpose of financial gain."

While the introduction talks specifically about webcasts, the above
statement seems to include the hansard itself, which makes hansard
scraping projects like howdtheyvote and ourparliament.ca suddenly
legitamate.

It's fun that this arrangement isn't given a name. 'Crown Copyright
for Parliamentary Proceedings'? Crown Copyright-minus-minus?

... and of course, I shake my head at the commercial restriction.
Heaven forbid anyone in this country should make money.

Anyway, mySociety's timestamping code is theoretically somewhere in
here:
https://secure.mysociety.org/cvstrac/dir?d=mysociety

As for: could Canada support the same thing?... if you look at their
timestamping stats[1], they basically have one really, really
committed person, and a few less committed people who represent the
bulk of the work. People who just breeze through represent ~1/20th of
the effort. So, if a really, really committed Cdn came forward, I'd
say the same thing would be possible here.

Jennifer
[1] http://www.theyworkforyou.com/video/

On May 11, 12:09 pm, Andy Kaplan-Myrth <a...@kaplan-myrth.ca> wrote:
> Mysociety.org in the UK crowdsourced the task of matching up hansard
> with video using a Flash game, with legendary success. I visit their
> site from time to time to try it out, but it's so successful that all
> their timestamping is always complete!
>
>    http://www.theyworkforyou.com/video/
>
> Can we get the code they built that on and set up something similar?
> Is Canada's population large enough to support the task? Or would
> others outside Canada help?
>
> Cheers,
> Andy
>
> Kevin McArthur wrote:
> > So, according tohttp://www.michaelgeist.ca/content/view/3953/135/this
> > morning.... looks like we've got the right permissions for Mycelium
> > finally. The direct link is

> >http://www2.parl.gc.ca/HousePublications/Publication.aspx?DocId=37121...

> ...
>
> read more »

[Kevin McArthur]

So, if a really, really committed Cdn came forward, I'd
say the same thing would be possible here.
  

Any volunteers :)

I took a quick peruse on the site, but couldn't actually find the flash game -- I think because there's nothing to tag right now. Would it have been just as successful without the matchup game, given that really, its one person doing the tagging?

I'm told there's a closed captioning feed on the parliamentary tv network (the cable system they have in the building) but, I've not seen anyone successfully tap into it. However, if we can get the CC data, plus some sort of algorithm -- maybe a Levenshtein based match -- could be possible to automate the tagging?

Anyone know a sympathetic MP who might be willing to put a Myth-Based PVR on the cable feed and/or know of a way to get it from a parlvu webcast [i dont think the data is in the stream, but could be wrong]...

While the introduction talks specifically about webcasts, the above
statement seems to include the hansard itself, which makes hansard
scraping projects like howdtheyvote and ourparliament.ca suddenly
legitamate.

Pretty awesome if you ask me...

K


Jennifer Bell wrote:

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "VisibleGovernment Discuss" group.
To post to this group, send email to visiblegover...@googlegroups.com

To unsubscribe from this group, send email to visiblegovernment-...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/visiblegovernment-discuss?hl=en
-~----------~----~----~----~------~----~------~--~---

  

[Andy Kaplan-Myrth]

Jennifer Bell wrote:
> It's fun that this arrangement isn't given a name. 'Crown Copyright
> for Parliamentary Proceedings'? Crown Copyright-minus-minus?

The "proceedings exemption"?

[Chris Taggart]

I think the parlvid/hansard flex stuff is here..
https://secure.mysociety.org/cvstrac/dir?d=mysociety/twfy/flex

I'd be interested in helping out. Less interested in tagging all this stuff.

-- 

Kevin McArthur

StormTide Digital Studios Inc.
http://www.stormtide.ca

--
Chris Taggart

[Jennifer Bell]

OK, so it's awesome. :-)

It would be awesome-er if the liscensing scheme had a name, so that it
could be argued for in other places. eg. 'I think that the Open
Government Data Exception' should be applied to data set X.'

As for tagged parliamentary voting records... I think the game is
still useful. What may have happened is that the very motivated
person was so motivated they beat everyone else to the punch. There
were only so many records to tag, after all.

What metavid.org does is really interesting. It apparently screen
scrapes CSPAN to get the name of the person talking, then matches that
up with the transcript to do the time sync. Presumably the same
thing, more or less, could be done with CC.

http://metavid.org/wiki/Help:FAQ#Where_does_this_video_and_metadata_come_from.3F

I'm totally interested in seeing searchable parliamentary video happen
in Canada....

Jennifer

On May 11, 1:31 pm, Kevin McArthur <ke...@stormtide.ca> wrote:
> > So, if a really, really committed Cdn came forward, I'd
> > say the same thing would be possible here.
>
> Any volunteers :)
>
> I took a quick peruse on the site, but couldn't actually find the flash
> game -- I think because there's nothing to tag right now. Would it have
> been just as successful without the matchup game, given that really, its
> one person doing the tagging?
>
> I'm told there's a closed captioning feed on the parliamentary tv
> network (the cable system they have in the building) but, I've not seen
> anyone successfully tap into it. However, if we can get the CC data,
> plus some sort of algorithm -- maybe a Levenshtein based match -- could
> be possible to automate the tagging?
>
> Anyone know a sympathetic MP who might be willing to put a Myth-Based
> PVR on the cable feed and/or know of a way to get it from a parlvu
> webcast [i dont think the data is in the stream, but could be wrong]...
>
> > While the introduction talks specifically about webcasts, the above
> > statement seems to include the hansard itself, which makes hansard
> > scraping projects like howdtheyvote and ourparliament.ca suddenly
> > legitamate.
>
> Pretty awesome if you ask me...
>
> K
>
> Jennifer Bell wrote:
> > That's wild.  The committee report is here:
>

> >http://www2.parl.gc.ca/HousePublications/Publication.aspx?DocId=37121...

> ...
>
> read more »
>
>  smime.p7s
> 4KViewDownload

[Kevin McArthur]

Yes, it should have a name. Regardless...

In light of the new clarity in the license, I've put my archives online. Ive got _almost_ every question period since Oct 16 2007.  You can view/download them here: http://cdn.stormtide.ca/video/?C=M;O=D ... it should be almost 200 sessions.

I've also added an all-slices section to the site, which has every clip made on Mycelium. Viewable here: http://mycelium.chanterelle.ca/index/all-slices

Unfortunately, I've not yet got the ability to make slices in older clips [technically, because my VPS which runs the app has a small drive, and my cdn which hosts the archive has crappy programming support]... I'm working on it though.

Let me know if you guys have any other good ideas to make the site better.

K


Jennifer Bell wrote:

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "VisibleGovernment Discuss" group.
To post to this group, send email to visiblegover...@googlegroups.com

To unsubscribe from this group, send email to visiblegovernment-...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/visiblegovernment-discuss?hl=en
-~----------~----~----~----~------~----~------~--~---

  

[Jennifer Bell]

Hey! Your recordings have the names of who's talking.... between
that, the hansard, and the open source metavid software, it might not
be too hard to make it searchable.

Do you want help with that? Let me know! Technically, I'm not
supposed to do anything without board approval, but I have a feeling
they'd be keen on this.

Jennifer

On May 14, 7:45 pm, Kevin McArthur <ke...@stormtide.ca> wrote:
> Yes, it should have a name. Regardless...
>
> In light of the new clarity in the license, I've put my archives online.
> Ive got _almost_ every question period since Oct 16 2007.  You can

> view/download them here:http://cdn.stormtide.ca/video/?C=M;O=D... it

> should be almost 200 sessions.
>
> I've also added an all-slices section to the site, which has every clip
> made on Mycelium. Viewable here:http://mycelium.chanterelle.ca/index/all-slices
>
> Unfortunately, I've not yet got the ability to make slices in older
> clips [technically, because my VPS which runs the app has a small drive,
> and my cdn which hosts the archive has crappy programming support]...
> I'm working on it though.
>
> Let me know if you guys have any other good ideas to make the site better.
>
> K
>
> Jennifer Bell wrote:
> > OK, so it's awesome.  :-)
>
> > It would be awesome-er if the liscensing scheme had a name, so that it
> > could be argued for in other places.   eg.  'I think that the Open
> > Government Data Exception' should be applied to data set X.'
>
> > As for tagged parliamentary voting records... I think the game is
> > still useful.  What may have happened is that the very motivated
> > person was so motivated they beat everyone else to the punch.  There
> > were only so many records to tag, after all.
>
> > What metavid.org does is really interesting.  It apparently screen
> > scrapes CSPAN to get the name of the person talking, then matches that
> > up with the transcript to do the time sync.  Presumably the same
> > thing, more or less, could be done with CC.
>

> >http://metavid.org/wiki/Help:FAQ#Where_does_this_video_and_metadata_c...

[Kevin McArthur]

Sure. The video data is all there, and I can make available whatever other data I have. Let me know what you come up with, and/or how I can help.

K


Jennifer Bell wrote:

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "VisibleGovernment Discuss" group.
To post to this group, send email to visiblegover...@googlegroups.com

To unsubscribe from this group, send email to visiblegovernment-...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/visiblegovernment-discuss?hl=en
-~----------~----~----~----~------~----~------~--~---

  

[Cory Horner]

On 15-May-09, at 11:46 AM, Jennifer Bell wrote:

> Hey! Your recordings have the names of who's talking.... between
> that, the hansard, and the open source metavid software, it might not
> be too hard to make it searchable.

Let's have some fun now...

http://www.howdtheyvote.ca/api.php?call=gethansards&session_id=10

and

http://www.howdtheyvote.ca/api.php?call=getquotes&hansard_id=605

should get you the raw data. Experimental, subject to breakage/change/
etc... and feedback wanted. I'm likely to put an api token on that
soon... but in the mean time, go crazy.

Cheers,
Cory.

[Jennifer Bell]

Brilliant. I'll do some testing this week to see how easily this can
all come together...

Jennifer

[Kevin McArthur]

Cory, or others

You don't by chance have a parser that cuts up the final hansard format to xml do you?

I've got a few ideas for this, but let me know if you've already done the parsing work.

Kevin McArthur

Cory Horner wrote:

should get you the raw data.  Experimental, subject to breakage/change/ 
etc... and feedback wanted.  I'm likely to put an api token on that  
soon... but in the mean time, go crazy.

Cheers,
Cory.

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "VisibleGovernment Discuss" group.
To post to this group, send email to visiblegover...@googlegroups.com

To unsubscribe from this group, send email to visiblegovernment-...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/visiblegovernment-discuss?hl=en
-~----------~----~----~----~------~----~------~--~---

  

[Cory Horner]

On 2-Jun-09, at 9:06 AM, Kevin McArthur wrote:

> You don't by chance have a parser that cuts up the final hansard
> format to xml do you?
>
> I've got a few ideas for this, but let me know if you've already
> done the parsing work.

The parser is specialized to stuff Hansard into my database, so I
recommend you just consume the end result:

http://www.howdtheyvote.ca/api.php?call=getquotes&hansard_id=605

I'll add a few more attributes, such as topic and heading... which
should make it equivalent to Hansard itself... am I missing anything
else?

Cheers,
Cory.

[Kevin McArthur]

Cory,

Thanks, I'll see what I can do with this.

Other than that, no, cant think of anything offhand other than maybe trying to convince you to opensource these parsers... maybe lgpl?

K

Cory Horner wrote:

I'll add a few more attributes, such as topic and heading... which  
should make it equivalent to Hansard itself... am I missing anything  
else?

Cheers,
Cory.

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "VisibleGovernment Discuss" group.
To post to this group, send email to visiblegover...@googlegroups.com

To unsubscribe from this group, send email to visiblegovernment-...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/visiblegovernment-discuss?hl=en
-~----------~----~----~----~------~----~------~--~---

  

[Cory Horner]

On 4-Jun-09, at 1:06 PM, Kevin McArthur wrote:

> Thanks, I'll see what I can do with this.
>
> Other than that, no, cant think of anything offhand other than maybe
> trying to convince you to opensource these parsers... maybe lgpl?

http://sourceforge.net/projects/howdtheyvote/

it's been under GPL for ~ 5 years... but I warn you, you don't want to
know how sausages are made!

[Luke Closs]

FWIW, I have just created and released a Perl module to scrape member, bill and vote data from parl.gc.ca into data structures.

The code is released on CPAN: http://search.cpan.org/dist/Net-Parliament

And it is up on github: http://github.com/lukec/cpan-net-parliament

Cheers,
Luke

[Kevin McArthur]

Cory, and maybe et al

How are you indexing the member id's. Does this come from somewhere?

I'm currently working off of http://www.parl.gc.ca/information/about/process/house/GeneralInformation/SeatingPlan.pdf ... which lists id's for the seats... but not anything unique to the members....

Kevin


Cory Horner wrote:

I'll add a few more attributes, such as topic and heading... which  
should make it equivalent to Hansard itself... am I missing anything  
else?

Cheers,
Cory.

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "VisibleGovernment Discuss" group.
To post to this group, send email to visiblegover...@googlegroups.com

To unsubscribe from this group, send email to visiblegovernment-...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/visiblegovernment-discuss?hl=en
-~----------~----~----~----~------~----~------~--~---

  

[Jennifer Bell]

Personally, I've found electoral district ids (EDID) to be useful for
cross-referencing. The ID is stable (though sometimes new ones are
added) and has a particular format based on province.

During the last election, Robin Millete did a meta-tool that allowed
people to look things up for different ridings by linking in to
different sites via the riding IDs. The interface was rough, but the
idea was there. For example:

http://cancan.waglo.com/dataface/district/35001/

Jennifer

On Jun 5, 1:10 pm, Kevin McArthur <ke...@stormtide.ca> wrote:
> Cory, and maybe et al
>
> How are you indexing the member id's. Does this come from somewhere?
>

> I'm currently working off ofhttp://www.parl.gc.ca/information/about/process/house/GeneralInformat...

> ... which lists id's for the seats... but not anything unique to the
> members....
>
> Kevin
>
>
>
> Cory Horner wrote:
> > On 2-Jun-09, at 9:06 AM, Kevin McArthur wrote:
>
> >> You don't by chance have a parser that cuts up the final hansard  
> >> format to xml do you?
>
> >> I've got a few ideas for this, but let me know if you've already  
> >> done the parsing work.
>
> > The parser is specialized to stuff Hansard into my database, so I  
> > recommend you just consume the end result:
>
> >http://www.howdtheyvote.ca/api.php?call=getquotes&hansard_id=605
>
> > I'll add a few more attributes, such as topic and heading... which  
> > should make it equivalent to Hansard itself... am I missing anything  
> > else?
>
> > Cheers,
> > Cory.
>
> > >

> --
>
> Kevin McArthur
>
> StormTide Digital Studios Inc.http://www.stormtide.ca
>

>  smime.p7s
> 4KViewDownload

[Kevin McArthur]

Unless I'm mistaken, the EDID's represent a riding though, not a member... I've got clips going back to 2007, and the make-up of the house has changed a lot in that time. I need a way to tag a member uniquely. The webinfo site seems to use a long key-identifier... eg ( http://webinfo.parl.gc.ca/MembersOfParliament/ProfileMP.aspx?Key=128634&Language=E ) but I'm not sure where this data comes from, or how it would be correlated with Cory's xml datafile for the quotes.

I'm tempted to just parse, http://webinfo.parl.gc.ca/MembersOfParliament/MainMPsCompleteList.aspx?TimePeriod=Historical&Language=E starting with 1 equaling the first member in the 39th parliament -- but thats only because thats how far back my datasets go. This, however, seems less than interoperable.

K

Jennifer Bell wrote:

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "VisibleGovernment Discuss" group.
To post to this group, send email to visiblegover...@googlegroups.com

To unsubscribe from this group, send email to visiblegovernment-...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/visiblegovernment-discuss?hl=en
-~----------~----~----~----~------~----~------~--~---

  

[Jennifer Bell]

Yes, the EDIDs do represent ridings. One idea to make unique MP ids
would be to combine the riding and the date the MP was elected eg.

3500120081104

for whoever was elected in riding 35001 this November.

It's worth a question on sunlightlabs to see what they're doing. From
conversations, I got the impression people were (right now) using
congress-person ids from govtrack.org as cross links. Likely
something better is in the pipes, from Sunlight or the w3c working
group.

Jennifer

On Jun 5, 1:52 pm, Kevin McArthur <ke...@stormtide.ca> wrote:
> Unless I'm mistaken, the EDID's represent a riding though, not a
> member... I've got clips going back to 2007, and the make-up of the
> house has changed a lot in that time. I need a way to tag a member

> uniquely. The webinfo site seems to use a long key-identifier... eg (http://webinfo.parl.gc.ca/MembersOfParliament/ProfileMP.aspx?Key=*128...

> ) but I'm not sure where this data comes from, or how it would be
> correlated with Cory's xml datafile for the quotes.
>

> I'm tempted to just parse,http://webinfo.parl.gc.ca/MembersOfParliament/MainMPsCompleteList.asp...

[Cory Horner]

On Fri, Jun 5, 2009 at 10:10 AM, Kevin McArthur <ke...@stormtide.ca> wrote:

How are you indexing the member id's. Does this come from somewhere?

This is internal to howdtheyvote.ca, and they refer to an individual regardless of session or house.

http://www.howdtheyvote.ca/api.php?call=getmembers&session_id=10

gives you something to cross reference... and it includes some useful info like EDID and riding name too.

[Luke Closs]

Where do the member numbers from parl.gc.ca come from?  They seem to be unique per-person.

http://webinfo.parl.gc.ca/MembersOfParliament/ProfileMP.aspx?Key=128634

Luke

[Kevin McArthur]

Cory,

Thanks, I was able to import this data.

Only one bug, which was, Josée Beaudin doesn't have an official website value, which i was using to scrape the Key=> identifier... her's happens to be 128335. So I've now got a table complete with webinfo Id's as well which should be useful for tying into the commons seating plans etc...

Do you have a similar table for caucus. I think I could scrap it from webinfo at this point, but if you have it, it will save me spidering the webinfo site.

K

Cory Horner wrote:

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "VisibleGovernment Discuss" group.
To post to this group, send email to visiblegover...@googlegroups.com
To unsubscribe from this group, send email to visiblegovernment-...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/visiblegovernment-discuss?hl=en
-~----------~----~----~----~------~----~------~--~---

[Cory Horner]

On 8-Jun-09, at 4:24 PM, Kevin McArthur wrote:

> Only one bug, which was, Josée Beaudin doesn't have an official
> website value, which i was using to scrape the Key=> identifier...
> her's happens to be 128335. So I've now got a table complete with
> webinfo Id's as well which should be useful for tying into the
> commons seating plans etc...

corrected... thanks!

> Do you have a similar table for caucus. I think I could scrap it
> from webinfo at this point, but if you have it, it will save me
> spidering the webinfo site.

Sure... i've exposed the "party" element now in the getmembers call.

Do you have video dumps for the period outside of question period? At
the moment it looks parlvu expects me to watch 8 hours of streaming
video (i can't jump to another time index), in order to figure out how
M. Paradis voted today (at least right now, they have him down as
voting both "yea" and being "paired")...

http://webinfo.parl.gc.ca/MembersOfParliament/ProfileMP.aspx?Key=128163&SubSubject=1006&Language=E

Cheers,
Cory.

[Kevin McArthur]

Do you have video dumps for the period outside of question period? 

No, because that would take up more disk space than I've got for hosting. The same process by which I record the QP stream could theoretically be used on the entire day though.

Sure... i've exposed the "party" element now in the getmembers call.

Thanks.

K

Cory Horner wrote:

Cheers,
Cory.
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "VisibleGovernment Discuss" group.
To post to this group, send email to visiblegover...@googlegroups.com

To unsubscribe from this group, send email to visiblegovernment-...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/visiblegovernment-discuss?hl=en
-~----------~----~----~----~------~----~------~--~---

  

[Cory Horner]

On 9-Jun-09, at 10:21 AM, Kevin McArthur wrote:

>> Do you have video dumps for the period outside of question period?
> No, because that would take up more disk space than I've got for
> hosting. The same process by which I record the QP stream could
> theoretically be used on the entire day though.

Maybe we should look at putting those up on Amazon S3... (it's quite
affordable for static content)

I'd definitely contribute to hosting costs for that since i really
want to be able to link to video for any given quote... and VG may
help put a campaign behind it... moving towards an "open hosting
costs" donation model.

[Kevin McArthur]

Cory,

See what you can do with this.... these are cut down versions of the two primary scripts for recording from parlvu that i use. (Same how-sausages-are-made remarks apply)

Notes:

endpos controls how long the script records for, in seconds.

------ asxparse.php -------
<?php
$file = file_get_contents("http://parlvu.parl.gc.ca/parlvu/asx/A001-V-EN-VH.asx");
preg_match('/"(http:\/\/[^"]+)"/',$file,$matches);
$file2 = file_get_contents($matches[1]);
preg_match('/"(mms:\/\/[^"]+)"/',$file2,$matches2);
echo $matches2[1];


------ recorder.script -------

#!/bin/bash
date=`date +%m-%d-%Y`
file=/usr/local/www/mycelium.chanterelle.ca/content/recordings/qp-$date.mpg
stream=`/usr/bin/php /usr/local/www/mycelium.chanterelle.ca/asxparse.php`
options=vbitrate=500:vcodec=mpeg2video:keyint=500
log=/var/log/hoc.log
mencoder=/usr/bin/mencoder
ffmpeg=/usr/bin/ffmpeg
#endpos=30
endpos=3360

$mencoder $stream -ovc lavc -oac mp3lame -vf harddup -fps 30 -lavcopts $options -o $file -of mpeg -endpos $endpos >> $log 2>&1
$ffmpeg -i $file -acodec mp3 -ar 44100 -ab 96 -vcodec mpeg2video -r 30hz -b 500k -f mpeg $file.scaled >> $log 2>&1
/bin/mv -f $file.scaled $file >> $log 2>&1


Cory Horner wrote:

Maybe we should look at putting those up on Amazon S3... (it's quite  
affordable for static content)

I'd definitely contribute to hosting costs for that since i really  
want to be able to link to video for any given quote...  and VG may  
help put a campaign behind it...  moving towards an "open hosting  
costs" donation model.

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "VisibleGovernment Discuss" group.
To post to this group, send email to visiblegover...@googlegroups.com

To unsubscribe from this group, send email to visiblegovernment-...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/visiblegovernment-discuss?hl=en
-~----------~----~----~----~------~----~------~--~---