Popolo and JSON-LD

[Andreas Kuckartz]

I want to support the publication and use of Linked Data.

I have a few questions regarding Popolo:

1. Which URLs point to data which is published using Popolo ?
Do lists of such URLs exist ?
Is the data provided as JSON-LD serialization or as pure JSON ?

2. Which applications can import Popolo data ?
Is import of JSON-LD supported or only pure JSON ?

3. In case the data is not published as JSON-LD:
What were or are the main reasons to prefer pure JSON to JSON-LD ?

4. Is anyone aware of tools to convert pure Popolo JSON to a JSON-LD
serialization ?

(I am certainly also interested if anybody uses Turtle ;-)

Cheers,
Andreas

[Tony Bowden]

On 8 April 2015 at 06:35, Andreas Kuckartz <a.kuc...@ping.de> wrote:
> 1. Which URLs point to data which is published using Popolo ?
> Do lists of such URLs exist ?
> Is the data provided as JSON-LD serialization or as pure JSON ?

EveryPolitician.org is trying to collect such a list. Its own list is
currently only an embedded JavaScript array, though we plan to
separate that out as a separate data file Real Soon Now.

> 2. Which applications can import Popolo data ?
> Is import of JSON-LD supported or only pure JSON ?

Other people can answer about others, but the one I'm most familiar
with at the minute is WriteIt. It imports by giving it a PopIt API
URL, from which it then downloads pure JSON for processing. To the
best of my knowledge PopIt does not export JSON-LD, so that wouldn't
be an option for WriteIt.

In general the entire Poplus eco-system to date appears to be
operating with JSON, rather than JSON-LD. Moving towards JSON-LD would
certainly be _possible_ IMO — but either someone would need to be a
compelling argument why everyone should do so, or a couple of the main
components would need to simply switch and force everyone to follow.
But again I don't expect this to happen unless someone can clearly and
convincingly explain why it should.

Tony

[Nick Jackson]

Although the whole Poplus ecosystem speaks JSON (which makes a lot more sense than JSON-LD for a lot of reasons), surely it's also a relatively small step to having a PopIt instance also make objects available in a variety of Linked Data formats such as JSON-LD, RDF/XML, N-Triples etc along with RDFa embedded directly in the page?

This seems the best of both worlds to me – people who just want Popolo-compliant data can grab the JSON, those who want to plug a PopIt instance into a more connected Linked Data world have the triples readily available without an intermediate step.

Nick

[James McKinney]

> On Apr 8, 2015, at 1:35 AM, Andreas Kuckartz <A.Kuc...@ping.de> wrote:
>
> I want to support the publication and use of Linked Data.
>
> I have a few questions regarding Popolo:
>
> 1. Which URLs point to data which is published using Popolo ?
> Do lists of such URLs exist ?
> Is the data provided as JSON-LD serialization or as pure JSON ?

All the Popolo-compliant data I’m aware of uses plain JSON. There’s no comprehensive registry - I’m not sure what the utility of such a registry would be, outside of focused projects/campaigns like EveryPolitician.

> 2. Which applications can import Popolo data ?
> Is import of JSON-LD supported or only pure JSON ?

All the tools I know import plain JSON, not JSON-LD. In general, if an application exports Popolo-compliant data, there’s a good chance it can import Popolo-compliant data. I can point to some.

> 3. In case the data is not published as JSON-LD:
> What were or are the main reasons to prefer pure JSON to JSON-LD ?

Since very few people who use legislative data use linked data tools, there hasn’t been a big incentive to use JSON-LD, which takes a little more work than plain JSON.

> 4. Is anyone aware of tools to convert pure Popolo JSON to a JSON-LD
> serialization ?

For any Popolo-compliant plain JSON document, you can (for most properties) just apply the appropriate JSON-LD context; in other words, most plain JSON documents are JSON-LD documents with an implicit @context. While there are some differences between plain JSON and JSON-LD serializations [1], many of those differences aren’t realized, since the JSON-LD contexts, for example, alias the common vote result values (which are the strings “pass” and “fail" in plain JSON) to an appropriate URI.

1. http://www.popoloproject.com/specs/#serialization

James

> (I am certainly also interested if anybody uses Turtle ;-)
>
> Cheers,
> Andreas
>

> --
> Poplus.org - Get involved: http://poplus.org/get-involved
> IRC: #poplus https://webchat.freenode.net
> Docs: http://bit.ly/poplusdrive
> ---
> You received this message because you are subscribed to the Google Groups "Poplus - Collaborative Civic Coding" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to poplus+un...@googlegroups.com.
> To post to this group, send an email to pop...@googlegroups.com.
> Visit this group at http://groups.google.com/group/poplus.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/poplus/5524BE2C.2040207%40ping.de.
> For more options, visit https://groups.google.com/d/optout.

[Andreas Kuckartz]

Am 08.04.2015 23:53, schrieb James McKinney:
> All the Popolo-compliant data I’m aware of uses plain JSON. There’s
> no comprehensive registry - I’m not sure what the utility of such a
> registry would be, outside of focused projects/campaigns like
> EveryPolitician.

Data can not be used if it can not be accessed. And it can not be
accessed if it is not known where it is or that it does exist at all.

I would like to analyse the data, convert it to RDF and do something
with that heap of Linked Data (and in combination with other Linked
Data). Currently I can do none of these.

>> 4. Is anyone aware of tools to convert pure Popolo JSON to a
>> JSON-LD serialization ?
>
> For any Popolo-compliant plain JSON document, you can (for most
> properties) just apply the appropriate JSON-LD context; in other
> words, most plain JSON documents are JSON-LD documents with an
> implicit @context.

I am interested in machine readable Linked Data. Implicit contexts are
out of band information and therefore prevent machine readability.

And if the implicit context does not apply to all properties there is
an information loss, something I do not find acceptable because the
JSON-LD serialization should contain at least as much information as
the pure JSON version.

> While there are some differences between plain JSON and JSON-LD
> serializations [1], many of those differences aren’t realized,
> since the JSON-LD contexts, for example, alias the common vote
> result values (which are the strings “pass” and “fail" in plain
> JSON) to an appropriate URI.

If there were no difference between these serializations then the only
difference would be the inclusion of a context - which is almost
trivial and does not really require JSON-LD know-how.

The critical differences are those where a JSON-LD context can not help.

Cheers,
Andreas

[Andreas Kuckartz]

Am 08.04.2015 08:46, schrieb Tony Bowden:
> EveryPolitician.org is trying to collect such a list. Its own list is
> currently only an embedded JavaScript array, though we plan to
> separate that out as a separate data file Real Soon Now.

Ok, thanks!

>> 2. Which applications can import Popolo data ?
>> Is import of JSON-LD supported or only pure JSON ?
>

> In general the entire Poplus eco-system to date appears to be
> operating with JSON, rather than JSON-LD. Moving towards JSON-LD would
> certainly be _possible_ IMO — but either someone would need to be a
> compelling argument why everyone should do so, or a couple of the main
> components would need to simply switch and force everyone to follow.
> But again I don't expect this to happen unless someone can clearly and
> convincingly explain why it should.

At the moment I am still trying to understand the situation, while
(hopefully) compelling arguments will come later.

Cheers,
Andreas

[Andreas Kuckartz]

Am 08.04.2015 15:32, schrieb Nick Jackson:
> Although the whole Poplus ecosystem speaks JSON (which makes a lot
> more sense than JSON-LD for a lot of reasons),

I would really like to know those reasons in detail.

> surely it's also a relatively small step to having a PopIt instance
> also make objects available in a variety of Linked Data formats
> such as JSON-LD, RDF/XML, N-Triples etc along with RDFa embedded
> directly in the page?
>
> This seems the best of both worlds to me – people who just want
> Popolo-compliant data can grab the JSON, those who want to plug a
> PopIt instance into a more connected Linked Data world have the
> triples readily available without an intermediate step.

I certainly agree! But I at the moment I think that it is even more
important to understand the situation to make improvements.

Cheers,
Andreas

[James McKinney]

On Apr 13, 2015, at 1:19 AM, Andreas Kuckartz <A.Kuc...@ping.de> wrote:

Am 08.04.2015 23:53, schrieb James McKinney:

All the Popolo-compliant data I’m aware of uses plain JSON. There’s
no comprehensive registry - I’m not sure what the utility of such a
registry would be, outside of focused projects/campaigns like
EveryPolitician.

Data can not be used if it can not be accessed. And it can not be
accessed if it is not known where it is or that it does exist at all.

I would like to analyse the data, convert it to RDF and do something
with that heap of Linked Data (and in combination with other Linked
Data). Currently I can do none of these.

Thanks for explaining your use case. Having access to all available Popolo data is not a common use case, which is why there is no registry of all Popolo data. More common use cases are, “I want a list of all Canadian MPs” or “I want an API for all votes in the House of Commons.” In such use cases, the format is not important; the person will use whatever data they can find. It’s a small number of people searching specifically for Popolo data. Also, in such cases, the use case is geographically scoped, so what matters is local awareness/promotion, not international awareness/promotion, which is more difficult.

So, for your use case, I think you’ll need to do some research yourself, which if you share with the community, others might contribute to. Besides various PopIt deployments, I also know of:

http://docs.opencivicdata.org/en/latest/api/index.html (same API is at http://scrapers.herokuapp.com/)

https://github.com/KohoVolit/api.parldata.eu

https://github.com/tmtmtmtm/eduskunta-popolo

4. Is anyone aware of tools to convert pure Popolo JSON to a
JSON-LD serialization ?

For any Popolo-compliant plain JSON document, you can (for most
properties) just apply the appropriate JSON-LD context; in other
words, most plain JSON documents are JSON-LD documents with an
implicit @context.

I am interested in machine readable Linked Data. Implicit contexts are
out of band information and therefore prevent machine readability.

JSON-LD has not been the preferred serialization for Popolo data. The only way to avoid out-of-band contexts is to eliminate the plain JSON serialization, which I think will be an unpopular choice.

And if the implicit context does not apply to all properties there is
an information loss, something I do not find acceptable because the
JSON-LD serialization should contain at least as much information as
the pure JSON version.

The JSON-LD serialization can contain the same information as the plain JSON serialization. It’s just that in some specific cases, the two serializations will look different. Those differences are not resolvable without making the plain JSON more complicated for no apparent reason. For example, a plain JSON Speech may be:

{

  “creator_id”: “john-f-kennedy”,

  “text”: “Ask not what your country can do for you…”,

  “audio”: "http://example.com/audio/jfk-inauguration.ogg”

}

But the JSON-LD Speech would be:

{

  “@context”: "http://www.popoloproject.com/contexts/speech.jsonld”,

  “@type”: “Speech”,

  “@id”: “http://example.com/speeches/jfk-inauguration”,

  “creator”: “http://example.com/speakers/john-f-kennedy”,

  “text”: “Ask not what your country can do for you…”,

  “audio”: {

    "url”: "http://example.com/audio/jfk-inauguration.ogg”

  }

}

The two differences are (1) that plain JSON allows using project-specific identifiers in the *_id fields, whereas JSON-LD shouldn’t use any *_id fields and (2) that “audio” must be a schema:AudioObject in JSON-LD. To my knowledge, JSON-LD contexts don’t allow you to say, “alias the property ‘audio' to a blank node of type schema:AudioObject with the property ‘url’.”

Allowing the use of internal identifiers without having to produce a URL for those identifiers has been a clear use case, hence the *_id fields. There has been no use case for adding additional properties about an audio file, which is why the plain JSON is just a direct link instead of an object as in JSON-LD.

To eliminate the difference on the audio property, I propose substituting og:audio for schema:audio.

James

[Andreas Kuckartz]

Am 13.04.2015 17:28, schrieb James McKinney:
>> On Apr 13, 2015, at 1:19 AM, Andreas Kuckartz

>> <A.Kuc...@ping.de> wrote: I would like to analyse the data,

>> convert it to RDF and do something with that heap of Linked Data
>> (and in combination with other Linked Data). Currently I can do
>> none of these.
>

> More common use cases are, “I want a list of all Canadian MPs” or
> “I want an API for all votes in the House of Commons.”

These are perfect examples of "do something with that heap of Linked
Data" ;-)

> I think you’ll need to do some research yourself, which if you
> share with the community, others might contribute to.

The is now a Wiki page on which I am collecting such information
(starting with the info you provided):
https://github.com/OpenGovLD/specs/wiki/List-of-data-providers

> The only way to avoid out-of-band contexts is to eliminate the
> plain JSON serialization, which I think will be an unpopular
> choice.

That can easily be misunderstood: The addition of support for media
type "application/ld+json" does *not* require the removal of support
for "application/json".

Cheers,
Andreas

[Martín Szyszlican]

Hi, this conversation was interesting. Did you guys reach an agreement on how to add json-ld to popolo?

I have no use case for it, but I like standardy standards.

--
Poplus.org - Get involved: http://poplus.org/get-involved
   IRC: #poplus https://webchat.freenode.net
   Docs: http://bit.ly/poplusdrive
---
You received this message because you are subscribed to the Google Groups "Poplus - Collaborative Civic Coding" group.
To unsubscribe from this group and stop receiving emails from it, send an email to poplus+un...@googlegroups.com.

To post to this group, send an email to pop...@googlegroups.com.
Visit this group at http://groups.google.com/group/poplus.

To view this discussion on the web, visit https://groups.google.com/d/msgid/poplus/552CA7C0.20700%40ping.de.

For more options, visit https://groups.google.com/d/optout.

--

Martín Szyszlican

Desarrollo web usable y accesible

martinszyszlican.com

[Paul Mackay]

James, just responding on a couple of your earlier points:

> (1) that plain JSON allows using project-specific identifiers in the *_id fields, whereas JSON-LD shouldn’t use any *_id fields

Isnt it possible to have both? A JSON-LD object could have "id" and "@id" if necessary.

> Allowing the use of internal identifiers without having to produce a URL for those identifiers has been a clear use case, hence the *_id fields.

Could you please expand a bit on these use cases specifically?

I'm currently doing some civic API work and Popolo has been a big inspiration. I'm also keen to encourage linked data where it makes sense. So this is a useful conversation.

Paul

To post to this group, send email to pop...@googlegroups.com.

Visit this group at http://groups.google.com/group/poplus.

To view this discussion on the web, visit https://groups.google.com/d/msgid/poplus/CAKTxXVV1W7v9gxzjKxaqk6fKs%2BTsMZ6bRFBrL5ew_LSWgwZbMw%40mail.gmail.com.

[James McKinney]

On Jun 2, 2015, at 4:42 AM, Paul Mackay <paulj...@gmail.com> wrote:

James, just responding on a couple of your earlier points:

> (1) that plain JSON allows using project-specific identifiers in the *_id fields, whereas JSON-LD shouldn’t use any *_id fields

Isnt it possible to have both? A JSON-LD object could have "id" and "@id" if necessary.

Sure, but the *_id properties are not aliased to any property in RDF.

> Allowing the use of internal identifiers without having to produce a URL for those identifiers has been a clear use case, hence the *_id fields.

Could you please expand a bit on these use cases specifically?

It’s as simple as most developers do not want to be concerned with authoring URLs for everything, and very few object-relational mappers (ORMs) facilitate this, so it would be additional work for those developers.

J

[Andreas Kuckartz]

James McKinney wrote:
>> On Jun 2, 2015, at 4:42 AM, Paul Mackay <paulj...@gmail.com

>> Isnt it possible to have both? A JSON-LD object could have "id"
>> and "@id" if necessary.
>
> Sure, but the *_id properties are not aliased to any property in
> RDF.

Such *_id properties can be mapped to RDF by extending the JSON-LD
context.

>>> Allowing the use of internal identifiers without having to
>>> produce
>> a URL for those identifiers has been a clear use case, hence the
>> *_id fields.
>>
>> Could you please expand a bit on these use cases specifically?
>
> It’s as simple as most developers do not want to be concerned with
> authoring URLs for everything, and very few object-relational
> mappers (ORMs) facilitate this, so it would be additional work for
> those developers.

I only agree with this in part. I do not doubt that in many cases some
additional effort might be required. But the main issue is that many
developers simply do not understand why it would make sense to invest
that effort.

If developers have the choice to either render pure JSON or JSON-LD
without understanding the benefits of JSON-LD they understandably will
decide against JSON-LD.

Explaining and demonstrating those benefits is key.

There probably will be a chance to move the balance in Europe a bit in
favor of JSON-LD soon, but I can not yet make any announcement.

Cheers,
Andreas