10 principles for open data

15 views
Skip to first unread message

Daniel Schuman

unread,
Aug 11, 2010, 11:45:33 AM8/11/10
to sunlig...@googlegroups.com, openhous...@googlegroups.com
Hi everyone,

Over the last few years, people have been trying to identify/refine/codify principles for making data (especially government data) open to the public. Several years ago a conference took place on identifying these open data principles, and additional efforts have been made since then to flesh out the ideas.

Standing on the shoulders of these efforts, I've put together (and slightly refined and expanded) these principles into a document that lays them out for the intelligent layperson. It's intended as a policy -- and not technology -- document, and should serve as an introduction for policymakers interested in this kind of stuff.

I've put up an explanatory blogpost here, and the full document is available here.

This isn't a finished product, but rather another step along the way. I welcome your suggestions and corrections.

Thanks,

Daniel

Daniel Schuman
Director, Advisory Committee on Transparency
Policy Counsel | The Sunlight Foundation
o: 202-742-1520 x 273 | c: 202-713-5795
Twitter: danielschuman

Josh Tauberer

unread,
Aug 11, 2010, 5:13:55 PM8/11/10
to sunlig...@googlegroups.com, Daniel Schuman, openhous...@googlegroups.com
I think you've seen it, but I thought for others I should just note that
I wrote a similar but much longer update to the original 8 principles
last year. It's on the revamped www.opengovdata.org site.

I've found that around 20 distinct principles have been proposed by
various people and groups and they're all interesting.

- Josh Tauberer
- CivicImpulse / GovTrack.us

http://razor.occams.info | www.govtrack.us | civicimpulse.com

"Members of both sides are reminded not to use guests of the
House as props."

On 08/11/2010 11:45 AM, Daniel Schuman wrote:
> Hi everyone,
>
> Over the last few years, people have been trying to
> identify/refine/codify principles for making data (especially government
> data) open to the public. Several years ago a conference took place on
> identifying these open data principles, and additional efforts have been
> made since then to flesh out the ideas.
>
> Standing on the shoulders of these efforts, I've put together (and
> slightly refined and expanded) these principles into a document that
> lays them out for the intelligent layperson. It's intended as a policy
> -- and not technology -- document, and should serve as an introduction
> for policymakers interested in this kind of stuff.
>

> I've put up an explanatory blogpost here <http://bit.ly/ctkygR>, and the
> full document is available here <http://bit.ly/bWAJ6A>.


>
> This isn't a finished product, but rather another step along the way. I
> welcome your suggestions and corrections.
>
> Thanks,
>
> Daniel
>
> Daniel Schuman
> Director, Advisory Committee on Transparency
> Policy Counsel | The Sunlight Foundation
> o: 202-742-1520 x 273 | c: 202-713-5795
> Twitter: danielschuman
>

> --
> You received this message because you are subscribed to the Google
> Groups "sunlightlabs" group.
> To post to this group, send email to sunlig...@googlegroups.com.
> To unsubscribe from this group, send email to
> sunlightlabs...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/sunlightlabs?hl=en.

Joseph Lorenzo Hall

unread,
Aug 12, 2010, 5:19:10 PM8/12/10
to openhouseproject, sunlightlabs, Daniel Schuman
Hey Josh, can we get a link? ::) best wishes, Joe

> "Open House Project" group.
> To post to this group, send email to openhous...@googlegroups.com.


> To unsubscribe from this group, send email to

> openhouseproje...@googlegroups.com.


> For more options, visit this group at

> http://groups.google.com/group/openhouseproject?hl=en.
>
>

--
Joseph Lorenzo Hall
ACCURATE Postdoctoral Research Associate
UC Berkeley School of Information
Princeton Center for Information Technology Policy
http://josephhall.org/

Josh Tauberer

unread,
Aug 12, 2010, 10:16:43 PM8/12/10
to openhous...@googlegroups.com, Joseph Lorenzo Hall, sunlightlabs, Daniel Schuman
On 08/12/2010 05:19 PM, Joseph Lorenzo Hall wrote:
> Hey Josh, can we get a link? ::) best wishes, Joe
>
> On Wed, Aug 11, 2010 at 5:13 PM, Josh Tauberer<taub...@govtrack.us> wrote:
>> I think you've seen it, but I thought for others I should just note that I
>> wrote a similar but much longer update to the original 8 principles last
>> year. It's on the revamped www.opengovdata.org site.

Here's the direct link:
http://razor.occams.info/pubdocs/opendataciviccapital.html

In summary, the total list of recommendations I found (and liked) were:

1. Available on the Internet for free (OKD's "access" and Sunlight's
Principles for Transparency in Government)

From the original 8 (some overlapping with the OKD):

2. Primary (and I'd throw "Complete" in here, in retrospect --- I never
really understood "complete" although I had a dream about it recently...)
3. Timely
4. Accessible (industry standards, bulk)
5. Machine Processable
6. Non-discriminatory
7. Non-proprietary
8. License-free

Clay Johnson first pointed out to me the obvious missing one, and Silona
Bonewald has looked at it more concretely:

9. Permanence / Preservation

From the Association of Computing Machinery's Recommendation on Open
Government:

10. Formats and Approaches that Promote Analysis
11. Safe file formats that don't contain executable content
12. Provenance and trust

From the Association of Government Accountants, and to some extent the
Open House Project and the 8 Principles:

13. Public Input
14. Public Review of the Data Collection Process

From me:

15. Engage in interagency coordination when it would be useful to
establish basic standards
16. Avoid singling out technologies that are also endorsements for
particular corporations

From Google: Use sitemaps, robots, and deep-searchable forms properly

Use globally unique identifiers for things in the data

From me, W3C: Use Linked Open Data (i.e. semantic web tech)

In my opinion, only the first 8 truly describe "open government data".
Everything else is icing on the cake --- how to do openness well, but
beyond at least the first hurdle of making it minimally open. That's why
I call these best practices for open government data, rather than simply
principles of open government data. So despite all of what I listed, I
don't find a compelling reason to actually "revise" the original 8
Principles when we're looking for a definition of openness that's
suitable for government. (The OKD is great but I think the bar is too
low for government.) But, clearly, there is a long list of things a data
publisher can do to make the data better and better for reuse, and so
there are a lot of potential best practices.

Since the Sebastopol meeting there had been a few attempts on the
related mail list and wiki to make the 8 Principles more reader
friendly. The results weren't terribly good IMO, which is why I went off
and wrote the doc above. (Thanks go to Gunnar, btw, for some help with
that.)

If others can make it even friendlier, great. (Am I going to get
involved in rehashing the details again? Probably not.) But, I would
like to suggest we avoid further fragmentation on this. We accidentally
fragmented when the Sebastopol meeting went forward without knowledge of
the work of the Open Knowledge Foundation in the UK. And lo and behold
today there are two sites on this topic, www.opengovdata.org which came
out of Sebastopol (and which I'm stewarding since no one else is) and
www.opengovernmentdata.org (!!) from the OKF guys. Plus the OpenMuni
Wiki, and Daniel's wiki page.

There are some legitimate reasons for keeping some of these things
separate. There's a real cultural difference between openness in the US
and openness in the UK and in what seems to be a European tradition ---
and so I don't think we here in the US will be able to reconcile exactly
with the OKF guys. But otherwise it would be a great idea to try to keep
these sorts of resources unified.

Daniel Schuman

unread,
Aug 19, 2010, 10:28:42 AM8/19/10
to Josh Tauberer, openhous...@googlegroups.com, Joseph Lorenzo Hall, sunlightlabs
Per Josh's excellent (as always) comment, I am making sure to link from the wiki page containing resources on the Open Data Principles to all other sources that I'm aware of. This won't fix the fragmentation that has occurred, but hopefully it will increase awareness of all the efforts that are going on.


Daniel

Daniel Schuman
Director, Advisory Committee on Transparency
Policy Counsel | The Sunlight Foundation
o: 202-742-1520 x 273 | c: 202-713-5795
Twitter: danielschuman


Reply all
Reply to author
Forward
0 new messages