Building on my earlier post about listing collaborative options for government or congressional agencies, I'm thinking about useful ways to distinguish between different types of government information, and what that implies about records management.
At the recent IPDI Politics Online panel on radical transparency<http://polc.ipdi.org/Agenda/radicaltransparency.htm>, Peggy Garvin made a great point about one fundamental distinction that can be made within government information. She suggested that all government information is either collected from regulated entities, or pertains to the operations of government itself.
Other fundamental distinctions could be made about government information generally; for example, all government information is either classified or not (attempt to equivocate notwithstanding.)
Reflecting on this point, I thought of the various Creative Commons<http://creativecommons.org/>designations to be asserted over works. These distinctions have been developed as technology made use and reuse of (especially) digital information easier by an order of magnitude, which, in turn, rendered previous regulatory systems inadequate both de jure and in spirit. The same could now be said of both privacy law and many privacy policies, whose phrasing is too vague to apply clearly to new digital circumstances. (Technology that changes rapidly can be a real headache for legislative drafting: for example, the line between terms of art and informal slang is fuzzy at best.)
Back to government information, the same thing that happened to music because of mixtapes, and that happened to video because of youtube, is also happening to government information. Regulatory schemes and records management practices need to be rethought.
What would a forward looking government information management system look like? FDSys, obviously, is the new federal documents delivery system, but I'm wondering about the new policy distinctions that are probably on their way.
For example, some documents have enough public import and enough consistent demand that they should be published online as quickly as possible, in as structured a way as possible. Votes in legislatures would fall under this distinction. Another set of documents would be those things that exist under the domain of the government, and maybe are FIOA-able, but are probably best not public. Personnel information and yearly reviews all online would make for a very complex work environment.
I suspect that people at the GSA or OMB spend their time making exactly this sort of documents management distinction for the federal government, and if anyone has public documents or OMB memos to this effect, I'd love to see them. These management practices, however, are probably based on old legislative mandates like the paperwork reduction act or the OMB A-130 circular.
I'm thinking about something more far-reaching, that could proactively assign distinctions to documents *with public access in mind*, in addition to other concerns like national security, or commercial or private or sensitive information. Another example, which information sets within the governmental (there are more than a few) are appropriate for APIs? Surely there's a better way to decide on this than on a case-by-case basis. Which documents should have permanent<http://seanmcgrath.blogspot.com/2008/03/permanent-urls-for-legislatio...>associated for them? If NIST can dictate that congressional districts will be referred to as "AL - 01" (or whatever), then shouldn't someone make similar statements about government URL/database structures?
If the US gets a CTO, (or, if the structures in place to carry out these functions are expanded), that person or those people should come into the job realizing that they're not just coordinating technology, but they're practical epistemologists, determining in practice what can be known about our national government.
(Full Disclosure: Larry Lessig <http://www.lessig.org/> is on Sunlight's Advisory board, although I had no contact with him about this post.) -- John Wonderlich
Program Director The Sunlight Foundation (202) 742-1520 ext. 234
I really like the way you are conceptualizing distinctions of types of information. I'd add one other idea to that mix.
As governments move to "e-government" I believe we are going to increasingly see government information provided as "transactions" as opposed to "instantiations." This distinction is a little different from the ones you are drawing, I think. You rightly focus on the information content and how to characterize it; I'm thinking of the delivery mechanisms.
But, I think the method of delivery is potentially very important to ensuring that the information content we access today is accessible tomorrow, and ensuring that it has not been altered, and ensuring that how the information can be used is not limited to the provisions of a particular transactional system (or even a particular API).
Here is a simple example: I can call 411 and get a phone number: that's a transaction and is a big improvement over having to locate and use a bulky telephone book which may not even be current. Lots of kinds of government information lend themselves to this kind of transaction delivery and make for better, more accurate, more timely service. But, if I am a journalist and I want to look at a directory of all employees in a department, or if I'm an historian and want to see who was in a particular office last year (or 10 or 50 years ago), or if I'm a demographer and I want to do an surname or given-name analysis of an agency's employees, then a current, up-to-date one- transaction-at-a-time system won't help me at all. I need an instantiation of the information from one or more time periods.
Transaction systems will be limited because of costs as to how robust and fully functional they are and what information they deliver and how it is delivered. Here's another example, if you want to read the Congressional Record online, it is pretty easy to get a page here and a page there. But if you want to download entire "volumes" of information to do textual analysis or build your own interface or index or just to browse... well, try it and let me know if you find a way to do any of that today!
There is a conceptually easy solution to this problem: require that information be "instantiated" at appropriate intervals and distributed in total in a reusable form so that the transaction systems made available by governments can be complemented by other systems developed by others. There is an existing example of this: The Census. We have moved from a completely print Census (instantiated) to an almost completely online (transactional) Census (at American FactFinder). But this is supplemented by the fact that the raw datafiles that are used to answer questions at American FactFinder are available for download. So, users get the best of both worlds: you can look up (in a transaction) the population of your zipcode, but you can also download all the data for all the zipcodes in your state and analyze the data any way you like.
Transactional e-government, if it is done well, will be very attractive to lots of people for lots of very good reasons. What we want to remember is that it will not serve all needs of all people for all time. For that we are going to need periodic distributions of raw information for preservation, analysis, and re-processing. Governments may not like the idea of doing this, though. They may want to keep control and may want to do so under the vise of "accuracy." (E.g., "Last year's phone book isn't accurate anymore. We don't want copies of it out in the world confusing people.") Indeed, we hear an argument something like that with regard to CRS reports. And the private sector may not like the idea of raw information being freely distributed because they want to control access so they can charge for it. (Indeed we see something like that with CRS reports!) It may be a challenge to get governments to understand this concept and, once they do, to embrace distribution.
jim jacobs data services librarian emeritus, UC San Diego freegovinfo.info
On Mar 12, 2008, at 2:22 PM, John Wonderlich wrote:
> Building on my earlier post about listing collaborative options for > government or congressional agencies, I'm thinking about useful ways > to distinguish between different types of government information, > and what that implies about records management.
> At the recent IPDI Politics Online panel on radical transparency, > Peggy Garvin made a great point about one fundamental distinction > that can be made within government information. She suggested that > all government information is either collected from regulated > entities, or pertains to the operations of government itself.
> Other fundamental distinctions could be made about government > information generally; for example, all government information is > either classified or not (attempt to equivocate notwithstanding.)
> Reflecting on this point, I thought of the various Creative Commons > designations to be asserted over works. These distinctions have > been developed as technology made use and reuse of (especially) > digital information easier by an order of magnitude, which, in turn, > rendered previous regulatory systems inadequate both de jure and in > spirit. The same could now be said of both privacy law and many > privacy policies, whose phrasing is too vague to apply clearly to > new digital circumstances. (Technology that changes rapidly can be > a real headache for legislative drafting: for example, the line > between terms of art and informal slang is fuzzy at best.)
> Back to government information, the same thing that happened to > music because of mixtapes, and that happened to video because of > youtube, is also happening to government information. Regulatory > schemes and records management practices need to be rethought.
> What would a forward looking government information management > system look like? FDSys, obviously, is the new federal documents > delivery system, but I'm wondering about the new policy distinctions > that are probably on their way.
> For example, some documents have enough public import and enough > consistent demand that they should be published online as quickly as > possible, in as structured a way as possible. Votes in legislatures > would fall under this distinction. Another set of documents would > be those things that exist under the domain of the government, and > maybe are FIOA-able, but are probably best not public. Personnel > information and yearly reviews all online would make for a very > complex work environment.
> I suspect that people at the GSA or OMB spend their time making > exactly this sort of documents management distinction for the > federal government, and if anyone has public documents or OMB memos > to this effect, I'd love to see them. These management practices, > however, are probably based on old legislative mandates like the > paperwork reduction act or the OMB A-130 circular.
> I'm thinking about something more far-reaching, that could > proactively assign distinctions to documents with public access in > mind, in addition to other concerns like national security, or > commercial or private or sensitive information. Another example, > which information sets within the governmental (there are more than > a few) are appropriate for APIs? Surely there's a better way to > decide on this than on a case-by-case basis. Which documents should > have permanent associated for them? If NIST can dictate that > congressional districts will be referred to as "AL - 01" (or > whatever), then shouldn't someone make similar statements about > government URL/database structures?
> If the US gets a CTO, (or, if the structures in place to carry out > these functions are expanded), that person or those people should > come into the job realizing that they're not just coordinating > technology, but they're practical epistemologists, determining in > practice what can be known about our national government.
> (Full Disclosure: Larry Lessig is on Sunlight's Advisory board, > although I had no contact with him about this post.) > -- > John Wonderlich
> Program Director > The Sunlight Foundation > (202) 742-1520 ext. 234
I posted recently<http://www.theopenhouseproject.com/2008/03/12/a-series-of-evolving-di...>that a new set of nuanced distinctions should be put into effect for government documents, setting priority levels and procedures for digital public access, much like security classifications and preservation schedules operate now. After James A. Jacobs thoughtful comment on "instantiation" to the google group<http://groups.google.com/group/openhouseproject/browse_thread/thread/...>(where he reminds us that transactional access only works for very specific kinds of public access needs), I went on a more detailed search for executive branch guidance on documents management.
There's far more detail available online than I expected, perhaps because the executive branch functions through centralized authority with centralized management structures. (Most notably the Office of Management and Budget, and the General Services Administration, possibly also the National Archives and Records Administration and the Government Printing Office.)
I also found M-98-09 <http://www.whitehouse.gov/omb/memoranda/m9809.html>, which guides federal agencies in creating "Government Information Locator Services", intended to offer "in one place the various ways by which a person can obtain public information from the agency, as well as the types and categories of information available." Since their publication online wasn't mandated, ("Although the law does not require that the handbook be available on-line, OMB encourages agencies to do so as a matter of policy…") I decided to look for agencies offering this sort of guide.
I took the Department of Commerce <http://www.commerce.gov/> as an example, and wasn't able to find any information outline (or GILS) on their site. I did, however, find resources relevant to my original inquiry, the one that sparked this investigation. I was looking for records management designations, and I found a host of resources on the Commerce Department's CIO's Web site<http://ocio.os.doc.gov/ITPolicyandPrograms/Information_Quality/index.htm> .
Well, the question is this. When we make laws about information, and make practical distinctions about kinds of data, are they made with the public end in mind? As the Internet forcibly exposes public information at a rate faster than we can make distinctions about it, we need to better understand the mechanisms of public information control. The strategies, successes, and shortcomings of the executive's information management history will illuminate the challenges of the legislative.
> John, > I really like the way you are conceptualizing distinctions of types of > information. I'd add one other idea to that mix.
> As governments move to "e-government" I believe we are going to > increasingly see government information provided as "transactions" as > opposed to "instantiations." This distinction is a little different from the > ones you are drawing, I think. You rightly focus on the information content > and how to characterize it; I'm thinking of the delivery mechanisms.
> But, I think the method of delivery is potentially very important to > ensuring that the information content we access today is accessible > tomorrow, and ensuring that it has not been altered, and ensuring that how > the information can be used is not limited to the provisions of a particular > transactional system (or even a particular API).
> Here is a simple example: I can call 411 and get a phone number: that's a > transaction and is a big improvement over having to locate and use a bulky > telephone book which may not even be current. Lots of kinds of government > information lend themselves to this kind of transaction delivery and make > for better, more accurate, more timely service. But, if I am a journalist > and I want to look at a directory of all employees in a department, or if > I'm an historian and want to see who was in a particular office last year > (or 10 or 50 years ago), or if I'm a demographer and I want to do an surname > or given-name analysis of an agency's employees, then a current, up-to-date > one-transaction-at-a-time system won't help me at all. I need an > instantiation of the information from one or more time periods.
> Transaction systems will be limited because of costs as to how robust and > fully functional they are and what information they deliver and how it is > delivered. Here's another example, if you want to read the Congressional > Record online, it is pretty easy to get a page here and a page there. But if > you want to download entire "volumes" of information to do textual analysis > or build your own interface or index or just to browse... well, try it and > let me know if you find a way to do any of that today!
> There is a conceptually easy solution to this problem: require that > information be "instantiated" at appropriate intervals and distributed in > total in a reusable form so that the transaction systems made available by > governments can be complemented by other systems developed by others. There > is an existing example of this: The Census. We have moved from a completely > print Census (instantiated) to an almost completely online (transactional) > Census (at American FactFinder). But this is supplemented by the fact that > the raw datafiles that are used to answer questions at American FactFinder > are available for download. So, users
