Does an Open Contracting Data Standard need a privacy policy?

[Tim Davies]

Hello all,

The Open Spending Community have recently been exploring the issue of privacy in relation to releasing government spending transactions, which got me thinking that this may be an issue we should consider during the development of the Open Contracting Data Standard.

The Open Spending draft principles for privacy can be found here: https://docs.google.com/document/d/1uAIXuKUL-L8GCI7ub01cJPxmdVO8taeR8hoPL7wyu9A/edit and are based on the idea that:

• Open Data should not contain private or personal information;
• But, some government transactions take place with individuals rather than companies, and are a matter of public record, and so will involve publication of personal information;

So - some questions we might need to think about:

• When might contracting data contain 'private' information - either personal information (e.g. disclosing an individuals address, salary or other information considered personal in a particular country), or information covered by some form of corporate confidentiality?
  
  
• Are there legitimate reasons why a publisher would not disclose certain details of the parties to a contract? E.g. not publishing name and address of a bidder or entity awarded a contract?
  .
  
• Should an Open Contracting Data Standard allow 'exclusions' like IATI does? http://iatistandard.org/how-to-publish/establish-publishing-policies/#exclusions
  
  
• Are there technical ways in which privacy could be protected, but the ability to follow the money retained? E.g. if two publishers have contracts with an individual, whose name and address details cannot be disclosed for some legitimate reason, could some hash of their details be used which would make it possible to know that these were still contracts with the same entity...

Any thought on these issues would be very welcome.

All best wishes

Tim

--

-- 

Tim Davies

Research Coordinator, Open Data Research Network

+44 7834 856 303

@timdavies | @odrnetwork | www.opendataresearch.org 

World Wide Web Foundation | 1110 Vermont Ave NW, Suite 500, Washington DC 20005, USA | www.webfoundation.org | Twitter: @webfoundation

[aball...@worldbank.org]

Hi Tim,

I can offer my thoughts on two of your questions:

1. Are there legitimate reasons why a publisher would not disclose certain details of the parties to a contract? E.g. not publishing name and address of a bidder or entity awarded a contract?

For part of my life I was head of customs in my country. We wanted to make public all information about imports and exports. Someone objected to publishing the name and addresses of suppliers in every import transaction because that would give an unfair advantage to the importer's competitors. If the importer had spent time and money (which could be substantial) doing worldwide research on the best provider, why should the State give out this information for free to the competitors? This seems reasonable to me, but not sure if it applies to government procurement contracts.

2. Are there technical ways in which privacy could be protected, but the ability to follow the money retained? E.g. if two publishers have contracts with an individual, whose name and address details cannot be disclosed for some legitimate reason, could some hash of their details be used which would make it possible to know that these were still contracts with the same entity...

Yes there are. The data owner can assign a code to each vendor and publish the contracting data under the vendor's code. This seems too simple. Am I missing something?

Regards

Amparo Ballivian Lead Economist, Development Data Group World Bank, 1818 H St. NW Washington, DC 20433 Tel: 202-458-4962 | aball...@worldbank.org | data.worldbank.org

Tim Davies ---06/10/2014 05:30:08 PM---Hello all, The Open Spending Community have recently been exploring the issue of

From:	Tim Davies <timd...@webfoundation.org>
To:	publi...@webfoundation.org
Date:	06/10/2014 05:30 PM
Subject:	[public-ocds] Does an Open Contracting Data Standard need a privacy policy?
Sent by:	publi...@webfoundation.org

---

All best wishes

Tim

--
You received this message because you are subscribed to the Google Groups "Public OCDS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to public-ocds...@webfoundation.org.
To post to this group, send email to publi...@webfoundation.org.
Visit this group at http://groups.google.com/a/webfoundation.org/group/public-ocds/.
To view this discussion on the web visit https://groups.google.com/a/webfoundation.org/d/msgid/public-ocds/CAOA4sPBPGvaFoaper_1j4EMyYPoZ1u0ZUGYy7BnrD3vpC%3DX6rg%40mail.gmail.com.

[Rachel Rank]

Hi Tim,

 

This is always a tricky issue – how to get the right balance between publishing useful information and protecting information that’s confidential or commercially sensitive (this is particularly relevant for maintaining a competitive procurement process).

 

My initial reaction is that you should develop an exclusions policy as this pre-empts some of the legitimate reasons you’ve already identified and sets out an approach for dealing with them, rather than allowing organisations to decide themselves what is/isn’t appropriate and therefore risking having a less ambitious standard. The policy needs to be specific to avoid inappropriate use and the reasons for any exemptions needs to be stated.

 

I suggest you follow some basic principles when drafting the policy:

 

(1)    No individual publisher should be granted exemptions. There should be a presumption that organisations publish everything, with exceptions only where a case can be made, for each specific contract.

(2)    In addition, publishers should ensure that any exclusions are detailed in their organisation’s information disclosure policy, open data policy or equivalent public document.

(3)    Groups of (particular types of) organisations should only be exempted entirely from particular information fields if, for 100% of their contracts, it would be logically impossible (not just difficult, or an issue of existing regulatory or confidentiality practices) or cause harm, subject to a public interest test, for all of the organisations to publish that field.

(4)    The OpenContracting Steering Committee should only endorse group exemptions if Principle (3) applies, plus:

a.       The exemption has been agreed in an open, consultative way, comprising both the relevant publishing organisations, and external stakeholders;

b.      The exemption has not only considered which information fields to remove, but also which fields to add instead.

(5)    Minimum requirements should be to provide all basic information that will be needed to make sense of a particular contract. However, these should be seen as the bare minimum, rather than “sufficient”. Publication should be ambitious and stretching; all organisations should be encouraged to aim for full compliance to the standard (otherwise you risk everyone doing the basics and not the tricky stuff that involves changes in current practice and updating disclosure policies).

 

I like Amparo’s suggestion of using codes for suppliers, although in most cases you should be able to name the contract awardee as it will be a registered company, be it an individual or a large provider. I don’t see why that would be a problem and that’s exactly the kind of information people want to see. The sum of the contract could be redacted, but I’d expect the identity of providers to be a less sensitive area for government procurement.

 

Happy to have a look through any policy you develop. I suggest you also reach out to the FOI community for advice on this. Let me know if you’d like some introductions to people at Article19, AccessInfo and the Centre for Law and Democracy.

 

Best wishes,

Rachel

 

Rachel Rank

Deputy Director

T: +44 (0)20 3176 2512

M: +44 (0)7983 409 406                    

rache...@publishwhatyoufund.org

Skype: rachel.rank

--

[Mihaly Fazekas]

Dear all,

while I cannot offer detailed responses to all the posed questions, I would like to highlight one crucial overarching issue with implementation and corruption risks.

We may well be able to define a consensus on the balance between open data and privacy policy, but then the question is whether any such balanced policy on paper would be abused in practice by those in power determined to avoid the accountability and supervision ideally generated by open contracting data. In general, it is hard to monitor from outside the nitty-gritty details of implementing specific privacy regulations especially in the absence of independent courts and strong civil society, exactly the kind of governance environments where open contracting data is likely to have the largest impact.

Taking into account the potential practical implementation risks of any ideal balanced policy, one potential solution would be to tip the balance towards more open data and less privacy considerations simply because it is more easy to implement and more resistant to abuse by powerful, often corrupt, elites. So, this approach would sacrifice some economic benefits of privacy regulations in order to escape its potential abuse. Something along these lines, at least implicitly, was implemented in Slovakia where even the submitted public procurement bids themselves are published besides many standard thing such as the identity of competing companies.

Best,
Mihaly

To view this discussion on the web visit https://groups.google.com/a/webfoundation.org/d/msgid/public-ocds/3d8fbd2453a0044cef08e9be618503d5%40mail.gmail.com.

--

Mihály Fazekas
Postdoctoral research fellow
University of Cambridge, Faculty of Human, Social and Political Sciences
www.mihalyfazekas.eu
http://ssrn.com/author=1713716

[Herb Lainchbury]

Hi All / Tim,

My sense is that the default should be open.  i.e. If you choose to do business with a public body then your business information should be public.  This would include business contact information whether or not you are operating as a corporation, partnership or sole proprietorship.  

In British Columbia, where I live, the definition of personal information excludes contact information such as what you would find on a business card.  From the BC Freedom of Information and Protection of Privacy Act:

"personal information" means recorded information about an identifiable individual other than contact information;

As far as commercially sensitive or competitive advantage, I think as long as everyone is held to the same rules then it is fair.  If a company can't compete without the additional advantage of secrecy then they are not competitive and the public shouldn't have to bear the cost of secrecy when other vendors are able to compete without it.  There may be circumstances when without secrecy no vendors would be willing to come forward (safety of employees?) so those areas may never be able to be considered "open".

Having said all that, my hope is that open contracting is itself seen as a competitive advantage.  So that given the choice between doing business with two public bodies, all things being equal except that one is open and one is not, the one that is open will be more attractive for companies wishing to do business - as the risks/costs of doing business will be lower.  And when it comes to leadership and elections, political leaders committed to open contracting principles will be seen as more trustworthy and competent and therefore will out-compete those who are not.

As such, being able to say "we adhere to open contacting principles" (or some suitable recognition) will have meaning and value.  That value will need to be protected by a rigorous standard, such as a "definition of open contracting", and parties may want to have the bar lowered so they can qualify and benefit from the label and still retain some secrecy.  I say this is the time to set the bar high and go for as open a standard as possible and that will help flush out where the exceptions actually lie.  

I am okay with some areas not being able to be open for safety reasons.  What I wouldn't want to have happen is the definition of open be watered down so these exceptions could qualify as open, when they are not.

Herb Lainchbury

Open Data Society of BC

--
You received this message because you are subscribed to the Google Groups "Public OCDS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to public-ocds...@webfoundation.org.
To post to this group, send email to publi...@webfoundation.org.
Visit this group at http://groups.google.com/a/webfoundation.org/group/public-ocds/.
To view this discussion on the web visit https://groups.google.com/a/webfoundation.org/d/msgid/public-ocds/CAOA4sPBPGvaFoaper_1j4EMyYPoZ1u0ZUGYy7BnrD3vpC%3DX6rg%40mail.gmail.com.

--

Herb Lainchbury

[aball...@worldbank.org]

One more thing. I understood Tim's initial question as related to individuals who are contracted by public agencies, not to companies contracted by public agencies. In the case of companies, including sole-proprietorship companies, the full company details are normally a matter of public record in the business registries, so I do not see a problem. The only problem would be with individuals. Right?

Amparo Ballivian Lead Economist, Development Data Group World Bank, 1818 H St. NW Washington, DC 20433 Tel: 202-458-4962 | aball...@worldbank.org | data.worldbank.org

Rachel Rank ---06/11/2014 07:12:40 AM---Hi Tim, This is always a tricky issue – how to get the right balance between

From:	Rachel Rank <rache...@publishwhatyoufund.org>
To:	publi...@webfoundation.org
Date:	06/11/2014 07:12 AM
Subject:

[public-ocds] Does an Open Contracting Data Standard need a privacy policy?

Sent by:	publi...@webfoundation.org

All best wishes

.
To view this discussion on the web visit https://groups.google.com/a/webfoundation.org/d/msgid/public-ocds/3d8fbd2453a0044cef08e9be618503d5%40mail.gmail.com.

[Friedrich Lindenberg]

Hey all, 

I wonder if this is a useful debate to have for practical reasons. I do not mean to reduce the importance of privacy considerations in the publication of data. 

Governments that publish procurement information will already have privacy legislation, and this will usually apply. So does adoption of Open Contracting imply that these laws need to be changed? I fear you then go yak shaving.

Meanwhile, I would really like to start using an open contracting data spec in my work with contracting data. This is where re-interpreting the term “standards” from a “data standard” to a “political standard” (i.e. to include policies and principles) becomes counter-productive.

Please ship a data standard.

Best wishes,

- Friedrich

p.s. The OpenSpending document is a takedown policy, not a publication policy. Thus the comparison is a bit off. 

p.p.s. I find it very concerning to find people in this thread mention contract amounts and supplier company names as examples of data affected by privacy principles; first because it mixes up privacy and commercial confidentiality; and second because it basically sets the bar for Open Contracting so low that the initiative’s impact on transparency will be null and hence this whole thing will be a huge waste of time and money. 

On 11 Jun 2014, at 18:04, aball...@worldbank.org wrote:

One more thing. I understood Tim's initial question as related to individuals who are contracted by public agencies, not to companies contracted by public agencies. In the case of companies, including sole-proprietorship companies, the full company details are normally a matter of public record in the business registries, so I do not see a problem. The only problem would be with individuals. Right?

<0E853131.jpg>

Amparo Ballivian Lead Economist, Development Data Group World Bank, 1818 H St. NW Washington, DC 20433 Tel: 202-458-4962 | aball...@worldbank.org | data.worldbank.org

<graycol.gif>Rachel Rank ---06/11/2014 07:12:40 AM---Hi Tim, This is always a tricky issue – how to get the right balance between

<ecblank.gif> From:	<ecblank.gif> Rachel Rank <rache...@publishwhatyoufund.org>
<ecblank.gif> To:	<ecblank.gif> publi...@webfoundation.org
<ecblank.gif> Date:	<ecblank.gif> 06/11/2014 07:12 AM
<ecblank.gif> Subject:	<ecblank.gif>

[public-ocds] Does an Open Contracting Data Standard need a privacy policy?

<ecblank.gif> Sent by:	<ecblank.gif> publi...@webfoundation.org
<ecblank.gif>	<ecblank.gif>
<ecblank.gif>	<ecblank.gif>

---

Hi Tim,
 
This is always a tricky issue – how to get the right balance between publishing useful information and protecting information that’s confidential or commercially sensitive (this is particularly relevant for maintaining a competitive procurement process).
 
My initial reaction is that you should develop an exclusions policy as this pre-empts some of the legitimate reasons you’ve already identified and sets out an approach for dealing with them, rather than allowing organisations to decide themselves what is/isn’t appropriate and therefore risking having a less ambitious standard. The policy needs to be specific to avoid inappropriate use and the reasons for any exemptions needs to be stated. 
 
I suggest you follow some basic principles when drafting the policy:
 

(1)    No individual publisher should be granted exemptions. There should be a presumption that organisations publish everything, with exceptions only where a case can be made, for each specific contract.
(2)    In addition, publishers should ensure that any exclusions are detailed in their organisation’s information disclosure policy, open data policy or equivalent public document.
(3)    Groups of (particular types of) organisations should only be exempted entirely from particular information fields if, for 100% of their contracts, it would be logically impossible (not just difficult, or an issue of existing regulatory or confidentiality practices) or cause harm, subject to a public interest test, for all of the organisations to publish that field.
(4)    The OpenContracting Steering Committee should only endorse group exemptions if Principle (3) applies, plus:
a.       The exemption has been agreed in an open, consultative way, comprising both the relevant publishing organisations, and external stakeholders;
b.      The exemption has not only considered which information fields to remove, but also which fields to add instead.
(5)    Minimum requirements should be to provide all basic information that will be needed to make sense of a particular contract. However, these should be seen as the bare minimum, rather than “sufficient”. Publication should be ambitious and stretching; all organisations should be encouraged to aim for full compliance to the standard (otherwise you risk everyone doing the basics and not the tricky stuff that involves changes in current practice and updating disclosure policies).

 
I like Amparo’s suggestion of using codes for suppliers, although in most cases you should be able to name the contract awardee as it will be a registered company, be it an individual or a large provider. I don’t see why that would be a problem and that’s exactly the kind of information people want to see. The sum of the contract could be redacted, but I’d expect the identity of providers to be a less sensitive area for government procurement.
 
Happy to have a look through any policy you develop. I suggest you also reach out to the FOI community for advice on this. Let me know if you’d like some introductions to people at Article19, AccessInfo and the Centre for Law and Democracy.
 
Best wishes,
Rachel
 

<0E135075.gif>

Rachel Rank
Deputy Director
T: +44 (0)20 3176 2512
M: +44 (0)7983 409 406                     
rache...@publishwhatyoufund.org
Skype: rachel.rank

<0E662526.gif><0E718750.gif>

 
From: publi...@webfoundation.org [mailto:publi...@webfoundation.org] On Behalf Of Tim Davies
Sent: 10 June 2014 22:30
To: publi...@webfoundation.org
Subject: [public-ocds] Does an Open Contracting Data Standard need a privacy policy?
 
Hello all,

The Open Spending Community have recently been exploring the issue of privacy in relation to releasing government spending transactions, which got me thinking that this may be an issue we should consider during the development of the Open Contracting Data Standard.
The Open Spending draft principles for privacy can be found here:https://docs.google.com/document/d/1uAIXuKUL-L8GCI7ub01cJPxmdVO8taeR8hoPL7wyu9A/edit and are based on the idea that:

• Open Data should not contain private or personal information;
• But, some government transactions take place with individuals rather than companies, and are a matter of public record, and so will involve publication of personal information;

So - some questions we might need to think about:

• When might contracting data contain 'private' information - either personal information (e.g. disclosing an individuals address, salary or other information considered personal in a particular country), or information covered by some form of corporate confidentiality?
• Are there legitimate reasons why a publisher would not disclose certain details of the parties to a contract? E.g. not publishing name and address of a bidder or entity awarded a contract?
  .

• Should an Open Contracting Data Standard allow 'exclusions' like IATI does?http://iatistandard.org/how-to-publish/establish-publishing-policies/#exclusions 

To view this discussion on the web visithttps://groups.google.com/a/webfoundation.org/d/msgid/public-ocds/OF8551E8D5.453512E4-ON85257CF4.005287A4-85257CF4.0052C411%40worldbank.org.

[Daniel Dudis]

Hello all,

 

From the perspective of civil society groups monitoring public procurements for corruption and fraud, more information is obviously better.  The only types of information I can think of off hand that might warrant protection might be 1) the addresses/contact information of any experts/consultants identified in bids/proposals and 2) highly complex technical descriptions of goods that truly contain trade secrets.  I realize that the second category becomes really hard to define, because almost every company will claim that their bids/proposals contain trade secrets.  The reality is of course otherwise, as unless the procurement is for some innovative computer system or weapons program or other innovative product, it is highly unlikely that bids/proposals will contain anything that truly warrants confidentiality.  The reality is that most of what governments procure, be it roads, buildings, office equipment, medicines, or consulting services is not going to involve legitimate claims for confidentiality.

 

Best,

 

Dan

 

Daniel Dudis

 

Senior Policy Director – Government Accountability

Transparency International – USA

1023 15th Street NW Suite 300

Washington, DC 20005

www.transparency-usa.org

ddu...@transparency-usa.org

202 589 1616

 

 

From: publi...@webfoundation.org [mailto:publi...@webfoundation.org] On Behalf Of Tim Davies

Sent: Tuesday, June 10, 2014 5:30 PM
To: publi...@webfoundation.org

--

[Tim Davies]

Hello all,

Thanks for all the deep insights into this issue.

So - if I summarize a couple of things I think are coming up in this discussion:

• There is a distinction between 'privacy', 'exclusions' and 'takedown' policies or practices. In some countries there is an assumption that any personal information involved in contracting is public information. Would could assume that privacy laws are in place in most countries to handle the relevant issues - but in practice many countries currently adopting open data practices lack robust privacy laws. It's not the place of a data standard to fill this gap - but it's development does need to be cognizant of the fact it may be used where we cannot assume good practices around personal data to be being practiced. 
  
  
• There may be times when it is justifiable to exclude information from the contracting data that is published (e.g. public safety or safety of individuals; some cases of commercial confidentiality) but a standard should assume openness as the very strong default, and could/should assume that the decisions over when to apply exclusions is (a) based on existing national law; and (b) something to address during adoption of a standard and not before.
  
  
• Any privacy/exclusion/takedown policies or features within a standard risk being abused - and if any are enacted - then there needs to be strong transparency about their use.
  
  
• We may be able to consider cases where safety / commercial confidentiality etc. would lead to missing or excluded data as edge cases, and to deal with them when they arise rather than in design of the standard. Or there may be some things, like encouraging publishers to always provide an internal ID for suppliers etc. even if they cannot provide full details etc. which would allow some use-cases (finding out when multiple contracts are going to the same supplier) to still be possible even when there are cases of missing/excluded data.
  

My greatest interest was in whether at this point there were technical approaches we could take to pre-empt the situation where, because of privacy/exclusion issues someone fails to put any information out at all in the standard, or to make handling inevitable exclusions easier. For example, when contract notices are published, should we consider a flag for 'Excluded data' and even request pointers to the party responsible for authorizing the exclusion? (That way, those interested in what data is being omitted can dig into what's missing - and if data appears to be missing without authorization - can address this through advocacy etc.)

I think in response to Friedrich's points, from a social scientists perspective, I would argue that that all data standards are, if they receive any level of adoption, inherently political, in that they make decisions about what is important to know or not and what gets represented or not - even down to the technical data level of field choice having consequences for who can or can't use the data to do X, or how easy it is or not for an approach to exclusions being included later which minimises the impact of exclusions on the ability of the standard to meet all the key use cases we're exploring.

That's why in this OCDS process we're trying to avoid just putting out something as a de-facto standard, but to, as rapidly as possible, get towards something that both works technically, and politically, to drive forward open contracting.

By way of a quick process update on that:

- Next week we're planning to publish the Conceptual Model for the standard for comments.

- By the end of August we hope to be shipping field level information on what Open Contracting Records (and other associated released of information) could contain for consultation

- After that we'll be working towards a fully documented standard by the end of the year. That will focus on the *data standard*, and if there are other policies etc. required (like privacy) I anticipate these will need to follow at that point / during piloting and adoption...

Thanks again

Tim

[Rachel Rank]

Hi Tim,

 

Thanks for the useful summary.

 

In response to your question below about exclusions, based on our experience with IATI, I would suggest waiting until you have some data before developing any kind of exclusions policy or flags. I wouldn’t recommend you try to pre-empt what they might be at this stage. I agree you need a process for deciding exemptions (see my earlier email) and a way of flagging what they are, but wait until you have the initial standard and the data is flowing would be my advice, then organisations can start addressing this themselves via the communities of practice.

 

Looking forward to seeing the conceptual model next week!

 

Rachel

 

Rachel Rank

Deputy Director

T: +44 (0)20 3176 2512

M: +44 (0)7983 409 406                    

rache...@publishwhatyoufund.org

Skype: rachel.rank

To view this discussion on the web visit https://groups.google.com/a/webfoundation.org/d/msgid/public-ocds/CAOA4sPB7mwLx_w8e49NtXUrhb2Zcb0Rw1KJ0sATaiWy87i9vkg%40mail.gmail.com.

[Bibhusan Bista]

Hello Everyone,

I totally agree with Rachel and this was going through my mind since the begining of this thread. Rather than trying to embrace exclusion policies beforehand, I think it would be nice to first see data coming from the countries based on the standard and then focus on finding out exclusions. 

Also, based on our ongoing work on developing an open contract visualizaiton platform in Nepal, we see that different organisations might perceive exclusions in different manner. The key to have public agencies agreeing to open contracting principle would be first building their capacities to follow standards in publishing data (might be based on what they are already doing but by following global standard) and then work on a case by case basis regarding privacy and exclusion stuffs. 

Cheers,

Bibhusan

To view this discussion on the web visit https://groups.google.com/a/webfoundation.org/d/msgid/public-ocds/a51a0eb5d75fc5e9a060531ead12cfdb%40mail.gmail.com.

--

Bibhusan Bista

CEO

-----------------------------

YoungInnovations Pvt. Ltd.

GPO 8976 CPC 241
Jawalakhel, Lalitpur, Nepal

-----------------------------

P: +977-1-2291892

W: http://yipl.com.np

E: in...@yipl.com.np

-----------------------------

Skype: bibhusan

-----------------------------

*DISCLAIMER*

This email contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system.

-----------------------------