Question about arrays

[Makx Dekkers]

Apologies for maybe misunderstanding the approach with arrays. I also made comments on this issue in the specification (0.3.3) ahead of yesterday's deadline.

At first I thought that arrays within a block were related in a one-to-one correspondence. For example, in the Award block there are two arrays, one for itemsAwarded and one for suppliers, so I thought that the meaning was that Item[i] was awarded to Organization[i], but when looking at the Tender block, where there are also arrays for itemsToBeProcured and bidders, such a 1-to-1 relationship can't be the intention -- after all, one item may attract more than one bidder and one bidder may bid for more than one item.

So is there some implicit rule about such correspondences, or is there some place in the documentation that explains this?

In a way, it would be clearer if the model was based on an approach where one Tender procured a single Item. Then the bidders on that Tender would unambiguously be associated with the bid for that particular item. 

Further down the process, it may also make sense to link one item to one supplier (or one consortium acting jointly as the supplier) in the Award, and the same could be the approach for the Contract. In the current specification of the Contract, I found it strange that there is no explicit relationship between the buyer, the supplier and the item -- I would have expected all three to appear in the attributes for the contract.

Probably useful to explain that I am looking at the model with the intention to express the conceptual model in RDF where each of the 'blocks' would be modelled as separate resources with links (URI references) between them. I realize that if you express the model in a tree-container structure, it may be easy to use data that is available a bit higher up in the record (such as the buyer in the Release block and the supplier in the Award block) and it may be awkward to repeat information that you have already included in an earlier block, but it makes the model maybe a little less clear.

Makx.

[Tim Davies]

Hello Makx, Myroslav and Mathias 

Thank you for the detailed comments around bids and other elements. 

I'll keep working through these and others and capturing these in the tracker. 

So - at present I don't think the model captures the full depth of statements that people might want to make about the relationship of bidders, bids, tenders and awards. We're aiming to always have the user side in mind as well as the publisher, and that's where the demand for numberOfBids / numberOfBidders comes from - although we recognise that won't always be possible to publish clearly in all cases.

I've started a discussion at https://github.com/open-contracting/standard/issues/78 about whether we should separate out a new 'bidding' block in the releases. This will likely be slightly controversial, as we've tried to keep the concept of the contracting process as simple as possible, and to use the names for stages commonly used in contracting conversations (e.g. tender, award, contract) as far as possible - but it does look like we might need this to be expressive enough to capture both 'bids' and 'registered bidders' as distinct groups - and to capture the idae that a bidder might only bid for one of N items named in a tender.

To deal with this array issue - I agree we need better documentation on this. As the definition of a unique 'Contracting Process' (i.e. something which gets it's own set of releases) is essentially something with a unique tender, it can be assumed that any award relates to the tender that it shares an OCID with. However, because there can be multiple awards, and thus multiple contracts, we have the notion of linking between these with the ability to have an awardID within the contract block.

However, at present, there is no way of cross-referencing Items named in the different blocks (apart from heuristically working out that the 10 things with a given CPV code in itemsToBeProcured, are the same ones as fund in itemsAwarded or itemsContracted). One response to this would be to just ask for an ID for items, which need not be globally unique, but could help tie together items between these stages for the linked modelling: https://github.com/open-contracting/standard/issues/79 

Makx - would this help with your goals?

Thanks again for all the feedback

All the best

Tim

--
You received this message because you are subscribed to the Google Groups "Public OCDS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to public-ocds...@webfoundation.org.
To post to this group, send email to publi...@webfoundation.org.
Visit this group at http://groups.google.com/a/webfoundation.org/group/public-ocds/.
To view this discussion on the web visit https://groups.google.com/a/webfoundation.org/d/msgid/public-ocds/CAPkhOK4%3Deq4p2udmcqLYSxRoY_pnqrp%2B8SHwDZTD3PNop16wnA%40mail.gmail.com.

--

-- 

Tim Davies

Research Coordinator, Open Data Research Network

+44 7834 856 303

@timdavies | @odrnetwork | www.opendataresearch.org 

World Wide Web Foundation | 1110 Vermont Ave NW, Suite 500, Washington DC 20005, USA | www.webfoundation.org | Twitter: @webfoundation

[Makx Dekkers]

Tim,

Thanks for your thoughts on how to identify items across blocks. 

In an RDF approach, you would assign globally unique identifiers (URIs) to everything, including the items, in any case, so in such an approach you would be able to cross-reference by design.

You would have statements (triples) like 

Organization-A bids-for Item-X

Organization-B bids-for Item-X

Organization-B gets-award-for Item-X

Organization-B supplies Item-X

etc.

All these things (Organizations, Items, but also 'bids-for', 'gets-award-for' and 'supplies') would be identified by globally unique URIs.

In a tree-container approach, I guess that you could indeed make the linkage explicit by giving every every item an ItemID that is unique in the the context of the Release.

Makx.

To view this discussion on the web visit https://groups.google.com/a/webfoundation.org/d/msgid/public-ocds/CAOA4sPB_q3xbY28UiX%3D4n9udX8q7Qjf2Tz%3DhUndnDRUSspiNPA%40mail.gmail.com.

[Tim Davies]

Thanks Makx,

Yes - we've had the RDF model in mind whilst constructing the JSON-Schema - with the idea that any JSON-LD @context that could get created should have the minimal number of blank nodes, whilst avoiding adding IDs to the data which are not also useful to JSON and CSV renderings of the data (a bit of balancing act...) 

One of the issues which might need some thought for an RDF rendering of the data is the issue of change over time. 

A few examples of change over time might include:

* When releases come from different systems at different stages (e.g. tenders might be in one systems; awards in another) then it is possible that data like an organisations address might vary between releases (because the supplying systems are not in sync)
* When amendments update the itemsContracted and contractValue over time or similar; 

In the merging strategies to generate summary OCDS records (see http://ocds.open-contracting.org/standard/r/0__3__3/#tabs-3) we've had to think a lot about the right logics to apply on whether later data should be additive, or should replace, earlier field values. 

I've not worked through in detail yet how the use of identifiers might factor into this (i.e. sometimes blank nodes may be better, to accept that whilst X and Y might be describing the same thing, it is possible they are not, and so any business logic, rather than the data, would have to work on reconciling them together). 

As I say - I've not thought this through in depth - but flagging in case relevant to your current explorations/thinking.

All the best

Tim

To view this discussion on the web visit https://groups.google.com/a/webfoundation.org/d/msgid/public-ocds/CAPkhOK5WgjxeyF_UN_W2sYr0eWtV8t-6mQxU4teN9tTUjV9_9A%40mail.gmail.com.

[Sarah Bird]

"So is there some implicit rule about such correspondences, or is there some place in the documentation that explains this?"

To be blunt - there is no correspondence of items in one array to items in another array at any point anywhere!

I'm really sorry that this isn't clear and its really interesting feedback to get, we will be working on our documentation.

Some notes:

1 - There is a "link" between an item in the contracts array and an item in the awards array, but this is explicitly documented by specifying the awardID in the contract object.

2 - How items and suppliers work through, in some different scenarios

Scenario A

• A tender goes out to purchase 6 pandas and 5 ninjas.
• The pandas are awarded to Company A, and the ninjas to Company B.
• You would have two awards in the award array
  
• one award is for Company A to sell 6 pandas 
• one award is for Company B to sell 5 ninjas
• You would have two contracts in the contract array, they would just have the additional contract information, no items information as it is the same as in the award array
  
• Note these can come out in four separate releases, the compiledRelease in the record would then show the two awards and two contracts all wrapped up together

Scenario B

• A tender goes out for a standing arrangement to purchase pandas and ninjas.
• Company A and B are awarded the right to sell pandas, Company C is awarded the right to sell ninjas, Company D is awarded the right to sell both.
• You could structure this as 4 or 5 awards: one for each company documenting the items, or company D's award could be split into two awards
• As this is a standing arrangement, lets say the buyer buys 15 pandas, one at a time, over a period of 2 years.
• There will be 15 contracts in the contracts array. Each one will specify the award which the contract is made against. They will also specify the items that were contracted as this will change from contract to contract and should be. They do not need to specify the supplier because the supplier because the award specifies the supplier.
• There are easily over 20 releases involved here. This is why the record and the compiledRelease are so important to users, so they can see all this information in one place.
  

There's an interesting question about why have a suppliers array. I am feeling like we might need to change this, the idea is that sometimes you have joint ventures or consortium awards and you want to capture the information about each company that is part of a single award. But I'm starting to think the suppliers array is the best way of capturing this - this is why we have a consultation period!

https://github.com/open-contracting/standard/issues/117

There is only a subtle distinction between an award and a contract. The award happens, and then there are contract negotiations to finalize the details, but the supplier doesn't change - otherwise that would be a different award.

I'm really hoping this clarifies things!

Thanks so much for all your feedback and questions Makx its super helpful.

Best,

Bird

To view this discussion on the web visit https://groups.google.com/a/webfoundation.org/d/msgid/public-ocds/CAOA4sPDXnZ1UL36yy-uBcdSojS8g-v1DadwkmVG9E0vJr1ijeQ%40mail.gmail.com.

--

sa...@aptivate.org
skype: birdsarah

www.linkedin.com/in/birdsarah/

Aptivate - Ethical IT for International Development

Aptivate | http://www.aptivate.org | Phone: +44 1223 967838
Citylife House, Sturton Street, Cambridge CB1 2QF

Aptivate is a not-for-profit company registered in England and Wales 
with company number 04980791.