Kia Ddms

0 views

Skip to first unread message

Percival Blanco

unread,

Aug 3, 2024, 6:12:05 PM8/3/24

to utdisseta

A Domain Data Management Service (DDMS) can be seen as any source of truth for data that manages the data life cycle, satisfies given mandatory data access concerns, and makes its data globally discoverable and retrievable through the OSDU. It could be a standalone service dedicated to a specific data type or a subcomponent of an application or platform. It simply enables its data to be retrieved outside of its regular scope.

OSDU solves these concerns primarily using Storage records. A Storage record is metadata pertaining to the bulk data stored in the DDMS. Every record created in Storage enforces that ACLs are assigned, checks compliance and then indexes the record into search, making it discoverable.

The first step is to register as a DDMS. This makes your DDMS discoverable to clients and presents them with an API definition that tells them how to retrieve the bulk data when a record from their DDMS is discovered.

Note that you can register as much of your API specification as you like. You only need to define the method clients should use to retrieve the bulk data using the custom property x-ddms-retrieve-entity: true.

When deploying your service, you should do the one-time operation of publishing the schema via the Schema API, the schema upload automation will happen through CSP automation scripts, this is an example of the osdu:wks:master-data--Wellbore:1.0.0.

This will then allow any Record that references this schema to be indexed in the DE search. Without this, the Record will be published but without any of the data and it will be hidden by default in search.

These act as well-known properties that should be added to the record by your DDMS. Clients can then use this information to retrieve the bulk data after discovery using the DDMS registration APIs. Every schema created should declare these properties to use this pattern of ingestion.

Unless you have a scenario where you know what legal tag and ACL should be applied to the data you are ingesting, you will need to expose the legal tag and ACL in your ingestion APIs. This allows your clients to supply the legal tag and ACL themselves.

Whenever bulk data is ingested, you need to create a shadow record within Storage. DDMS may have the feature to create the record and ingest it automatically such as api/os-wellbore-ddms/ddms/v3/wellbores. This shadow record represents the specific bulk data instance in a 1:1 relationship and makes each instance globally discoverable.

First, you should store the bulk data, and then create the shadow Record. This way, a global piece of data is not discoverable before the bulk data is available. If this is not successful, e.g. because an invalid legal tag is provided, the request will fail and you should return this response to the client and attempt to clean up the bulk data.

As mentioned, a DDMS should create a shadow record for every instance of bulk data ingested into their data store. This can have advantages beyond global discover-ability. Whenever you request a storage record, both compliance and entitlements are checked before returning the data. A DDMS can use this to their advantage.

By forwarding on any request by the client to retrieve the record, you can delegate these responsibilities to the Storage service. If OSDU returns the Record, the client can access both this and the bulk data, and so you can return the same to the client or only the Record.

They can then use the entityType property to work out which API definition to use, and then use the localId property to work out how to create the API call to retrieve the bulk data from the DDMS using the API defined.