Problems with PUT /{eventId}/metadata

[Ferran C]

Hello everyone,
I am new with OpenCast and I am having some problems modifying the event metadata.

Specifically with the API call, PUT/{eventId}/metadata.
I have seen in the forum a post similar to my problem but I can't solve it by following the recommendations : https://groups.google.com/a/opencast.org/g/users/c/vTDXFLwCQws/m/hVAUI7IvBwAJ

What I am doing:
1. I get the event metadata with GET/{eventId}/metadata, with type dublincore/episode.

I get the following data:

[
  {
    "readOnly": false,
    "id": "title",
    "label": "EVENTS.EVENTS.DETAILS.METADATA.TITLE",
    "type": "text",
    "value": "Demo event",
    "required": true
  },
  {
    "readOnly": false,
    "id": "subjects",
    "label": "EVENTS.EVENTS.DETAILS.METADATA.SUBJECT",
    "type": "text",
    "value": [
      "demo"
    ],
    "required": false
  },
  {
    "readOnly": false,
    "id": "description",
    "label": "EVENTS.EVENTS.DETAILS.METADATA.DESCRIPTION",
    "type": "text_long",
    "value": "demo",
    "required": false
  },
  {
    "translatable": true,
    "readOnly": false,
    "id": "language",
    "label": "EVENTS.EVENTS.DETAILS.METADATA.LANGUAGE",
    "type": "text",
    "value": "spa",
    "required": false
  },
  {
    "readOnly": false,
    "id": "rightsHolder",
    "label": "EVENTS.EVENTS.DETAILS.METADATA.RIGHTS",
    "type": "text",
    "value": "public",
    "required": false
  },
  {
    "translatable": true,
    "readOnly": false,
    "id": "license",
    "label": "EVENTS.EVENTS.DETAILS.METADATA.LICENSE",
    "type": "text",
    "value": "",
    "required": false
  },
  {
    "translatable": false,
    "readOnly": false,
    "id": "isPartOf",
    "label": "EVENTS.EVENTS.DETAILS.METADATA.SERIES",
    "type": "text",
    "value": "",
    "required": false
  },
  {
    "translatable": false,
    "readOnly": false,
    "id": "creator",
    "label": "EVENTS.EVENTS.DETAILS.METADATA.PRESENTERS",
    "type": "mixed_text",
    "value": [
      "demo"
    ],
    "required": false
  },
  {
    "translatable": false,
    "readOnly": false,
    "id": "contributor",
    "label": "EVENTS.EVENTS.DETAILS.METADATA.CONTRIBUTORS",
    "type": "mixed_text",
    "value": [
      "demo"
    ],
    "required": false
  },
  {
    "readOnly": false,
    "id": "startDate",
    "label": "EVENTS.EVENTS.DETAILS.METADATA.START_DATE",
    "type": "date",
    "value": "2022-05-19",
    "required": false
  },
  {
    "readOnly": false,
    "id": "startTime",
    "label": "EVENTS.EVENTS.DETAILS.METADATA.START_TIME",
    "type": "time",
    "value": "08:52",
    "required": false
  },
  {
    "readOnly": false,
    "id": "duration",
    "label": "EVENTS.EVENTS.DETAILS.METADATA.DURATION",
    "type": "text",
    "value": "00:01:00",
    "required": false
  },
  {
    "readOnly": false,
    "id": "location",
    "label": "EVENTS.EVENTS.DETAILS.METADATA.LOCATION",
    "type": "text",
    "value": "pyca",
    "required": false
  },
  {
    "readOnly": false,
    "id": "source",
    "label": "EVENTS.EVENTS.DETAILS.METADATA.SOURCE",
    "type": "text",
    "value": "local",
    "required": false
  },
  {
    "readOnly": true,
    "id": "created",
    "label": "EVENTS.EVENTS.DETAILS.METADATA.CREATED",
    "type": "date",
    "value": "2022-05-19T08:52:00.000Z",
    "required": false
  },
  {
    "readOnly": true,
    "id": "publisher",
    "label": "EVENTS.EVENTS.DETAILS.METADATA.PUBLISHER",
    "type": "text",
    "value": "",
    "required": false
  },
  {
    "readOnly": true,
    "id": "identifier",
    "label": "EVENTS.EVENTS.DETAILS.METADATA.ID",
    "type": "text",
    "value": "79a44174-10ae-469d-9719-bc67a5d5f0a5",
    "required": false
  }
]


2. I execute the call, PUT/{eventId}/metadata, with type dublincore/episode and the metadata I just got, modifying only the title.

I get a 400 response with the following error: Unable to parse type 'null' as a flavor so unable to find the matching catalog.


I have also tried executing the call, GET /{eventId}/metadata, without specifying the type and I get this metadata:

[
  {
    "flavor": "dublincore/episode",
    "title": "EVENTS.EVENTS.DETAILS.CATALOG.EPISODE",
    "fields": [
      {
        "readOnly": false,
        "id": "title",
        "label": "EVENTS.EVENTS.DETAILS.METADATA.TITLE",
        "type": "text",
        "value": "Demo event",
        "required": true
      },
      {
        "readOnly": false,
        "id": "subjects",
        "label": "EVENTS.EVENTS.DETAILS.METADATA.SUBJECT",
        "type": "text",
        "value": [
          "demo"
        ],
        "required": false
      },
      {
        "readOnly": false,
        "id": "description",
        "label": "EVENTS.EVENTS.DETAILS.METADATA.DESCRIPTION",
        "type": "text_long",
        "value": "demo",
        "required": false
      },
      {
        "translatable": true,
        "readOnly": false,
        "id": "language",
        "label": "EVENTS.EVENTS.DETAILS.METADATA.LANGUAGE",
        "type": "text",
        "value": "spa",
        "required": false
      },
      {
        "readOnly": false,
        "id": "rightsHolder",
        "label": "EVENTS.EVENTS.DETAILS.METADATA.RIGHTS",
        "type": "text",
        "value": "public",
        "required": false
      },
      {
        "translatable": true,
        "readOnly": false,
        "id": "license",
        "label": "EVENTS.EVENTS.DETAILS.METADATA.LICENSE",
        "type": "text",
        "value": "",
        "required": false
      },
      {
        "translatable": false,
        "readOnly": false,
        "id": "isPartOf",
        "label": "EVENTS.EVENTS.DETAILS.METADATA.SERIES",
        "type": "text",
        "value": "",
        "required": false
      },
      {
        "translatable": false,
        "readOnly": false,
        "id": "creator",
        "label": "EVENTS.EVENTS.DETAILS.METADATA.PRESENTERS",
        "type": "mixed_text",
        "value": [
          "demo"
        ],
        "required": false
      },
      {
        "translatable": false,
        "readOnly": false,
        "id": "contributor",
        "label": "EVENTS.EVENTS.DETAILS.METADATA.CONTRIBUTORS",
        "type": "mixed_text",
        "value": [
          "demo"
        ],
        "required": false
      },
      {
        "readOnly": false,
        "id": "startDate",
        "label": "EVENTS.EVENTS.DETAILS.METADATA.START_DATE",
        "type": "date",
        "value": "2022-05-19",
        "required": false
      },
      {
        "readOnly": false,
        "id": "startTime",
        "label": "EVENTS.EVENTS.DETAILS.METADATA.START_TIME",
        "type": "time",
        "value": "08:52",
        "required": false
      },
      {
        "readOnly": false,
        "id": "duration",
        "label": "EVENTS.EVENTS.DETAILS.METADATA.DURATION",
        "type": "text",
        "value": "00:01:00",
        "required": false
      },
      {
        "readOnly": false,
        "id": "location",
        "label": "EVENTS.EVENTS.DETAILS.METADATA.LOCATION",
        "type": "text",
        "value": "pyca",
        "required": false
      },
      {
        "readOnly": false,
        "id": "source",
        "label": "EVENTS.EVENTS.DETAILS.METADATA.SOURCE",
        "type": "text",
        "value": "local",
        "required": false
      },
      {
        "readOnly": true,
        "id": "created",
        "label": "EVENTS.EVENTS.DETAILS.METADATA.CREATED",
        "type": "date",
        "value": "2022-05-19T08:52:00.000Z",
        "required": false
      },
      {
        "readOnly": true,
        "id": "publisher",
        "label": "EVENTS.EVENTS.DETAILS.METADATA.PUBLISHER",
        "type": "text",
        "value": "",
        "required": false
      },
      {
        "readOnly": true,
        "id": "identifier",
        "label": "EVENTS.EVENTS.DETAILS.METADATA.ID",
        "type": "text",
        "value": "79a44174-10ae-469d-9719-bc67a5d5f0a5",
        "required": false
      }
    ]
  }
]

Then I execute the call, PUT /{eventId}/metadata, with type dublincore/episode and the metadata I just got, modifying only the title.

In this case I get a 400 error: One of the metadata fields is missing an id or value. The id was '' and the value was ''.


I have tried to send only one field in the metadata to modify only that field. But I get the same response saying that an id or value is missing (all fields I specify have value...)

Do all metadata fields have to have a value to be able to update them?
Or must all the metadata be modified at once, and cannot be done individually? I have read the documentation and it says that the metadata that is not included in the form will not be updated.

Any help on why I can't update the data and why I'm getting these errors? I don't understand these errors.

Best regards, Ferran.

[Andreas Lindner]

Hey there,

while I cannot directly help you with metadata, I recently had to work with the REST Endpoint to quickly change ACLs.

For the ACLs, the req body looks like this:
acl=%5B+++%7B+++++%22allow%22%3A+true%2C+++++%22action%22%3A+%22write%22%2C+++++%22action%22%3A+%22read%22%2C+++++%22role%22%3A+%22ROLE_ADMIN%22+++%7D%2C+++%7B+++++%22allow%22%3A+true%2C+++++%22action%22%3A+%22read%22%2C+++++%22role%22%3A+%22ROLE_USER%22+++%7D+%5D

Perhaps this is of some use for you.

Best regards,

Andreas | TH Deggendorf

[Karen Dolan]

Hi Ferran,

I was curious about your problem because we also update OC metadata from a 3rd party source-of-truth system. However, my site uses an older version OC.

My short comment is that I could not duplicate your issue, but did the [unencoded] JSON string that you tried to update the event title look like this?

 

[   {     "id": "title",     "value": “My new title"   } ]

An aside, the exception log text you saw “id and the value was” from the IllegalArgumentException  is problematic, because the “id” field's value will always be missing from the exception text when the “id” field has an empty value. Which was the reason the exception was thrown. This makes Andreas’s response to your post relevant that maybe your JSON input was not properly encoded when sent in the PUT body's form.

Link to the OC code that parsed the JSON that threw the error about missing “id” field value:  https://github.com/opencast/opencast/blob/develop/modules/index-service/src/main/java/org/opencastproject/index/service/util/RequestUtils.java#L67-L82

Best of luck,

Karen

====================

Possibly unrelated. While trying to test what format is used by the API (modeling the input on the Java Unit test input data in the OC code), I got an unexpected error using Opencast stable External API endpoint. The PUT metadata of the endpoint docs returns text that should have only come from a GET metadata… I couldn’t duplicate the issue via Opencast stable website...

Using https://stable.opencast.org/docs.html?path=/admin-ng/event#updateeventmetadata-19

:method: PUT

:path: /api/events/ID-weitsprung/metadata

Body Content:

type: dublincore/episode

metadata: [   {     "id": "title",     "value": "Weitsprung - updated with External API's API"   } ]

Response from the PUT is a 400 with text:

`Unable to parse type 'null' as a flavor so unable to find the matching catalog.` <— This text is Only called from a GET per the OC code (develop branch)!!!!

[Karen Dolan]

Hahah, I just realized what that error looks like when both the “id” field’s value and the “value” field’s value are missing/unreadable because of the single quotes in the exception ...

 

   `One of the metadata fields is missing an id or value. The id was ‘<empty string>' and the value was '<empty string>’`

Which is…

   `One of the metadata fields is missing an id or value. The id was " and the value was"`

https://github.com/opencast/opencast/blob/develop/modules/index-service/src/main/java/org/opencastproject/index/service/util/RequestUtils.java#L67-L82

[Ferran C]

Thanks for your help. I'm testing both local, versión 11.6, and stable.opencast.

In my case I receive 2 different errors depending on the format with which I send the metadata. In both cases, I always specify that the type of the metadata is dublincore/episode.

Case 1: If I send the metadata with dublincore/episode format

[ { "id": "title", "value": "test" }]

I get a 400 error: Unable to parse type 'null' as a flavor so unable to find the matching catalog.

Case 2: If I send the metadata without dublincore/episode format

[ { "flavor": "dublincore/episode", "title": "EVENTS.EVENTS.DETAILS.CATALOG.EPISODE", "fields": [ { "id": "title", "value": "test" } ] } ]

I get a 400 error: One of the metadata fields is missing an id or value. The id was ' ' and the value was ' '.

[Andreas Lindner]

Hey Ferran,

I can reproduce your problem, later today I´m gonna run a fiew tests on our test server, perhaps i can find something.

Take care,

Andreas | TH Deggendorf

[Andreas Lindner]

Ok - i just ran some tests, and kinda got it to work.

Thanks to Karen, I realized that /api/event is the wrong Endpoint to use.

If I work with  the /admin-ng/event Endpoint changing the Metadata works just like Greg wrote here: https://groups.google.com/a/opencast.org/g/users/c/vTDXFLwCQws/m/hVAUI7IvBwAJ

As I´m still hessitant to use the admin-ng Endpoints I would still like to use the /api/events Endpoint, so I ran a view tests with this:

Test System:

1 admin VM, 1 worker vm, 1 engage vm

Opencast Version 11.4

OS: CentOS 7 (Red Hat 4.85-39)

Test runs: 

Differnt entries inside the metadata for the /api/events Endpoint. https://stable.opencast.org/docs.html?path=/api/events#updateeventmetadata-8

eventID: <existing event id>

type: dublincore/episode

1st try: metadata: [   {     "id": "title",     "value": “My new title"   } ]

2nd try: metadata: {     "id": "title",     "value": “My new title"   }

3rd try: metadata: "id": "title",     "value": “My new title"

4th try: metadata: ["id": "title",     "value": “My new title"]

5th try: metadata: "[   {     "id": "title",     "value": “My new title"   } ]"

All tests got me the response: Status: 400 (Bad Request)

Inside the opencast log I found this. 

(as this was basically the same for every test run, this is the log for the first test)

2022-06-03T10:55:36,712 | DEBUG | (UrlSigningFilter:100) - The request 'http://<our test server>/api/events/b3ca3399-5de1-4fc3-8df7-1f34b627b5b5/metadata' is not a GET or HEAD request so skipping the filter.
2022-06-03T10:55:36,712 | DEBUG | (TransparentProxyFilter:59) - Found X-Forwarded-For header. Resetting source IP
2022-06-03T10:55:36,712 | DEBUG | (JsonpFilter:112) - No json padding requested from org.opencastproject.kernel.filter.proxy.TransparentProxyRequestWrapper@704323e1
2022-06-03T10:55:36,713 | DEBUG | (EventsEndpoint:1391) - Unable to update event 'b3ca3399-5de1-4fc3-8df7-1f34b627b5b5' with metadata type 'null' and content '[   {     "id": "title",     "value": My new title"   } ]'


Well atleast I have a way to quickly update metadata for now.

The /admin-ng/event endpoint works like a charm for me.

Take care,

Andreas | TH Deggendorf

[Ferran C]

I'm happy to hear that it works for you Andreas, updating metadata with the /admin-ng/event endpoint also works for me.

The problem is that I had read in the documentation or in the forum that it was recommended to use the /api/ endpoints, because they will try to maintain them and modify them as little as possible in the next versions.

That's why I was interested in the /api/metadata endpoint.

Thank you for your help and dedicated time.

[Andreas Lindner]

I am 100% with you on this point.

I strongly desire to use the /api/events endpoint too.
If I can afford to put some more time into this, I will try around some more.

Take Care

Andreas | TH Deggendorf

[Greg Logan]

A quick suggestion for those who come across this thread and want to update things via our REST endpoints: I've found it easier to build the contents of ugly json blobs like this by working from the other direction.  First, build what you *want* to see in the UI in a new workflow.  Then query what that metadata *should* look like from the REST endpoints.  Now modify what you got to fit the workflow you're trying to update, and send the whole blob.  Rarely partial blobs will work, and that's usually on endpoints with explicit fields for that partial blob.  If in doubt, you probably need the whole thing, even if you're only updating a single field.

G

--
To unsubscribe from this group and stop receiving emails from it, send an email to users+un...@opencast.org.

[Andreas Lindner]

Hey again,

I found a way without using the admin-ng Endpoints.

Using the  /lti-service/ Endpoint it is easy to get the metadata for an event as json.

The whole json can then be modified, and using POST  /lti-service/<uuid> the metadata is "updated".

This works in CO 11.4.


It isn´t PUT, but it works for me.
Careful though, you probably want to make sure to not loose any data doint this.
(And a reminder, you still need to republish the video after this.)

Take care,

Andreas | TH Deggendorf