Events ending up as empty lines in Admin UI after OC 11 reindexing

34 views
Skip to first unread message

itz.feli...@gmail.com

unread,
Mar 26, 2022, 5:49:17 AM3/26/22
to Opencast Users
Hi,

some events end up as blank lines (except for the "published" column) in the Admin UI after a reindex operation in Opencast 11.

There are no issues reported about this event while reindexing.

Screenshot 2022-03-26 at 10.39.19.png

What could be the cause of this lack of meta data in the Admin UI? A missing DC Catalog?

/b53285e0-f9d7-4b3e-9d21-58445a20c2ca$ ls -lha 1
total 7.6M
drwxr-xr-x 2 800 800 4.0K Jun 16  2021 .
drwxr-xr-x 4 800 800   24 Jun 16  2021 ..
-rw-r--r-- 3 800 800  58K Jun 16  2021 09bd2649-7847-4c2d-91cb-51d59c943b46.png
-rw-r--r-- 2 800 800 843K Jun 16  2021 265179a3-437b-4fdb-8623-0ee35016022a.m4a
-rw-r--r-- 4 800 800 1.8M Jun 16  2021 46343293-9575-4f12-8324-5a0265fb7e7b.mp4
-rw-r--r-- 2 800 800 671K Jun 16  2021 dbeb76f7-beb0-4177-b91e-f20ecc63641f.ogg
-rw-r--r-- 2 800 800 826K Jun 16  2021 fafd9891-1b78-4073-89a5-b4d052406dde.mp3
-rw-r--r-- 1 800 800 7.3K Jun 16  2021 manifest.xml
-rw-r--r-- 1 800 800 1.4K Jun 16  2021 s-a7237e14-5b25-45bf-a97e-053a5709ce48.smil
-rw-r--r-- 4 800 800 1.8M Jun 16  2021 track-16.mp4
-rw-r--r-- 4 800 800 1.8M Jun 16  2021 track-17.mp4


Can missing DC catalogs somehow be generated from the manifest.xml (where at least a title is denoted)?

Here is the 1/manifest.xml (latest version thereof):

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<mediapackage duration="56400" id="b53285e0-f9d7-4b3e-9d21-58445a20c2ca" start="2021-06-16T13:36:16Z" xmlns="http://mediapackage.opencastproject.org">
    <title>Audioonly Tests (+Laud,-Vol,+Editor,3)</title>
    <media>
        <track id="fafd9891-1b78-4073-89a5-b4d052406dde" type="audioonly/audio">
        <mimetype>audio/mpeg</mimetype>
        <tags>
            <tag>archive</tag>
            <tag>mp3</tag>
        </tags>
        <url>urn:matterhorn:b53285e0-f9d7-4b3e-9d21-58445a20c2ca:1:fafd9891-1b78-4073-89a5-b4d052406dde:out.mp3</url>
        <size>845668</size>
        <checksum type="md5">6c659d7767b07149302a46c93563e98f</checksum>
        <duration>52819</duration><audio id="audio-1">
            <device />
            <encoder type="MP3 (MPEG audio layer 3)" />
            <channels>2</channels>
            <samplingrate>44100</samplingrate>
            <bitrate>128000.0</bitrate>
        </audio>
        <live>false</live>
        </track>
        <track id="dbeb76f7-beb0-4177-b91e-f20ecc63641f" type="audioonly/audio">
        <mimetype>audio/ogg</mimetype>
        <tags>
            <tag>archive</tag>
            <tag>opus</tag>
        </tags>
        <url>urn:matterhorn:b53285e0-f9d7-4b3e-9d21-58445a20c2ca:1:dbeb76f7-beb0-4177-b91e-f20ecc63641f:out.ogg</url>
        <size>686632</size>
        <checksum type="md5">16c0c1a6bb7c72e7b3422c5e910672fb</checksum>
        <duration>52791</duration><audio id="audio-1">
            <device />
            <encoder type="Opus (Opus Interactive Audio Codec)" />
            <channels>2</channels>
            <samplingrate>48000</samplingrate>
        </audio>
        <live>false</live>
        </track>
        <track id="265179a3-437b-4fdb-8623-0ee35016022a" type="audioonly/audio">
        <mimetype>audio/m4a</mimetype>
        <tags>
            <tag>archive</tag>
            <tag>m4a</tag>
        </tags>
        <url>urn:matterhorn:b53285e0-f9d7-4b3e-9d21-58445a20c2ca:1:265179a3-437b-4fdb-8623-0ee35016022a:out.m4a</url>
        <size>862804</size>
        <checksum type="md5">01878143716ec44356699b75470bd023</checksum>
        <duration>52808</duration><audio id="audio-1">
            <device />
            <encoder type="AAC (Advanced Audio Coding)" />
            <framecount>2275</framecount>
            <channels>2</channels>
            <samplingrate>44100</samplingrate>
            <bitrate>129207.0</bitrate>
        </audio>
        <live>false</live>
        </track>
        <track id="46343293-9575-4f12-8324-5a0265fb7e7b" type="audioonly/video">
        <mimetype>video/mp4</mimetype>
        <tags>
            <tag>archive</tag>
        </tags>
        <url>urn:matterhorn:b53285e0-f9d7-4b3e-9d21-58445a20c2ca:1:46343293-9575-4f12-8324-5a0265fb7e7b:result.mp4</url>
        <size>1804827</size>
        <checksum type="md5">8f071967c3f00bc56b4c35f4a8732f6a</checksum>
        <duration>56400</duration><audio id="audio-1">
            <device />
            <encoder type="AAC (Advanced Audio Coding)" />
            <framecount>2275</framecount>
            <channels>2</channels>
            <samplingrate>44100</samplingrate>
            <bitrate>129252.0</bitrate>
        </audio><video id="video-1">
            <device />
            <encoder type="H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10" />
            <framecount>564</framecount>
            <bitrate>131387.0</bitrate>
            <framerate>10.0</framerate>
            <resolution>1920x1080</resolution>
        </video>
        <live>false</live>
        </track>
        <track id="track-16" type="presenter/work" ref="track:46343293-9575-4f12-8324-5a0265fb7e7b">
        <mimetype>video/mp4</mimetype>
        <tags>
            <tag>archive</tag>
        </tags>
        <url>urn:matterhorn:b53285e0-f9d7-4b3e-9d21-58445a20c2ca:1:track-16:result.mp4</url>
        <size>1804827</size>
        <checksum type="md5">8f071967c3f00bc56b4c35f4a8732f6a</checksum>
        <duration>56400</duration><audio id="audio-1">
            <device />
            <encoder type="AAC (Advanced Audio Coding)" />
            <framecount>2275</framecount>
            <channels>2</channels>
            <samplingrate>44100</samplingrate>
            <bitrate>129252.0</bitrate>
        </audio><video id="video-1">
            <device />
            <encoder type="H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10" />
            <framecount>564</framecount>
            <bitrate>131387.0</bitrate>
            <framerate>10.0</framerate>
            <resolution>1920x1080</resolution>
        </video>
        <live>false</live>
        </track>
        <track id="track-17" type="presenter/source" ref="track:track-16">
        <mimetype>video/mp4</mimetype>
        <tags>
            <tag>archive</tag>
        </tags>
        <url>urn:matterhorn:b53285e0-f9d7-4b3e-9d21-58445a20c2ca:1:track-17:result.mp4</url>
        <size>1804827</size>
        <checksum type="md5">8f071967c3f00bc56b4c35f4a8732f6a</checksum>
        <duration>56400</duration><audio id="audio-1">
            <device />
            <encoder type="AAC (Advanced Audio Coding)" />
            <framecount>2275</framecount>
            <channels>2</channels>
            <samplingrate>44100</samplingrate>
            <bitrate>129252.0</bitrate>
        </audio><video id="video-1">
            <device />
            <encoder type="H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10" />
            <framecount>564</framecount>
            <bitrate>131387.0</bitrate>
            <framerate>10.0</framerate>
            <resolution>1920x1080</resolution>
        </video>
        <live>false</live>
        </track>
    </media>
    <metadata>
        <catalog id="s-a7237e14-5b25-45bf-a97e-053a5709ce48" type="presenter/silence">
            <mimetype>text/xml</mimetype>
            <tags>
                <tag>archive</tag>
            </tags>
            <url>urn:matterhorn:b53285e0-f9d7-4b3e-9d21-58445a20c2ca:1:s-a7237e14-5b25-45bf-a97e-053a5709ce48:smil.smil</url>
            <checksum type="md5">1565ce19c67cbb10538d7ee6730df657</checksum>
        </catalog>
    </metadata>
    <attachments>
        <attachment id="09bd2649-7847-4c2d-91cb-51d59c943b46" type="audioonly/image">
            <mimetype>image/png</mimetype>
            <tags>
                <tag>archive</tag>
                <tag>waveform</tag>
            </tags>
            <url>urn:matterhorn:b53285e0-f9d7-4b3e-9d21-58445a20c2ca:1:09bd2649-7847-4c2d-91cb-51d59c943b46:waveform.png</url>
            <checksum type="md5">49138213143fd087794cca394864b2b2</checksum>
        </attachment>
    </attachments>
    <publications>
        <publication id="6da679f1-83de-4620-90e7-5fb8287105f2" channel="engage-player">
            <mimetype>text/html</mimetype>
            <tags />
            <url>https://staging-engage.opencast.example.com/play/b53285e0-f9d7-4b3e-9d21-58445a20c2ca</url>
            <media />
            <attachments />
            <metadata />
        </publication>
        <publication id="e2a20e8c-0df3-4f2b-9a88-48db52bed5a4" channel="internal">
            <tags />
            <url>/admin-ng/index.html#/events/events/b53285e0-f9d7-4b3e-9d21-58445a20c2ca/tools/editor</url>
            <media>
                <track id="1d13a31c-962a-4506-8d4d-c37d5e98c86e" type="presenter/preview" ref="track:track-16">
                <mimetype>video/mp4</mimetype>
                <tags>
                    <tag>preview</tag>
                </tags>
                <url>https://staging-engage.opencast.example.com/static/mh_default_org/internal/b53285e0-f9d7-4b3e-9d21-58445a20c2ca/85d46957-639f-40ec-a6fd-c435536c2234/result.mp4</url>
                <size>497278</size>
                <checksum type="md5">8e8e3f17994bdacf8461b8472d99faa0</checksum>
                <duration>56400</duration><audio id="audio-1">
                    <device />
                    <encoder type="AAC (Advanced Audio Coding)" />
                    <framecount>1138</framecount>
                    <channels>2</channels>
                    <samplingrate>22050</samplingrate>
                    <bitrate>64706.0</bitrate>
                </audio><video id="video-1">
                    <device />
                    <encoder type="H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10" />
                    <framecount>564</framecount>
                    <bitrate>7865.0</bitrate>
                    <framerate>10.0</framerate>
                    <resolution>640x360</resolution>
                </video>
                <live>false</live>
                </track>
            </media>
            <attachments>
                <attachment id="3cd9ab44-50c0-4bf9-a266-4fc9b46dddd9" type="presenter/audio+preview">
                    <mimetype>image/png</mimetype>
                    <tags>
                        <tag>preview</tag>
                    </tags>
                    <url>https://staging-engage.opencast.example.com/static/mh_default_org/internal/b53285e0-f9d7-4b3e-9d21-58445a20c2ca/9b92e7e0-9490-40db-a9f6-efedabca66d6/waveform.png</url>
                </attachment>
                <attachment id="3b899ab9-debf-426f-8b37-5359c3542b5d" type="presenter/video+preview" ref="track:track-16">
                    <mimetype>image/jpeg</mimetype>
                    <tags>
                        <tag>preview</tag>
                    </tags>
                    <url>https://staging-engage.opencast.example.com/static/mh_default_org/internal/b53285e0-f9d7-4b3e-9d21-58445a20c2ca/56040515-bc7e-45bb-a704-313b356efdf8/result_50_0p_preview.jpg</url>
                    <size>66723</size>
                </attachment>
                <attachment id="351ddacd-511f-459b-a595-598234fd27e6" type="presenter/waveform">
                    <mimetype>image/png</mimetype>
                    <tags>
                        <tag>preview</tag>
                    </tags>
                    <url>https://staging-engage.opencast.example.com/static/mh_default_org/internal/b53285e0-f9d7-4b3e-9d21-58445a20c2ca/29ed02da-c620-4bd4-b9a3-08c6ceb3ab5f/waveform.png</url>
                </attachment>
            </attachments>
            <metadata />
        </publication>
    </publications>
</mediapackage>

Greg Logan

unread,
Apr 1, 2022, 4:08:34 PM4/1/22
to Opencast Users
Hi Felix,

Missing DC catalogs will cause exactly this issue.  As far as I know there's no automated way of fixing this, so you'd need to manually go through and edit the recordings to fix the metadata.

G

--
To unsubscribe from this group and stop receiving emails from it, send an email to users+un...@opencast.org.

Maximiliano Lira Del Canto

unread,
Apr 4, 2022, 3:32:07 AM4/4/22
to Opencast Users, itz.feli...@gmail.com
Have you checked if the xml files still exists in the hard drive? if yes, you should be able to add them again.

Matthias Vollroth

unread,
Nov 2, 2022, 9:35:15 AM11/2/22
to us...@opencast.org, itz.feli...@gmail.com
Hi Greg,
Hi Maximaliano,

we are facing the same issue with nearly 4000 events in our admin ui after reindexing :(

Could you please tell us, how to proceed in detail? Which xml files has to be moved and where should they go (From archive to mediapackage)?
Is an index rebuild necessary after moving these files manually?

Thanks in advance,

Best,
Matthias

Andreas Lindner

unread,
Nov 7, 2022, 7:21:03 AM11/7/22
to Opencast Users, thd.ma...@gmail.com, itz.feli...@gmail.com

Hello everyone,

as Matthias already wrote, we have quite a lot of empty entries in our Admin – UI.

 

Our System:

We are currently running OC 11.8 and updated in September from OC 8.5.

We have 3 Server: admin – node, worker – node and a presentation node.

 

Our Problem: 

These videos can be viewed with the paella player without problems.

Even the Metadata gets displayed inside the paella player.

But if you want to do anything to these videos except watching them, these entries do not work in any way, shape or form.

 

(Standalone Video Editor):

This is a big Problem for us, as we want to enable our lecturers to edit their videos by themselves.

But currently videos with empty entries in the admin – ui, cannot be edited at all.

We tested a lot of these videos and with all of them, only the preview video and the video title can be loaded.

Everything else will not be loaded (video – waveform, video – length, smil catalogue, metadata,…).

 

As the metadata fields inside the standalone video editor are empty, we tried to change the metadata and publish the video with the new information. After this we receive a 500 (Server Error).

ERROR:

POST https://<admin-node>/editor/a21a169f-7f21-4562-9a5d-ee1d6fda62fd/metadata.json 500 (Server Error)

This error does not leave an entry in our worker – log or in our admin – log, but does leave a warning in our presentation – node (see attachment).

 2022-11-07T09:10:07,715 | WARN  | (WebApplicationExceptionMapper:73) - javax.ws.rs.InternalServerErrorException: HTTP 500 Internal Server Error


(Admin – UI):

In addition to the standalone video editor, changing information inside the admin – ui does not work either.

Neither Metadata changes, nor changes to the ACL can be applied, although we receive the “metadata saved” message.

But if we close the video details window and open it again, all changes are lost.

Republishing or deleting these videos does not work (“republish metadata”, “publish” and “delete” workflow).

As this is our production system, I cannot simply take one of these videos and run tests, but the one video that I could run some tests on, Opencast wrote an ERROR into the log:


2022-10-26T14:06:44,892 | ERROR | (DublinCoreUtil:100) - Unable to load metadata from catalogue 'https://<admin-node>/assets/assets/eccad833-b6ec-4007-bb54-45a999a82f5f/d759521b-c413-407f-a855-0230ac5b686b/6/dublincore.xml'

But I cannot be sure if this error is the real culprit, as the video was processed by a lecturer, and I only go this error after the "publish" workflow was started… (the whole error – log can be found in the attachments), and I cannot even be sure how the database entry of this video was looking like, as I cannot access this video anymore.

 

The only operation that does work is to retract a video (“retract” – workflow).

 

Further Investigation:

(Database):

These videos (with empty entries in the admin – ui) have a corresponding entry inside our database.

While these videos contain a similar number of mediapackage.xml files (compared to videos that used the same workflow, belong to the same series and come from the same source), these mediapackage.xml files themselves are shorter.

After checking these mefdiapackage.xml files, I found that they contain all the needed metadata for these videos.

It seems like the database should not be our problem.

 

(File System):

Because of this I continued with our filesystem.

As far as I can tell, the video files are located in 4 directories:

/mh_default_org/api

/mh_default_org/engage-player

/mh_default_org/internal

/mh_default_org/oaipmh-default

 

These directories contain subdirectories, each with their own associated file.

(These subdirectories get saved in some mediapackage.xml for reference).

We found that /api, /engage-player and /oaipmh-default contain at least one doublincore.xml file.

But /internal is missing this doublincore.xml file.

 

Currently we are suspecting that this is the reason for these empty admin – ui entries as I suspect /internal may correlate to the admin – ui.

  

Our questions:

So, we still have the doublincore.xml files with all needed metadata, but we cannot get this data into the admin – ui.

Greg wrote that we would go through these recordings to fix the metadata, but I do not understand how to do this, as the workflows do not work (I could only test this with one video though, as I cannot simply risk disabling a video that may still be used).

Is there another way to change the metadata?

Or is it as simple as copying a directory from e.g. /mh_default_org/api/<uuid> to /mh_default_org/internal/<uuid>?

But these directories will then be missing inside the mediapackage.xml file in our database, and I would suspect this would not work?

 

Can anyone help me with this problem?


Take care,

Andreas | TH Deggendorf

22_11_07-admin_error_log-doblin_core_error.txt
22_11_07-presentation_error_log_500.txt

Dietmar Zenker

unread,
Nov 7, 2022, 10:09:02 AM11/7/22
to Opencast Users, thd.l...@gmail.com, thd.ma...@gmail.com, itz.feli...@gmail.com
Hi Andreas,

concerning your file system structure: where is your archive directory?? The structure described is inside the /downloads subdir (at least in our installation).

Greetings,
Dietmar



Andreas Lindner

unread,
Nov 7, 2022, 11:29:52 AM11/7/22
to Opencast Users, dzenk...@online.de, Andreas Lindner, thd.ma...@gmail.com, itz.feli...@gmail.com
Hi Dietmar,

sorry, I actually forgot to add that.

basically we have this structure:

-srv
--opencast
----archive
--------mh_default_org
----------------<uuid> (manifest, security policy and <uuid>.xml  is in all mediapackages included
----downloads
----------------api
---------------------------<uuid> (see above)
----------------engage-player
---------------------------<uuid> (see above)
----------------internal
---------------------------<uuid> (see above)
----------------oaipmh-default
---------------------------<uuid> (see above)
----files
----------------collection
----------------mediapackage (contains security-policies)
----staticfiles (empty)
----streams (empty)
----tmp
----------------zip (empty)
----workspace (well... the workspace)


I hope this helps.


Take care,

Andreas | TH Deggendorf

Andreas Lindner

unread,
Nov 16, 2022, 7:25:41 AM11/16/22
to Opencast Users, Andreas Lindner, dzenk...@online.de, thd.ma...@gmail.com, itz.feli...@gmail.com

We managed to solve our problem.

So we have about 4000 videos that showed up as empty lines inside the admin-ui.

But the opencast media portal and the paella player could access the metadata, strangely enough neither the Opencast editor, nor the Standalone Editor did work.

If we wanted to run a workflow on one of these videos like “publish” or “republish-metadata” these workflows failed.

In addition, we could not use the api to get any metadata from any of the empty videos with /api/events/<video-id>

 

We always received the following error (API, workflows, metadata):

Exception message WorkflowOperationException: org.opencastproject.assetmanager.api.AssetManagerException: An asset with checksum ec2ade4e91e4d277ca4b1988f03a4781 (md5) has already been archived but trying to copy or link asset StoragePath(orgId=mh_default_org, mpId=8ff51d53-a7b3-4b06-bd3b-3a8eb646a886, version=3, mpeId=5a4f80c4-0f65-4d2b-b55a-60617e241711) to it failed

 

 

We think this error first occurred during our update from opencast 8.5 to opencast 11.8.

 

After quite some investigation we concluded that this problem arose from the archive and the stored snapshots.

Let me explain:

Usually, the archive stores all snapshots that are created for one video.  

As we had a very “custom” workflow sometimes these snapshots contain 6 video files. In addition, we saw that these snapshots were sometimes not referencing other snapshots, as some video files were edited.

In addition to that, many videos were in some way, shape or form changed and stored upwards of 10 snapshots.

Because of this we use an asset-delete workflow operation, to reduce our snapshots.

 

So, after updating opencast the index rebuild did not find the archive snapshot. Although the archive stored a snapshots, the rebuild somehow always searched for the wrong snapshot number.

(if the snapshot was named “4” the index rebuild looked for snapshot “3”)

This is due to the database, where the wrong snapshot is stored.

As we delete previous snapshots, this snapshot cannot be found.

 

And this produced our empty lines inside the admin – ui.

As these videos were published though, the media portal and the paella player could access the dublincore.xml and extract the metadata.

 

Our solution:

Inside the Archive we rename the snapshot (“4” gets renamed to “3”).

After this we republish (“publish” workflow) the video. If you just want to have the metadata back, the “republish-metadata” workflow is insufficient, as the “publish” workflow pulls the video files back into the archives and rewrites the manifest.xml files.

So, with the “publish” workflow everything inside the admin – ui is fine, the publications work and the assets are mapped correct.

Reply all
Reply to author
Forward
0 new messages