Extron SMP 35x cannot ingest: 400 Bad Request

[K. S.]

Hi Extron SMP users,

has anybody managed to get Extron SMP capture agents to work with Opencast 9/10?

For us the initial POST request from the SMP to the /ingest/addCatalog endpoint always results in 400 Bad request. The error log indicates that Opencast is not happy with the XML format:

2021-08-04T15:55:52,924 | DEBUG | (IngestRestService:556) - Invalid catalog data
java.lang.IllegalArgumentException: Catalog XML is invalid
[...]

Caused by: org.xml.sax.SAXParseException: Content is not allowed in prolog.
        at org.apache.xerces.parsers.DOMParser.parse(Unknown Source) ~[?:?]
        at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source) ~[?:?]
        at org.opencastproject.util.XmlSafeParser.parse(XmlSafeParser.java:187) ~[!/:?]
        at org.opencastproject.util.XmlSafeParser.parse(XmlSafeParser.java:150) ~[!/:?]
        at org.opencastproject.ingest.impl.IngestServiceImpl.addCatalog(IngestServiceImpl.java:950) ~[!/:?]
        ... 116 more

Looking at the content of the actual POST request, the XML part seems fine. At least I cannot see any weird special chars or something like that (I have attached the content of the request in case someone wants to have a look).

I also found a recent pull request in the github repo that changes the behavior with XML parsing (https://github.com/opencast/opencast/pull/2535), could this be related?

Any help is appreciated!

Katrin

[Christoph Reifers]

Hi Katrin,

I just updated the PR which "fixes" this issue  with my findings on this, hopefully you can fix the issue with those.
https://github.com/opencast/opencast/pull/2849

Regards

Christoph

--
To unsubscribe from this group and stop receiving emails from it, send an email to users+un...@opencast.org.

[K. S.]

Hi Christoph,

thanks for the pointer, I will definitely try this!

By the way, we also reached out to our Extron Contact and they have referred the issue to their dev team. It's of course hard to tell how long it will take until we (hopefully) have access to a firmware fix.

-Katrin

[Lars Kiesow]

Hi everyone,
to complete the loop: Extron reaced out to me they are working on the
issue.

An exception for this in Opencast will also be part of Opencast 10.2
which is scheduled to be released on August 18.


To give you a little bit of background to what's happening: Catalogs in
Opencast are supposed to be XML and having broken XML in this can cause
really annoying issues which is why Opencast now validates the input to
not accept broken things in the first place.

Unfortunately, the SMP ingests some JSON data as catalog. It is not
actually used anywhere and thus does not cause any internal problems,
but is still rejected now that the verification of data is in place.
The SMPs should ingest something like this as attachment instead, which
they will likely do in the future.

But since it is not nice to break capture agents and these data aren't
actually causing any harm to the system, there is now also an exception
for these data in Opencast. Essentially, if it seems to be JSON,
Opencast will not verify it and just let it pass.

That fix will be in the 10.2 release.


We would like to remove that exception eventually, but will keep the
exception around for now, revisiting the issue in a year or so to give
the firmware time to catch up ;-)

Best regards,
Lars