Multiserver install Opencast 3.0 beginner questions

[X-Net Services]

Hi

I tried to install opencast on a "multi" (three) server setup.

1.) opencast-dist-adminpresentation (Database, ActiveMQ, ...)
2.) opencast-dist-worker
3.) NFS Server

I used the the multiserver installation guide and nearly everything else I have found.
https://docs.opencast.org/r/2.2.x/admin/installation/multiple-servers/#install-across-multiple-servers


What it currently does:
Both servers can connect to the database and the ActiveMQ server.
If I start the servers with "sudo -u opencast ./start-opencast" everything looks fine.
I can connect to the admin interface and login.

After some time I can see this messages in the log file:
...
2017-09-11 13:59:46,686 | WARN  | (ServiceRegistryJpaImpl$JobProducerHeartbeat:3220) - Service org.opencastproject.composer@http://opencast.mydomain.org:8080 is not working as expected: HTTP/1.1 404 Not Found
2017-09-11 13:59:46,687 | WARN  | (ServiceRegistryJpaImpl$JobProducerHeartbeat:3239) - Added org.opencastproject.composer@http://opencast.mydomain.org:8080 to the watch list
2017-09-11 13:59:46,702 | WARN  | (ServiceRegistryJpaImpl$JobProducerHeartbeat:3220) - Service org.opencastproject.coverimage@http://opencast.mydomain.org:8080 is not working as expected: HTTP/1.1 404 Not Found
2017-09-11 13:59:46,703 | WARN  | (ServiceRegistryJpaImpl$JobProducerHeartbeat:3239) - Added org.opencastproject.coverimage@http://opencast.mydomain.org:8080 to the watch list
2017-09-11 13:59:46,848 | WARN  | (ServiceRegistryJpaImpl$JobProducerHeartbeat:3220) - Service org.opencastproject.inspection@http://opencast.mydomain.org:8080 is not working as expected: HTTP/1.1 404 Not Found
2017-09-11 13:59:46,848 | WARN  | (ServiceRegistryJpaImpl$JobProducerHeartbeat:3239) - Added org.opencastproject.inspection@http://opencast.mydomain.org:8080 to the watch list
2017-09-11 13:59:46,913 | WARN  | (ServiceRegistryJpaImpl$JobProducerHeartbeat:3220) - Service org.opencastproject.publication.youtube@http://opencast.mydomain.org:8080 is not working as expected: HTTP/1.1 404 Not Found
2017-09-11 13:59:46,913 | WARN  | (ServiceRegistryJpaImpl$JobProducerHeartbeat:3239) - Added org.opencastproject.publication.youtube@http://opencast.mydomain.org:8080 to the watch list
2017-09-11 13:59:46,952 | WARN  | (ServiceRegistryJpaImpl$JobProducerHeartbeat:3220) - Service org.opencastproject.silencedetection@http://opencast.mydomain.org:8080 is not working as expected: HTTP/1.1 404 Not Found
2017-09-11 13:59:46,953 | WARN  | (ServiceRegistryJpaImpl$JobProducerHeartbeat:3239) - Added org.opencastproject.silencedetection@http://opencast.mydomain.org:8080 to the watch list
2017-09-11 13:59:46,968 | WARN  | (ServiceRegistryJpaImpl$JobProducerHeartbeat:3220) - Service org.opencastproject.sox@http://opencast.mydomain.org:8080 is not working as expected: HTTP/1.1 404 Not Found
2017-09-11 13:59:46,968 | WARN  | (ServiceRegistryJpaImpl$JobProducerHeartbeat:3239) - Added org.opencastproject.sox@http://opencast.mydomain.org:8080 to the watch list
2017-09-11 13:59:46,985 | WARN  | (ServiceRegistryJpaImpl$JobProducerHeartbeat:3220) - Service org.opencastproject.textanalyzer@http://opencast.mydomain.org:8080 is not working as expected: HTTP/1.1 404 Not Found
2017-09-11 13:59:46,985 | WARN  | (ServiceRegistryJpaImpl$JobProducerHeartbeat:3239) - Added org.opencastproject.textanalyzer@http://opencast.mydomain.org:8080 to the watch list
2017-09-11 13:59:47,000 | WARN  | (ServiceRegistryJpaImpl$JobProducerHeartbeat:3220) - Service org.opencastproject.videoeditor@http://opencast.mydomain.org:8080 is not working as expected: HTTP/1.1 404 Not Found
2017-09-11 13:59:47,000 | WARN  | (ServiceRegistryJpaImpl$JobProducerHeartbeat:3239) - Added org.opencastproject.videoeditor@http://opencast.mydomain.org:8080 to the watch list
2017-09-11 13:59:47,011 | WARN  | (ServiceRegistryJpaImpl$JobProducerHeartbeat:3220) - Service org.opencastproject.videosegmenter@http://opencast.mydomain.org:8080 is not working as expected: HTTP/1.1 404 Not Found
2017-09-11 13:59:47,011 | WARN  | (ServiceRegistryJpaImpl$JobProducerHeartbeat:3239) - Added org.opencastproject.videosegmenter@http://opencast.mydomain.org:8080 to the watch list
2017-09-11 13:59:47,022 | WARN  | (ServiceRegistryJpaImpl$JobProducerHeartbeat:3220) - Service org.opencastproject.waveform@http://opencast.mydomain.org:8080 is not working as expected: HTTP/1.1 404 Not Found
...


I have added "opencast.mydomain.org" to both "/etc/hosts" files with the local IP address.


But I think I misunderstood some things in the installation description.

1.) Do I need to compile opencast myself?
Currently I simply downloaded the binarys from here and don't compiled anything:
https://bitbucket.org/opencast-community/opencast/downloads/?tab=downloads
opencast-dist-adminpresentation-3.0.tar.xz
opencast-dist-worker-3.0.tar.xz
(I think I should try it with 3.2 in the future)

I simply extracted the tar files to "/opt/opencast" and started it there.
But maybe I miss some dependencies on this way?


2.) Questions to some configs:
Opencast should run on 8080 for external usage I use an apache proxy (for http and https)

In "/opt/opencast/etc/custom.properties"
org.opencastproject.server.url=http://opencast.mydomain.org
I'm pretty sure I should not use "http://opencast.mydomain.org:8080" here?
This option represents the URL?
I use the same value on admin and worker.

In "/opt/opencast/etc/org.opencastproject.organization-mh_default_org.cfg"
prop.org.opencastproject.admin.ui.url=http://opencast.mydomain.org:8080
prop.org.opencastproject.engage.ui.url=http://opencast.mydomain.org:8080
Here I think I should use the ":8080" URL? (The Opencast worker should connect dirictly to the admin server without the apache proxy -> sounds better for me)
Or do need here the same value as "org.opencastproject.server.url"?
Does the "opencast-dist-adminpresentation-3.0.tar.xz" include "engage"?

In "/opt/opencast/etc/org.ops4j.pax.web.cfg"
org.ops4j.pax.web.listening.addresses=0.0.0.0
I need to set this so the worker can connect directly?

Regards
xnet

[Greg Logan]

Hi xnet,

First off, the installation documentation you linked is for 2.2.x, not 3.x.  Things have changed a fair bit from there, so please use the correct documentation!

Re: /etc/hosts, you should not have to modify your hosts file if things are correctly set up, but it also should not *hurt* either.

Re: Compilation vs download: You can just download the binaries from BitBucket, however we encourage everyone who is not a developer to use the RPM (or upcoming Deb) packages instead.  Take a look at https://docs.opencast.org/r/3.x/admin/installation/, under Installation from Repository.

Re configs: In general I set org.opencastproject.server.url with :8080, although the documentation in the file suggests the opposite.  In the org config file you want to set the URLs that your *users* will type, so in this case you do *not* want to include the ports.  Workers don't use the org config file to figure out where their admin server is.  Every node in an Opencast install runs (by default) the core Service Registry, which does the job dispatching and ties the nodes together.  This lets the workers (and every other node) find the core admin node bits using just their database connection.

Re things including "engage": Engage is an old name for the current Presentation node/profile, so yes adminpresentation would include Engage :)

Re pax.web.cfg: You could set the listening address to your public IP as well, however 0.0.0.0 is much easier to explain!  The default only allows connections on the localhost, which would not be terribly useful!

G

--
You received this message because you are subscribed to the Google Groups "Opencast Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to users+un...@opencast.org.

[X-Net Services]

Hi Greg,

it looks like I'm doing something wrong.

I'm working on Ubuntu, sadly there is currently no repository I can use.
To make sure everything should work I now compiled everything myself (two times -> on Admin Server and on Worker).

I have done again all steps from the 3.x docu and now I have the same error.
...
2017-10-09 14:18:37,949 | WARN  | (ServiceRegistryJpaImpl$JobProducerHeartbeat:3220) - Service org.opencastproject.composer@http://opencast.mydomain.org:8080 is not working as expected: HTTP/1.1 404 Not Found
2017-10-09 14:18:37,950 | WARN  | (ServiceRegistryJpaImpl$JobProducerHeartbeat:3239) - Added org.opencastproject.composer@http://opencast.mydomain.org:8080 to the watch list
2017-10-09 14:18:37,967 | WARN  | (ServiceRegistryJpaImpl$JobProducerHeartbeat:3220) - Service org.opencastproject.coverimage@http://opencast.mydomain.org:8080 is not working as expected: HTTP/1.1 404 Not Found
2017-10-09 14:18:37,968 | WARN  | (ServiceRegistryJpaImpl$JobProducerHeartbeat:3239) - Added org.opencastproject.coverimage@http://opencast.mydomain.org:8080 to the watch list
2017-10-09 14:18:38,061 | WARN  | (ServiceRegistryJpaImpl$JobProducerHeartbeat:3220) - Service org.opencastproject.execute@http://opencast.mydomain.org:8080 is not working as expected: HTTP/1.1 404 Not Found
2017-10-09 14:18:38,062 | WARN  | (ServiceRegistryJpaImpl$JobProducerHeartbeat:3239) - Added org.opencastproject.execute@http://opencast.mydomain.org:8080 to the watch list
2017-10-09 14:18:38,122 | WARN  | (ServiceRegistryJpaImpl$JobProducerHeartbeat:3220) - Service org.opencastproject.inspection@http://opencast.mydomain.org:8080 is not working as expected: HTTP/1.1 404 Not Found
2017-10-09 14:18:38,122 | WARN  | (ServiceRegistryJpaImpl$JobProducerHeartbeat:3239) - Added org.opencastproject.inspection@http://opencast.mydomain.org:8080 to the watch list
2017-10-09 14:18:38,165 | WARN  | (ServiceRegistryJpaImpl$JobProducerHeartbeat:3220) - Service org.opencastproject.publication.youtube@http://opencast.mydomain.org:8080 is not working as expected: HTTP/1.1 404 Not Found
...

If I remove the "8080" from the "org.opencastproject.server.url" option the error stays the same (without the 8080 in the url)
...
2017-10-09 14:18:37,949 | WARN  | (ServiceRegistryJpaImpl$JobProducerHeartbeat:3220) - Service org.opencastproject.composer@http://opencast.mydomain.org is not working as expected: HTTP/1.1 404 Not Found
2017-10-09 14:18:37,950 | WARN  | (ServiceRegistryJpaImpl$JobProducerHeartbeat:3239) - Added org.opencastproject.composer@http://opencast.mydomain.org to the watch list
...

I see this messages on the admin and on the worker server.

I can login to the admin webinterface.
If I go to "Systems -> Server" I can see the admin server but not the worker.
Short after the start the status dot is green. After the warn messages the dot goes red.

I don't understand why the warning messages point to a "404 Not Found".
There is a "/etc/hosts" entry for "opencast.mydomain.org" which points to its own local IP address (on the worker the hosts entry points to the local IP of the admin server).
I don't think this is an IP related issue. There is also no Firewall between the servers (and there is no local firewall on the admin server).
Maybe I forgot something to install or to activate/configure but I don't have an idea what this could be.

Regards
x-net

[Greg Logan]

Hi X-net,

Not sure if you got this worked out, but here are the settings I use at a production site.  The site has nginx reverse proxies in front of the admin and presentation nodes, but the workers do not have these proxies.

On the admin and presentation nodes:

custom.properties

 - org.opencastproject.server.url=http://(admin|presentation).example.com

 - org.opencastproject.jobs.url=http://admin.example.com

 - org.opencastproject.file.repo.url=http://admin.example.com

org.opencastproject.organization-mh_default_org.cfg

 - server=(admin|presentation).example.com

 - port=-1

 - prop.org.opencastproject.admin.ui.url=http://admin.example.com

 - prop.org.opencastproject.engage.ui.url=http://presentation.example.com

On the worker nodes:

custom.properties

 - org.opencastproject.server.url=http://workerN.example.com:8080

 - org.opencastproject.jobs.url=http://admin.example.com

 - org.opencastproject.file.repo.url=http://admin.example.com

org.opencastproject.organization-mh_default_org.cfg

 - server=workerN.example.com

 - port=8080

 - prop.org.opencastproject.admin.ui.url=http://admin.example.com

 - prop.org.opencastproject.engage.ui.url=http://presentation.example.com

Were I to add the reverse proxies to the workers, it would be simple change of removing the :8080 from the server url, and changing the organization file's port to -1.

Does that make things clearer?

G