Cannot build KMS 7.0 from sources

[jordan...@gmail.com]

Good Evening,

I have a custom built version of KMS 7.0 which I build from sources. I haven't had cause to build the Docker image for a couple of months but for some reason the Dockerfile no longer work. Has something been changed?

My Dockerfile contains

RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 5AFA7A83 \

&& echo "deb http://ubuntu.openvidu.io/bionic-gstreamer bionic kms6" >/etc/apt/sources.list.d/kurento.list

which gives the error:

Step 4/50 : RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 5AFA7A83  && echo "deb http://ubuntu.openvidu.io/bionic-gstreamer bionic kms6" >/etc/apt/sources.list.d/kurento.list
 ---> Running in d9be4712a245
Warning: apt-key output should not be parsed (stdout is not a terminal)
Executing: /tmp/apt-key-gpghome.luwnyy68GV/gpg.1.sh --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 5AFA7A83
gpg: key FC8A16625AFA7A83: public key "Openvidu - Kurento TEAM <open...@gmail.com>" imported
gpg: Total number processed: 1
gpg:               imported: 1
Removing intermediate container d9be4712a245
 ---> 612de079d37b
Step 5/50 : RUN apt-get update
 ---> Running in ae3030d59a29
Err:1 http://ubuntu.openvidu.io/bionic-gstreamer bionic InRelease
  403  Forbidden [IP: 52.218.45.112 80]
Hit:2 http://security.ubuntu.com/ubuntu bionic-security InRelease
Hit:3 http://archive.ubuntu.com/ubuntu bionic InRelease
Hit:4 http://archive.ubuntu.com/ubuntu bionic-updates InRelease
Hit:5 http://archive.ubuntu.com/ubuntu bionic-backports InRelease
Reading package lists...
E: Failed to fetch http://ubuntu.openvidu.io/bionic-gstreamer/dists/bionic/InRelease  403  Forbidden [IP: 52.218.45.112 80]
E: The repository 'http://ubuntu.openvidu.io/bionic-gstreamer bionic InRelease' is not signed.
The command '/bin/sh -c apt-get update' returned a non-zero code: 100

How do I get around this?

Jordan

[Juan Navarro]

Hi,

In preparation of a Release Candidate version that is almost ready now, everything related to the Kurento 7.0 build has been renamed from the old "bionic-gstreamer" names to simply "7.0.0". Also, some restructuring has been done to the way Docker images are published.

This changes several things:

* The git branches in all source code repos are now called "7.0.0"
* The apt-get repo is now "deb http://ubuntu.openvidu.io/dev-7.0.0 bionic kms6" (by the way that "kms6" will become "main" with the final release)
* The Docker images are now "kurento/kurento-media-server:dev-7.0.0"

Note however that right now, some of the Ubuntu apt-get repositories are broken (they fail when running apt-get), so we're currently unable to publish new versions of the Docker images, until Ubuntu fixes this issue in their debug packages repositories (ddeb). But if you'll be using our repo instead of the Docker images, this issue doesn't affect you.

--
Juan Navarro
Kurento developer
@j1elo at GitHub & Twitter

--
You received this message because you are subscribed to the Google Groups "kurento" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kurento+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/e12cc42d-adc0-47a3-8d8b-cd7fc8ebf8dcn%40googlegroups.com.

[Juan Navarro]

Sorry, the apt-get repo line should be like this (it's a repo for Ubuntu Focal, not Bionic):

deb [arch=amd64] http://ubuntu.openvidu.io/dev-7.0.0 focal kms6

[jordan...@gmail.com]

Thanks Juan,

I'm going to presume that I'm going to be unable to build a custom version of KMS via Docker myself then?

Running a Dockerfile with:

RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 5AFA7A83 \

&& echo "deb http://ubuntu.openvidu.io/dev-7.0.0 focal kms6" >/etc/apt/sources.list.d/kurento.list

RUN apt-get update

RUN apt-get install --no-install-recommends --yes \

kurento-media-server-dev \

kurento-media-server

Causes a:

The following packages have unmet dependencies:
 kurento-media-server : Depends: kms-core (>= 7.0.0) but it is not going to be installed etc etc etc

[Juan Navarro]

Hi,

maybe you just caught the very first repo build after the rename from "bionic-gstreamer" to "7.0.0"; please try again now, as this should be fixed and all packages available as required. I just tried to install kurento-media-server-dev on a clean Docker container with Ubuntu 20.04 and it all worked fine, and the Kurento code compiled fine too.

To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/8ef6a115-7329-4ce6-a8d4-f93934371f3an%40googlegroups.com.

[Charchris Sloan]

Hi Juan,

I have been trying to build from sources as of the past week, using KMS 7.0
I have a question regarding the gstreamer version used.

For the following repository is gstreamer 1.16 used?

deb [arch=amd64] http://ubuntu.openvidu.io/dev-7.0.0 focal kms6

I have also tried adding this repository for bionic. I assume this uses gstreamer 1.14?

deb [arch=amd64] http://ubuntu.openvidu.io/dev-7.0.0 bionic kms6

But it does not seem to work.

Do you still hold repositories that work with KMS 7.0 and bionic with gstreamer 1.14?

Thanks

[Juan Navarro]

Hi,

deb [arch=amd64] http://ubuntu.openvidu.io/dev-7.0.0 focal kms6

This repo is to be used with an Ubuntu 20.04 system or Docker image ("ubuntu:20.04"). The packages deployed in this repo have been generated with the branch "7.0.0" that exists in most (but not all) of the individual Kurento repositories. This includes a) some of the forked repos (like libsrtp, libnice, etc, but excluding the gstreamer repos) and the actual kurento repositories (kms-core and friends, kurento-module-creator, etc).

However, all these repos have been stalled and they will be archived soon.

Following a similar change made by GStreamer, Kurento is moving everything into a single monorepo:

https://github.com/Kurento/kurento

This way, almost _everything_ related to Kurento Media Server, Java / JavaScript clients, docs, docker images, etc... will reside in a single place. This will make maintenance much easier, now that the server is not undergoing too much development any more (apart from basic maintenance to keep it up to date).

A hidden but huge change also is that we're leaving our self-hosted Jenkins server for CI, and moving everything to Github Actions, to reduce the burden on server maintenance. And it just happens that _today_ we finished everything needed to build its first successful run and publish new Debian packages and Docker images.

So from now on, please use this repo:

deb [arch=amd64] http://ubuntu.openvidu.io/dev focal main

And you'll find corresponding Docker images published as

kurento/kurento-media-server:dev-7.0

For developing and building from source, please refer to the new version of the doc files (not yet deployed to ReadTheDocs because the CI job hasn't been migrated yet to GH Actions):

https://github.com/Kurento/kurento/blob/main/doc-kurento/source/dev/dev_guide.rst

We're just getting ready to make a full release, and soon before that we'll make an announcement in this mailing list in order to introduce all changes.

Other stuff that will also change is the bug repository (it will be the Issues section of the new monorepo) AND probably the mailing list itself (which will move to the Discussion section of the repository). As you see, the main target here is to unify every resource so all different parts of the project can be found under the same place.

Regards

-- 
Juan Navarro
Software Development Engineer
Kurento & OpenVidu WebRTC platforms | https://openvidu.io/

To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/aaa41124-549f-4d89-a7ca-cb19d7a6563fn%40googlegroups.com.

[Neil Young]

Great move. Can't wait to give it a try on the weekend :)

Thanks for sharing. Good news

[Alejandro Sellero Rodríguez]

Hi,

I got a bit worried about the sentence "now that the server is not undergoing too much development any more (apart from basic maintenance to keep it up to date)."

Does that mean that Kurento is in maintenance mode and will not get new features? Or does it mean only during this transisition phase to KMS 7 and the monorepo infrastructure?

Thanks
Alex

[Juan Navarro]

Kurento has been in what I'd call semi-maintenance mode since at least 2016. Still, it has been kept up to date and constantly fine tuned, with lots of bug fixes, security considerations, documentation improvements, etc. Tons of already existing features have been improved upon over time, but there are no major *big* new features planned for it (such as e.g. WebRTC insertable streams, or support for more than 1 video + 1 audio in the same WebRTC peer connection).

As such, Kurento works very well for whom it works, and it falls short for who needs all that newer stuff. It is not the fastest media server in existence, but it is mostly the only one able to seamlessly integrate OpenCV processing. Some people built their services on top of Kurento, others opted for lighter-weight alternatives like mediasoup, or higher-level platforms like OpenVidu that hides to some extent the complexity of WebRTC.

Note that OpenVidu is also built by us the same team, and it is itself built partly on top of Kurento, so the financial incentives to keep the engine running are there.

Regards

-- 
Juan Navarro
Software Development Engineer
Kurento & OpenVidu WebRTC platforms | https://openvidu.io/

To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/edd62d00-7c4a-4ca5-99e7-7c100d13d6a9n%40googlegroups.com.

[Alejandro Sellero Rodríguez]

Ok, now I feel like an idiot :D. But I don't remember seeing explicitly this info anywhere in the docs or the repos (maybe I missed it). Would it be ok for you if I create a PR in your new monorepo to add exactly this text in your README.md?

[Juan Navarro]

I think now that we're restructuring the whole project and creating a single point of truth for all resources and documentation, would also be a great time to make clarifications regarding the current state of the project and its future roadmap.

Better than a PR, I will talk about this with the team and decide on a text that could be seen in the project's readme and/or (probably "and") the documentation site. It's good that people know the current state of things and use that to decide; while we have been straightforward and honest in this regard whenever people asked us about it, it's true that I cannot see any clear-cut statement in the docs.

-- 
Juan Navarro
Software Development Engineer
Kurento & OpenVidu WebRTC platforms | https://openvidu.io/

To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/7aff5fbd-dd98-4715-8ee2-26a633b0a968n%40googlegroups.com.

[Alejandro Sellero Rodríguez]

Very nice to hear, thank you very much Juan!

Have a nice weekend :)

[Neil Young]

@Juan: I notice that the Installation guide pointed to from the new repo is still referring to 6.18. Not 7?

[Neil Young]

I'm a bit confused now. The repo, which is supposed to be "the future of KMS" is this https://github.com/Kurento/kurento (if I understand this correctly). In that repo there is a README, which describes the installation from source. This refers to 6.18 (which is maybe not 7) and requires Ubuntu 18.04. I thought the purpose of all that re-organization was to make it run on more recent versions of Ubuntu, namely 20.04 and to use the system installed GStreamer for this. Up to now I cannot see any trace of this.

Then there is this https://github.com/Kurento/kurento/blob/main/doc-kurento/source/dev/dev_guide.rst

There are now explicit instructions to build from source on Ubuntu 20.04. 

Could you please confirm, that this final gist is now the gist to follow to build KMS 7 from source on Ubuntu 20.04? Currently there is unnecessarily too much buzz in the air around this, IMHO. But I really would appreciate to have finally something built from source running on Ubuntu 20.04. Including my OpenCV plugins. Including all the new stuff (namely support for the request intra-frame)

[Neil Young]

@Juan: Install build dependencies, Option 1, quick setup is not going to work:

ubuntu@kms7:~$ sudo apt-get update ; sudo apt-get install --no-install-recommends \

>     kurento-media-server-dev

[sudo] password for ubuntu: 

Hit:1 http://de.archive.ubuntu.com/ubuntu focal InRelease

Get:2 http://de.archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB]

Get:3 http://de.archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB]

Get:4 http://de.archive.ubuntu.com/ubuntu focal-security InRelease [114 kB]

Get:5 http://de.archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2,405 kB]

Fetched 2,741 kB in 11s (245 kB/s) 

Reading package lists... Done

Reading package lists... Done

Building dependency tree       

Reading state information... Done

E: Unable to locate package kurento-media-server-dev

ubuntu@kms7:~$ 

Option2 is also not going to work (or I didn't understand how to use it):

ubuntu@kms7:~$ sudo apt-get update ; sudo mk-build-deps --install --remove \

>     --tool="apt-get -o Debug::pkgProblemResolver=yes --no-install-recommends --no-remove" \

>     debian/control

Hit:1 http://de.archive.ubuntu.com/ubuntu focal InRelease

Hit:2 http://de.archive.ubuntu.com/ubuntu focal-updates InRelease

Hit:3 http://de.archive.ubuntu.com/ubuntu focal-backports InRelease

Hit:4 http://de.archive.ubuntu.com/ubuntu focal-security InRelease

Reading package lists... Done

mk-build-deps: You must have equivs installed to use this program.

ubuntu@kms7:~$ 

After sudo apt-get install -y equivs and re-run:

    E: You must put some 'deb-src' URIs in your sources.list

After this:

sudo cp /etc/apt/sources.list /etc/apt/sources.list~
    sudo sed -Ei 's/^# deb-src /deb-src /' /etc/apt/sources.list
    sudo apt-get update

finally 

W: Unable to locate package debian/control

mk-build-deps: Unable to find package name in `apt-cache showsrc debian/control'

And I gave up :)

[Neil Young]

System is a fresh Ubtuntu 20.04.5 server VM

I'm missing statements like 

apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 5AFA7A83 echo "deb [arch=amd64] http://ubuntu.openvidu.io/6.13.0 bionic kms6" \ | tee /etc/apt/sources.list.d/kurento.list

in the gist

[Neil Young]

And finally: The ":ref:" in the gist are not working either.

I will be patient then

Nice weekend

[Juan Navarro]

Hi, yes I can confirm that https://github.com/Kurento/kurento/blob/main/doc-kurento/source/dev/dev_guide.rst is the guide to build Kurento from source code. I cannot confirm that it's _final_, because it is not. Kurento 7 has not been released, thus everything is just to be considered as an in-progress development preview. As such, there are broken things, and links to doc-kurento.readthedocs.io will still point to docs in the Read The Docs server, which contains info for Kurento 6.x; we don't even yet have the CI jobs to upload new content to ReadTheDocs, much less to build new doc versions in that site.

To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/eed6e780-bda5-48dc-95e8-4cfa9aceead0n%40googlegroups.com.

[Neil Young]

Well,  I'm only interested in this https://github.com/Kurento/kurento/blob/main/doc-kurento/source/dev/dev_guide.rst#build-from-sources

Isn't it possible to have a gist, which works for just this?

[Juan Navarro]

Just today I've finished the CI workflow that builds the docs, so you should now be able to see the latest version of the text:

https://doc-kurento.readthedocs.io/en/latest/dev/dev_guide.html#build-from-sources

For now, when the text mentions to install or configure Kurento repositories, make sure to use the development option, as the release doesn't exist yet for version 7.0.0.

Regards,

-- 
Juan Navarro
Software Development Engineer
Kurento & OpenVidu WebRTC platforms | https://openvidu.io/

To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/5bc9c235-38d3-495a-bf56-a78982d953f6n%40googlegroups.com.

[Neil Young]

Thanks.

> For now, when the text mentions to install or configure Kurento repositories, make sure to use the development option

Means what?

To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/9b630ecf-f4e3-5d56-b7a5-f8e9f8026d32%40gmx.es.

[Neil Young]

OK, hoping I figured out what you meant.

Did this so far:

sudo apt-get update
sudo apt-get install --no-install-recommends build-essential ca-certificates cmake git gnupg pkg-config
source /etc/upstream-release/lsb-release 2>/dev/null || source /etc/lsb-release

# Note: adapted key server URL, other didn't work
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80  --recv-keys 234821A61B67740F89BFD669FC8A16625AFA7A83

sudo tee "/etc/apt/sources.list.d/kurento.list" >/dev/null <<EOF
# Kurento Media Server - Nightly packages
deb [arch=amd64] http://ubuntu.openvidu.io/dev $DISTRIB_CODENAME main
EOF

sudo apt-get update ; sudo apt-get install --no-install-recommends kurento-media-server

git clone https://github.com/Kurento/kurento.git
cd kurento/
git submodule update --init --recursive
cd server
export MAKEFLAGS="-j$(nproc)"
bin/build-run.sh

# First attempt failed, since mvn was missing so I installed maven:

sudo apt install maven

# and re-run the build script
bin/build-run.sh

It ran a while then

-- Try GNU SanitizerBlacklist flag = [-fsanitize-blacklist=/home/ubuntu/kurento/server/bin/sanitizers/blacklist.txt]
-- Performing Test SanBlist_FLAG_DETECTED
-- Performing Test SanBlist_FLAG_DETECTED - Failed
CMake Warning at cmake-utils/3rdparty/sanitizers-cmake/cmake/sanitize-helpers.cmake:150 (message):
  SanitizerBlacklist is not available for GNU compiler.  Targets using this
  compiler will be compiled without SanitizerBlacklist.
Call Stack (most recent call first):
  cmake-utils/3rdparty/sanitizers-cmake/cmake/FindSanitizers.cmake:51 (sanitizer_check_compiler_flags)
  CMakeLists.txt:98 (sanitizer_add_blacklist_file)


-- CMake project: kurento-jsonrpc
-- Found KurentoUtils: 7.0.0~.gf1665e4f1 (found version "7.0.0~.gf1665e4f1")
-- [FindKurentoUtils.cmake] Found: /home/ubuntu/kurento/server/build-Debug/cmake-utils
fatal: ambiguous argument '6.18.0..HEAD': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
-- Version info from git: 7.0.0~.gf1665e4f1
-- Project version: kurento-jsonrpc-7.0.0~.gf1665e4f1
-- Checking for module 'jsoncpp>=1.7.2'
--   No package 'jsoncpp' found
CMake Error at /usr/share/cmake-3.16/Modules/FindPkgConfig.cmake:463 (message):
  A required package was not found
Call Stack (most recent call first):
  /usr/share/cmake-3.16/Modules/FindPkgConfig.cmake:643 (_pkg_check_modules_internal)
  jsonrpc/CMakeLists.txt:39 (pkg_check_modules)


-- Configuring incomplete, errors occurred!
See also "/home/ubuntu/kurento/server/build-Debug/CMakeFiles/CMakeOutput.log".
See also "/home/ubuntu/kurento/server/build-Debug/CMakeFiles/CMakeError.log".
[build-run.sh] ERROR (1)
ubuntu@kms7:~/kurento/server$

Stopped here

[Juan Navarro]

Hi, let's see if we can solve the build issues.

For starters, my recommendation for verifying that Kurento builds on our supported base system (Ubuntu 20.04 Focal), is to use a pristine, clean system, where past installations or user customizations won't affect the result. For this, a new virtual machine would do, but I find it more handy to use Docker containers.

As long as the following script runs successfully, I would be confident that the whole process is healthy and any issue or error is caused by external factors not related with Kurento itself. Please check it out and see if you can reproduce the same set of steps and commands in your system; otherwise you'd need to clean up your system or better yet, please use a disposable environment that can guarantee a clean system without remains from previous installs or other projects.

If Docker is installed in your machine, you should be able to copy-paste and run this whole block (between the BEGIN and END lines) in one single go; I just did before sending this email and it finished successfully:

# In-place Docker container commands BEGIN
docker run --rm -i ubuntu:20.04 /bin/bash <<'DOCKERCOMMANDS'

# Bash options for strict error checking
set -o errexit -o errtrace -o pipefail -o nounset

# Trace all commands
set -o xtrace

# Run apt-get/dpkg without interactive dialogue.
export DEBIAN_FRONTEND=noninteractive

# Get DISTRIB_* env vars.

source /etc/upstream-release/lsb-release 2>/dev/null || source /etc/lsb-release

echo "# Install required tools"

apt-get update ; apt-get install --no-install-recommends --yes \
    build-essential \
    ca-certificates \
    cmake \
    git \
    pkg-config



echo "# Install Kurento Media Server"

apt-get update ; apt-get install --no-install-recommends --yes \
    gnupg

# Add Kurento repository key for apt-get.
apt-key adv \
    --keyserver keyserver.ubuntu.com \
    --recv-keys 234821A61B67740F89BFD669FC8A16625AFA7A83

# Add Kurento repository line for apt-get.
tee "/etc/apt/sources.list.d/kurento.list" >/dev/null <<EOF

deb [arch=amd64] http://ubuntu.openvidu.io/dev $DISTRIB_CODENAME main
EOF

# Install Kurento Media Server development packages.
apt-get update ; apt-get install --no-install-recommends --yes \
    kurento-media-server-dev



echo "# Download Kurento source code"

git clone https://github.com/Kurento/kurento.git

cd kurento/

git submodule update --init --recursive

echo "# Build and run Kurento"

cd server/

export MAKEFLAGS="-j$(nproc)"
bin/build-run.sh --build-only

echo "Done! Everything got installed and built successfully"

DOCKERCOMMANDS
# In-place Docker container commands END

Regards,

-- 
Juan Navarro
Software Development Engineer
Kurento & OpenVidu WebRTC platforms | https://openvidu.io/

To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/535b17af-2f3a-4e14-8861-3ec46b5a74cen%40googlegroups.com.

[Neil Young]

Hehe. Thanks for dealing with this, Juan. But I abysmally hate docker and there is no way to make me use it :)

Anyway, it _should_ be possible to run the DOCKERCOMMANDS on a fresh VM as is. I thought I did this, but I will give it another try now

[Neil Young]

But anyway, I see no _major_ difference to my commands, and I was missing maven. Can't see how you install this.

[Neil Young]

Yepp, this time it worked. Maybe my previous VM wasn't that "clean" as I thought it would be. There was just one little issue, which I could overcome. Also - no maven post-installation required as for the first time. Strange, anyway.

For those, who would like to follow, these are the steps:

I was starting from a fresh Ubuntu Server 20.04.5 instance in a VirtualBox VM, 4 cores, 8 GB RAM, 60 GB HD, LAN connection.

1)

source /etc/upstream-release/lsb-release 2>/dev/null || source /etc/lsb-release

2)
sudo apt-get update
sudo apt-get install --no-install-recommends --yes build-essential ca-certificates cmake git pkg-config gnupg

3)
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 234821A61B67740F89BFD669FC8A16625AFA7A83

Step 3 fails (I already reported), but this is not uncommon (can be googled):

ubuntu@kms7:~$ sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 234821A61B67740F89BFD669FC8A16625AFA7A83
Executing: /tmp/apt-key-gpghome.cOtG38DRDG/gpg.1.sh --keyserver keyserver.ubuntu.com --recv-keys 234821A61B67740F89BFD669FC8A16625AFA7A83
gpg: keyserver receive failed: Server indicated a failure

This is an alternative Step 3, which works:

ubuntu@kms7:~$ sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80  --recv-keys 234821A61B67740F89BFD669FC8A16625AFA7A83
Executing: /tmp/apt-key-gpghome.nzMOsh5XQv/gpg.1.sh --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 234821A61B67740F89BFD669FC8A16625AFA7A83

gpg: key FC8A16625AFA7A83: public key "Openvidu - Kurento TEAM <open...@gmail.com>" imported
gpg: Total number processed: 1
gpg:               imported: 1

4)

sudo tee "/etc/apt/sources.list.d/kurento.list" >/dev/null <<EOF

deb [arch=amd64] http://ubuntu.openvidu.io/dev $DISTRIB_CODENAME main
EOF

5)
sudo apt-get update
sudo apt-get install --no-install-recommends --yes kurento-media-server-dev

git clone https://github.com/Kurento/kurento.git
cd kurento/
git submodule update --init --recursive

cd server/
export MAKEFLAGS="-j$(nproc)"
bin/build-run.sh --build-only

After successful build KMS is started (check with "sudo service kurento-media-server status")

Check versions:

kurento-media-server --version
Kurento Media Server version: 7.0.0-dev
Found modules:
   'core' version 7.0.0-dev
   'elements' version 7.0.0-dev
   'filters' version 7.0.0-dev

That's it. 

Thanks.

[Juan Navarro]

Glad to hear it worked!

Maven should not need to be installed explicitly, as it is already a dependency of Kurento. If it wasn't installed in your system that's a very weird state that the package manager was in, because the dependencies go like this:

kurento-media-server-dev -> kurento-module-creator -> maven

you can check them yourself with:

$ apt-cache depends kurento-media-server-dev
kurento-media-server-dev
  Depends: kurento-module-creator

$ apt-cache depends kurento-module-creator
kurento-module-creator
  Depends: maven

So I'm curious about how you managed to apt-get install kurento-media-server-dev but not get maven installed.

In any case it's good you made it work.

I'd say that Docker is a _very_ good tool to create environments where you have 100% certainty of it being a clean slate, in a 1) very easy and fast way, and which are 2) ephemeral and disposable. Those two things proved to be extremely valuable for development, and were worth the learning curve. For me anyways :)

Regards

To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/3ae8c1f2-d862-4ed3-9487-d8631f135411n%40googlegroups.com.

[Neil Young]

> So I'm curious about how you managed to apt-get install kurento-media-server-dev but not get maven installed.

I would like to know that too, but it is too late. The VM is dead. It maybe have been "infected" by a previous installation I tried some days ago (the one I reported earlier in this thread)

> I'd say that Docker is a _very_ good tool to create environments where you have 100% certainty of it being a clean slate, in a 1) very easy and fast way, and which are 2) ephemeral and disposable. Those two things proved to be extremely valuable for development, and were worth the learning curve. For me anyways :)

I have no doubts. I don't like it. I'm more for the straight approach. And this one I have now is fine. When I need a clean state I fire up a VM, well mostly at least :)

[Neil Young]

Good, now I'm having a running KMS instance on that box.

But I still seem not to be fully setup as I was with previous versions. Especially w.r.t. compiling my own plugins on that box.

In the past I was used to find some CMake "plugins" (?) like so for instance (here I was patching an opencv4 dependency into the file in question):

sed -i 's/set (REQUIRED_LIBS "opencv")/set (REQUIRED_LIBS "opencv4")/' /usr/share/"`cmake --version | awk '{split($0,a," "); split($a[3],a,"."); print "cmake-"a[1]"."a[2];exit}'`"/Modules/FindKMSFILTERS.cmake

whereas the path resolves to:

/usr/share/cmake-3.16/Modules/FindKMSFILTERS.cmake

Now - there is no such "Modules" directory anymore. In fact, all the little helpers, like this one or "FindKurentoModules.cmake" are not there. I suppose (not tried yet) I will not be able to compile my modules anymore

What am I missing?

[Neil Young]

To be more specific: Maybe I'm not forced to patch any opencv4 dependency anymore, but during the cmake build process I was requiring these modules like so:

cmake \

-DCMAKE_MODULE_PATH=/usr/local/share/"`cmake --version | awk '{split($0,a," "); split($a[3],a,"."); print "cmake-"a[1]"."a[2];exit}'`"/Modules \

-DCMAKE_BUILD_TYPE=Release \

-DCMAKE_INSTALL_PREFIX=/usr \

..

Did that change now?

So the general question is: What do I need to do in order to compile a 6.x custom module for KMS7?

[Neil Young]

Found this https://doc-kurento.readthedocs.io/en/latest/user/writing_modules.html. Let's see if it gives some answers.

[Neil Young]

Ok, first: I'm sorry in case I'm too verbose for the community here, but I hope this is will be balanced by the fact, that I'm not only asking, but also contributing... In fact, these are probably "special" problems and it would be better to discuss it with the Kurento team directly. But I don't have this direct wire, so I have to be fine with the help Juan gives here (and in fact, I am).

Situation is: I have a couple of "old" custom OpenCV modules (2018/19), running perfectly under 6.x versions of KMS. Now 7 is "ante portas" and it seems, that my build scripts are no longer fully working.

So I followed the quoted documentation, created a dummy opencv plugin on a KMS 7 instance and compared the output with my old modules.

Right now I'm able to build my modules under KMS 7, but I'm currently not setup to test it (next step). 

What I was forced to do was this:

- I was forced to align my old CMakeLists.txt to the new CMakeLists.txt generated in order to get rid of all the old dependencies (KurentoHelpers, GST 1.5 etc). This was pretty easy, because you can still recognize some old structures.

- I was forced to change `/usr/share/cmake/Kurento/VersionHelpers.cmake` since it failed with 

CMake Error at /usr/share/cmake/Kurento/VersionHelpers.cmake:301 (message):

  Version does not match <7.0.0 with 7.0.0-dev

- This is for sure caused by the fact, that I still have to have a 7.0.0-dev environment, while release is expected here. Release is not released yet :). I commented the check starting at around line 301 in the mentioned file
- After that I was able to `cmake ..` and to compile my module

- Usually my installation are finishing with `sudo make install` which in turn copies my binaries to `/usr/local/lib` (amongst some meta-data elsewhere). 

Install the project...
-- Install configuration: ""
-- Installing: /usr/local/lib/libkmswebrtccbrinterface.a
-- Installing: /usr/local/include/kurento/modules/webrtccbr/webrtccbr.hpp
-- Installing: /usr/local/include/kurento/modules/webrtccbr/onResult.hpp
-- Installing: /usr/local/lib/pkgconfig/kmswebrtccbr.pc
-- Installing: /usr/local/lib/libkmswebrtccbrimpl.so.0.1.6~7.gb9ddf5e
-- Installing: /usr/local/lib/libkmswebrtccbrimpl.so.0
-- Installing: /usr/local/lib/libkmswebrtccbrimpl.so
-- Installing: /usr/local/include/kurento/modules/webrtccbr/webrtccbrImpl.hpp
-- Installing: /usr/local/include/kurento/modules/webrtccbr/webrtccbrOpenCVImpl.hpp
-- Installing: /usr/local/include/kurento/modules/webrtccbr/webrtccbrImplFactory.hpp
-- Installing: /usr/local/lib/kurento/modules/libkmswebrtccbrmodule.so
-- Set runtime path of "/usr/local/lib/kurento/modules/libkmswebrtccbrmodule.so" to ""
-- Installing: /usr/local/share/cmake/Kurento/FindKMSWEBRTCCBR.cmake
-- Installing: /usr/local/share/kurento/modules/webrtccbr.kmd.json

This doesn't work anymore. Following the documentation quoted above it is now not even "allowed" anymore to have ".so" installed. Instead there is a focus to ".deb" installations. I don't have means for that, so I tried the suggestion to place the path to the module into the file `/etc/default/kurento`. But whatever I tried, the setting there was not recognized at start of the KMS service.

Only this worked at command line:

export KURENTO_MODULES_PATH="/home/ubuntu/webrtc-calibration/build/src/server"

I'm not too happy with it, also because all the same files are available at /usr/local/lib already... If someone could clarify, what the proper location would be for a pure binary installation.

Finally I got my module loaded:

Kurento Media Server version: 7.0.0-dev
Found modules:
'core' version 7.0.0-dev
'elements' version 7.0.0-dev
'filters' version 7.0.0-dev

'webrtccbr' version 0.1.6~7.gb9ddf5e

So questions are:

What is the proper location of the plain module *.so" files so that those get loaded as before?

Sorry for the long post. I hope it is of help for somebody else having the same issues.

[Neil Young]

OK, end of lane for today: Deb build is not working at all. Not even with the sample project:

// Install dpkg-dev

ubuntu@kms7:~/opencv-module$ sudo apt-get update ; sudo apt-get install --no-install-recommends     dpkg-dev

Hit:1 http://de.archive.ubuntu.com/ubuntu focal InRelease
Get:2 http://de.archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB]

Hit:3 http://ubuntu.openvidu.io/dev focal InRelease    
Get:4 http://de.archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB]
Get:5 http://de.archive.ubuntu.com/ubuntu focal-security InRelease [114 kB]
Get:6 http://de.archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2420 kB]
Get:7 http://de.archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1037 kB]
Fetched 3793 kB in 11s (342 kB/s)                        

Reading package lists... Done
Reading package lists... Done
Building dependency tree      
Reading state information... Done

dpkg-dev is already the newest version (1.19.7ubuntu3.2).
0 upgraded, 0 newly installed, 0 to remove and 26 not upgraded.

// Run deb build

ubuntu@kms7:~/opencv-module$ dpkg-buildpackage -us -uc
dpkg-buildpackage: info: source package opencv-module
dpkg-buildpackage: info: source version 0.0.1~rc1
dpkg-buildpackage: info: source distribution testing
dpkg-buildpackage: info: source changed by Maintainer Name <maintai...@example.com>
dpkg-buildpackage: info: host architecture amd64
dpkg-buildpackage: warning: debian/rules is not executable; fixing that
 dpkg-source --before-build .
dpkg-checkbuilddeps: error: Unmet build dependencies: debhelper-compat (= 13)
dpkg-buildpackage: warning: build dependencies/conflicts unsatisfied; aborting
dpkg-buildpackage: warning: (Use -d flag to override.)

Nose full for today. Have a nice weekend

[Neil Young]

Found it finally. The file to manipulate is not `/etc/default/kurento` (this might work in another world, not in mine), but `/etc/default/kurento-media-server`.

There is a commented entry in that file

## Path to the directory where Kurento modules can be found

#export KURENTO_MODULES_PATH="/usr/lib"

Uncomment and change to `/usr/local/lib` and done.

[Tho Nguyen]

Cảm ơn các bạn của tôi rất tuyệt vời . Chúc ngày nghỉ cuối tuần an nhiên vui vẻ . Sang tuần chúng ta làm việc nhé 🥰🥰🥰🥰🥰

Vào Th 7, 11 thg 3, 2023 lúc 21:40 'Neil Young' via kurento <kur...@googlegroups.com> đã viết:

--

Nguyen tho

[Juan Navarro]

Hi Neil,

were you able to make your module work as desired?

There is no desire to push users to install custom modules with .deb packages if just deploying .so files would be enough, so if docs are lacking in that regard please let me know so maybe I could add a section touching this (or maybe restore it if it got deleted with all the movements that happened for the release of Kurento 7).

Now that almost everything has been properly packaged and released it should be easier to make things work; please note that the release of the media server is happening under a branch named "release-7.0.0" so every fix related to it will exist on that branch. The apt-get and Docker tag names for this are "dev-release-7.0.0". Next Tuesday this will all be merged into the main branch and final releases built.

The CMake modules were always manually installed in version-specific subdirs so that different versions of CMake would have different copies of the same files; this was not maintenable and actually caused a lot of problems, for example if the Kurento packages came with .cmake files installed for CMake 3.20 but you had installed CMake 3.21 from wherever (like, manually installed from the official website, or maybe from a 3rd-party Ubuntu PPA repository), then you wouldn't be able to use _that_ version of CMake with Kurento because the system wouldn't have Kurento's .cmake files in a directory specific to the 3.21 version of CMake.

The solution to this problem came from an external contribution which used the ability of CMake to look in generic directories for files named like Find<ModuleName>.cmake, and this file itself would contain the actual path to the .cmake files; this way it is possible to have a single place where .cmake files are located, regardless of the version of CMake in use. This also allows that people working in other systems with other unexpected versions of CMake will be able to build Kurento without having to deal with issues with the CMake modules search path.

You should now find all .cmake files at /usr/share/cmake/Kurento/


Given all changes that have happened, I think a sensible strategy to update custom plugins is to create a new one with the latest kurento-module-creator, and adapt the code to the new scripts.

Regards

To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/39dca5e0-c85d-4477-beb3-4be2909d083en%40googlegroups.com.

[Neil Young]

> were you able to make your module work as desired?

Yes, everything fine.

>There is no desire to push users to install custom modules with .deb packages if just deploying .so files would be enough, so if docs are lacking in that regard please let me know so maybe I could add a section touching this (or maybe restore it if it got deleted with all the movements that happened for the release of Kurento 7).

Sure. I learned that. I was confused from a previous installation. I always install my modules to /usr/local/lib and there they are found by the service. Just the command line module needed a bit help with "export KURENTO_MODULES_PATH=/usr/local/lib" and then "kurento-media-server --version". If I don't have that export, my modules will not be loaded (just for testing the proper installation)

> CMake...etc.

Thanks for the explanation. I just had to remove all my "CMake" path magic from the module builds and fine. 

> Given all changes that have happened, I think a sensible strategy to update custom plugins is to create a new one with the latest kurento-module-creator, and adapt the code to the new scripts.

That was my initial approach to find the diffs. In fact, it is just the root CMakeList.txt which needs to be updated.  Very easy. Everything else can be kept. Maybe here and there an include for OpenCV missing, but there is always a quick replacement somehwere.

The "main work" was the adaptation of the root CMakeList.txt. But this is that generic, that you can copy and paste one for all.

[Neil Young]

I have now tried to install the release version of KMS 7, to no avail, unfortunately. I this time did NOT attempt a source installation, also because it is nowhere documented, how.

1) I first uninstalled all old versions from this machine as indicated

sudo aptitude remove '?installed?version(kurento)'

2) I checked sudo service kurento-media-server status and found it stopped, but the entire service environment was still there. Hmm. Full uninstall?

3) I then changed to root dir and "find . -name "*kurento*" 2>/dev/null"

This came up:

./etc/rc1.d/K01kurento-media-server
./etc/kurento
./etc/kurento/modules/kurento
./etc/kurento/kurento.conf.json
./etc/rc2.d/S01kurento-media-server
./etc/default/kurento-media-server
./etc/init.d/kurento-media-server
./etc/rc6.d/K01kurento-media-server
./etc/rc0.d/K01kurento-media-server
./etc/rc5.d/S01kurento-media-server
./etc/rc4.d/S01kurento-media-server
./etc/rc3.d/S01kurento-media-server
./etc/apt/sources.list.d/kurento.list
./sys/kernel/slab/radix_tree_node/cgroup/radix_tree_node(859:kurento-media-server.service)
./sys/kernel/slab/dentry/cgroup/dentry(859:kurento-media-server.service)
./sys/kernel/slab/kmalloc-rcl-64/cgroup/kmalloc-rcl-64(859:kurento-media-server.service)
./sys/kernel/slab/:a-0000104/cgroup/buffer_head(859:kurento-media-server.service)
./sys/kernel/slab/ext4_inode_cache/cgroup/ext4_inode_cache(859:kurento-media-server.service)
./var/lib/kurento
./var/lib/dpkg/info/kurento-media-server.postrm
./var/lib/dpkg/info/kurento-module-core.list
./var/lib/dpkg/info/kurento-media-server.list
./var/lib/dpkg/info/kurento-module-elements.list
./var/log/kurento-media-server
./var/cache/apt/archives/libnice10_0.1.18-0kurento1.20.04~20230221140701.git50db4fd_amd64.deb
./var/cache/apt/archives/gir1.2-nice-0.1_0.1.18-0kurento1.20.04~20230221140701.git50db4fd_amd64.deb
./var/cache/apt/archives/kurento-jsonrpc_7.0.0-1kurento1.20.04~20230221140913_amd64.deb
./var/cache/apt/archives/libsrtp-kurento_1.0.0-0kurento1.20.04~20230221140701.git631e84c_amd64.deb
./var/cache/apt/archives/kurento-media-server_7.0.0-1kurento1.20.04~20230221143000_amd64.deb
./var/cache/apt/archives/gstreamer1.0-nice_0.1.18-0kurento1.20.04~20230221140701.git50db4fd_amd64.deb
./var/cache/apt/archives/gstreamer1.0-plugins-good-kurento_1.0.0-0kurento1.20.04~20230221140700.git22480e9_amd64.deb
./var/cache/apt/archives/openh264_2.3.0-0kurento1.20.04~20230221140703.git64aa79a_amd64.deb
./var/cache/apt/archives/libgstreamer-plugins-good1.0-dev_1%3a1.16.3-0ubuntu1.1kurento1_amd64.deb
./var/cache/apt/archives/kurento-jsonrpc-dev_7.0.0-1kurento1.20.04~20230221140913_amd64.deb
./var/cache/apt/archives/kurento-module-creator_7.0.0-1kurento1.20.04~20230221140729_all.deb
./var/cache/apt/archives/kurento-module-core-dev_7.0.0-1kurento1.20.04~20230221141121_amd64.deb
./var/cache/apt/archives/kurento-module-elements_7.0.0-1kurento1.20.04~20230221141638_amd64.deb
./var/cache/apt/archives/libsrtp2-1_1%3a2.3.0-2kurento1_amd64.deb
./var/cache/apt/archives/kurento-cmake-utils_7.0.0-1kurento1.20.04~20230221140659_all.deb
./var/cache/apt/archives/kurento-media-server-dev_7.0.0-1kurento1.20.04~20230221143000_amd64.deb
./var/cache/apt/archives/libgstreamer-plugins-good1.0-0_1%3a1.16.3-0ubuntu1.1kurento1_amd64.deb
./var/cache/apt/archives/openh264-gst-plugin_1.0.0-0kurento1.20.04~20230221140746.gitc66af05_amd64.deb
./var/cache/apt/archives/kurento-module-filters_7.0.0-1kurento1.20.04~20230221142154_amd64.deb
./var/cache/apt/archives/kurento-module-filters-dev_7.0.0-1kurento1.20.04~20230221142154_amd64.deb
./var/cache/apt/archives/libnice-dev_0.1.18-0kurento1.20.04~20230221140701.git50db4fd_amd64.deb
./var/cache/apt/archives/kurento-module-elements-dev_7.0.0-1kurento1.20.04~20230221141638_amd64.deb
./var/cache/apt/archives/gstreamer1.0-plugins-good_1%3a1.16.3-0ubuntu1.1kurento1_amd64.deb
./var/cache/apt/archives/kurento-module-core_7.0.0-1kurento1.20.04~20230221141121_amd64.deb
./run/systemd/generator.late/kurento-media-server.service
./run/systemd/generator.late/graphical.target.wants/kurento-media-server.service
./run/systemd/generator.late/multi-user.target.wants/kurento-media-server.service
./usr/share/kurento
./usr/lib/x86_64-linux-gnu/kurento
./usr/include/kurento

I removed every single item.

4) I followed the steps under "Local installation" https://doc-kurento.readthedocs.io/en/latest/user/installation.html#local-installation

4a) I again found this statement not working:

sudo apt-key adv \
     --keyserver keyserver.ubuntu.com \
     --recv-keys 234821A61B67740F89BFD669FC8A16625AFA7A83

Executing: /tmp/apt-key-gpghome.M53zisD60O/gpg.1.sh --keyserver keyserver.ubuntu.com --recv-keys 234821A61B67740F89BFD669FC8A16625AFA7A83

gpg: keyserver receive failed: Server indicated a failure

... but this:

sudo apt-key adv \
    --keyserver hkp://keyserver.ubuntu.com:80 \
    --recv-keys 234821A61B67740F89BFD669FC8A16625AFA7A83
   

I believe the latter is the statement which will most likely work always.

4b)

sudo tee "/etc/apt/sources.list.d/kurento.list" >/dev/null <<EOF

# Kurento Media Server - Release packages
deb [arch=amd64] http://ubuntu.openvidu.io/7.0.0 $DISTRIB_CODENAME main
EOF

4c)

sudo apt-get update ; sudo apt-get install --no-install-recommends \
    kurento-media-server
   

ended like so:

ubuntu@kms7:~$ sudo apt-get update ; sudo apt-get install --no-install-recommends \
>     kurento-media-server
Hit:1 https://deb.nodesource.com/node_14.x focal InRelease
Hit:2 http://de.archive.ubuntu.com/ubuntu focal InRelease                                        
Err:3 http://ubuntu.openvidu.io/7.0.0 focal InRelease                                            
  403  Forbidden [IP: 52.218.30.35 80]
Hit:4 http://de.archive.ubuntu.com/ubuntu focal-updates InRelease          
Hit:5 http://de.archive.ubuntu.com/ubuntu focal-backports InRelease        
Hit:6 http://de.archive.ubuntu.com/ubuntu focal-security InRelease        
Get:7 https://pkgs.tailscale.com/stable/ubuntu focal InRelease
Reading package lists... Done
E: Failed to fetch http://ubuntu.openvidu.io/7.0.0/dists/focal/InRelease  403  Forbidden [IP: 52.218.30.35 80]
E: The repository 'http://ubuntu.openvidu.io/7.0.0 focal InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.

Reading package lists... Done
Building dependency tree      
Reading state information... Done

Package kurento-media-server is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'kurento-media-server' has no installation candidate


Hmm. End of lane for today I guess.

[Neil Young]

Double checked from a clean VM

[Neil Young]

O sh...t. I recall it was supposed to appear no Tuesday.Today is Monday. 

My bad obviously

[Juan Navarro]

Note that the release process is being held by service breakdown on the Ubuntu ddebs repositories, which are currently returning error 503 in some operations (not easily reproducible by hand, but happening on the CI server), there is a bug report opened regarding this issue with the package repos:

https://bugs.launchpad.net/ddeb-retriever/+bug/1976233

As soon as the release finishes I'll try to make changes to avoid needing the ddebs repo, which seems to be too fragile to depend on.


Re: the point 3), you've found the difference between "remove" and "purge", only the latter cleans configuration files up from /etc. "remove" leaves them there in case you want to install the same software later.

This has been taken into account and the cleanup section in the docs now point to this subtle difference too:

https://doc-kurento.readthedocs.io/en/stable/dev/dev_guide.html#clean-up-your-system

I.e. from the state you showed, if you run:

sudo aptitude purge '?config-files'

then you'll end up with a clean system. Otherwise, just run this in one single step:

sudo aptitude purge '?installed?version(kurento)'


Regards

To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/9abc6456-1693-4c5e-b4b2-b09c8a4336b9n%40googlegroups.com.

[Neil Young]

> Re: the point 3), you've found the difference between "remove" and "purge", only the latter cleans configuration files up from /etc. "remove" leaves them there in case you want to install the same software later.

Ehem. No. I was aware of the difference. I was just following the instructions.

>As soon as the release finishes I'll try to make changes to avoid needing the ddebs repo, which seems to be too fragile to depend on.

Could you ping here, once it works?

[Juan Navarro]

It finally worked and the version "7.0.0" is now published for both apt-get packages and Docker images. Will be posting an announcement in the next hours.

To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/d8cad7bd-806a-4ce6-a1ef-a82109d27395n%40googlegroups.com.