Packaging janus-gateway for Fedora

[Renich Bon Ćirić]

Buona sera!

As stated in the subject, I am. The effort is being reviewed here:

https://bugzilla.redhat.com/show_bug.cgi?id=2121585

I have a few questions that the reviewer threw at me and I wanted to
ask you, guys about. The questions are at the end of comment #6:
https://bugzilla.redhat.com/show_bug.cgi?id=2121585#c6

In short:


c) May want to check codecs used are freely licensed
d) Can one use the libraries independently of the application?

I answered in comment #7:
https://bugzilla.redhat.com/show_bug.cgi?id=2121585#c7

Briefly, I answered:

> c) It's WebRTC; they don't do transcoding. Some plugin might, though.
I presume they use open standards. No non-open/non-free licenses.

Is that true?

> d) I don't think they offer re-usable libraries. All .so are just
components (transports, event handlers, etc).

Is this also true?


OK, in any case, I have some other questions:

- Sub packages

In Fedora, we like to keep packages modular. What I mean is, we like to
sub-package plugins and stuff so that users can choose which
functionality they want. Is this possible? Can I partition the package
in multiple sub-packages so that the user can pick which ones to
install? For example, put the event handlers in their own sub-package?

Is there any of these that is required by the main app?


- unversioned .so files

I've been questioned about the unversioned .so files. Fedora requires
me to ask you to version them. A broader explanation can be found here:
https://docs.fedoraproject.org/en-US/packaging-guidelines/#_devel_packages

The question being: can you start versioning those?


OK, that's all for now. I thank you for lending your eyes and time to
reply to this email. Thank you for such great software too. :D


--
Renich Bon Ćirić <ren...@woralelandia.com>

[Lorenzo Miniero]

Il giorno martedì 20 settembre 2022 alle 21:49:23 UTC+2 ren...@woralelandia.com ha scritto:

Buona sera!

As stated in the subject, I am. The effort is being reviewed here:

https://bugzilla.redhat.com/show_bug.cgi?id=2121585

Ciao! That's great news, as I've been a Fedora user since Fedora 4, so seeing Janus in there when opening dnfdragora would be amazing :-)

 

I have a few questions that the reviewer threw at me and I wanted to
ask you, guys about. The questions are at the end of comment #6:
https://bugzilla.redhat.com/show_bug.cgi?id=2121585#c6

In short:


c) May want to check codecs used are freely licensed

d) Can one use the libraries independently of the application?

I answered in comment #7:
https://bugzilla.redhat.com/show_bug.cgi?id=2121585#c7

Briefly, I answered:

> c) It's WebRTC; they don't do transcoding. Some plugin might, though.
I presume they use open standards. No non-open/non-free licenses.

Is that true?

You're right, Janus itself doesn't transcode, so it doesn't use any codec itself. The only plugin that performs transcoding is the AudioBridge, which is an audio mixer and so needs to decode and encode audio packets to do its job. That plugin only supports G.711 (a-law, mu-law) and Opus, so nothing proprietary or problematic license-wise. All the other plugins only relay packets or save them locally (recordings).

The recordings post-processor may look a bit more ambiguous, but that one also never touches the media themselves, since it uses FFmpeg as a library to just save the content of RTP packets in containerized media formats (mkv, mp4, etc.) without transcoding. As such, even when problematic codecs like H.264 or H.265 are processed, we're just copying the data as it is from RTP packets to, e.g., mp4.

 

> d) I don't think they offer re-usable libraries. All .so are just
components (transports, event handlers, etc).

Is this also true?

Yes, all our .so only make sense as Janus modules, in the sense that they extend the Janus functionality but cannot be used on their own, or as libraries of a different application (the header signatures are specific to Janus). That said, Janus and modules are also independent of each other, meaning you can have one Janus instance load some but not all and still do its job: some may only want HTTP support, others only WebSockets support, and some will only do SIP while others will want the SFU for videoconferencing. As such, it can be seen as a generic modular application.

OK, in any case, I have some other questions:

- Sub packages

In Fedora, we like to keep packages modular. What I mean is, we like to
sub-package plugins and stuff so that users can choose which
functionality they want. Is this possible? Can I partition the package
in multiple sub-packages so that the user can pick which ones to
install? For example, put the event handlers in their own sub-package?

Is there any of these that is required by the main app?

You definitely can, and I was indeed wondering if that was an option (as it's used for Asterisk and OpenSIPS already, looking at the list, which share a similar architecture). The Janus core can start even with no module loaded at all. It will exit right away since it needs at least one transport loaded, for instance, but it doesn't care which one, so all modules are by definition optional. I think it's indeed a good idea to have different sub-packages for media plugins, transport plugins, event handler plugins and logger plugins, where then each module can also be installed separately if needed (since you may not be interested in all event handlers but just one).

Janus is already packaged in Debian and openSUSE, so having a look at how they chose to package things may also help:

https://janus.conf.meetecho.com/docs/resources.html#repos

Debian, for instance, apparently has a janus package that contains core and plugins, but then different packages for the generated Doxygen documentation, demos, header files, and tools (postprocessing ones, I assume). I guess those may also be different packages in Fedora.

 

- unversioned .so files

I've been questioned about the unversioned .so files. Fedora requires
me to ask you to version them. A broader explanation can be found here:
https://docs.fedoraproject.org/en-US/packaging-guidelines/#_devel_packages

The question being: can you start versioning those?

That's a good point. We do version them in the code itself, but I think our build process is still generating code for them as if they weren't. I guess the easiest approach is to use the same versioning of Janus itself for all modules as well: this may result in generating new versions of a plugin even when the plugin itself hasn't changed, but I'm not sure that would be that big of a deal (unless you think it is). I'll have a look at how that works from a build perspective and let you know.

 

OK, that's all for now. I thank you for lending your eyes and time to
reply to this email. Thank you for such great software too. :D

Thanks for taking care of this, it's really appreciated!

Lorenzo

[Renich Bon Ćirić]

On Wed, 2022-09-21 at 02:52 -0700, Lorenzo Miniero wrote:
> Il giorno martedì 20 settembre 2022 alle 21:49:23 UTC+2  
> >

> > > d) I don't think they offer re-usable libraries. All .so are just
> > components (transports, event handlers, etc).
> >
> > Is this also true?
> >
>
>
> Yes, all our .so only make sense as Janus modules, in the sense that
> they
> extend the Janus functionality but cannot be used on their own, or as
> libraries of a different application (the header signatures are
> specific to
> Janus). That said, Janus and modules are also independent of each
> other,
> meaning you can have one Janus instance load some but not all and
> still do
> its job: some may only want HTTP support, others only WebSockets
> support,
> and some will only do SIP while others will want the SFU for
> videoconferencing. As such, it can be seen as a generic modular
> application.

OK, good to know. Thank you for confirming the prior points as well.

> >
> > OK, in any case, I have some other questions:
> >
> > - Sub packages

Thank you for these references. I'll check them out definitely and do
my best to package it in a modular way.

> >
> > - unversioned .so files
> >
> > I've been questioned about the unversioned .so files. Fedora
> > requires
> > me to ask you to version them. A broader explanation can be found
> > here:
> > https://docs.fedoraproject.org/en-US/packaging-guidelines/#_devel_packages
> >  
> >
> > The question being: can you start versioning those?
> >
>
>
> That's a good point. We do version them in the code itself, but I
> think our
> build process is still generating code for them as if they weren't. I
> guess
> the easiest approach is to use the same versioning of Janus itself
> for all
> modules as well: this may result in generating new versions of a
> plugin
> even when the plugin itself hasn't changed, but I'm not sure that
> would be
> that big of a deal (unless you think it is). I'll have a look at how
> that
> works from a build perspective and let you know.

It's OK to have the plugins versioned as per janus is. The only thing
is that it would be kind of difficult for you to identify when a
plugin/component actually changed. It's more up to you in that regard. 

> > OK, that's all for now. I thank you for lending your eyes and time
> > to
> > reply to this email. Thank you for such great software too. :D
> >
> >
>
> Thanks for taking care of this, it's really appreciated!
> Lorenzo

I am happy to help. The feeling is mutual. :)

[Renich Bon Ćirić]

¡Ciao, Lorenzo!

OK, so, here's the file list. I know you must be very busy but, if you
have time, let me know if it seems OK to you:

https://paste.centos.org/view/e4fe9539

The list will be deleted in less than 24 hours. So, if you need more
time, just download it.

Now, it would be good if you could tell me what depends on what.

For example, janus-gateway-demos should depend on everything but the
devel packages, right?

janus-gateway doesn't depend on anything but every component will
require it. Correct?

In any case, thank you for your assistance and patience. :D

In the meantime, I will be checking if there are any runtime
dependencies that I missed.

[Lorenzo Miniero]

Hi Renich,

first of all, if possible I'd choose a different name for the package (and as a consequence paths). While the repo is still called "janus-gateway" because that's how it was called historically, and renaming it could cause confusion, we don't call it a gateway anymore, but a WebRTC server. So maybe something like janus-webrtc-server or more simply janus-webrtc would be a better choice? I guess only calling it janus (as Debian does) may be risky, since it's a common name for software projects of different kinds, but would also be easier to find.

On the contents, from a first glance:

• I think that the janus.js file can be removed from the main package, since it's not an actual SDK (Janode is closer to being that), and it's also available in the demos package anyway, where it indeed belongs.
  
• The "/usr/share/janus/recordings/" and "/usr/share/janus/streams/" samples, instead, should probably be moved away from the demos, and to the respective plugins packages instead (recordings to recordandplay, streams to streaming), since those files are actually used only by those plugins as part of the default (sample) configuration files. More precisely, the sample recording is the only recording that's available right away when the Record&Play plugin is loaded, while the streams are static samples that are made available as static mountpoints in the Streaming plugin. This is similar to how the Duktape and Lua plugins have their own samples (which you already correctly bundled with the respective plugins). As a side note, make also sure that the paths you chose for all of them them do match the paths listed in the respective .jcfg files, or they won't be found at startup.
• For the same reason, the "test_gstreamer*" files all belong to the Streaming plugin package, and not the -devel one. I guess the -devel package should only include the headers that can be used to builld plugins externally.
  
• pcap2mjr seems to be missing in the tools? That one depends on libpcap(-devel), so in case it wasn't installed, it may not have been built for you which is why you may have missed it.
• The docs paths seem incorrect: I see a "/usr/share/doc/janus-gateway/janus-gateway-1.0.4" folder is created, but is then unused, since all docs are added to the parent "/usr/share/doc/janus-gateway" folder instead, meaning that updates would replace the files rather than putting them in the folder with the new version path. I also see the configuration samples added to the docs package too, but they're already part of the respective packages (core, plugins, etc.), wouldn't that cause a conflict when installing?
  
• Not sure if it makes sense, but do you think macro packages like janus-gateway-plugins to automatically install all media plugins (and same for the other groups) may be useful? IIRC it's the approach used for GStreamer packages to group-install some of its plugins, although I may be wrong. At any rate, a dnf install on janus-gateway-plugins-* would get the same result, so it's probably not needed.
  

I hope this helps, and thanks for the great work!

Lorenzo

[Lorenzo Miniero]

Apologies, forgot to answer on the dependencies.

 

Now, it would be good if you could tell me what depends on what.

For example, janus-gateway-demos should depend on everything but the
devel packages, right?

Not sure what you mean by that? That installing the demos would include all packages and all plugins except for -devel? If so, I think that might be overkill: the demos can depend only on the main package as all others do, I guess: trying to run a demo when a plugin is not installed will return an error in the web page, and that's expected.

 

janus-gateway doesn't depend on anything but every component will
require it. Correct?

Yep, that's correct. Looking at the packages you shared, I guess the dependency tree could actually be quite simple, since all packages are independent of each other, meaning that all packages only depend on the main one and nothing else (unless macro packages for the different categories are added, as imagined in the previous response).

As a side note, do you need info on the library dependencies for each package too, or have you that figured out already?

L.

[Lorenzo Miniero]

As a side note, I've prepared a PR to add versioning to .so files as you asked:

https://github.com/meetecho/janus-gateway/pull/3075

This would start from 1.1.0, which will be the next tagged version, after the 1.0.4 you're working with right now (which will still have the .0.0.0). Please let me know if the way it's implemented actually works the way you need it to, or if I missed something.

Lorenzo

[Renich Bon Ćirić]

On Thu, 2022-09-22 at 03:18 -0700, Lorenzo Miniero wrote:
> Hi Renich,
>
> first of all, if possible I'd choose a different name for the package
> (and
> as a consequence paths). While the repo is still called "janus-
> gateway"
> because that's how it was called historically, and renaming it could
> cause
> confusion, we don't call it a gateway anymore, but a WebRTC server.
> So
> maybe something like janus-webrtc-server or more simply janus-webrtc
> would
> be a better choice? I guess only calling it janus (as Debian does)
> may be
> risky, since it's a common name for software projects of different
> kinds,
> but would also be easier to find.

No problem by me. I've never changed the name of a package in the
middle of a review, though. Let me run this by the reviewer. The worst
case scenario would be to rename it afterwards, which is possible.

janus seems like a good name. Besides, debian already did it. I think
we can too.

> On the contents, from a first glance:
>

>    - I think that the janus.js file can be removed from the main

> package,
>    since it's not an actual SDK (Janode is closer to being that), and
> it's
>    also available in the demos package anyway, where it indeed
> belongs.

Done. No more janus.js.

>    - The "/usr/share/janus/recordings/" and

> "/usr/share/janus/streams/"
>    samples, instead, should probably be moved away from the demos,
> and to the
>    respective plugins packages instead (recordings to recordandplay,
> streams
>    to streaming), since those files are actually used only by those
> plugins as
>    part of the default (sample) configuration files. More precisely,
> the
>    sample recording is the only recording that's available right away
> when the
>    Record&Play plugin is loaded, while the streams are static samples
> that are
>    made available as static mountpoints in the Streaming plugin. This
> is
>    similar to how the Duktape and Lua plugins have their own samples
> (which
>    you already correctly bundled with the respective plugins). As a
> side note,
>    make also sure that the paths you chose for all of them them do
> match the
>    paths listed in the respective .jcfg files, or they won't be found
> at
>    startup.

This is done as well. I will have to check the configuration files,
though. i'll make sure they're correct.

>    - For the same reason, the "test_gstreamer*" files all belong to

> the
>    Streaming plugin package, and not the -devel one. I guess the -
> devel
>    package should only include the headers that can be used to builld
> plugins
>    externally.

OK. Done as well.

>    - pcap2mjr seems to be missing in the tools? That one depends on

>    libpcap(-devel), so in case it wasn't installed, it may not have
> been built
>    for you which is why you may have missed it.

I've added libpcap-devel to the requirements and pcap2mjr was generated
and put in place. Thank you! :D

>    - The docs paths seem incorrect: I see a

>    "/usr/share/doc/janus-gateway/janus-gateway-1.0.4" folder is
> created, but
>    is then unused, since all docs are added to the parent
>    "/usr/share/doc/janus-gateway" folder instead, meaning that
> updates would
>    replace the files rather than putting them in the folder with the
> new
>    version path. I also see the configuration samples added to the
> docs
>    package too, but they're already part of the respective packages
> (core,
>    plugins, etc.), wouldn't that cause a conflict when installing?

Yeah, good catch. Removed the duplicate files and directory. Fedora,
doesn't version documentation. It gets replaced with every new version
and doesn't leave lingering files, AFAIK.

Is this a problem? Would you prefer it to be located in
"/usr/share/doc/janus/1.0.4/" instead?

In any case, I ran into a few issues here. First, even if I pass 
"--docdir=/usr/share/doc/janus" to "configure", it, still, creates:
"/usr/share/doc/janus/janus-gateway-1.0.4".

It's not a big deal but, given what you told me about the old name,
maybe it would be good to "softcode" the name and/or eliminate it
completely even. For example, just generate
"/usr/share/doc/janus/1.0.4/" or something similar.

>    - Not sure if it makes sense, but do you think macro packages like

>    janus-gateway-plugins to automatically install all media plugins
> (and same
>    for the other groups) may be useful? IIRC it's the approach used
> for
>    GStreamer packages to group-install some of its plugins, although
> I may be
>    wrong. At any rate, a dnf install on janus-gateway-plugins-* would
> get the
>    same result, so it's probably not needed.

Oh, it does! I was leaving that one for last. 

And, for the sake of clarity, janus-plugins would install janus-
plugins-*, but not janus-eventhandlers-* or janus-loggers-*. The reason
for this would be that you will rarely need support for all transports
or eventhandlers at the same time. If you do, you're a special case and
should just install them all manually. 

Do we agree on this?

> I hope this helps, and thanks for the great work!
> Lorenzo

You have been of great help indeed, my friend. :D 

Thank you for all this.

I will make the relevant proposals to the reviewer and get back to you
afterwards.


A few useful links to check:

* file list of new packages: https://termbin.com/9z69
* file diff (modified to fit the purpose): https://termbin.com/yydi

[Renich Bon Ćirić]

On Thu, 2022-09-22 at 03:24 -0700, Lorenzo Miniero wrote:
>
> Apologies, forgot to answer on the dependencies.

No worries! :D

> > Now, it would be good if you could tell me what depends on what.
> >
> > For example, janus-gateway-demos should depend on everything but
> > the
> > devel packages, right?
> >
>
> Not sure what you mean by that? That installing the demos would
> include all
> packages and all plugins except for -devel? If so, I think that might
> be
> overkill: the demos can depend only on the main package as all others
> do, I
> guess: trying to run a demo when a plugin is not installed will
> return an
> error in the web page, and that's expected.

Yeah, forgive my poor explanation. You got it right, though and I take
your advice. packages will only depend on janus.

> > janus-gateway doesn't depend on anything but every component will
> > require it. Correct?
> >
>
> Yep, that's correct. Looking at the packages you shared, I guess the
> dependency tree could actually be quite simple, since all packages
> are
> independent of each other, meaning that all packages only depend on
> the
> main one and nothing else (unless macro packages for the different
> categories are added, as imagined in the previous response).

OK, thanks. I am interested in the "macro packages for the different
categories" thing you're talking about.

It would be interesting to hear your proposal on this.

> As a side note, do you need info on the library dependencies for each
> package too, or have you that figured out already?

Well, AFAIK, Fedora does a pretty good job on digging up runtime
dependencies automatically. That said, I've prepared a list of the
dependencies it found. It has happened that it misses a few.

* per rpm dependencies: https://termbin.com/5jdn

Sorry to put so much of the work on you, Lorenzo. Janus is pretty
comprehensive and I've done my best to do my homework but been at this
for a few weeks only... :S

I appreciate your effort and help.

[Renich Bon Ćirić]

On Thu, 2022-09-22 at 04:15 -0700, Lorenzo Miniero wrote:
> As a side note, I've prepared a PR to add versioning to .so files as
> you
> asked:
> https://github.com/meetecho/janus-gateway/pull/3075
>
> This would start from 1.1.0, which will be the next tagged version,
> after
> the 1.0.4 you're working with right now (which will still have the
> .0.0.0).
> Please let me know if the way it's implemented actually works the way
> you
> need it to, or if I missed something.
>

Great news! It seems OK to me. As long as you can keep track of what
changed, that's fine.

In case of a change, I imagine, you would tag a revision and, Fedora,
would just rebuild everything against the dependencies so we're good.

[Lorenzo Miniero]

Snipping to just keep the open questions, as we're spanning multiple threads now :-)

Oh, it does! I was leaving that one for last. 

And, for the sake of clarity, janus-plugins would install janus-
plugins-*, but not janus-eventhandlers-* or janus-loggers-*. The reason
for this would be that you will rarely need support for all transports
or eventhandlers at the same time. If you do, you're a special case and
should just install them all manually. 

Do we agree on this?

Yep, that's indeed what I was thinking about! I was also thinking of other macro-packages like, i.e., janus-transports, janus-eventhandlers, and janus-loggers, to keep plugins of different categories packaged in different "boxes". But even just janus-plugins would be fine, as those are the plugins you're most likely to want more instances of (while you'll typically only need 1-2 transports and 1 handler).

 

OK, thanks. I am interested in the "macro packages for the different
categories" thing you're talking about.

It would be interesting to hear your proposal on this.

It would basically be what I wrote above, with different packages for different categories of plugins.

 

Well, AFAIK, Fedora does a pretty good job on digging up runtime
dependencies automatically. That said, I've prepared a list of the
dependencies it found. It has happened that it misses a few.

* per rpm dependencies: https://termbin.com/5jdn

That looks quite accurate to me! The only thing I noticed is that gstreamer1.0-plugins-good is not needed for janus-tools, since the post-processor only uses ffmpeg as a library.

 

Great news! It seems OK to me. As long as you can keep track of what
changed, that's fine.

Yep, we keep a changelog so that should be fine!

 

Sorry to put so much of the work on you, Lorenzo. Janus is pretty
comprehensive and I've done my best to do my homework but been at this
for a few weeks only... :S

I appreciate your effort and help.

No worries at all! I'm really excited about the possibility of getting Janus in Fedora, so whatever I can do to facilitate that, I'm happy to do.

Lorenzo

[Renich Bon Ćirić]

On Fri, 2022-09-23 at 03:20 -0700, Lorenzo Miniero wrote:
> > That looks quite accurate to me! The only thing I noticed is that >
> > gstreamer1.0-plugins-good is not needed for janus-tools, since the
> > > post-processor only uses ffmpeg as a library. 

OK. Do you, still, need me to add it or should we leave it as it is?

>
> > No worries at all! I'm really excited about the possibility of >
> > getting Janus in Fedora, so whatever I can do to facilitate that,
> > I'm > happy to do.

:D You rock, man!


Additionally, I've checked the paths in the files. Here's proof:

https://youtu.be/pMUcEumwYAk?t=11115

BTW, that is a looooong hacking session me and some guys had in order
to deploy Janus as part of the components required by NextCloud. We're
wroking openly in order to develop an ansible playbook to enable people
to just deploy it (in an efficient manner).

https://git.softwarelibre.mx/strepsirrhini/lemur/-/tree/master/nextcloud/ansible

The video is in Spanish since we're in México. :D

The link I provide, shows how we install Janus (from our own repo) and
show the files.

We're, by no means, experts in Janus, so we spend a long time figuring
out the configuration. :D

Well, I think that does it for the package getting into Fedora. I
really appreciate your help. I will let you know when it's there and,
obviously, feel free to file bugs and stuff. We'll try to maintain it
as best as we can.

I will me messaging you if I need more help, hehe. ;D

[Lorenzo Miniero]

Il giorno lunedì 26 settembre 2022 alle 09:19:06 UTC+2 ren...@woralelandia.com ha scritto:

On Fri, 2022-09-23 at 03:20 -0700, Lorenzo Miniero wrote:
> > That looks quite accurate to me! The only thing I noticed is that >
> > gstreamer1.0-plugins-good is not needed for janus-tools, since the
> > > post-processor only uses ffmpeg as a library. 

OK. Do you, still, need me to add it or should we leave it as it is?

It should actually be removed from the list of dependencies, since it's not needed by the janus-tools package.

 

>
> > No worries at all! I'm really excited about the possibility of >
> > getting Janus in Fedora, so whatever I can do to facilitate that,
> > I'm > happy to do.

:D You rock, man!

Glad to be of help!

 

Additionally, I've checked the paths in the files. Here's proof:

https://youtu.be/pMUcEumwYAk?t=11115

BTW, that is a looooong hacking session me and some guys had in order
to deploy Janus as part of the components required by NextCloud. We're
wroking openly in order to develop an ansible playbook to enable people
to just deploy it (in an efficient manner).

https://git.softwarelibre.mx/strepsirrhini/lemur/-/tree/master/nextcloud/ansible

The video is in Spanish since we're in México. :D

The link I provide, shows how we install Janus (from our own repo) and
show the files.

We're, by no means, experts in Janus, so we spend a long time figuring
out the configuration. :D

Kudos, that looks like a lot of work!

 

Well, I think that does it for the package getting into Fedora. I
really appreciate your help. I will let you know when it's there and,
obviously, feel free to file bugs and stuff. We'll try to maintain it
as best as we can.

I will me messaging you if I need more help, hehe. ;D

Of course, I'll be there if needed. Feel also free to reach me privately in case.

Lorenzo 

[Renich Bon Ćirić]

Hello, Lorenzo!

Just wanted to write to you to bring you up to date on what's up with the Fedora
Packaging effort.

I had a strange issue when I tried updating to v1.1.0. Given this commit: 

https://github.com/meetecho/janus-gateway/commit/d9f1ffce78a626a7b035b0fc626c3afc9786a517

I assumed that the .so versions would end up being: so.1.1.0 but, to my
surprise, they ended up differently:


<mock-chroot> sh-5.2# find . -type f -iname '*.so.*.?'
./build/BUILD/janus-gateway-1.1.0/src/events/.libs/libjanus_nanomsgevh.so.1.0.1
./build/BUILD/janus-gateway-1.1.0/src/events/.libs/libjanus_gelfevh.so.1.0.1
...


I dunno why this went the way it did. Maybe you do. :D


Another issue. Fedora doesn't distribute html resources that include external
resources (js, images, etc).


> https://docs.fedoraproject.org/en-US/packaging-guidelines/JavaScript/
> https://docs.fedoraproject.org/en-US/packaging-guidelines/Web_Assets/

Doxygen generates documentation with several includes. Also, the demos have
those. It seems that I cannot package them but, the reviewer; Mr. Benson Muite,
had the idea of tweaking doxygen in order for it to generate man pages.

So I did but, now, I have a ton of documentation that is, either:

* irrelevant
* relevant but badly named

For a full list, check this out:
https://bugzilla.redhat.com/show_bug.cgi?id=2121585#c22

Now, I removed src/mainpage.dox from the dox config and, more less, this:


diff --git a/.gitignore b/.gitignore
index 2f3eae6a..b03b8e3e 100644
--- a/.gitignore
+++ b/.gitignore
@@ -2,6 +2,7 @@ cmdline.c
cmdline.h
version.c
docs/html/
+docs/man/

janus
janus-cfgconv
diff --git a/docs/Makefile.am b/docs/Makefile.am
index 7e334931..2fd14fd8 100644
--- a/docs/Makefile.am
+++ b/docs/Makefile.am
@@ -4,7 +4,7 @@ doxygendir = $(htmldir)/janus-gateway-$(VERSION)

EXTRA_DIST = html

-all: html-local
+all: html-local man-local

html-local:
doxygen janus-doxygen.cfg
@@ -15,14 +15,19 @@ html-local:
cp ../html/favicon.ico html/
cp ../html/meetecho-logo.png html/

-install-data-local: html-local
+man-local:
+ doxygen janus-doxygen-man.cfg
+
+install-data-local: html-local man-local
$(MKDIR_P) $(DESTDIR)$(doxygendir)
cp -r html/ $(DESTDIR)$(doxygendir)
+ cp -r man/ $(DESTDIR)$(doxygendir)

uninstall-local:
rm -rf $(DESTDIR)$(doxygendir)

clean-local:
rm -rf $(builddir)/html
+ rm -rf $(builddir)/man

endif

And, that, removed most of the irrelevant files (README and stuff).

In any case, I am hesitant to rename documents myself. I dunno if you agree with
this or, if you do, have any naming scheme in mind in order to make janus' man
pages unique (since they have to live besides all other man pages in
/usr/share/man/man3...

Maybe just prepending janus_ or janus-?.

Let me know what you think and what do you propose to do in order to be able to
include the documentation to Fedora.

[Lorenzo Miniero]

Il giorno giovedì 13 ottobre 2022 alle 02:34:11 UTC+2 ren...@woralelandia.com ha scritto:

Hello, Lorenzo!

Just wanted to write to you to bring you up to date on what's up with the Fedora
Packaging effort.

Hey there! Answering inline.

 

I had a strange issue when I tried updating to v1.1.0. Given this commit: 

https://github.com/meetecho/janus-gateway/commit/d9f1ffce78a626a7b035b0fc626c3afc9786a517

I assumed that the .so versions would end up being: so.1.1.0 but, to my
surprise, they ended up differently:


<mock-chroot> sh-5.2# find . -type f -iname '*.so.*.?'
./build/BUILD/janus-gateway-1.1.0/src/events/.libs/libjanus_nanomsgevh.so.1.0.1
./build/BUILD/janus-gateway-1.1.0/src/events/.libs/libjanus_gelfevh.so.1.0.1
...


I dunno why this went the way it did. Maybe you do. :D

I think that's "normal", due to the way versioning is computed, which works differently for .so files from what I understood. There's an explainer here, even though I can't recall if it's the one I used to prepare my patch: https://www.sourceware.org/autobook/autobook/autobook_61.html

It may be I'll need to use different numbers to have them match the Janus version, but I don't know exactly how. Hints would be welcome!

Another issue. Fedora doesn't distribute html resources that include external
resources (js, images, etc).


> https://docs.fedoraproject.org/en-US/packaging-guidelines/JavaScript/
> https://docs.fedoraproject.org/en-US/packaging-guidelines/Web_Assets/

That's unfortunate. We explicitly went for external resources at the time, since they were all libraries we just used in demos, and so didn't want to clutter the repo with a ton of files that were not relevant to the project itself.

 

Doxygen generates documentation with several includes. Also, the demos have
those. It seems that I cannot package them but, the reviewer; Mr. Benson Muite,
had the idea of tweaking doxygen in order for it to generate man pages.

How would that help the demos, though? Those cannot be converted to man pages. Would they simply be left out as a result?

You can't remove mainpage.dox: that's where the vast majority of the documentation resides. Without it, you're just left with the bits of documentation Doxygen generates out of the source code itself, which covers individual plugins and code structures, but would be marginally useful.

 

In any case, I am hesitant to rename documents myself. I dunno if you agree with
this or, if you do, have any naming scheme in mind in order to make janus' man
pages unique (since they have to live besides all other man pages in
/usr/share/man/man3...

Maybe just prepending janus_ or janus-?.

Let me know what you think and what do you propose to do in order to be able to
include the documentation to Fedora.

I'm afraid I don't understand exactly what you're asking here, but I'm not sure I like the man approach. I didn't even know you could man-ify Doxygen documentation, and I'm not sure if the end result would be broken as a result: besides, as you noticed it would generate an awful lot of files, which would be awkward to put in a place shared with the whole system. Maybe you can try to convert the documentation to latex and then PDF itself, which would be easier to share? Or just put a single txt file that links to the online documentation instead (suboptimal but would get you rid of the problem of how to package it)? I don't have a clear preference: I can try to to see if the existing doxygen configuration we have generates a decent PDF and let you know. I'm more concerned about demos, at this point, since by the rules you shared it looks like they'll have to be dropped, which would be a pity.

Lorenzo

[Renich Bon Ćirić]

On Thu, 2022-10-13 at 01:54 -0700, Lorenzo Miniero wrote:
> I think that's "normal", due to the way versioning is computed, which works
> differently for .so files from what I understood. There's an explainer
> here, even though I can't recall if it's the one I used to prepare my
> patch: https://www.sourceware.org/autobook/autobook/autobook_61.html

This clears it up for me a bit. so, the library/plugin/shared object versions
are independent from the main project. I get it.

I you would sync it that's why I was asking.

> It may be I'll need to use different numbers to have them match the Janus
> version, but I don't know exactly how. Hints would be welcome!

Well, as far as I read, it would be up to you to sync them. '1:1:1' turned into
'1.0.1' for some reason. I was just expecting 1.1.0.

This is not a huge problem. I'm just trying to interpret the links I sent you
regarding this from Fedora's packaging guidelines.

If you're fine the way they are, I'll just package them as they are and see if
they pass the review. :)

>
> That's unfortunate. We explicitly went for external resources at the time,
> since they were all libraries we just used in demos, and so didn't want to
> clutter the repo with a ton of files that were not relevant to the project
> itself.

Yeah, I dunno for sure but I think Fedora does this to prevent vectors of
attack. Also, to be able to provide documentation/demos in offline
installations.

If they're important, well, I can just try and create a huge patch to apply at
build-time.

I'm not much of a web programmer though and I'm not sure the jquery versions
provided by Fedora will meet your requirements and stuff. I'll see what I can
do.

>
> How would that help the demos, though? Those cannot be converted to man
> pages. Would they simply be left out as a result?

Yeah, well, if we want the demos, I need to patch them so that they use what
Fedora provides.

I'll try my best with this.

To tell you the truth, I wanted Janus in the repos as soon as possible and asked
the reviewer to omit this and that, later on, I could add them (calmly and not
in a hurry) but he insisted. :D

> You can't remove mainpage.dox: that's where the vast majority of the
> documentation resides. Without it, you're just left with the bits of
> documentation Doxygen generates out of the source code itself, which covers
> individual plugins and code structures, but would be marginally useful.

OK. I won't remove src/mainpage.dox. :S

Sorry. I was just trying out stuff.

In regards to the file name of the resulting man pages, I am still not sure
about the name of the resulting man pages, though. I am afraid they will clash
with other files in man3/. All other man pages just get put there and, if the
name is too generic, they're bound to collide with others.

> I'm afraid I don't understand exactly what you're asking here, but I'm not
> sure I like the man approach. I didn't even know you could man-ify Doxygen
> documentation, and I'm not sure if the end result would be broken as a
> result: besides, as you noticed it would generate an awful lot of files,
> which would be awkward to put in a place shared with the whole system.
> Maybe you can try to convert the documentation to latex and then PDF
> itself, which would be easier to share? Or just put a single txt file that
> links to the online documentation instead (suboptimal but would get you rid
> of the problem of how to package it)? I don't have a clear preference: I
> can try to to see if the existing doxygen configuration we have generates a
> decent PDF and let you know. I'm more concerned about demos, at this point,
> since by the rules you shared it looks like they'll have to be dropped,
> which would be a pity.

OK, I don't want to pressure you into the man-page approach. I'll just drop it
if it for now. 

If you, ever, have some time available and want to try the man-page approach and
see what it produces, then do so and let me know if you'd like these too.

Also, it's my first time using doxygen so... :S

Given what you've told me, I will focus my effort on patching demos and the
documentation so that they use Fedora's offline resources. This way, they can be
made available offline as well and we keep you happy. ;D

Thank you for your time and patience.

p.s. Work was crazy these past days and had no time at all to reply or package.
Sorry for the late reply.

[Lorenzo Miniero]

Il giorno mercoledì 19 ottobre 2022 alle 00:26:28 UTC+2 ren...@woralelandia.com ha scritto:

On Thu, 2022-10-13 at 01:54 -0700, Lorenzo Miniero wrote:
> I think that's "normal", due to the way versioning is computed, which works
> differently for .so files from what I understood. There's an explainer
> here, even though I can't recall if it's the one I used to prepare my
> patch: https://www.sourceware.org/autobook/autobook/autobook_61.html

This clears it up for me a bit. so, the library/plugin/shared object versions
are independent from the main project. I get it.

I you would sync it that's why I was asking.

> It may be I'll need to use different numbers to have them match the Janus
> version, but I don't know exactly how. Hints would be welcome!

Well, as far as I read, it would be up to you to sync them. '1:1:1' turned into
'1.0.1' for some reason. I was just expecting 1.1.0.

This is not a huge problem. I'm just trying to interpret the links I sent you
regarding this from Fedora's packaging guidelines.

If you're fine the way they are, I'll just package them as they are and see if
they pass the review. :)

I think I _may_ have figured out a way to have them match: at least in 1.1.2 (new version, not tagged yet) it looks like the .so files now have 1.1.2 too. I'll have to see if I can get it to do the same in 1.1.3 as well, when the time comes :-D

 

Any update on the status of the packaging, as a side note? Please do let me know if you need me to do anything else, or if I can help with something.

Lorenzo

[Renich Bon Ćirić]

- all

 

09.12.2022, 05:32, "Lorenzo Miniero" <lmin...@gmail.com>:

I think I _may_ have figured out a way to have them match: at least in 1.1.2 (new version, not tagged yet) it looks like the .so files now have 1.1.2 too. I'll have to see if I can get it to do the same in 1.1.3 as well, when the time comes :-D

Hey Lorenzo, good to hear from you. :)

 

Great news! :D

Any update on the status of the packaging, as a side note? Please do let me know if you need me to do anything else, or if I can help with something.

 

I am a bit stuck with the documentation and demos part. All resources must be offline (jquery, themes, etc); one cannot distribute code that uses CDNs. I was supposed to package those resources as independent resources for Fedora but, the further I looked into it, the more resources popped out (bootstrap's theme, bootstrap itself, the sepcific jquery version, etc).

 

Then, I asked around (some time ago, honestly) in the packaging mailing list and I got a good answer: https://bugzilla.redhat.com/show_bug.cgi?id=2121585#c32 where I refer to this comment:  https://lists.fedoraproject.org/archives/list/pack...@lists.fedoraproject.org/message/G4SR25UUCDF4EJCWUF5T5H4WENSUUO6B/

 

I think I have a path forward with that option, yet, I've had way too much work lately and haven't been able to dedicate time in order to finish the packaging. 90% of it is done, though. :)

 

I swear I will find some time this weekend and try to finish what I started. I am sorry I haven't been able to do so.

 

[Lorenzo Miniero]

No need to apologise! You're doing the project a favour and in your free time, that's how open source works :-)  That's why I asked if I could help somehow. 

Cheers,

Lorenzo

[Renich Bon Ćirić]

On Fri, 2022-12-09 at 03:32 -0800, Lorenzo Miniero wrote:
> Any update on the status of the packaging, as a side note? Please do let me
> know if you need me to do anything else, or if I can help with something.

OK so, as promised, I think I nailed it.

Here's what I did. I've generated two patches:

0001-janus-docs-make-web-resources-local.patch
0002-janus-demos-make-web-resources-local.patch

They patch docs and demos so that we:

* do not use remote web resources (CDNs and stuff)
* use Fedora's jquery (js-jquery) and AwesomeFont (awesomefont-fonts-web).

The reason is basically that Fedora cannot ship software that connects to
resources it cannot trust (not official; my interpretation completely). So it
wants us to ship all sources required and use them offline.

It was a ton of them. I had to download each. Also, some of them hand name
clashes so I went with versioning them all.

Everything is in the patches.

It would be good if your code included everything and no CDNs. Think about it.
Those sources, even if trustworthy, have no guarantee of being available in the
future. Then, the software isn't future-proof. One cannot reproduce the software
later on. (100 years later... ;D).

My point is, it would be good not to use external sources and just keep
everything local.

I, also, noticed that you use bootswatch v3.4.0 while using bootstrap's js
v3.4.1.

I think there are several versions of jquery as well. Might want to normalize
this too.

Just sayin' (as they say).

In any case, all this work hasn't been reviewed yet. The reviewer might turn me
down on this and ask me to modify it. In any case, my point stands. Please
consider it (making all web resources local).

[Lorenzo Miniero]

Il giorno lunedì 19 dicembre 2022 alle 01:03:17 UTC+1 ren...@woralelandia.com ha scritto:

It would be good if your code included everything and no CDNs. Think about it.
Those sources, even if trustworthy, have no guarantee of being available in the
future. Then, the software isn't future-proof. One cannot reproduce the software
later on. (100 years later... ;D).

That's what we did in the past, and we decided to stop doing that, exactly because it was a ton of files that really didn't belong in the repo. Besides, IIRC from the PR where we changed this, part of the reason was also related to licensing: https://github.com/meetecho/janus-gateway/pull/780

Long story short, sorry but for now I'm not going to include all those files in the repo itself again.

Lorenzo

[Mirko Brankovic]

JS files for frontend? Or for js plugin?

Can't you just exclude it from installation?

--
You received this message because you are subscribed to the Google Groups "meetecho-janus" group.
To unsubscribe from this group and stop receiving emails from it, send an email to meetecho-janu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/meetecho-janus/06cd5aae-e18d-447b-b926-c452df0f160cn%40googlegroups.com.

[Renich Bon Ćirić]

On Mon, 2022-12-19 at 03:29 -0800, Lorenzo Miniero wrote:
> That's what we did in the past, and we decided to stop doing that, exactly
> because it was a ton of files that really didn't belong in the repo.
> Besides, IIRC from the PR where we changed this, part of the reason was
> also related to licensing:
> https://github.com/meetecho/janus-gateway/pull/780
>
> Long story short, sorry but for now I'm not going to include all those
> files in the repo itself again.

Right. Had to try. I will have to use the patches I mentioned in order to make
docs and demos available 100% offline.

One thing, though, is it possible to upgrade demos and docs to use the latest
jquery (v3.6)? Currently, you're using v1.9. Kind of old if you ask me.

Also, another issue, the use of OpenSSL has been contested. Here's what the
reviewer had to say:

"The cipher warning seems to come from line 491 in src/dtls.c see
https://docs.fedoraproject.org/en-US/packaging-guidelines/CryptoPolicies/#_cc_applications
Probably an exemption or some coordination will be required with upstream."

source: https://bugzilla.redhat.com/show_bug.cgi?id=2121585#c38 (near the end of
the comment)

In any case, let me know if it is convenient. Otherwise, I will have to figure
it out.

Thank you, Lorenzo. And may you spend a happy Isaac Newton's birthday (in the
old calendar, at least). :D

[Renich Bon Ćirić]

On Mon, 2022-12-19 at 14:49 +0100, Mirko Brankovic wrote:
> JS files for frontend? Or for js plugin?
> Can't you just exclude it from installation?

Hello Mirko,

I would have to exclude demos and doc. It would be good to have them.

[Renich Bon Ćirić]

On Fri, 2022-12-23 at 13:40 -0600, Renich Bon Ćirić wrote:
> Also, another issue, the use of OpenSSL has been contested. Here's what the
> reviewer had to say:
>
> "The cipher warning seems to come from line 491 in src/dtls.c see
> https://docs.fedoraproject.org/en-US/packaging-guidelines/CryptoPolicies/#_cc_applications
> Probably an exemption or some coordination will be required with upstream."
>
> source: https://bugzilla.redhat.com/show_bug.cgi?id=2121585#c38 (near the end
> of
> the comment)
>
> In any case, let me know if it is convenient. Otherwise, I will have to figure
> it out.

This might be all we need.

BTW, I'm not a C developer. Just using common sense.

[Lorenzo Miniero]

I'm not familiar with PROFILE=SYSTEM, and if it can impact the setup of PeerConnections. Have you checked if this causes things to break already? I may but only in the next few days.

L.

[Renich Bon Ćirić]

On Sat, 2022-12-24 at 00:46 -0800, Lorenzo Miniero wrote:
> I'm not familiar with PROFILE=SYSTEM, and if it can impact the setup of
> PeerConnections. Have you checked if this causes things to break already? I
> may but only in the next few days.

Well, I did run make check but that's all. It compiled fine.

Yet, due to family parties and stuff, haven't been able to properly test it.
Will try to find the time in the following days to do so as well.

Thanks and may you spend a happy Isaac Newton's birtdhay ;D

[Renich Bon Ćirić]

Hey, Lorenzo!

I hope everything is going well. I'm almost done with the packaging, I hope.
Sorry for the huge delays and all...

The only problem my reviewer and me can find is that Fedora 36 isn't building
the documentation properly.

When `make html` is done, at `docs/`, while using Fedora 36, some files go
missing.

The error can be viewed in this build.log file from Fedora 36:
https://download.copr.fedorainfracloud.org/results/@fedora-review/fedora-review-2121585-janus/fedora-36-x86_64/05238279-janus/build.log.gz

The missing files are:

/usr/share/doc/janus/html/bc_sd.png
/usr/share/doc/janus/html/dir_000004_000000.html
/usr/share/doc/janus/html/dir_000004_000001.html
/usr/share/doc/janus/html/dir_000004_000002.html
/usr/share/doc/janus/html/dir_000004_000005.html
/usr/share/doc/janus/html/docd.png
/usr/share/doc/janus/html/nav_fd.png
/usr/share/doc/janus/html/nav_hd.png
/usr/share/doc/janus/html/search/mag.svg
/usr/share/doc/janus/html/search/mag_d.svg
/usr/share/doc/janus/html/search/mag_seld.svg
/usr/share/doc/janus/html/splitbard.png
/usr/share/doc/janus/html/tab_ad.png
/usr/share/doc/janus/html/tab_bd.png
/usr/share/doc/janus/html/tab_hd.png
/usr/share/doc/janus/html/tab_sd.png

On the other hand, when the exact same build is done in Fedora 37, this doesn't
happen.

I've created two scripts to reproduce this. The scripts are here:

# gendocs.bash
https://termbin.com/z3rd

# podman-generate_documentation.bash
https://termbin.com/7av2


gendocs.bash is the script that builds the html docs.

podman-generate_documentation.bash uses gendocs.bash to automate the process in
Fedora 36 and Fedora 37 containers.

In both cases, the script, at the end, tries to find the files.

It does find the files in Fedora 37 but it doesn't find them in Fedora 36.

Instructions:

With podman available: 

* put the janus source code in ~/src/janus-gateway
* put the gendocs.bash script in ~/src/janus-gateway
* run the podman-generate_documentation.bash script

The result we're looking for is at the end of each run. It looks something like
this to me:

# Fedora 36
# Finding the following files: bc_sd.png dir_000004_000000.html
dir_000004_000001.html dir_000004_000002.html dir_000004_000005.html docd.png
nav_fd.png nav_hd.png mag.svg mag_d.svg mag_seld.svg splitbard.png tab_ad.png
tab_bd.png tab_hd.png tab_sd.png:


# Fedora 37
# Finding the following files: bc_sd.png dir_000004_000000.html
dir_000004_000001.html dir_000004_000002.html dir_000004_000005.html docd.png
nav_fd.png nav_hd.png mag.svg mag_d.svg mag_seld.svg splitbard.png tab_ad.png
tab_bd.png tab_hd.png tab_sd.png:

/usr/src/janus/docs/html/bc_sd.png
/usr/src/janus/docs/html/dir_000004_000000.html
/usr/src/janus/docs/html/dir_000004_000001.html
/usr/src/janus/docs/html/dir_000004_000002.html
/usr/src/janus/docs/html/dir_000004_000005.html
/usr/src/janus/docs/html/docd.png
/usr/src/janus/docs/html/nav_fd.png
/usr/src/janus/docs/html/nav_hd.png
/usr/src/janus/docs/html/search/mag.svg
/usr/src/janus/docs/html/search/mag_d.svg
/usr/src/janus/docs/html/search/mag_seld.svg
/usr/src/janus/docs/html/splitbard.png
/usr/src/janus/docs/html/tab_ad.png
/usr/src/janus/docs/html/tab_bd.png
/usr/src/janus/docs/html/tab_hd.png
/usr/src/janus/docs/html/tab_sd.png


I conclude, from this test, that those files are generated in Fedora 37 but not
in Fedora 36.

The doxygen versions of both are:

# podman exec -it f36-janus rpm -q doxygen
doxygen-1.9.1-13.fc36.x86_64

podman exec -it f37-janus rpm -q doxygen
doxygen-1.9.5-2.fc37.x86_64


Any ideas on what is the issue?

[Lorenzo Miniero]

Is the error coming from assumptions about files that should be there and aren't? If so, not sure if that would work, since different versions of Doxygen may be producing different files, with new ones added in 1.9.5, for instance (I see that the doxygen versions differ from your snippet). I also remember some time ago we had to fence out some specific versions of Doxygen as they were causing broken results: looking at the configure it's for 1.8.12 --> 1.8.14 though, so probably unrelated. I was on Fedora 36 myself just a few weeks ago and I don't remember errors, which is why I'm thinking that maybe the missing files are not really needed for the output produced in the F36 case (unless they are actually needed because of changes you added to make them really offline).

Apart from this, I honestly have no clue, as I'm not familiar with the Doxygen internals :-(

L.

[Renich Bon Ćirić]

On Mon, 2023-01-23 at 01:53 -0800, Lorenzo Miniero wrote:
> Is the error coming from assumptions about files that should be there and
> aren't? If so, not sure if that would work, since different versions of
> Doxygen may be producing different files, with new ones added in 1.9.5, for
> instance (I see that the doxygen versions differ from your snippet). I also
> remember some time ago we had to fence out some specific versions of
> Doxygen as they were causing broken results: looking at the configure it's
> for 1.8.12 --> 1.8.14 though, so probably unrelated. I was on Fedora 36
> myself just a few weeks ago and I don't remember errors, which is why I'm
> thinking that maybe the missing files are not really needed for the output
> produced in the F36 case (unless they are actually needed because of
> changes you added to make them really offline).
>
> Apart from this, I honestly have no clue, as I'm not familiar with the
> Doxygen internals :-(

Thank you for your insight. I will create an f36 exception for now then and not
include those files when building the f36 package.

Almost there! :D
--
Renich Bon Ćirić <ren...@woralelandia.com>