s2s connections

[Damian Minkov]

Hey all,

How does s2s works, isn't it supposed to have one connection per host pair?

# prosodyctl shell "s2s:show()" | wc -l
461

# prosodyctl shell "s2s:show()"
Session ID           | Host                   | Dir  | Remote                 | IPv  | Security  | SASL       | Dialback
s2sin5558e296def0    | meet.jit.si            | <->  | conference.v0.meet.ji… | IPv4 | TLSv1.3   | Succeeded  | Not used
s2sin5558e3a52aa0    | meet.jit.si            | <->  | conference.v0.meet.ji… | IPv4 | TLSv1.3   | Succeeded  | Not used
s2sin5558e2bce550    | meet.jit.si            | <->  | conference.v0.meet.ji… | IPv4 | TLSv1.3   | Succeeded  | Not used

And so on, almost the whole list is like that. Is this leaking the connections? Any idea what can be the reason for that?

Thank you

[Damian Minkov]

To update:

the configuration of prosody that is v0, v1 and so on can be seen here: 

https://github.com/jitsi/jitsi-meet/blob/master/resources/extra-large-conference/prosody.cfg.lua.visitor.template

And the configuration of the main prosody is mentioned here: https://github.com/jitsi/jitsi-meet/blob/master/resources/extra-large-conference/README.md

The modules about s2s that are enabled are:

      "s2s_bidi";
      "certs_s2soutinjection";
      "s2soutinjection";
      "s2s_whitelist";

Thanks

[Kim Alvefur]

Hi Damian and everyone else,

As discussed in the chat room, this in itself isn't really abnormal.
There is no limit in XMPP on the number of connections per host pair,
only a limit in Prosody in that it can only have a single outgoing
connection per remote host. Incoming connections (as these are) can have
any number, which may be uncommon but e.g. would be normal for a
clustered setup.

Often however it can mean that connections are not closed correctly, or
time out on one side without the other side noticing, in which case they
should normally time out after some amount of time. This amount of time
can however be quite long and mostly depends on kernel settings.

Finally, as it turned out in this case, it could be that the remote
somehow opens a new connection instead of using an existing connection.

On Tue, May 23, 2023 at 03:43:19PM -0700, Damian Minkov wrote:
>To update:
>the configuration of prosody that is v0, v1 and so on can be seen here:
>https://github.com/jitsi/jitsi-meet/blob/master/resources/extra-large-conference/prosody.cfg.lua.visitor.template
>
>And the configuration of the main prosody is mentioned
>here: https://github.com/jitsi/jitsi-meet/blob/master/resources/extra-large-conference/README.md
>The modules about s2s that are enabled are:
> "s2s_bidi";
> "certs_s2soutinjection";
> "s2soutinjection";
> "s2s_whitelist";
>

Based on discussions in the chat room, this appears to be a problem with
mod_s2soutinjection, which is something of a hack that likely has become
incompatible with the latest prosody developments. Unclear how.

I have published work from a while ago meant to allow for a cleaner way
to accomplish what this module does in the form of this new event:
https://hg.prosody.im/trunk/rev/d5f322dd424b and this new module using
it: https://modules.prosody.im/mod_s2sout_override.html

This works by switching out the "connection resolver" instance that is
responsible for finding targets to connect to.

--
Regards,
Zash

[Damian Minkov]

Hey Zash,

Thanks for the help. So I updated everything to use the resolver (I needed to enable s2sout_override under the muc component). But we are still experiencing the problem where multiple s2s connections are established and never closed between the prosody instances. 

I'm attaching such a log where 3 connections are created (conference.v1.meet.jitsi->myserver.com). 

The first one is s2sin, I think, then one s2sout is created again for conference.v1.meet.jitsi->myserver.com that overrides the first one in hosts[to].s2sout[from] and at the end after the room is destroyed a new s2sout is created (conference.v1.meet.jitsi->myserver.com) that overrides the previous one. 

Is this a bug or something on our end that is not working as expected or not configured as expected WDYT?

Thanks

damencho

--
You received this message because you are subscribed to the Google Groups "prosody-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prosody-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prosody-dev/ZG5YtnzU9Z8d%2B4Ma%40diploria.zash.se.

[Damian Minkov]

I think the problem is provoked from our code. So if you have filters on s2s session and if you filter a stanza - that will skip this if:

https://github.com/bjc/prosody/blob/180daf56a1f0934ba8043901f4b565382058cdd1/plugins/mod_s2s.lua#L200

And route_to_existing_session will return undefined which will execute the next in chain which is to create a new s2s session and override the previous. 

[Matthew Wild]

On Wed, 31 May 2023 at 16:29, Damian Minkov <dame...@damencho.com> wrote:
>
> I think the problem is provoked from our code. So if you have filters on s2s session and if you filter a stanza - that will skip this if:
> https://github.com/bjc/prosody/blob/180daf56a1f0934ba8043901f4b565382058cdd1/plugins/mod_s2s.lua#L200
> And route_to_existing_session will return undefined which will execute the next in chain which is to create a new s2s session and override the previous.

Ah, that sounds plausible indeed, congrats on tracking that down.

Obviously we need to be careful with core code such as this, but one
possible solution is to just return the result of host.sends2s()
there. The main downside I see is that it would cause a delivery error
instead of establishing a new connection if the write() fails while
sending a stanza. But we also don't want to spawn unlimited outgoing
connections in that case either, so we probably need to think about it
anyway.

Regards,
Matthew