Document AMQP config in Kamailio

[fred]

I just found out that there are apparently additional AMQP configs required in Kamailio.  If that is documented anywhere I cannot find it.

The ONLY way I am even aware of it is by finding a presentation someone made in 2015.

https://www.slideshare.net/2600hzmarketing/tuning-kazoo-to-10000-handsets-kazoocon-2015

On page 9.  

If that multizone configuration is documented anywhere I cannot find it.  Would it be possible to document that in addition to Primary/Secondary AMQP?  I tried to submit a pull request to add comments for Primary/Secondary AMQP to the Kamailio local.cfg and the response I got was "why would you want to do this" and then the pull request was closed.  Maybe I am wrong to think that these things should be documented.

[Luis Azedo]

it seems the pr is open https://github.com/2600hz/kazoo-configs/pull/150

if you know of any other pr please post here

--
You received this message because you are subscribed to the Google Groups "2600hz-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to 2600hz-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Darren Schreiber]

Looks like the pull request is still open but asking questions to you to have answered.

--

To unsubscribe from this group and stop receiving emails from it, send an email to 2600hz-users...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

 

--

You received this message because you are subscribed to the Google Groups "2600hz-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to 2600hz-users...@googlegroups.com.

[fred]

What is this for anyways?  I believe I have tested multizone without this config and everything seemed to work in v4.0 including BLF across zones. 

I know that zones is a more advanced config but why are the basics not documented anywhere?  In this case it would only require a few additional comments in the kamailio local.cfg file.  Are these things supposed to be obvious?  I figured out the Primary Secondary AMQP by looking through Kamailio configs.  This zone configuration is not as obvious to me.  Is the expectation of this project that we find configurations by interpreting Erlang code?

It's not just this.  There is almost no documentation for configuring zones in /etc/kazoo/core/config.ini either.  I had to figure that out by finding bits and pieces in comments scattered around the internet.

[Darren Schreiber]

> What is this for anyways?  I believe I have tested multizone without this config and everything seemed to work in v4.0 including BLF across zones. 

This was an attempt at failover from a while back, we are not actively promoting this method at this time, it was experimental. It should probably not be documented frankly (for now). As you noted, the preso was from 2015, much has changed since then.

 

> I know that zones is a more advanced config but why are the basics not documented anywhere?  In this case it would only require a few additional comments in the kamailio local.cfg file.  Are these things supposed to be obvious?

Most things that are lacking in the project is a result of lack of time/resources to do the documentation. This is why we encourage you to help out where you see things that are missing. Sometimes we use the stuff all day and forget that some of these items are not obvious. So your contributions are most helpful here.

 

I figured out the Primary Secondary AMQP by looking through Kamailio configs.  This zone configuration is not as obvious to me.  Is the expectation of this project that we find configurations by interpreting Erlang code?

> No, and this is C code anyway, not Erlang code. Nonetheless, this particular item isn’t documented because it’s experimental, so you can just ask on here if you’re unsure.

 

It's not just this.  There is almost no documentation for configuring zones in /etc/kazoo/core/config.ini either.  I had to figure that out by finding bits and pieces in comments scattered around the internet.

> Do you want to post what you found? Then we can reformat it into something more user friendly.

[fred]

Thanks for your patience in answering Darren.  

What is the recommended multizone and multi-AMQP per zone configuration for AMQP in Kamailio in that case?  

That primary/secondary failover did work when I tested in on v4. Obviously not something I want to use if you don't recommend it. 

[Darren Schreiber]

Hi Fred,

               

Thanks for your patience in answering Darren.  

> Sure, I’m not unaware the project is complicated and sometimes we rush things so it appears inconsistent. It’s not nefarious, just time and resource related.

 

What is the recommended multizone and multi-AMQP per zone configuration for AMQP in Kamailio in that case?  

> This is one of the reasons this isn’t well documented, frankly, because the answer (which nobody likes) is, “it depends”. Basically it depends what your tolerance for failure is. There’s multiple ways to skin the cat.

 

> In reality, we are now running many installations with 3,000+ phones on just 3 servers (something we never used to recommend because of memory leaks or CPU hogs, but most of that has been optimized now). So, if I had to make a recommendation, I’d do 3 zones minimum. I would not use multi-AMQP per zone yet frankly. We don’t (yet). We just do hot standby for now. It’s the only component that doesn’t auto-failover, for now.

 

That primary/secondary failover did work when I tested in on v4. Obviously not something I want to use if you don't recommend it. 

> So the problem is that it doesn’t fail back properly, and while it may appear to work, things like BLFs and other items will become out of sync. Also, the original intent was to use clustered Rabbit, but that continues to prove slow and we haven’t had time to go back and figure out why it’s slow (I’m sure it’s solvable). At some point the idea will be each zone has a local cluster of Rabbit’s (and then failback doesn’t matter) but for now, we’re not using that.

[fred]

Assuming 3 zones and one Kamailio per zone and one RabbitMQ per zone.  Does each Kamailio just need the one RabbitMQ in that zone configured and that's it?  That was how I originally thought it worked and then I was sort of led to believe that all 3 RabbitMQs in from the 3 zones needed to be added to each and every Kamailo in each zone and that slide presentation I found re-inforced that idea.  

I am ok with one RabbitMQ per zone with hot standby so I am not complaining about the lack of redundancy.

[Darren Schreiber]

For now, yes, that’s been the recommendation.

[Andre S.]

Just for crystal clarity the current recommendation by 2600hz is One Kamailio + RabbitMQ per zone with hot standby. Do not reference the other zone RabbitMQs in the zone.

Zone A: Kamailio + RabbitMQ, no reference to Zone B RabbitMQ

Zone B: Kamailio + RabbitMQ, no reference to Zone A RabbitMQ

[Luis Azedo]

AFAIK, 2600hz does not recommend 'RabbitMQ per zone with hot standby'

it may work, but it is not endorsed

To unsubscribe from this group and stop receiving emails from it, send an email to 2600hz-users+unsubscribe@googlegroups.com.

[Andre S.]

Thanks Luis, I believe myself and the community may be confused on the recommended configuration as Darren wrote this in this thread: "I would not use multi-AMQP per zone yet frankly. We don’t (yet). We just do hot standby for now. It’s the only component that doesn’t auto-failover, for now."

We absolutely do *not* have the right to know what 2600hz is using internally, however I think it'd make all feel more comfortable if we knew what configuration was currently recommended by the developers. I think some of us like fred and myself read every post on 2600hz-users, 2600hz-dev and reference https://botbot.me/freenode/2600hz/ as much as possible, and it's not always clear what the current recommendations are.

[Darren Schreiber]

Hi Andre,

                We are attempting to clarify any items here in the docs so they are more permanent. It would be most helpful if you stubbed out an article and then we corrected it. You can submit a starter to Github and then Luis & I can work to add to it.

[Andre Soileau]

Darren, would this be the correct place? https://github.com/2600hz/sysadmin/blob/master/doc/kazoo/cluster-guide.md

--

PRIME65

Advanced Technology Solutions

10 4th St, Petaluma, CA 94952

http://www.p65.org

[Darren Schreiber]

Yeah that would work, thanks for your help, much appreciated.

[fred]

Now I am confused again....lol.

I think you are just talking about Kamailio config but just to be crystal clear, I am pretty sure the RabbitMQ's in the different zones need to know about each other and the way you do that is in /etc/kazoo/core/config.ini

[zone]

name = "z1"

amqp_uri = "amqp://guest:guest@z1_AMQP_IP"

[zone]

name = "z2"

amqp_uri = "amqp://guest:guest@z2_AMQP_IP"

Not sure if the hot standby in each zone could also be added to the above config in each zone.  However, based on what Darren said, it sounds like it should not be added to Kamailio config.

[Darren Schreiber]

Correct.

 

I guess this is indeed why we need a doc J

[fred]

Hi Darren,

I could not get BLF working across zones.  So someone in this thread 

https://groups.google.com/forum/#!topic/2600hz-dev/IXnk5aUXJ7A

suggested I do the alternate zone AMQP thing in Kamailio and it worked.  So I am back to being confused again.

[Darren Schreiber]

Heh well then I am too. I admit I don’t know everything J It’s possible I’ve made a mistake.

 

Let me speak with core engineering and we’ll draft some changes to whatever you guys submit as a base document. Then we’ll finally have this in a document!

[Arek]

Hello,

I confirmed zone configuration with 2600hz guys on IRC channel. Was told that's required config to make kamailio aware of other zones in 4.1. 

Also, failover amqp works only if both rabbitmq servers are clustered (in 4.1) and if they are not you will see wired things happening with BLF. 

Tested and confirmed on my cluster with multi-zone and multi kamailio per zone setup.

Arek

[Darren Schreiber]

How much more begging do I hav to do for someone to start a doc and then we can formally document this appropriately? J

 

Then this thread becomes less jenky/needed.

[fred]

I assumed 2600hz are the only ones qualified to document this.  If someone from 2600hz can explain to me exactly how it works then I can type it out and do a pull request into the cluster guide.
https://github.com/2600hz/sysadmin/blob/master/doc/kazoo/cluster-guide.md

Right now I am still trying to understand how it works myself.

[Darren Schreiber]

Sure, perhaps we setup a call or something and go over it, then you can post the document? Probably easier to just talk it through. Let me talk to engineering about this. Maybe a public conf call.

[Arek Fryz]

I'm interested in this call too. Let me know if you had anything scheduled. 

 

Regards,

Arek Fryz

REMAC ENTERPRISES, LTD.

Office: (773) 888-5395 

Email: sup...@remacenterprises.com

Fax: (224) 655-6556

www.remacenterprises.com

[Darren Schreiber]

I am traveling this week, I’ll see if I can get something put together for next week or the week after.

 

From: Arek Fryz <ar...@remacenterprises.com>

Date: Wednesday, June 14, 2017 at 4:55 PM
To: Darren Schreiber <dschr...@2600hz.com>