Restcomm Weekly meeting notes June 8th, 2016
34 views
Skip to first unread message
Jean Deruelle
Jun 9, 2016, 4:29:31 AM6/9/16
to mobicents-public, rest...@googlegroups.com
Attendees:
Jean, George, Orestis, Antonis, Jaime, Tran, Sergey V, Sergey P., Kostyantyn,, Charles, Gui, Henrique, Lefteris, Despina, Maria, Lucas, Fernando, Fernando
Join in at https://gitter.im/RestComm/Restcomm-discuss
Summary
#1 JAIN SLEE: TelScale release performance CI job automation and integration with PerfCorder completed. Found a mem leak to investigate and setup similar CI jobs for HA with LB. Planning release next week again and Preparing roadmap for 3.0
#2 jEIR: No news
#3 Load Balancer: Added SMPP node automatic discovery. Working on SMPP perf job and new conf file for LB
#4 Restcomm Core and RVD: Moving forward on 7.8.0. Conference API ready for review. Load Testing Restcomm and found 2 mem leaks and 1 thread leak. PerfCorder integration in progres. Fixes in progress. Migration of docker configuration scripts to binary so that binary is easier to setup without docker. Multiple Media Server Support in progress
#5 Restcomm SSO: Moving forward. Planning to have it ready for 8.0.0
#6 Restcomm as a Service: Customer work. Upgrade to 7.7.0 done and Mayday app deployed too. Auto-configuration script re factoring first phase completed.
#7 Restcomm Apps, App Store: Mayday app progress
#8 Restcomm SDKs/Clients/WebRTC Test Tool: Customer work. Resumed work on Cellular data calling for Android
#9 Restcomm UI/UX Design: Finalized Olympus new UI/UX mockups. Internal review triggered new ideas. Statred investigation of Olympus desing for iOS and Android. Iterating on Mayday mockups
#10 Media Server: Customer Work. New 4.1.1 release https://github.com/RestComm/mediaserver/releases/tag/4.1.1.1
#11 SIP Servlets/SIP Stack/PerfCorder : Customer Work. Merged requests simple solution in testing by customer. Testsuite failures investigation. Planning next release before end of week.
PerfCorder progress.
#12 jSS7/SMSC/USSD/CAMEL GW: jSS7: 7.0.0 release is out https://github.com/RestComm/jss7/releases/tag/7.0.1383. PR reviews.
CSC moving forward on first contribution SMSC: Preparing TelScale 6.2.3.GA release
#13 GeoLocation/GMLC: Fernando trying to fix GMLC Build and iterating on 1.0.0 release. Loactrix preparing for demo at MWC Shanghai
#14 jDiameter/Charging Server: reviewing jDiameter Pull Requests.
Log:
---------------
Agenda
#0 Documentation
#1 JAIN SLEE
#2 jEIR
#3 Load Balancer
#4 Restcomm Core & RVD
#5 RestComm SSO
#6 Restcomm as a Service
#7 Restcomm Apps, App Store
#8 Restcomm SDKs/Clients
#9 Restcomm UI/UX Design
#10 Media Server
#11 SIP Servlets/SIP Stack
#12 jSS7/SMSC/USSD
#13 GeoLocation/GMLC
#14 jDiameter/Charging Server
#2 jEIR
#3 Load Balancer
#4 Restcomm Core & RVD
#5 RestComm SSO
#6 Restcomm as a Service
#7 Restcomm Apps, App Store
#8 Restcomm SDKs/Clients
#9 Restcomm UI/UX Design
#10 Media Server
#11 SIP Servlets/SIP Stack
#12 jSS7/SMSC/USSD
#13 GeoLocation/GMLC
#14 jDiameter/Charging Server
#0 Documentation
new documentation website is live
@desipepova will be setting up redirection from old website
there is still some content to migrate
notably JAIN SLEE RAs, Enablers and Tools to asciidoc
and about 10 pages from the old website
Thanks to @jaimecasero for the PR
on making the image boxes clickable
first time in our 15y history
where we have all documentation in one place
#1 JAIN SLEE
@SergeyLee can you update ?
but before my report a few words about asciidoc for JSLEE :)
there is not pdf file in Release, is it good?
we had docs/container/user-guide folder in earlier versions
the asciidoc generates 3 outputs
you can use html-book and pdf
for bundling into the release
so make changes there
so, this week was week of Performance tests with PerfCorder and reading/investigating JSLEE spec/code
I created 3 tests
UAS, B2BUA and Registrar
yes
were you able to investigate the mem leak found in one of those test ?
memory wise
only B2BUA seems leaking
btw customer have not access on our zendesk because he need to extend support
like 2 minutes
Test Duration(seconds)= 2734
still a bit short
Test Duration(seconds)= 378
6 minutes test is not representative
we need at least 30 minutes to 1 hour tests
you can see #33 build
btw I need some advices how to create same test with LB @jaimecasero
RestComm/jain-slee@9505c32
may not be triggering the leak
this is how it looks for those with no access
because some refs are not removed anymore as some nulls are commented
but profiler will tell which objects are leaking
we have one since the project is open source
ok, I have question: where I can see JSLEE app in github for example? for investigation
not small
perhaps open-source?
ok
do we provide any with the release ?
it may be interesting to provide a number of examples by default
by example the SIP apps we use for perf tests
yes, we have many examples: SIP, Media, HTTP... ok, I start with them
and I had some problems with EclipSLEE :) I will create many issues on it
ok, that's all!
we shouldn't delay the release because of EclipSLEE
especially as it will change a lot for 3.X release
let's prioritize the mem leak
go to JProfiler
#2 jEIR
no news here
#3 Load Balancer
@knosach can you update ?
Last week I worked on issue: add SMPP port of Restcomm to the ping which Restcom sends to Load balancer.
currently I am continue working on issue RestComm/load-balancer#30 (Create a new Configuration File). I am finishing JUnit test's changes
that is all
where do we stand on performance tests for the LB
are those complete and integrated with PerfCorder too ?
this is more important than new features right now
for UDP and TCP transport
why do you say you will take care after issue 30 then ?
can we try to run it for longer like 2 hours
I think SMPP perf test should be done too
before the configuration
file
I did not know
diff with G1
mem looks better
@SergeyLee we should run JAIN SLEE perf test with G1 as well
again, we need to wait to hit the GC threshold to see if mem is reduced
somtimes it takes time
depends on JVM settings
#4 Restcomm Core & RVD
So last week I was mainly load testing restcomm to check for mem leaks and cpu performance.
I have a new call flow that I will add
as CI perf job
in ther process I discovered two mem leaks,
one for outbound call akka actors
and one for the MediaSession object
the first one for outbound call akka actors has been patched and now I am working for the MediaSession mem leak
Also I discovered an issue with ConvergedSessions when the container serves JSP files and will further investigate this one
also I worked on couple of customer tickets and helped @atsakiridis to check web rtc load test results
I expect tomorrow to have the mem leaks patched and so I will proceed to create the CI jobs
and integrate PerfRecorder
PerfRecorder is used only in one of the CI job we currently have, it was configured by @jaimecasero
JAIN SIP, and sip-servelts
:D
not yet, I was head down to the load testing and didn't update them yet. Will update asap
I had some inconsistent behaviour in the test class and took care of #1166 too
Currently all bits are there apart from a last test that still fails.
will get finish that tomorrow and create a PR
also, i'm investigating RVD's behaviour regarding ES errors, it's relation with thread leaks etc.
also i fixed some AdminUI regressions namely #1146 and #1018
included in cloud deployment
and finally did some investigation on application friendly-name-based uniqueness issues
that's all on Restcomm for me.
Will check the PR for 1111 when you are ready
@otsakir
also, i'm investigating RVD's behaviour regarding ES errors, it's relation with thread leaks etc.
any insihgts there yet ?
well, so far i experimented with Jprofiler and different numbers of concurrent clients
i can see that when i switch from 50 to 100 clients, there are 50 more threads created
that will be re-used next time 100 concurrent clients fire requests
i'm not sure if this is leak or a behaviour
wdyt @deruelle ?
it caps to the max number of concurrent clients
but that seems bad
in a sense that if we have 1K clients
we will create 1K threads
which is non sense
we should use a ThreadPool
dynamic Thread pool is possible
which is bad for performance and CPU
I would recommend to fix that behavior
ok, let me try to see if i can shutdown manually these resources
there is a related issue recently reported by @gvagenas
MS,Restcomm,client tool?
at least i attached jprofiler to JBoss
yes
not sure if its related to latest issue with JSP,Jean...
something is going on sure
HTTP-connector-*?
yes, the name o threads will tell wchich component is responsible
in this case is tomcat, but then we need to check the whole E2E scenario, in terms of http client/server
Restcomm calling an external service
@otsakir I believe the problem here is with the version of the apache client you use. The latest apache client that I moved Restcomm to behave much better and provides better methods for resource cleanup
if you check the stack trace youll see is tomcat threads for http traffic
that sounds better George
well, don't know guys, the big number of thread is in SIP sevlets
there are some hundreds of them.
is that normal ?
in mss-sip-stack.properties
it's a thread pool initialized by default
@jaimecasero this is what I was referring to
send your mss-sip-stack.properties
http-/127.0.0.1e coming from org.apache.tomcat.util.net.JIoEndpoint
I think @otsakir is reffering to the UDP SIP Servlets threads
of course we share same JVM with other containers
they are named Restcomm-SIP-Servlets- UDPMessageChannelThread-XXX
those are controlled by
13:29:57,767 ERROR [org.apache.tomcat.util] (http-/127.0.0.1:8080-Acceptor-0) JBWEB003011: Error allocating socket processor: java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method) [rt.jar:1.7.0_79]
at java.lang.Thread.start(Thread.java:714) [rt.jar:1.7.0_79]
at org.apache.tomcat.util.net.JIoEndpoint$Worker.start(JIoEndpoint.java:940) [jbossweb-7.2.0.Final.jar:7.2.0.Final]
at org.apache.tomcat.util.net.JIoEndpoint.newWorkerThread(JIoEndpoint.java:1162) [jbossweb-7.2.0.Final.jar:7.2.0.Final]
at org.apache.tomcat.util.net.JIoEndpoint.createWorkerThread(JIoEndpoint.java:1141) [jbossweb-7.2.0.Final.jar:7.2.0.Final]
at org.apache.tomcat.util.net.JIoEndpoint.getWorkerThread(JIoEndpoint.java:1173) [jbossweb-7.2.0.Final.jar:7.2.0.Final]
at org.apache.tomcat.util.net.JIoEndpoint.processSocket(JIoEndpoint.java:1211) [jbossweb-7.2.0.Final.jar:7.2.0.Final]
at org.apache.tomcat.util.net.JIoEndpoint$Acceptor.run(JIoEndpoint.java:312) [jbossweb-7.2.0.Final.jar:7.2.0.Final]
at java.lang.Thread.start0(Native Method) [rt.jar:1.7.0_79]
at java.lang.Thread.start(Thread.java:714) [rt.jar:1.7.0_79]
at org.apache.tomcat.util.net.JIoEndpoint$Worker.start(JIoEndpoint.java:940) [jbossweb-7.2.0.Final.jar:7.2.0.Final]
at org.apache.tomcat.util.net.JIoEndpoint.newWorkerThread(JIoEndpoint.java:1162) [jbossweb-7.2.0.Final.jar:7.2.0.Final]
at org.apache.tomcat.util.net.JIoEndpoint.createWorkerThread(JIoEndpoint.java:1141) [jbossweb-7.2.0.Final.jar:7.2.0.Final]
at org.apache.tomcat.util.net.JIoEndpoint.getWorkerThread(JIoEndpoint.java:1173) [jbossweb-7.2.0.Final.jar:7.2.0.Final]
at org.apache.tomcat.util.net.JIoEndpoint.processSocket(JIoEndpoint.java:1211) [jbossweb-7.2.0.Final.jar:7.2.0.Final]
at org.apache.tomcat.util.net.JIoEndpoint$Acceptor.run(JIoEndpoint.java:312) [jbossweb-7.2.0.Final.jar:7.2.0.Final]
that is stack trace
it will depend on UDP transport mode
please carry on @gvagenas
@ghjansen any update from you for the Organization work?
@maria-farooq can you update ?
hm actually nothing yet @gvagenas , i just participated in some discussions about Application uniqueness, Organizations and created issues RestComm/RestComm-Connect#1156 and RestComm/RestComm-Connect#1158
@maria-farooq what is the progress on the issue your are working to support multiple RMS server?
yea right, i thought i would update about that in RaaS section, sorry
let me update
ok so i am working on RestComm/Restcomm-Connect#932 and My progress contains: modifying configuration to read multiple media servers from restcomm.xml, make them ready to choose by ControllerFactories.
next step which i am currently working is: to modify call, conference and bridge Controller factories
to be able to choose from available media servers which i converted into a list
currently i am modifying in MMS aka RMS realted controllers, after that will move to external (jsr-309)
not sure we need to modify the JSR309 controller for now, I believe you can stick to the RMS controllers only, @deruelle wdyt?
@maria-farooq when do you will have it ready ?
Well according to plan we should have both #932 and #1154 ready and tested on staging by end of June but if you ask me, answer is unsure to me, i am discovering, learning restcomm and making changes according to task’s requirement, maybe when i am done with #932 i will have a clear picture, wdyt?
@gvagenas well we can modify jsr 309 to keep support for that for dialogic too
or when we later switch RMS to JSR 309
@deruelle I was thinking that Dialogic has different means to support a cluster of media servers and there is no need to provide that at Restcomm side. Maybe we should support multiple jsr309 server after we finish with RMS
to support cluster of Media Servers
I'm fine with avoiding JSR 309 for now
not to forget
if we don't do it now
please provide your comment
while watching a related video i found that twilio has a dedicated DB to keep track of what requests are being fulfilled by what MS
like what call is right now connected to which MS, it is called “registrar"
we will need a similar DB table or modify existing tables, so we know which RMS server serves which call/conference
this is especially needed for conferences so we can connect calls to the same conference to the correct RMS server
true , for Live Call Modifcation we need Call SID and the RMS server
thanks @maria-farooq , lets sync on the issue as you move forward
@deruelle thats all for Restcomm
I have no update on that, I am 100% on the load testing and customer issues troubleshooting, to patch for mem leaks and prepare the new binary
#5 RestComm SSO
worked on #994 (add 'linked' account attribute), #1157 (addition of IdentityInstance entity, dao etc.)
i'm porting graudlly changes from old sso branch to a fork of master
i plan to spend half effort on SSO this month
I guess the merge back will be easier now that no shiro is merged
right ?
and what's left
Thanks @otsakir
@leftyb @maria-farooq can you update ?
so this weekend we updated cloud to latest release, 7.7.0
@leftyb should we mention about upgrade scripts as well?
?
Please explain..
@jjsenay if you build an app you can even record your calls to customers
and then with a little bit of dev
something went wrong with the DB uprgade scripts ?
db upgrade scripts had some problems, and we skipped bcz for this particular release there were no db changes..
Looks like fore some reason we get errors when trying the scritpts to the cloud DBs , either staging or cloud.. (copies of them)
@gvagenas might be worth considering a patch release for 7.7 with mem leaks fixed and DB upgrade scripts fixed
for Telscale
actually only for the DB engine at cloud, same DB same data when testing locally , scripts are working fine
agree @deruelle
1.) On Client project
- Troubleshooting and gave support to the team.
- Participated on some organizational meetings
- Prepared Client RestComm-Docker for Graylog.
- Installed and run Graylog (at the moment monitoring RC01 & RC02 (CPU & MEM & Calls stats for RestCOmm))
- prepared binaries for an alternative RC run.
:D
2.) Raas - Cloud
- Prepared the update of cloud server
- Cloud training with Maria
3.) RestComm
- 1st cleanup and re-factoring of RestComm configuration scripts
PR RestComm/Restcomm-Connect#1161
that is all from me..
Mysql support
Graylog support
USESBC
DTMFDBI
activateAddressMapping (for SMPP)
(S3_BUCKET configuration)
INIT_PASSWORD
SMTP_USER -LOG_LEVEL (fix logs confiuguration) -RESTCOMMHOST
PROD_MODE
load balancer configuration
Graylog support
USESBC
DTMFDBI
activateAddressMapping (for SMPP)
(S3_BUCKET configuration)
INIT_PASSWORD
SMTP_USER -LOG_LEVEL (fix logs confiuguration) -RESTCOMMHOST
PROD_MODE
load balancer configuration
thanks @maria-farooq
I'm glad we take care of that now
before it gets even more insance
but is not a very complicated to integrate. The most important was SSL
let's try to complete that for the next patch release of TelScale restcomm
sure..
when will that be @gvagenas
?
SSL is huge :D Been wasting so much time in latest load runs for that. Congrats!
thanks for confirming @atsakiridis
;)
gireesh is working on the Mayday app
to improve it
along with @ipsilantide
#8 Restcomm SDKs/Clients
@atsakiridis can you update ?
so last week been working on:
- Customer troubleshooting and follow ups for most of the time
- Load testing, seems that gvagenas was able to spot the issue in Restcomm, so once he has I fix I plan to start a new run. Was thinking to do docker this time to check how the whole thing performs there
Just wondering @gvagenas would you like to do a non-docker first to make sure the issue is actually fixed? The problem here is that there runs take too long (1-2 days)
when you need to validate that the fix is working properly
let's do non docker first
and then docker
- Following up with community inquiries as much as possible
A bit on Android PR and some new enquiries on iOS
- Following up with Partners on MayDay and Android-QoS
- Helping Despina in some of her inquiries
- Finally just resumed my work on Android Cellular Data to help partner with their demo
That's all from me @deruelle
#9 Restcomm UI/UX Design
@ipsilantide can you update ?
Last week I was working to finalise things in Olympus mockups for web. I have finished most of the things and now I am waiting to take some feedback so I can make some improvements
Also I have start making research since I will start working on Olympus mock-ups for Android and IOS
I guess that from now on we will be ready to start the implementation of the first release Olympus for web. And may be in parallel I should work as well to create the final graphics for Olympus.
I mean the skin we will use.
Also last week I was working in MayDay the customers side and I have created a first flow of mockups having as priority to simplify the appearance of the client.
Also I have started working on the agents part of the client.
I will sync with Gireesh and Amit from now on and I will make improvement in both sides.
I will sync with Gireesh and Amit from now on and I will make improvement in both sides.
Apart of that last week I spend some of my time t working with and for the marketing team.
I also make some
little but I think interesting changes to the logos
thats from my side
:D
what do you think of the room idea
@ipsilantide ??
Lets find a great name!
let's try to do mockups of that with that name
while you find a new greek name to use
:D
:)
yeah not forums
#10 Media Server
I think we should stick to sth simple, everybody is looking for fancy ways to call such stuff! room or channel I good I think
or discussion
screen share etc
not sure it captures that
room reminds of conference room too
chat room
@hrosa @ghjansen can you update on RMS ?
go @hrosa
I've released a new version of Media Server, mainly containing patches for some concurrency problems
last week i worked on RestComm/mediaserver#111 and created PR RestComm/mediaserver#168
worked on RestComm/mediaserver#163 and created PR RestComm/mediaserver#169
this week i worked on RestComm/mediaserver#144 and created PR RestComm/mediaserver#172
created issue RestComm/mediaserver#171
did few adjustments for PR RestComm/mediaserver#169
worked on RestComm/mediaserver#145 and created PR RestComm/mediaserver#173
and now im working on RestComm/mediaserver#134 and in a customer issue to reduce the time limit of the calls
and this is all from my side
worked on RestComm/mediaserver#163 and created PR RestComm/mediaserver#169
this week i worked on RestComm/mediaserver#144 and created PR RestComm/mediaserver#172
created issue RestComm/mediaserver#171
did few adjustments for PR RestComm/mediaserver#169
worked on RestComm/mediaserver#145 and created PR RestComm/mediaserver#173
and now im working on RestComm/mediaserver#134 and in a customer issue to reduce the time limit of the calls
and this is all from my side
all small issues
#163 was included on version 4.1.1.1
if im not mistaken
what's in store for next week ?
cranking up the complexity of tasks ?
well, i have to finish the list of simple tasks and probably have some talks with @hrosa
if there is no need of a dominatrix he may trust some complexity on my shoulders :P
there will some smaller tasks for next week as well, mostly loose ends
so that we can finally focus on resuming work for new MGCP stack
with no distractions
#11 SIP Servlets/SIP Stack
Gui will also do some study on MGCP and how RestComm uses it to setup calls on MS side
back to school ;)
also reviewed test-suite in CI environment
since it was failing
created a bunch of issues to track progress there, since we have different problems
I need to rework SIPServlet Roadmap
still not ready
when do you plan to rework the roadmap for at least upcoming release ?
when do you plan the next release ?
got good feedback on PerfCorder from the team, specially Henrique, so i iwll deliver some enhancements there too
well, roadmap for upcoming release should be ready this week, unless more things happens
next release, my intention was to list and estimate main points ( support EAP7, replciation,sipServlet2.0) and decide what we want
depending what we want to include may be 3 months or more
lets wait for it to be ready, then we may dicuss dates
if that is what you ask for
again, we should target at 1-3 moneths release
?
your call
you're the project lead ;)
ping me when ready with roadmaps
so we can brainstorm
#12 jSS7/SMSC/USSD
@vetss @nhanth87 @f-alonso-vendrell can you update ?
JSS7:
1) we have just released a first community release for JSS7 stack and resource adaptors (RAs)
and corresponded JASN and SCTP releases:
JASN 2.1.0
SCTP 1.7.0
JSS7 7.0.0
JSS7 RA 7.0.0
long time since the last release of jSS7
Also I need to announse that I will rename soon "master" branch to "non-netty" branch and "netty-2" branch to "master"
we are starting of preparing of a next iteartion in JSS7 stack life - adding support for netty stack at SCTP level and congestion control
that from now on the new master is going to be netty based
we need a specail announsment I think
ok, let me continue
for JSS7 stack:
2) I made some discovering of a dialogic board part code for may be we do not proper process incoming messages
I have added "GctLib.relm(gctmsg);" for incoming messages from a board
3) accepted community pull requests:
- adding anyTimeSubscriptionInterrogation MAP operation
- adding XML serializing into 3 CAP operations
- adding a new CAP operation splitLeg
- updates for ISUP RA
- gmlc update - bug with wrong SCCP address creating
- smsc gw - decrease of logging level at one of the place
SMSC GW:
I am preparing SMSC GW release 6.2.3.GA
this is a Telscale release
This release will cover many enchansements for last half year
There will not be correspondent github release for this Telscale release
We will make public release little bit later
I'm glad now with CI/CD on SS7 side
this should be easier too
@nhanth87 can you update ?
this week I have many support activities so I dont have any update :(
in turn
@nhanth87 ok :(
@f-alonso-vendrell still there ?
can you update ?
:)
yes, my updates.
we can try that next week
right after #1
as it is tied to JAIN SLEE as well
I have made progress on the simulator, I created the form and setted up the outline.
basically 1) hardcode operation parameters on SRIforLCS, test end2end (so I would need also SERVER Part), then pull request
I am to the point on actually making the request message and sending it. Lots of copy/paste from existing code.
Additionally I wanted to request a feature on the Simulator, that would be a "Load Default Values for Side A/B" at top level, so once you have selected Layer 1, Layer 2, Layer 3 and TestTask you click that button and all layers get populated at once.
I dont know why prev message is pink.
Load Default Values for Side A/B - is not important for your concrete testing task, do them same
maybe create a separate issue for that
as enhancement
so this is tracked and committed separately
it will be easier to review your pull requests
I agree, this was just a suggestion, nothing to do right now.
I expect to have the PR by next wed, given the availability we have. BTW one of my team member is already getting up to speed with this, so we will divide the work.
and that's all from me
pink keeps showing, dont know if you read me.
I do read you
ok great
thanks @f-alonso-vendrell for the update
looking forward to the first PR
#13 GeoLocation/GMLC
@FerUy can you update ?
yes... well, in the meantime Rescomm Geolocation API is on hold, squeezing my head with the GMLC right now
also exchanged info with @f-alonso-vendrell about the task we just mentioned for the jSS7 simulation of location in 3G networks
when do you think you can turn the CI job blue again https://mobicents.ci.cloudbees.com/job/RestComm-GMLC/ ?
I think that would be a good first step in bringing things back on track
and with @vetss also, regarding the MAP operations needed, as not only SRIforLCS and PSL will be useful, but subscriberLocationReport for notifcation type of Geolocation (but that's for a later stage)
before we add anything more
to make it blue you neet to point GMLC to a released JSS7 as a first action @FerUy
are already setup
let's execute on them now
well, that's what I was referring to with squeezing my head... yes, of course, that's why I said for a later stage
it doesn't compile anymore ?
instead of using an old release ?
but @vetss is right
use that
it's pointless to debug old stuff
that were not stable anyway
I would love to get it ready asap, just need some days to come with an answer on that regard Jean
#14 jDiameter/Charging Server
due to time zones
ok thanks everyone
that's a wrap up
sorry for the long one today
team is growing :D
Reply all
Reply to author
Forward
0 new messages