Unit test nucleus mme

[heavyd]

Hello guys im working on deployement.

Can you guys give me any pointers on how to unit test nucleus mme first.

 

thank you

[Ajay Thakur]

HI,

Nucleus (mme) is matured and works well in the live network. We do interop with multiple real eNodeB.

We are in the process of getting UT framework for multiple omec projects including Nucleus. Would you

be interested in proposing/developing/discussing the UT framework for nucleus ?

Thanks

Ajay

--
You received this message because you are subscribed to the Google Groups "OMEC Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to omec-dev+u...@opennetworking.org.
To view this discussion on the web visit https://groups.google.com/a/opennetworking.org/d/msgid/omec-dev/efde2db7-ecc9-4cae-90be-f535b8200457n%40opennetworking.org.

[Harald Welte]

Hi Ajay,

On Fri, Jan 29, 2021 at 08:08:10AM -0800, Ajay Thakur wrote:

> We are in the process of getting UT framework for multiple omec projects
> including Nucleus. Would you
> be interested in proposing/developing/discussing the UT framework for
> nucleus ?

I would like to strongly recommend to consider TTCN-3 based testing for your
network elements.

TTCN-3 is a domain-specific programming language specifically designed by ITU and
ETSI to perform all kinds of protocol and conformance testing. It has a number
of quite unique capabilities. It is used heavily within the telecom industry,
including Ericsson and Nokia.

There's an open source TTCN-3 compiler (Eclipse TITAN) which runs on all major
platforms (Windows, Linux, ...) and which is also part of Linux distributions
like Debian for quite some time.

You can find some background on TTCN-3 and TITAN at:

* https://projects.eclipse.org/projects/tools.titan
* http://www-public.imtbs-tsp.eu/~maag/P7/TTCN3_P.pdf
* https://www.youtube.com/watch?v=GSosAKreMBI (video)
** https://netdevconf.info/2.2/papers/welte-ttcn3-talk.pdf (paper)
** https://netdevconf.info/2.2/slides/welte-ttcn3-talk.pdf (slides)

One of the nice parts about TITAN is that the also provide encoders/decoders
for all major telecom protocols - whether (in your context) LTE NAS or DIAMETER.
Testing against such a third-party implementation created by (most often) Ericsson
is much more useful than testing back-to-back against your own implementation.
See e.g.
* https://git.eclipse.org/c/titan/titan.ProtocolModules.NAS_EPS_15.2.0.1.git/
* https://git.eclipse.org/c/titan/titan.ProtocolModules.GTPv2_v15.2.0.git/

We use TTCN-3 quite heavily at Osmocom to test all our 2G/3G network elements,
such as BTS, BSC, MSC, HLR, SGSN, GGSN, etc. See
http://osmocom.org/projects/cellular-infrastructure/wiki/Titan_TTCN3_Testsuites

The fundamental approach is that you emulate all the surrounding network elements
of your implementation under test. For the MME, that would be emulation of eNB,
S-GW, HSS, but also MSC (SGs), CBC (SBc) or whatever other external interface you have.

You then implement test cases that test individual procedures, both for succesful
and unsuccessful cases. TITAN can generate junit-xml and be executed in jenkins,
see for example the results for the Osmocom BSC:
https://jenkins.osmocom.org/jenkins/view/TTCN3/job/ttcn3-bsc-test/test_results_analyzer/
[click on 'expand all']

Some time ago I started a very early MME test suite, whose code can be found at
https://git.osmocom.org/osmo-ttcn3-hacks/tree/mme
as well as e.g. some PGW tests at
https://git.osmocom.org/osmo-ttcn3-hacks/tree/pgw

Those mme/pgw are very early 'proof of concept' test suites, as the focus in
Osmocom still is primarily on 2G technology, as that's where most of the
existing commercial user base is paying us for developing tests suites.

Let me know if you have any related questions.

Regards,
Harald
--
- Harald Welte <laf...@gnumonks.org> http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
(ETSI EN 300 175-7 Ch. A6)

[heavyd]

Thanks 

Im busy with some other work as well.

Ill update our testing team with this.

[Ajay Thakur]

HI Harald,

Thanks for detailed response. I will go through it and get back to you. Is this UT framework or generic testing framework ?

Thanks

Ajay

[Harald Welte]

Hi Ajay and OMEC community,

On Mon, Feb 01, 2021 at 06:43:43AM -0800, Ajay Thakur wrote:
> Thanks for detailed response. I will go through it and get back to you. Is
> this UT framework or generic testing framework ?

It depends a bit on the terminology.

Unit testing to me is typically considered about testing individual functions
from libraries etc. inside your project. This is not what TTCN-3 and TITAN
are intended for. you could possibly do it (TITAN can link C/C++ code),
but it's not a natural fit.

I am not an expert in related terminology, but I would typically call the
kind of testing that TITAN/TTCN-3 can do well as 'functional integration testing'.
Depending on your requirements, it may also be possible to do performance testing,
but reallly it's the integration part.

* you run one complete, unmodified network element (e.g. MME) as IUT
* your tester attaches to all of its external interfaces (S1, GTP, DIAMETER, ...)
* your tester simulates the 'rest of the world' around the IUT, e.g. in the above
example the minimally required parts of UE, ENB, HSS, SGW, ... for the related test

This type of testing focuses on what typically matters most in 3GPP type use casese:
The externally visible behavior on the 3GPP-specified interfaces. One would
normally implement tests for individual procedures, in different variations,
both successful and erroneous/timeout situations.

btw: TTCN-3 is used for all the ETSI/3GPP UE conformance testing, too. There
are large amounts of TTCN-3 test suites released for that purpose. However,
they are very much air interface centric and require a lot of integration
with the so-called 'system emulator' (emulating the RAN+CN), and hence I don't
think there's much to be learned from them for eNB/EPC/CN testing.

Unfortunately neither 3GPP nor ETSI make available any test suites for the
network side. I once saw some very simplistic eNB tests being released, but
nothing really worth mentioning. It seems the various vendors don't have
an interest to disclose their actual tests suites. Only the TTCN-3 compilers
and the encoders/decoders for the various protocols are released in TITAN.