5G CN elements not healthy

[sayaz...@gmail.com]

Hello Powder team,

I am using oai-indoor-ota profile for 5G indoor OTA lab experiment.

After login in to all nodes (CN/gNB/UEs), to check if the core-network elements are up and healthy, I do get negative results (errors attached). I have rebooted the CN node couple of time, but it did not change the scenario. Can you please let me know what should be the next troubleshooting steps? 

Thank you,

Ayaz

[Kirk Webb]

Is this immediately after instantiating the experiment, or later on
after working with the experiment, restarting services, and/or making
configuration changes?

> --
> You received this message because you are subscribed to the Google Groups "Powder Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to powder-users...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/powder-users/dec401a8-f2aa-44fe-bba1-4115aa2860een%40googlegroups.com.

[sayaz...@gmail.com]

Hi Kirk,

It worked fine the very first time. The problem started from 2nd experiment.

Ayaz

[sayaz...@gmail.com]

Note: No config change was made. 

[Kirk Webb]

To be clear, do you mean you started having issues when you
instantiated a new POWDER experiment using the same profile, or do you
mean the second time you restarted the OAI services within the same
POWDER experiment?

One quick suggestion: Try rebooting the CN node. It's possible that
some setup script did not run to completion. In general, this is
something to watch out for on a newly-instantiated experiment; the
nodes may be "ready", but setup scripts may still be running (the
experiment view on the web UI will show you this). Be sure to wait for
the setup scripts to complete.

-Kirk

> To view this discussion on the web visit https://groups.google.com/d/msgid/powder-users/96191dee-0b9a-4253-8255-7ca4fb5ee3fdn%40googlegroups.com.

[sayaz...@gmail.com]

The very first time I instantiated a new experiment, the CN elements showed healthy, except one of the nuc(4) did not came out from airplane-mode. Also I did not save the trace. (I followed the steps from the video tutorial by Dustin).

Without making any configuration change for any node, I exited from all the process (ctrl + z) running in each nodes, and re-strated doing the same experiment (using same commands for each node). Only change I did is instead of tshark, I did tcpdump to save the trace in a file. At this point I got health error on the elements and rebooted only the CN node.

Waited till the node came 'green' and the status 'ready'. Waited for another 5~8 min. Re-run the experiment, but the error persist.

[Kirk Webb]

I'm not familiar with the Docker environment, and our expert on this
is out of the office for the holidays. If you rebooted the CN node
(this is not clear to me), and that did not help, the only other
suggestion I have is to terminate the POWDER experiment entirely, and
re-instantiate a new one.

> To view this discussion on the web visit https://groups.google.com/d/msgid/powder-users/558ac8a4-51f9-40dc-b254-74f0c5339806n%40googlegroups.com.

[sayaz...@gmail.com]

Hi,

I had a discussion with Dustin. It is now solved. The network elements was not terminated/killed correclty.

Thanks

Ayaz