ser-Plane Connectivity Issue (Ping/Data Path Failure) in Distributed 3-Node 5G SA Setup

[Yasir Nawaz]

Hello Powder Community,

We are currently building a 3-node 5G testbed for research on NexRAN slicing and closed-loop PRB allocation, but we have hit a persistent blocker regarding user-plane connectivity (ping failure).

Our Setup

• Profile: O-RAN (PowderProfiles/O-RAN)

• Hardware: 3 nodes, d430, 10Gb/s LAN

• Node-0 (10.10.1.1): Open5GS Core + Near-RT RIC (J-release) + NexRAN xApp

• Node-1 (10.10.1.2): srsRAN_Project gNodeB (commit e5d5b44b92)

• Node-2 (10.10.1.3): srsRAN_4G srsUE (ZeroMQ RF bridging)

Current Status

The Control Plane is working perfectly. The UE successfully attaches, and the AMF logs confirm full registration:

• SUPI: imsi-999990123456789

• DNN: internet | S-NSSAI: SST:1 SD:0x1

• UE Interface: tun_srsue receives IP 10.45.0.3/24

The Problem

We cannot get any data through the user plane. A ping from the UE namespace to the core gateway fails with 100% packet loss: sudo ip netns exec ue1 ping -c 4 10.45.0.1

What We Have Tried

1. UPF Config: Added 10.10.1.1 to the gtpu and pfcp server sections in upf.yaml. Confirmed UPF is listening on 10.10.1.1:2152.

2. Routing: Verified the ogstun interface is UP on Node-0 (10.45.0.1/16) and an iptables MASQUERADE rule exists. Added a route for 10.45.0.0/16 via 10.10.1.1 on Node-1.

3. SMF Attempt: We tried changing the SMF upf address to 10.10.1.1, but this resulted in a PDU Session Establishment Reject (HTTP 404). We reverted it to 127.0.0.7.

Our Hypothesis

We suspect the SMF is advertising the localhost address (127.0.0.7) to the gNodeB as the UPF's N3/GTP-U endpoint. Since the gNodeB is on a separate physical node (Node-1), it cannot route GTP-U traffic to a localhost address on Node-0.

Specific Questions

1. How do we configure smf.yaml or upf.yaml to ensure the SMF advertises the LAN IP (10.10.1.1) to the gNodeB for the N3 interface while keeping internal PFCP communication on localhost?

2. Is there a specific "advertise" parameter in Open5GS for distributed setups where the gNodeB and Core are on different physical nodes?

3. Are there any POWDER-specific MTU or routing nuances we should be aware of when bridging ZMQ over the 10.10.x.x network?

Any insights or example configuration files for a distributed Open5GS + srsRAN setup would be greatly appreciated!

Best regards,

Yasir Nawaz

[Dustin Maas]

Thank you for the detailed message in the other thread! (Just returned from a conference and had time to take a look.)

I have once again fixed the ue side zmq config. And I disabled the RIC connection to test because it doesn’t appear to currently be running (or that part of the current gnb config is incorrect).

I think the confusion here is partly around the use of a separate network namespace for the UE… you don’t need that when the UE, gNB, and core network are running on separate nodes, so I removed that bit from the ue config. The UE now attaches and pinging 10.45.0.1 from the gateway now works.

I’ve killed my test processes. So, back over to you guys.

-Dustin

--
You received this message because you are subscribed to the Google Groups "Powder Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to powder-users...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/powder-users/3ca18a04-84a0-428f-bc16-742dc7327ea7n%40googlegroups.com.

[Yasir Nawaz]

Hello Powder Community,

Following up on our previous thread where Dustin helped us resolve the user-plane connectivity issue ,  thank you again for that fix! We now have a stable 3-node 5G testbed with the UE attaching successfully and ping working through the tunnel.

We have moved on to the next goal: enabling NexRAN closed-loop RAN slicing, and have hit a new blocker.


Our Setup (unchanged from before)

- Node-0 (10.10.1.1): Open5GS Core + Near-RT RIC (J-release) + NexRAN xApp v2.0.0
- Node-1 (10.10.1.2): srsRAN_Project gNodeB (commit e5d5b44b92, version 24.10.0)
- Node-2 (10.10.1.3): srsRAN_4G srsUE over ZMQ



Current Status

The basic 5G path is fully working:
- UE attaches, gets IP 10.45.0.x on tun_srsue
- Ping from UE to core gateway (10.45.0.1) works
- iperf3 through the tunnel gives ~25 Mbps downlink
- E2 interface: gNB connects to Near-RT RIC successfully on 10.10.1.1:32222 (NodePort 32222 mapped to E2 term SCTP 36422)
- NexRAN xApp shows gNB as CONNECTED
- Two slices (fast: SST:1 SD:1, slow: SST:1 SD:2) created and bound to gNB successfully



The Problem

When we try to bind a UE to a slice, the xApp returns:

  cannot bind slice to ue on node gnbd_999_099_00019b_0: cannot find ran function id!

And when we try to bind slices to the gNB via the REST API:

  {"errors":["node does not support rc/nexran e2sm"]}

The gNB advertises these RAN function OIDs over E2:
- 1.3.6.1.4.1.53148.1.2.2.2 (KPM)
- 1.3.6.1.4.1.53148.1.1.2.3 (NR RC)

The NexRAN xApp appears to expect a different OID for RC — likely the NexRAN-specific custom E2SM, not the standard NR RC OID (.3).



What We Have Tried

1. Re-registered the gNB in the xApp with OID 1.3.6.1.4.1.53148.1.1.2.2 (LTE RC) instead of .3 — still got "node does not support rc/nexran e2sm"
2. Checked /local/setup/srsRAN_Project/build/apps/gnb/gnb — same commit e5d5b44b92 as system gnb, so the custom NexRAN-patched binary does not appear to be built
3. The run-nexran-slicing-5G.sh script runs successfully and shows CONNECTED, but slice-to-UE binding fails due to the OID mismatch


Specific Questions

1. Does the NexRAN xApp v2.0.0 require a custom-patched srsRAN build with a NexRAN-specific E2SM OID? If so, where can we find that patched binary or source?
2. Is there a working combination of srsRAN_Project commit + NexRAN xApp version that supports closed-loop PRB slicing on the 5G (NR) path?
3. Should we be using the LTE path (srsenb + run-nexran-slicing-4G.sh) instead for slicing demos, and if so can we still use the 5G core (Open5GS) or do we need to revert to srsEPC?
4. Are there any POWDER-specific setup steps for the NexRAN 5G slicing path that are not covered in the demo scripts?

Our ultimate goal is a stable multi-UE 5G testbed with closed-loop PRB slicing controlled by the NexRAN xApp, with the UE, gNB, and core on separate nodes.

Any guidance would be greatly appreciated!

Best regards,
Yasir
Powder profile login:  mbasit

[David M Johnson]

On 5/21/26 21:24, Yasir Nawaz wrote:
> Hello Powder Community,
>

When using NexRAN v2 with 5G and RC service model support, you should
not bind UEs to slices via the API. NexRAN v2 uses the RC service model
to configure slices by their NSSAI, and UEs are admitted to slices via
the 5G core. The binding of UEs to slices is an LTE-only workaround.

> And when we try to bind slices to the gNB via the REST API:
>
>   {"errors":["node does not support rc/nexran e2sm"]}
>
> The gNB advertises these RAN function OIDs over E2:
> - 1.3.6.1.4.1.53148.1.2.2.2 (KPM)
> - 1.3.6.1.4.1.53148.1.1.2.3 (NR RC)
>
> The NexRAN xApp appears to expect a different OID for RC — likely the
> NexRAN-specific custom E2SM, not the standard NR RC OID (.3).

No, if you grep through the code, you will see that we expect the same
RC OID as you cite above:

./lib/e2sm/include/e2sm_rc_v6.h:18:#define E2SM_RC_OID
"1.3.6.1.4.1.53148.1.1.2.3"

This is the same OID used by srsRAN_Project and OCUDU.

> What We Have Tried
>
> 1. Re-registered the gNB in the xApp with OID 1.3.6.1.4.1.53148.1.1.2.2
> (LTE RC) instead of .3 — still got "node does not support rc/nexran e2sm"

The 1.3.6.1.4.1.53148.1.1.2.2 OID is the KPM service model, and the
srsRAN gNB should already be advertising that OID for the KPM ran
function in its E2Setup request.

> 2. Checked /local/setup/srsRAN_Project/build/apps/gnb/gnb — same commit
> e5d5b44b92 as system gnb, so the custom NexRAN-patched binary does not
> appear to be built

We do not provide a custom patch for srsRAN_Project, since it has native
E2 support and 5G slicing. In the past we added an E2 agent to srsLTE
(http://gitlab.flux.utah.edu/powderrenewpublic/srslte-ric), and you can
still use NexRAN v1 with that if needed. But it sounds like you are
trying to stand up a gNB.

> 3. The run-nexran-slicing-5G.sh script runs successfully and shows
> CONNECTED, but slice-to-UE binding fails due to the OID mismatch

The run-nexran-slicing-5G.sh script does not attempt slice-to-UE
binding; only the run-nexran-slicing-4G.sh does that.

> Specific Questions
>
> 1. Does the NexRAN xApp v2.0.0 require a custom-patched srsRAN build
> with a NexRAN-specific E2SM OID? If so, where can we find that patched
> binary or source?

No, if you try out the POWDER O-RAN profile, and run through the demo
instructions, that uses srsRAN_Project and NexRAN v2, and works fine.
You can also try the most recent OCUDU release, but the OCUDU E2 RC
implementation changed slightly to require the dedicated_prb_ratio RAN
parameter, and we haven't yet found a valid value that can be sent that
does not result in a complete loss of connectivity on the PDU session...
so we have not made OCUDU the default in our profile, pending resolution.

> 2. Is there a working combination of srsRAN_Project commit + NexRAN xApp
> version that supports closed-loop PRB slicing on the 5G (NR) path?

No, we did not add support for the closed-loop PRB slicing in v2 via the
RC service model. That remains an LTE only feature with our custom
service model.

> 3. Should we be using the LTE path (srsenb + run-nexran-slicing-4G.sh)
> instead for slicing demos, and if so can we still use the 5G core
> (Open5GS) or do we need to revert to srsEPC?

If you want to use the closed loop policies, yes, you must use LTE and
the 4G script.

No, you can continue using Open5GS. If you look at the POWDER O-RAN
profile, you will see instructions for also running srsLTE and the 4G
slicing script. If you select the profile parameter to install srsLTE,
we make sure the installed Open5GS is setup for the eNB. That is
something we tested when we updated the profile to 5G as the default,
but have not checked since. I don't expect a problem, but YMMV.

> 4. Are there any POWDER-specific setup steps for the NexRAN 5G slicing
> path that are not covered in the demo scripts?

No, those scripts are not POWDER-specific, other than using the
get-env.sh script to discover k8s service endpoints.

> Our ultimate goal is a stable multi-UE 5G testbed with closed-loop PRB
> slicing controlled by the NexRAN xApp, with the UE, gNB, and core on
> separate nodes.

In this case, you'll have to "port" the implementation of the
closed-loop policies to the RC path in the xApp. We did not do this
because we were more interested in pushing in the ML direction for
future closed-loop policies.

> Any guidance would be greatly appreciated!
>
> Best regards,
> Yasir

David

> Powder profile login:  mbasit
>
> On Thursday, May 14, 2026 at 11:57:12 PM UTC-5 Dustin Maas wrote:
>

> __

>
> Thank you for the detailed message in the other thread! (Just
> returned from a conference and had time to take a look.)
>
> I have once again fixed the ue side zmq config. And I disabled the
> RIC connection to test because it doesn’t appear to currently be
> running (or that part of the current gnb config is incorrect).
>
> I think the confusion here is partly around the use of a separate
> network namespace for the UE… you don’t need that when the UE, gNB,
> and core network are running on separate nodes, so I removed that
> bit from the ue config. The UE now attaches and pinging 10.45.0.1
> from the gateway now works.
>

> I’ve killed my test processes. So, back over to you guys.
>
> -Dustin
>
> On 14 May 2026, at 21:45, Yasir Nawaz wrote:
>
> Hello Powder Community,
>
> We are currently building a 3-node 5G testbed for research on
> NexRAN slicing and closed-loop PRB allocation, but we have hit a
> persistent blocker regarding user-plane connectivity (ping failure).
>

> *Our Setup*
>
> *
>
> *Profile:* O-RAN (PowderProfiles/O-RAN)
>
> *
>
> *Hardware:* 3 nodes, d430, 10Gb/s LAN
>
> *
>
> *Node-0 (10.10.1.1):* Open5GS Core + Near-RT RIC (J-release)
> + NexRAN xApp
>
> *
>
> *Node-1 (10.10.1.2):* srsRAN_Project gNodeB (commit e5d5b44b92)
>
> *
>
> *Node-2 (10.10.1.3):* srsRAN_4G srsUE (ZeroMQ RF bridging)
>
> *Current Status*

>
> The Control Plane is working perfectly. The UE successfully
> attaches, and the AMF logs confirm full registration:
>

> *
>
> *SUPI:* imsi-999990123456789
>
> *
>
> *DNN:* internet | *S-NSSAI:* SST:1 SD:0x1
>
> *
>
> *UE Interface:* tun_srsue receives IP 10.45.0.3/24
> <http://10.45.0.3/24>
>
> *The Problem*

>
> We cannot get any data through the user plane. A ping from the
> UE namespace to the core gateway fails with 100% packet loss:
> sudo ip netns exec ue1 ping -c 4 10.45.0.1
>

> *What We Have Tried*
>
> 1.
>
> *UPF Config:* Added 10.10.1.1 to the gtpu and pfcp server

> sections in upf.yaml. Confirmed UPF is listening on

> 10.10.1.1:2152 <http://10.10.1.1:2152>.
>
> 2.
>
> *Routing:* Verified the ogstun interface is UP on Node-0
> (10.45.0.1/16 <http://10.45.0.1/16>) and an iptables

> MASQUERADE rule exists. Added a route for 10.45.0.0/16

> <http://10.45.0.0/16> via 10.10.1.1 on Node-1.
>
> 3.
>
> *SMF Attempt:* We tried changing the SMF upf address to

> 10.10.1.1, but this resulted in a PDU Session Establishment
> Reject (HTTP 404). We reverted it to 127.0.0.7.
>

> *Our Hypothesis*

>
> We suspect the SMF is advertising the localhost address
> (127.0.0.7) to the gNodeB as the UPF's N3/GTP-U endpoint. Since
> the gNodeB is on a separate physical node (Node-1), it cannot
> route GTP-U traffic to a localhost address on Node-0.
>

> *Specific Questions*
>
> 1.

>
> How do we configure smf.yaml or upf.yaml to ensure the SMF

> advertises the *LAN IP* (10.10.1.1) to the gNodeB for the N3

> interface while keeping internal PFCP communication on
> localhost?
>

> 2.

>
> Is there a specific "advertise" parameter in Open5GS for
> distributed setups where the gNodeB and Core are on
> different physical nodes?
>

> 3.

>
> Are there any POWDER-specific MTU or routing nuances we
> should be aware of when bridging ZMQ over the 10.10.x.x network?
>
> Any insights or example configuration files for a distributed
> Open5GS + srsRAN setup would be greatly appreciated!
>

> *Best regards,*

>
> Yasir Nawaz
>
> --
> You received this message because you are subscribed to the
> Google Groups "Powder Users" group.
> To unsubscribe from this group and stop receiving emails from
> it, send an email to powder-users...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/
> powder-users/3ca18a04-84a0-428f-

> bc16-742dc7327ea7n%40googlegroups.com <https://
> groups.google.com/d/msgid/powder-users/3ca18a04-84a0-428f-
> bc16-742dc7327ea7n%40googlegroups.com?
> utm_medium=email&utm_source=footer>.

>
> --
> You received this message because you are subscribed to the Google
> Groups "Powder Users" group.
> To unsubscribe from this group and stop receiving emails from it, send

> an email to powder-users...@googlegroups.com <mailto:powder-
> users+un...@googlegroups.com>.

> To view this discussion visit https://groups.google.com/d/msgid/powder-

> users/5e765c2b-6226-49b1-af62-54704b4ccbfdn%40googlegroups.com <https://
> groups.google.com/d/msgid/powder-users/5e765c2b-6226-49b1-
> af62-54704b4ccbfdn%40googlegroups.com?utm_medium=email&utm_source=footer>.