Setting Up Aether-in-a-Box

[Pedro Rito]

Hi all,

We are starting with the Aether project.

First step we are trying is to set up Aether-in-a-Box.

Our machine is Ubuntu 18.04, 4 cores, 8GB RAM and 64GB storage.

We run make roc-5g-models without problems but when we run make 5g-test we have the following error:

Release "5g-core-up" does not exist. Installing it now. Error: timed out waiting for the condition

And the output of kubectl -n omec describe pods upf-0 is:

Normal Created 9m14s (x2 over 9m17s) kubelet Created container bessd Normal Pulled 9m14s (x2 over 9m17s) kubelet Container image "registry.aetherproject.org/proxy/omecproject/upf-epc-bess:master-103b198" already present on machine Normal Started 9m13s (x2 over 9m17s) kubelet Started container bessd Normal Killing 9m13s (x2 over 9m17s) kubelet FailedPostStartHook Warning FailedPostStartHook 9m13s kubelet Exec lifecycle hook ([/etc/bess/conf/bessd-poststart.sh]) for Container "bessd" in Pod "upf-0_omec(f1756faf-a96f-428c-89d9-258131a28a57)" failed - error: command '/etc/bess/conf/bessd-poststart.sh' exited with 126: , message: "cannot exec in a stopped state: unknown\r\n"

Could you help us interpreting what is the issue in this step?

Thank you,

Pedro Rito

[Andy Bavier]

Hi Pedro,

I’m not sure why the upf-0 pod would be unable to successfully start.  Can you share more details about how you are invoking the “make 5g-test” target?  For example, are you specifying “CHARTS=latest make 5g-test” to install from the latest published Helm charts?

Andy

--
You received this message because you are subscribed to the Google Groups "Aether-Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aether-dev+...@opennetworking.org.
To view this discussion on the web visit https://groups.google.com/a/opennetworking.org/d/msgid/aether-dev/5b202cf5-aa1e-4baa-aafb-113a617d8e59n%40opennetworking.org.

[Pedro Rito]

Hi Andy,

Thanks for your answer. We tried both "CHARTS=latest make 5g-test" and "CHARTS=release-1.6 make 5g-test", having always the same issue.

Is there anything else we can check to try to understand the problem?

Pedro Rito

[Andy Bavier]

Hi Pedro,

Can you see any logs produced by the bessd container in the upf-0 pod?

$ kubectl -n omec logs upf-0 bessd

Andy

[Pedro Rito]

Hi Andy,

This is the output:

Error from server (BadRequest): container "bessd" in pod "upf-0" is waiting to start: PodInitializing

Thanks,

Pedro

[Andy Bavier]

OK, thanks.  Looks like it’s not able to start at all.

Can you send me the entire output of “kubectl -n omec describe pod upf-0”?

Andy

[Stevens, Craig]

Without the detailed upf-0 logs it's probably hard to tell, but I did have the same problem when I ran it all inside a virtual machine.  Basically the UPF-0 container is running DPDK libraries and needs certain flags in order to start, in my case it needed the F16C cpu flag and that wasn't being passed through in the hypervisor that I was using (i.e. VirtualBox wasn't passing it thru).  I switched to running libvirt with a KVM hypervisor and this worked as the F16C cpu flag then got passed through and UPF-0 container was able to start.

As I say not sure if this is your problem, but given you're install is having issues starting UPF-0 it could be some kind of DPDK problem like mine.  Look at the UPF-0 logs and as per Andy's email, it should give you hint as to why UPF-0 is failing.

regards,

craig

---

From: Andy Bavier <an...@opennetworking.org>
Sent: Thursday, 13 January 2022 11:36 AM
To: Aether-Dev <aethe...@opennetworking.org>; Pedro Rito <pedr...@av.it.pt>
Subject: Re: [aether-dev] Setting Up Aether-in-a-Box

 

[EXTERNAL EMAIL]

Hi all,

Normal Created 9m14s (x2 over 9m17s) kubelet Created container bessd Normal Pulled 9m14s (x2 over 9m17s) kubelet Container image "registry.aetherproject.org/proxy/omecproject/upf-epc-bess:master-103b198 [registry.aetherproject.org]" already present on machine Normal Started 9m13s (x2 over 9m17s) kubelet Started container bessd Normal Killing 9m13s (x2 over 9m17s) kubelet FailedPostStartHook Warning FailedPostStartHook 9m13s kubelet Exec lifecycle hook ([/etc/bess/conf/bessd-poststart.sh]) for Container "bessd" in Pod "upf-0_omec(f1756faf-a96f-428c-89d9-258131a28a57)" failed - error: command '/etc/bess/conf/bessd-poststart.sh' exited with 126: , message: "cannot exec in a stopped state: unknown\r\n"

Could you help us interpreting what is the issue in this step?

Thank you,

Pedro Rito

--
You received this message because you are subscribed to the Google Groups "Aether-Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aether-dev+...@opennetworking.org.

To view this discussion on the web visit https://groups.google.com/a/opennetworking.org/d/msgid/aether-dev/5b202cf5-aa1e-4baa-aafb-113a617d8e59n%40opennetworking.org [groups.google.com].

--
You received this message because you are subscribed to the Google Groups "Aether-Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aether-dev+...@opennetworking.org.

To view this discussion on the web visit https://groups.google.com/a/opennetworking.org/d/msgid/aether-dev/6abfd1d6-9347-41ae-9a25-ce50f067ae5e%40Spark [groups.google.com].

Internal Use - Confidential

[Pedro Rito]

Hi Stevens,

It seems that was really the problem. We were working with a VM in an older machine. We migrated now to a newer machine, the CPU has the f16c flag and we overcame the issue.

Thank you for the support,

Pedro Rito

[Stevens, Craig]

Great news, happy I could help!!

Get Outlook for iOS

---

From: Pedro Rito <pedr...@av.it.pt>
Sent: Thursday, January 20, 2022 10:02:06 PM
To: Aether-Dev <aethe...@opennetworking.org>
Cc: Stevens, Craig <Craig_...@Dell.com>; an...@opennetworking.org <an...@opennetworking.org>

Internal Use - Confidential