BlueField image management

[Tom Barbette]

Hi all,

On the r7525 there are some NVIDIA BlueField connected, which are ARM-based SmartNICs. Those NICs can be booted with different images.

It seems according to what machines I connect to, the BlueField is in different state. Sometimes the default login ubuntu/ubuntu works (ouch), sometimes I'm left with a previous user's image I think because no login works. Once I installed the NVIDIA Doca and could define my own login, but this takes hours...

I don't think CloudLab should develop a way to handle this special case but we would benefit from sharing user experience :) At least I would like to know how other BlueField users manage their images. What is your efficient setup to get your experiment ready as fast as possible? Especially that the install.bfb image is now buried in this huge mostly useless doca package?

Thanks,

Tom

[Wei Jin]

Hi Tom,

Sorry that I can't answer your questions. I'm new to CloudLab and also using r7525 recently. 

I want to install NVIDIA DOCA but SDK Manager reports that there isn't enough system root disk space (35GB required, but only 12B provided by CloudLab). Have you encountered the same problem, and how did you solve it?

(Manual installation with bfb image is faster, but requires extra efforts to configure the environment, so I'm trying to install NVIDIA DOCA with SDK Manager.)

Thanks,

Wei

[Mike Hibler]

We do not currently re-image the Bluefield between experiments, so you do
need to re-image it yourself to get a known starting point.

While there is only 16GB in the root filesystem, those nodes have a couple
of 1TB drives each that you can use for local storage. You could also load
the DOCA software in an iSCSI-based remote dataset so you do not have to
download it every time you create a new experiment. See the storage section
of the manual:

http://docs.cloudlab.us/advanced-storage.html

which has links to example profiles.

> --
> You received this message because you are subscribed to the Google Groups
> "cloudlab-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email
> to cloudlab-user...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/
> cloudlab-users/c26d5305-07c1-4b88-bdcf-41270e7bdb66n%40googlegroups.com.

[Tom Barbette]

Actually it seems DOCA is a bit of a bloatware... You just need the Mellanox OFED package, install the rshim driver, and you're good to go to access the BF2 and launch your own e.g. DPDK-based software. Before that I used a supplementary disk as proposed in Mike's link.