How to obtain a shared storage between nodes

[CALEB FONYUY-ASHERI S]

Hello,

I am carrying out some experiments on migration of virtual machines and need a shared storage accessible to both servers in order for the migration to go through. Is there a mechanism to achieve NFS storage on the platform? I didn't see anything in this light on the docs. 

Also, I have noticed a certain drive /proj/<project-name> that automatically gets mounted on nodes when I create an experiment. The <project-name> part is actually not the same on all nodes and even when it's the same it seems to be a different storage each time. I did try using it for a few experiments and sometimes it's the same on both nodes. I haven't understood the logic. Please can you tell me more about this and if it is possible to use it ? If yes, how? 

Thank you

Caleb

[Leigh Stoller]

> I am carrying out some experiments on migration of virtual machines and need a shared storage accessible to both servers in order for the migration to go through. Is there a mechanism to achieve NFS storage on the platform? I didn't see anything in this light on the docs.

Hi. Click on the Docs button in the upper right, then on Example Profiles.
There are a couple of NFS profiles there, use the search button. One of
these might be what you need. Note that these are “repository backed” profiles,
which are described in section 4.3 of the Cloudlab manual.

> Also, I have noticed a certain drive /proj/<project-name> that automatically gets mounted on nodes when I create an experiment. The <project-name> part is actually not the same on all nodes and even when it's the same it seems to be a different storage each time. I did try using it for a few experiments and sometimes it's the same on both nodes. I haven't understood the logic. Please can you tell me more about this and if it is possible to use it ? If yes, how?

That name will be different on remote clusters (remote from Emulate cluster).
It has a 100G quota, is not backed up of course, and should not be used for
high volume traffic. Check out the storage chapter of the Cloudlab manual for
more info on storage options.

Leigh

[CALEB FONYUY-ASHERI S]

Hello,

Thank you for your response. This NFS profile partially gives me what I desire. 

I forked and updated it to be able to choose the resource types I need and use the same operating system on them (https://github.com/Calebfonyuy/nfs-simple/blob/master/profile.py).

I however still experience a serious issue where I cannot be able to create an experiment with resource types from two different clusters. How can I handle this? 

Here is an example of an experiment that failed to start when using two different clusters: https://www.cloudlab.us/memlane.php?uuid=316e608d-2703-11ef-9f39-e4434b2381fc

Please can you let me know why the experiment fails? I don't get any failure message on the experiment page. 

--
You received this message because you are subscribed to the Google Groups "cloudlab-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cloudlab-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloudlab-users/3D7B4A63-E7CB-42A4-8D56-7A3462188A2D%40gmail.com.

[Leigh Stoller]

> I however still experience a serious issue where I cannot be able to create an experiment with resource types from two different clusters. How can I handle this?
>
> Here is an example of an experiment that failed to start when using two different clusters: https://www.cloudlab.us/memlane.php?uuid=316e608d-2703-11ef-9f39-e4434b2381fc

Hi. This experiment uses two nodes at the same cluster. Do you have an example
that uses nodes at different clusters, and fails?

Thanks
Leigh

[Leigh Stoller]

Forgot to mention that you might want to roll in some of the code in this
example profile:

https://www.cloudlab.us/show-profile.php?project=PortalProfiles&profile=multisite-lan

which creates a link that spans multiple clusters.

Leigh

[CALEB FONYUY-ASHERI S]

Thank you. I'll take a look and see what I can gather. 

which creates a link that spans multiple clusters.

Leigh

Here is a link with a multi-site experiment that fails: https://www.cloudlab.us/status.php?uuid=b190acdd-2726-11ef-9f39-e4434b2381fc

Sorry I mixed up the links in my previous mail. 

--
You received this message because you are subscribed to the Google Groups "cloudlab-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cloudlab-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloudlab-users/460063B1-D1F2-4674-B2FA-55EE047CD56A%40gmail.com.

[Leigh Stoller]

> Here is a link with a multi-site experiment that fails: https://www.cloudlab.us/status.php?uuid=b190acdd-2726-11ef-9f39-e4434b2381fc
> Sorry I mixed up the links in my previous mail.

Hi. No worries. I am not seeing anything wrong at the moment, I will look
further today and let you know.

Thanks
Leigh

[CALEB FONYUY-ASHERI S]

I found the following lines in the portal log. Seems like the issue could be linked to the networking. 

06/10 07:43:01 DEBUG    stitchhandler.py:2735 Sending SCS options {'geni_workflow_paths_merged': True}
06/10 07:43:01 DEBUG    stitchhandler.py:2483 Calling SCS with options {'geni_workflow_paths_merged': True}
06/10 07:43:01 DEBUG    stitchhandler.py:2489 Error from slice computation service: StitchingServiceFailedError: Error from Stitching Service: code 1: GeniRSpec::CreateApiRequestMessage - no stitching path or multi-aggregate link in request RSpec.
06/10 07:43:01 DEBUG    stitchhandler.py:3556 No AMs in AM list to process, so not creating amlist file
-----------------------------------------
Stitcher failed on all aggregates!
RunStitcher error: Stitcher failed!

Caleb

--
You received this message because you are subscribed to the Google Groups "cloudlab-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cloudlab-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloudlab-users/4FF4CA72-B7A2-44B6-B803-ABE27C8B5035%40gmail.com.

[Leigh Stoller]

> On Jun 10, 2024, at 6:50 AM, CALEB FONYUY-ASHERI S <fony...@gmail.com> wrote:
>
> I found the following lines in the portal log. Seems like the issue could be linked to the networking.

Ah, a long forgotten bug in the “stitcher” module. That will not get fixed
in the near future, so here is a minor reorg of your code. The goal is to
move the creation of the LAN till after the nodes. This is a little cumbersome
of course, but the easiest path to getting your experiment going.

Leigh

# Always need this when using parameters
params = pc.bindParameters()

# The NFS server.
nfsServer = request.RawPC("snode")
nfsServer.disk_image = params.osImage
nfsServer.hardware_type = params.serverType
# Attach server to lan.
iface0 = nfsServer.addInterface('interface-0', pg.IPv4Address('192.168.6.2','255.255.255.0'))
#nfsLan.addInterface(iface0)
# Storage file system goes into a local (ephemeral) blockstore.
nfsBS = nfsServer.Blockstore("nfsBS", nfsDirectory)
nfsBS.size = params.nfsSize
# Initialization script for the server
nfsServer.addService(pg.Execute(shell="sh", command="sudo /bin/bash /local/repository/nfs-server.sh"))

# The NFS client, also attached to the NFS lan.
nfsClient = request.RawPC("tnode")
nfsClient.disk_image = params.osImage
nfsClient.hardware_type = params.clientType
iface2 = nfsClient.addInterface('interface-1', pg.IPv4Address('192.168.6.3','255.255.255.0'))
#nfsLan.addInterface(iface2)
# Initialization script for the clients
nfsClient.addService(pg.Execute(shell="sh", command="sudo /bin/bash /local/repository/nfs-client.sh"))

# The NFS network. All these options are required.
nfsLan = request.LAN(nfsLanName)
# Must provide a bandwidth. BW is in Kbps
nfsLan.bandwidth = 100000
nfsLan.best_effort = True
nfsLan.vlan_tagging = True
nfsLan.link_multiplexing = True
nfsLan.addInterface(iface0)
nfsLan.addInterface(iface2)

[CALEB FONYUY-ASHERI S]

Thanks a lot for your assistance.  Reordering my instructions as you asked works. There is still an error message though but i don't know why since my nodes are connected and the NFS works. 

Thanks again. 

--
You received this message because you are subscribed to the Google Groups "cloudlab-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cloudlab-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloudlab-users/6205E258-749A-4869-A5E3-2E22F8325A86%40gmail.com.