MultiSite experiment issue

[Ehsan]

I am trying to run a multisite experiment. I used both my own profile (https://www.cloudlab.us/show-profile.php?uuid=2624a7d6-980b-11ed-994d-e4434b2381fc) and the sample multisite profile (https://www.cloudlab.us/show-profile.php?uuid=f4a83dac-5eef-11ea-9203-e4434b2381fc) but I'm getting errors on both. Any Idea about these errors and how can I create a multi-site experiment? Is there any limitation for the sites or the servers types?
Thanks.

This is my error when I'm trying to use Wisconsin and Utah as my sites:

"RunStitcher error: /usr/testbed/venv/stitcher/lib/python2.7/site-packages/OpenSSL/crypto.py:14: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release. from cryptography import utils, x509 Stitching Failed. Stitching reservation failed 5 times. Last error: Circuit reservation failed at <Aggregate utah-stitch3: https://www.stitch.geniracks.net:12369/protogeni/xmlrpc/am/3.0> (AMAPIError: Slice urn:publicid:IDN+emulab.net:anomaly+slice+tewtmultideaf expires within 1 day(s) on 2023-01-20 07:38:04 UTC Allocation of slivers in slice urn:publicid:IDN+emulab.net:anomaly+slice+tewtmultideaf at utah-stitch3 failed: Error from Aggregate: code 2. protogeni AM code: 2: Error creating container experiment (PG log url - look here for details on any failures: https://www.stitch.geniracks.net/spewlogfile.php3?logfile=008614b4493248d24c6590daa2809ffc399a557a).. Allocate slivers in slice urn:publicid:IDN+emulab.net:anomaly+slice+tewtmultideaf failed at https://www.stitch.geniracks.net:12369/protogeni/xmlrpc/am/3.0 Error from Aggregate: code 2. protogeni AM code: 2: Error creating container experiment.). Try again from the SCS Allocation of slivers in slice urn:publicid:IDN+emulab.net:anomaly+slice+tewtmultideaf at utah-stitch3 failed: Error from Aggregate: code 2. protogeni AM code: 2: Error creating container experiment (PG log url - look here for details on any failures: https://www.stitch.geniracks.net/spewlogfile.php3?logfile=b281c9b5ccf46bab729fa924658ea25874d43e5a).."
This is the error when Using Wisconsin and Emulab for the sites:
"RunStitcher error: /usr/testbed/venv/stitcher/lib/python2.7/site-packages/OpenSSL/crypto.py:14: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release. from cryptography import utils, x509 Stitching Failed. Stitching reservation failed 5 times. Last error: Circuit reservation failed at <Aggregate utah-stitch3: https://www.stitch.geniracks.net:12369/protogeni/xmlrpc/am/3.0> (AMAPIError: Slice urn:publicid:IDN+emulab.net:anomaly+slice+test2 expires within 1 day(s) on 2023-01-20 07:50:48 UTC Allocation of slivers in slice urn:publicid:IDN+emulab.net:anomaly+slice+test2 at utah-stitch3 failed: Error from Aggregate: code 2. protogeni AM code: 2: Error creating container experiment (PG log url - look here for details on any failures: https://www.stitch.geniracks.net/spewlogfile.php3?logfile=eb1b66a3f2638764714894037d0260249657f94a).. Allocate slivers in slice urn:publicid:IDN+emulab.net:anomaly+slice+test2 failed at https://www.stitch.geniracks.net:12369/protogeni/xmlrpc/am/3.0 Error from Aggregate: code 2. protogeni AM code: 2: Error creating container experiment.). Try again from the SCS Allocation of slivers in slice urn:publicid:IDN+emulab.net:anomaly+slice+test2 at utah-stitch3 failed: Error from Aggregate: code 2. protogeni AM code: 2: Error creating container experiment (PG log url - look here for details on any failures: https://www.stitch.geniracks.net/spewlogfile.php3?logfile=53b834da100910893e389fcfca80d3e9448722d4).." 

[Leigh Stoller]

> I am trying to run a multisite experiment. I used both my own profile (https://www.cloudlab.us/show-profile.php?uuid=2624a7d6-980b-11ed-994d-e4434b2381fc) and the sample multisite profile (https://www.cloudlab.us/show-profile.php?uuid=f4a83dac-5eef-11ea-9203-e4434b2381fc) but I'm getting errors on both. Any Idea about these errors and how can I create a multi-site experiment? Is there any limitation for the sites or the servers types?
> Thanks.

Yep, we are having a problem with some of the infrastructure. We are
working on it and will let you know soon.

Thanks
Leigh

[Ehsan]

Thanks!

[Leigh Stoller]

> Yep, we are having a problem with some of the infrastructure. We are
> working on it and will let you know soon.

Can you give it another try please.

Thanks
Leigh

[Ehsan]

I'm still getting a similar error.
https://www.cloudlab.us/spewlogfile.php?logfile=034bc641e0bf5e30b0a416f3225ecb460197307d

"RunStitcher error: /usr/testbed/venv/stitcher/lib/python2.7/site-packages/OpenSSL/crypto.py:14: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release. from cryptography import utils, x509 Reservation request impossible at <Aggregate wisconsin-clab3: https://www.wisc.cloudlab.us:12369/protogeni/xmlrpc/am/3.0>. Malformed request or insufficient resources: AMAPIError: Slice urn:publicid:IDN+emulab.net:anomaly+slice+lustremultisite expires within 1 day(s) on 2023-01-20 11:33:... Allocation of slivers in slice urn:publicid:IDN+emulab.net:anomaly+slice+lustremultisite at utah-stitch3 failed: Error from Aggregate: code 2. protogeni AM code: 2: Error creating container experiment (PG log url - look here for details on any failures: https://www.stitch.geniracks.net/spewlogfile.php3?logfile=9b2d0208d9740dee689c54e7c3734ade87469739)..
"

[Leigh Stoller]

> I'm still getting a similar error.
> https://www.cloudlab.us/spewlogfile.php?logfile=034bc641e0bf5e30b0a416f3225ecb460197307d

Yep, more fallout from the previous mess. I fixed up another problem,

[Ehsan]

Unfortunately, still I'm getting this error:
"RunStitcher error: /usr/testbed/venv/stitcher/lib/python2.7/site-packages/OpenSSL/crypto.py:14: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release. from cryptography import utils, x509 Aggregate al2s unreachable: Operation timed out"

I'm using this profile to test the multisite experiment: https://www.cloudlab.us/show-profile.php?uuid=f4a83dac-5eef-11ea-9203-e4434b2381fc
Not sure If I have to define something extra on the profile or not tho.

Thanks,
Ehsan

[Leigh Stoller]

> Unfortunately, still I'm getting this error:

> "RunStitcher error: /usr/testbed/venv/stitcher/lib/python2.7/site-packages/OpenSSL/crypto.py:14: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release. from cryptography import utils, x509 Aggregate al2s unreachable: Operation timed out”

This error is different. And beyond our control, looks like the
stitching service at Internet2 is suddenly offline. I will contact
them, but no idea when it will be resolved. Sorry about that.

Leigh

[Ehsan]

Thanks for the help and guidance!