AWS libfabric (nitro)

[Bryan B. Thompson]

Hello,

I am wondering what would be the path to integrate and tune GASNET with the AWS libfabric library [1,2,3]?  The AWS library integrates with AWS specific hardware in the NIC ("nitro").  It is part of the recent AWS HPC announcements [1,2].

Are there specific benchmarks that people use to assess correctness and completeness of the GASNET functionality, assess performance, and identify significant performance options for the target applications?

Thanks in advance,

Bryan

[1] https://aws.amazon.com/blogs/aws/now-available-elastic-fabric-adapter-efa-for-tightly-coupled-hpc-workloads/

[2] https://aws.amazon.com/about-aws/whats-new/2019/04/elastic-fabric-adapter-is-now-generally-available/

[3] https://github.com/aws/libfabric

-- 

[Paul Hargrove]

Bryan,

In response to your first question about use of GASNet with the AWS libfabric library:

I believe a colleague of mine recently tried to use GASNet over the AWS libfabric library and found the "efa" provider to lack the necessary features (at least FI_RMA if I understood correctly).

I am uncertain from your email if you are asking because you've seen a similar issue and desire help, or if you are engaged in some effort to provide this missing functionality in the provider.

If you are just seeking help, then I am afraid I can only suggest setting FI_PROVIDER=sockets in your runtime to use that libfabric provider.

That will not be as fast, in my estimation, as any eventual direct support for the AWS-specific provider, but will hopefully allow you to run.

If you are working to provide the missing support, I encourage you to contact us via gasnet...@lbl.gov, (rather than this public gasnet-users list) where we can discuss non-trivial technical details.  In particular, we currently have support for libfabric only in GASNet-1 (our legacy releases) and not yet in GASNet-EX (our current releases).  So, some coordination would be required.

To answer the second question about tests:

GASNet provides a test suite intended to asses both correctness and performance of a "conduit" (our analogue to a libfabric "provider").

Running "make run-tests-seq" or "make run-tests-par" in a conduit's build directory will compile and run all the tests for single- and multi-threaded library builds, respectively.

These make targets, and others, are described in more detail in GASNet's README file (in the sources or our git repository).

-Paul

--
You received this message because you are subscribed to the Google Groups "gasnet-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gasnet-users...@lbl.gov.
To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/gasnet-users/CAC-oqZ%3Dw_P74MgZVHVs09B4GG2oGRXZUJq%3DFpV_6Tu-QH2XQ%3Dw%40mail.gmail.com.

--

Paul H. Hargrove <PHHar...@lbl.gov>

Computer Languages & Systems Software (CLaSS) Group

Computer Science Department

Lawrence Berkeley National Laboratory

[Bryan B. Thompson]

Paul,

Thanks.  It's a bit of both.  I am interested in being a consumer of GASNET over the AWS libfabric.  But I also know the relevant people at AWS.  I will follow up at  gasnet...@lbl.gov.

Thanks,

Bryan