OpenMP multithreading degrading performance

[Jonathan Takeshita]

We have a scenario where using OpenMP to parallelize some code running on SGX via Graphene is actually degrading performance. Is there a known reason for such behavior?

Thanks,

Jonathan S. Takeshita

Department of Computer Science and Engineering

University of Notre Dame

Notre Dame, IN

https://sites.nd.edu/jonathan-takeshita/

[Taeho Jung]

As a further information:

We noticed the sum calculation of 1M floating numbers takes the least time when we use one thread only, and it takes more time when we use more threads to do divide-and-conquer (i.e., with 2 threads, we compute the sum of halves in parallel and add them at the end; with 4 threads, we compute the sum of quarters and add them at the end).

t refers to the number of threads used.

t = 1: 4.1195 seconds

t = 2: 4.548463 seconds

t = 4: 5.010339 seconds

t = 8: 6.182503 seconds

t = 16: 6.207797 seconds

p.s., we used read() to read in all the floating points from a file instead of fread() to measure the impacts of system calls, and then summed up the floating numbers using the divide-and-conquer approach with multiple threads except in case t=1.

Best,

Taeho

--

Assistant Professor

Department of Computer Science and Engineering, University of Notre Dame

Notre Dame, IN, United States

https://sites.nd.edu/taeho-jung

[Michał Kowalczyk]

Hi,

Could you also share the timings for both native run and graphene-direct? (i.e. non-SGX)

Best,
Michał

--
You received this message because you are subscribed to the Google Groups "Graphene Support Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to graphene-suppo...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/graphene-support/CA%2BYg0BCsifZQrEk6417dCkmf1Hbq-hF%3DiO5y_uCcbRVxbzsnKA%40mail.gmail.com.

[Taeho Jung]

Hi Michał,

Do you mean the plain running wo anything and plain running with graphene but no sgx?

Best,

Taeho

[Michał Kowalczyk]

Yes.

[Taeho Jung]

Okie dokie. Will try that and share the results with you -- might take some time because my team is on leave now :)

Thanks for looking into this.

Best,

Taeho

[Taeho Jung]

Hi Michał,

We did what you suggested, and dug more deeply into the program. We found the issue was in the IO that was not parallelized properly. After handling that, we observed speedup with multithreading the applications running inside SGX via Graphene.

Best,

Taeho

[Sankaranarayanan Venkatasubramanian]

Hi Taeho,

Could you also use numactl to limit the execution to only one NUMA (both CPU and memory)? Also, could you use the KMP_AFFINTY parameters? Something like below

KMP_AFFINITY=granularity=fine,noverbose,compact,1,0 numactl --cpubind=0 --membind=0 graphene-sgx...

~Sankar V 

To view this discussion on the web visit https://groups.google.com/d/msgid/graphene-support/CA%2BYg0BDF4ts4rQL8S1p6x%2BCaibjsjqLki2kJXr4JWTZh-d1w1g%40mail.gmail.com.