making software detection work faster on NFS in CK

0 views
Skip to first unread message

Grigori Fursin

unread,
Oct 30, 2018, 10:23:00 AM10/30/18
to Collective Knowledge, ctuning-discussions
Just a note that when preparing a demo workflow for SCC18
(https://github.com/ctuning/ck-scc18) using Collective Knowledge
framework, Spack package manager and Flux framework during my visit to
LLNL, we've noticed that CK software plugins
(http://cknowledge.org/shared-soft-detection-plugins.html) work
extremely slowly on NFS, i.e. 3..4 minutes to detect each software
dependency.

This is normal since we perform the full and recursive search of all
directories by default, however maybe we should provide a flag like
--quick to speed up search in such case by looking only in a few most
commonly used locations. We can then let a user specify the full path to
a given soft if it's not found...

You can test it as follows:

$ ck detect soft --tags=compiler,llvm
$ ck virtual env --tags=compiler,llvm

I opened a ticket about that: https://github.com/ctuning/ck-env/issues/82 .

Will tell you if we fix that,
Grigori

Reply all
Reply to author
Forward
0 new messages