Possible reasons to why concurrent.futures.ProcessPoolExecutor in Python takes so much longing bazel test

[Alfred53]

I have a simple unit test that uses concurrent.futures.ProcessPoolExecutor in Python. I noticed it can be over 50x slower when I run the test with bazel test.

Trying to understand why, I profiled the performances (sortedby cumtime):

With Bazel:

         11562 function calls (11442 primitive calls) in 2.673 seconds Ordered by: cumulative time ncalls tottime percall cumtime percall filename:lineno(function)
5/1 0.000 0.000 2.673 2.673 {built-in method builtins.exec}
1 0.000 0.000 2.672 2.672 <string>:1(<module>)
1 0.000 0.000 2.672 2.672 utils.py:172(apply_parallel)
8 2.606 0.326 2.606 0.326 {method 'acquire' of '_thread.lock' objects}
1 0.000 0.000 2.606 2.606 _base.py:635(__exit__)
1 0.000 0.000 2.606 2.606 process.py:679(shutdown)
2 0.000 0.000 2.605 1.302 threading.py:979(join)
2 0.000 0.000 2.605 1.302 threading.py:1017(_wait_for_tstate_lock)
1 0.000 0.000 0.050 0.050 process.py:650(map)
1 0.000 0.000 0.050 0.050 _base.py:575(map)
1 0.000 0.000 0.050 0.050 _base.py:600(<listcomp>)

Without Bazel

         3818 function calls (3767 primitive calls) in 0.018 seconds Ordered by: cumulative time ncalls tottime percall cumtime percall filename:lineno(function)
2/1 0.000 0.000 0.018 0.018 {built-in method builtins.exec}
1 0.000 0.000 0.018 0.018 <string>:1(<module>)
1 0.000 0.000 0.018 0.018 utils.py:172(apply_parallel)
8 0.010 0.001 0.010 0.001 {method 'acquire' of '_thread.lock' objects}
1 0.000 0.000 0.010 0.010 _base.py:635(__exit__)
1 0.000 0.000 0.010 0.010 process.py:679(shutdown)
2 0.000 0.000 0.010 0.005 threading.py:979(join)
2 0.000 0.000 0.010 0.005 threading.py:1017(_wait_for_tstate_lock)
1 0.000 0.000 0.004 0.004 process.py:650(map)
1 0.000 0.000 0.004 0.004 _base.py:575(map)
1 0.000 0.000 0.004 0.004 _base.py:600(<listcomp>)

If I sort the stats by tottime, which means “for the total time spent in the given function (and excluding time made in calls to sub-functions)“, {method 'acquire' of '_thread.lock' objects} would be the one that takes the most of the time (bazel vs no-bazel: 2.606s vs 0.010s)

I suspect it's related to the bazel sanboxing, but I don't know enough to tell what exactly is going on,  and how could I make the test faster.

Any insights would be appreciated.

[Alexandre Rostovtsev]

You can check how long it takes with sandboxing disabled: bazel test --spawn_strategy=standalone //my:test

By the way, are you seeing the performance issue on Linux or on macOS?

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/0500531c-2954-408d-bdcd-c03c09a40eb2n%40googlegroups.com.