Several questions about parallel programming

[Marco Caselli]

Hello,

I would like to parallel my code but I am facing some issues. So far I have just used the decorator @parallel. Some parts of my code needs to access at some sets/lists stored in the memory and the refresh them.

Q1- Is there any method that let me to share memory between process? 

Other blocks of my code do not need to access to the memory so I manage to parallelised them but I have some timing issues: now the code is slower! 

I looked at the timing of each step (writing the list of inputs, the parallelised block, access to the outputs) and the last one (access to the outputs) consists roughly in the 99% of the total time. The function which I am referring to is the following 

def parallelcount(vector,paralleloutput):
    for i,j in paralleloutput:
        vector=[x + y for x, y in zip(vector, j)]
    return vector  

Q2- Is there any faster/smart way to access at the paralleloutput?

I got that the output of a parallel process is a generator so, since I am using it inside a quite big loop, it could be not the best efficient way to speed up the code. My feeling is that at each loop's step sage reads my parallel block, builds a generator from it but it doesn't compute anything, once the generator is called by "parallelcount" it starts to compute everything (this should explain why all the time is used just for accessing to the outputs). If it works in this way building the generators should be the main reason of the slowness, indeed my loop has roughly 10^5 steps but the parallel count receives just 10 inputs, so probably is not worth building 10^5 generators.

Q3- Am I right? If not why is it slower then the not parallelised code? If yes may force sage to do not build a generator each time but just compute the outputs (since they are just 10)?

A positive answer to Q1 may let me avoid to build so many generators (indeed I may parallelise the above loop instead of the inner one).

Cheers,

Marco