Hi folks,
I am not sure if this question is related to the previous topic of multi-threading not working properly on the command line but here goes:
When running the following section of code
#    @jobs_limit(8)
    @files(makeRuffusParams(sampleToPairsDict))
    def runConcurrent(readPair, rdp, sampleName):
        QTAtomic_cmd =  "python2.7 %s -c %s -1 %s -2 %s -s %s"\
                    %(cfgDict[str.lower('qtAtomic')], options.config, readPair[0], readPair[1], sampleName)
        logFH.write(QTAtomic_cmd + '\n')
        executeCommand(QTAtomic_cmd, cwd = os.getcwd(), verbose = False)
    pipeline_printout(logFH, [])
    pipeline_run([], verbose = 8, multiprocess = 5, logger = ruffusLoggerProxy)
I am getting the following error:
Exception in thread Thread-2:
Traceback (most recent call last):
  File "/bioinformatics/sgi/asmopt/lib/python2.7/threading.py", line 808, in __bootstrap_inner
    self.run()
  File "/bioinformatics/sgi/asmopt/lib/python2.7/threading.py", line 761, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/bioinformatics/sgi/asmopt/lib/python2.7/multiprocessing/pool.py", line 342, in _handle_tasks
    put(task)
PicklingError: Can't pickle <type 'function'>: attribute lookup __builtin__.function failed
But when I run the following, my code runs sequentially:
    @jobs_limit(8)
    @files(makeRuffusParams(sampleToPairsDict))
    def runConcurrent(readPair, rdp, sampleName):
        QTAtomic_cmd =  "python2.7 %s -c %s -1 %s -2 %s -s %s"\
                    %(cfgDict[str.lower('qtAtomic')], options.config, readPair[0], readPair[1], sampleName)
        logFH.write(QTAtomic_cmd + '\n')
        executeCommand(QTAtomic_cmd, cwd = os.getcwd(), verbose = False)
    pipeline_printout(logFH, [])
    pipeline_run([], verbose = 8,  logger = ruffusLoggerProxy)
Any ideas please?