So I did want the script to cap the jobs at 4 (but in this case the machine could apparently handle running all 15 at once).
1 May-22 18:27:15.943 [main] DEBUG nextflow.script.CliRunner - nextflow
run_trinity.nf 2 May-22 18:27:16.164 [main] DEBUG nextflow.script.CliRunner - nextflow version 0.7.3.1805 ~ host:
szdedicated.solazyme.com [127.0.0.1]
3 May-22 18:27:16.232 [main] DEBUG nextflow.script.CliRunner - Installed File System: 'file' [LinuxFileSystemProvider]
4 May-22 18:27:16.239 [main] DEBUG nextflow.script.CliRunner - Installed File System: 'jar' [ZipFileSystemProvider]
5 May-22 18:27:16.240 [main] DEBUG nextflow.script.CliRunner - DxFileSystemProvider NOT available
6 May-22 18:27:16.860 [main] DEBUG nextflow.Session - Executor pool size: 65
7 May-22 18:27:16.945 [main] DEBUG nextflow.Session - Script base path does not exist or is not a directory: /data/bifx/tlaurent/trinity_wd/pipline/bin
8 May-22 18:27:16.950 [main] DEBUG nextflow.script.CliRunner - Script bin dir: null
9 May-22 18:27:16.989 [main] DEBUG nextflow.Session - Session start > phaser register (session)•
10 May-22 18:27:16.998 [main] DEBUG nextflow.processor.TaskDispatcher - Dispatcher > start
11 May-22 18:27:17.863 [main] DEBUG nextflow.Channel - files for syntax: glob; folder: /data/bifx/Sequence_data/TXOME/; pattern: Project*/*/*_R1_*.fastq.gz; skipHidden: false
12 May-22 18:27:18.005 [main] DEBUG nextflow.Channel - files for syntax: glob; folder: /data/bifx/Sequence_data/TXOME/; pattern: Project*/*/*_R2_*.fastq.gz; skipHidden: false
13 May-22 18:27:18.300 [main] DEBUG nextflow.processor.ProcessFactory - << taskConfig executor: null
14 May-22 18:27:18.300 [main] DEBUG nextflow.processor.ProcessFactory - >> processorType: 'local'
15 May-22 18:27:18.335 [main] INFO nextflow.executor.Executor - [warm up] executor > local
16 May-22 18:27:18.436 [main] DEBUG n.processor.TaskPollingMonitor - Creating task monitor for executor 'local' > capacity: 63; pollInterval: 100ms; dumpInterval: 5min•
17 May-22 18:27:18.449 [main] DEBUG nextflow.processor.TaskDispatcher - Starting monitor: TaskPollingMonitor
18 May-22 18:27:18.451 [main] DEBUG n.processor.TaskPollingMonitor - >>> phaser register (scheduler)
19 May-22 18:27:18.538 [main] DEBUG nextflow.script.BaseParam - output > channel unknown: Samps -- creating a new instance
20 May-22 18:27:18.599 [main] DEBUG n.processor.ParallelTaskProcessor - Creating operator > prepare_samples -- maxForks: 65
21 May-22 18:27:18.610 [main] DEBUG nextflow.Session - >>> phaser register (process)
22 May-22 18:27:18.651 [Actor Thread 19] DEBUG n.processor.ParallelTaskProcessor - <prepare_samples> Poison pill arrived
23 May-22 18:27:18.663 [main] DEBUG nextflow.processor.ProcessFactory - << taskConfig executor: null
24 May-22 18:27:18.664 [main] DEBUG nextflow.processor.ProcessFactory - >> processorType: 'local'
25 May-22 18:27:18.667 [Actor Thread 20] DEBUG nextflow.processor.StateObj - <prepare_samples> State before poison: StateObj[submitted: 15; completed: 0; poisoned: false; allScalar: false]
26 May-22 18:27:18.667 [main] DEBUG nextflow.script.BaseParam - output > channel unknown: trimout -- creating a new instance
27 May-22 18:27:18.738 [main] DEBUG n.processor.ParallelTaskProcessor - Creating operator > trim_fastas -- maxForks: 65
28 May-22 18:27:18.740 [main] DEBUG nextflow.Session - >>> phaser register (process)
29 May-22 18:27:18.771 [main] DEBUG nextflow.processor.ProcessFactory - << taskConfig executor: null
30 May-22 18:27:18.771 [main] DEBUG nextflow.processor.ProcessFactory - >> processorType: 'local'
31 May-22 18:27:18.774 [main] DEBUG nextflow.script.BaseParam - output > channel unknown: trinity -- creating a new instance
32 May-22 18:27:18.828 [main] DEBUG n.processor.ParallelTaskProcessor - Creating operator > run_trinity -- maxForks: 4
33 May-22 18:27:18.829 [main] DEBUG nextflow.Session - >>> phaser register (process)
34 May-22 18:27:18.831 [main] DEBUG nextflow.Session - Session await > processors completion
35 May-22 18:27:19.152 [Actor Thread 31] INFO n.processor.ParallelTaskProcessor - [b3/2e81a7] Submitted process > prepare_samples (11)
36 May-22 18:27:19.152 [Actor Thread 24] INFO n.processor.ParallelTaskProcessor - [ce/b34402] Submitted process > prepare_samples (5)
37 May-22 18:27:19.152 [Actor Thread 32] INFO n.processor.ParallelTaskProcessor - [46/693b4c] Submitted process > prepare_samples (12)
38 May-22 18:27:19.152 [Actor Thread 21] INFO n.processor.ParallelTaskProcessor - [87/41e5db] Submitted process > prepare_samples (1)
39 May-22 18:27:19.152 [Actor Thread 25] INFO n.processor.ParallelTaskProcessor - [4d/b20ad3] Submitted process > prepare_samples (3)
40 May-22 18:27:19.152 [Actor Thread 34] INFO n.processor.ParallelTaskProcessor - [4c/385234] Submitted process > prepare_samples (14)
41 May-22 18:27:19.154 [Actor Thread 23] INFO n.processor.ParallelTaskProcessor - [53/214639] Submitted process > prepare_samples (2)
42 May-22 18:27:19.157 [Actor Thread 29] INFO n.processor.ParallelTaskProcessor - [f3/80e97b] Submitted process > prepare_samples (9)
43 May-22 18:27:19.161 [Actor Thread 35] INFO n.processor.ParallelTaskProcessor - [ef/9eb87c] Submitted process > prepare_samples (15)
44 May-22 18:27:19.165 [Actor Thread 26] INFO n.processor.ParallelTaskProcessor - [14/b73cb1] Submitted process > prepare_samples (6)
45 May-22 18:27:19.166 [Actor Thread 22] INFO n.processor.ParallelTaskProcessor - [47/580223] Submitted process > prepare_samples (4)
46 May-22 18:27:19.167 [Actor Thread 33] INFO n.processor.ParallelTaskProcessor - [cf/7a3e23] Submitted process > prepare_samples (13)
47 May-22 18:27:19.168 [Actor Thread 27] INFO n.processor.ParallelTaskProcessor - [5c/5a2ae6] Submitted process > prepare_samples (7)
48 May-22 18:27:19.169 [Actor Thread 30] INFO n.processor.ParallelTaskProcessor - [aa/d158cd] Submitted process > prepare_samples (10)
49 May-22 18:27:19.173 [Actor Thread 28] INFO n.processor.ParallelTaskProcessor - [03/9cbaed] Submitted process > prepare_samples (8)
50 May-22 18:27:19.621 [Actor Thread 20] DEBUG nextflow.processor.TaskProcessor - <prepare_samples> Sending poison pills and terminating process
51 May-22 18:27:19.625 [Actor Thread 4] DEBUG n.processor.ParallelTaskProcessor - <trim_fastas> Poison pill arrived
52 May-22 18:27:19.626 [Actor Thread 41] DEBUG nextflow.processor.StateObj - <trim_fastas> State before poison: StateObj[submitted: 15; completed: 0; poisoned: false; allScalar: false]
53 May-22 18:27:19.626 [Actor Thread 20] DEBUG nextflow.Session - <<< phaser deregister (process)
54 May-22 18:27:19.630 [Actor Thread 5] DEBUG n.processor.ParallelTaskProcessor - <prepare_samples> After stop -- shareObjs [:]
55 May-22 18:27:19.658 [Actor Thread 44] INFO n.processor.ParallelTaskProcessor - [cd/ee88b4] Submitted process > trim_fastas (3)
56 May-22 18:27:19.659 [Actor Thread 52] INFO n.processor.ParallelTaskProcessor - [6d/a181a6] Submitted process > trim_fastas (7)
57 May-22 18:27:19.661 [Actor Thread 39] INFO n.processor.ParallelTaskProcessor - [8f/8605b5] Submitted process > trim_fastas (1)
58 May-22 18:27:19.663 [Actor Thread 42] INFO n.processor.ParallelTaskProcessor - [a5/76a2ce] Submitted process > trim_fastas (2)
59 May-22 18:27:19.664 [Actor Thread 48] INFO n.processor.ParallelTaskProcessor - [36/c77408] Submitted process > trim_fastas (5)
60 May-22 18:27:19.665 [Actor Thread 50] INFO n.processor.ParallelTaskProcessor - [dc/2e52c9] Submitted process > trim_fastas (6)
61 May-22 18:27:19.666 [Actor Thread 46] INFO n.processor.ParallelTaskProcessor - [a6/309c4b] Submitted process > trim_fastas (4)
62 May-22 18:27:19.666 [Actor Thread 60] INFO n.processor.ParallelTaskProcessor - [66/45b43f] Submitted process > trim_fastas (11)
63 May-22 18:27:19.667 [Actor Thread 58] INFO n.processor.ParallelTaskProcessor - [dc/e149ca] Submitted process > trim_fastas (10)
64 May-22 18:27:19.668 [Actor Thread 66] INFO n.processor.ParallelTaskProcessor - [99/3148fc] Submitted process > trim_fastas (14)
65 May-22 18:27:19.669 [Actor Thread 64] INFO n.processor.ParallelTaskProcessor - [77/6311e3] Submitted process > trim_fastas (13)
66 May-22 18:27:19.670 [Actor Thread 62] INFO n.processor.ParallelTaskProcessor - [36/8539e4] Submitted process > trim_fastas (12)
67 May-22 18:27:19.671 [Actor Thread 56] INFO n.processor.ParallelTaskProcessor - [66/e6ee6a] Submitted process > trim_fastas (9)
68 May-22 18:27:19.672 [Actor Thread 54] INFO n.processor.ParallelTaskProcessor - [5b/90b016] Submitted process > trim_fastas (8)
69 May-22 18:27:19.673 [Actor Thread 3] INFO n.processor.ParallelTaskProcessor - [52/d2625a] Submitted process > trim_fastas (15)
70 May-22 18:27:19.806 [Actor Thread 1] INFO n.processor.ParallelTaskProcessor - [fb/f0272b] Submitted process > run_trinity (1)
71 May-22 18:27:19.820 [Actor Thread 8] INFO n.processor.ParallelTaskProcessor - [ec/2f5ec4] Submitted process > run_trinity (2)
72 May-22 18:27:19.828 [Actor Thread 10] INFO n.processor.ParallelTaskProcessor - [0b/f6d31d] Submitted process > run_trinity (3)
73 May-22 18:27:19.840 [Actor Thread 12] INFO n.processor.ParallelTaskProcessor - [d8/a7360f] Submitted process > run_trinity (4)
74 May-22 18:27:19.856 [Actor Thread 15] INFO n.processor.ParallelTaskProcessor - [28/8486de] Submitted process > run_trinity (5)
75 May-22 18:27:19.867 [Actor Thread 17] INFO n.processor.ParallelTaskProcessor - [bb/e54117] Submitted process > run_trinity (6)
76 May-22 18:27:19.873 [Actor Thread 6] INFO n.processor.ParallelTaskProcessor - [7e/6847b3] Submitted process > run_trinity (7)
77 May-22 18:27:19.884 [Actor Thread 18] INFO n.processor.ParallelTaskProcessor - [50/10f3d8] Submitted process > run_trinity (8)
78 May-22 18:27:19.901 [Actor Thread 36] INFO n.processor.ParallelTaskProcessor - [b2/205bc3] Submitted process > run_trinity (9)
79 May-22 18:27:19.909 [Actor Thread 31] INFO n.processor.ParallelTaskProcessor - [d7/7e86b0] Submitted process > run_trinity (10)
80 May-22 18:27:19.913 [Actor Thread 41] DEBUG nextflow.processor.TaskProcessor - <trim_fastas> Sending poison pills and terminating process
81 May-22 18:27:19.915 [Actor Thread 41] DEBUG nextflow.Session - <<< phaser deregister (process)
82 May-22 18:27:19.916 [Actor Thread 26] DEBUG n.processor.ParallelTaskProcessor - <trim_fastas> After stop -- shareObjs [:]
83 May-22 18:27:19.919 [Actor Thread 21] INFO n.processor.ParallelTaskProcessor - [27/ba9720] Submitted process > run_trinity (11)
84 May-22 18:27:19.921 [Actor Thread 32] DEBUG n.processor.ParallelTaskProcessor - <run_trinity> Poison pill arrived
85 May-22 18:27:19.922 [Actor Thread 41] DEBUG nextflow.processor.StateObj - <run_trinity> State before poison: StateObj[submitted: 15; completed: 0; poisoned: false; allScalar: false]
86 May-22 18:27:19.930 [Actor Thread 25] INFO n.processor.ParallelTaskProcessor - [a9/49829d] Submitted process > run_trinity (12)
87 May-22 18:27:19.946 [Actor Thread 29] INFO n.processor.ParallelTaskProcessor - [37/296bd8] Submitted process > run_trinity (13)
88 May-22 18:27:19.955 [Actor Thread 35] INFO n.processor.ParallelTaskProcessor - [fa/80a373] Submitted process > run_trinity (14)
89 May-22 18:27:19.961 [Actor Thread 22] INFO n.processor.ParallelTaskProcessor - [de/1b88f7] Submitted process > run_trinity (15)
90 May-22 18:32:18.652 [Thread-5] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 15 -- first: TaskHandler[id: 31; name: run_trinity (1); status: RUNNING; exit: -; workDir: /data/bifx/tlaurent/trinity_wd/pipline/work/fb/f0272ba246243eb5a561052838755e]
91 May-22 18:37:18.692 [Thread-5] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 15 -- first: TaskHandler[id: 31; name: run_trinity (1); status: RUNNING; exit: -; workDir: /data/bifx/tlaurent/trinity_wd/pipline/work/fb/f0272ba246243eb5a561052838755e]
92 May-22 18:42:18.710 [Thread-5] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 15 -- first: TaskHandler[id: 31; name: run_trinity (1); status: RUNNING; exit: -; workDir: /data/bifx/tlaurent/trinity_wd/pipline/work/fb/f0272ba246243eb5a561052838755e]
93 May-22 18:47:18.754 [Thread-5] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 15 -- first: TaskHandler[id: 31; name: run_trinity (1); status: RUNNING; exit: -; workDir: /data/bifx/tlaurent/trinity_wd/pipline/work/fb/f0272ba246243eb5a561052838755e]
94 May-22 18:52:18.818 [Thread-5] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 15 -- first: TaskHandler[id: 31; name: run_trinity (1); status: RUNNING; exit: -; workDir: /data/bifx/tlaurent/trinity_wd/pipline/work/fb/f0272ba246243eb5a561052838755e]
95 May-22 18:57:18.908 [Thread-5] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 15 -- first: TaskHandler[id: 31; name: run_trinity (1); status: RUNNING; exit: -; workDir: /data/bifx/tlaurent/trinity_wd/pipline/work/fb/f0272ba246243eb5a561052838755e]
96 May-22 19:02:18.980 [Thread-5] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 15 -- first: TaskHandler[id: 31; name: run_trinity (1); status: RUNNING; exit: -; workDir: /data/bifx/tlaurent/trinity_wd/pipline/work/fb/f0272ba246243eb5a561052838755e]
97 May-22 19:07:19.055 [Thread-5] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 15 -- first: TaskHandler[id: 31; name: run_trinity (1); status: RUNNING; exit: -; workDir: /data/bifx/tlaurent/trinity_wd/pipline/work/fb/f0272ba246243eb5a561052838755e]
98 May-22 19:12:19.107 [Thread-5] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 15 -- first: TaskHandler[id: 31; name: run_trinity (1); status: RUNNING; exit: -; workDir: /data/bifx/tlaurent/trinity_wd/pipline/work/fb/f0272ba246243eb5a561052838755e]
99 May-22 19:17:19.167 [Thread-5] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 15 -- first: TaskHandler[id: 31; name: run_trinity (1); status: RUNNING; exit: -; workDir: /data/bifx/tlaurent/trinity_wd/pipline/work/fb/f0272ba246243eb5a561052838755e]
100 May-22 19:22:19.189 [Thread-5] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 15 -- first: TaskHandler[id: 31; name: run_trinity (1); status: RUNNING; exit: -; workDir: /data/bifx/tlaurent/trinity_wd/pipline/work/fb/f0272ba246243eb5a561052838755e]
101 May-22 19:27:19.203 [Thread-5] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 15 -- first: TaskHandler[id: 31; name: run_trinity (1); status: RUNNING; exit: -; workDir: /data/bifx/tlaurent/trinity_wd/pipline/work/fb/f0272ba246243eb5a561052838755e]
102 May-22 19:32:19.289 [Thread-5] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 15 -- first: TaskHandler[id: 31; name: run_trinity (1); status: RUNNING; exit: -; workDir: /data/bifx/tlaurent/trinity_wd/pipline/work/fb/f0272ba246243eb5a561052838755e]
103 May-22 19:37:19.337 [Thread-5] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 15 -- first: TaskHandler[id: 31; name: run_trinity (1); status: RUNNING; exit: -; workDir: /data/bifx/tlaurent/trinity_wd/pipline/work/fb/f0272ba246243eb5a561052838755e]
104 May-22 19:42:19.415 [Thread-5] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 15 -- first: TaskHandler[id: 31; name: run_trinity (1); status: RUNNING; exit: -; workDir: /data/bifx/tlaurent/trinity_wd/pipline/work/fb/f0272ba246243eb5a561052838755e]
105 May-22 19:47:19.483 [Thread-5] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 15 -- first: TaskHandler[id: 31; name: run_trinity (1);
```