Running for a long time and stuck in "Finding the best partitioning scheme"

[Nayeli Gutiérrez]

Hi Rob, thanks for this great software! It is my first time running PartitionFinder2 with a big dataset (12,997 loci, 40 spp) and would like to know if it is normal that it is taking more than 14 days to finish using 28 cpus in a cluster. It finished analyzing the subsets and its been stuck in the "Finding the best partitioning scheme" part for five days with no error message. 

Command:

python /home/ngutierrez/array1/software/partitionfinder-2.1.1/PartitionFinder.py /home/ngutierrez/array1/trees/allspp/partitionfinder/ --raxml -p 28

The log file is attached (I had to zip it because it was too big to attach here).

Thanks!

Nayeli

[Rob Lanfear]

Hi Nayeli,

I think the answer is in the logfile...

```

INFO     | 2022-06-15 22:01:01,359 | analysis_m |    PartitionFinder will have to analyse 1469956744 subsets to complete this analyses
WARNING  | 2022-06-15 22:01:01,365 | progress   |    1469956744 is a lot of subsets, this might take a long time to analyse. Perhaps consider using a different search scheme instead (see the Manual)
```

1.4 billion is a lot of subsets to analyse. I'd suggest taking a look through the manual and choosing the fastest possible approach for your dataset.

Rob

[Nayeli Gutiérrez]

Hi Rob, thanks for your answer. Do you think the combination below will do? I am using 80 cpus and 500gb of memory in a cluster.

--raxml -p 80 --rcluster-max 1000 --rcluster-percent 10

Best,

Nayeli

[Nayeli Gutiérrez]

It did: Total processing time: 1 day, 12:39:56 (h:m:s) !!!

:D 

[Rob Lanfear]

Cool!

R