Windows error when running PartitionFinder

[John Soghigian]

Hello all,

Following directions in the manual exactly but am running into an error when I use the nucleotide example:

Traceback (most recent call last):

  File "C:\partitionfinder\partitionfinder-paul_develop\PartitionFinder.py", line 23, in <module>

    sys.exit(main.main("PartitionFinder", "DNA"))

  File "C:\partitionfinder\partitionfinder-paul_develop\partfinder\main.py", line 354, in main

    results = anal.analyse()

  File "C:\partitionfinder\partitionfinder-paul_develop\partfinder\analysis.py", line 75, in analyse

    self.do_analysis()

  File "C:\partitionfinder\partitionfinder-paul_develop\partfinder\analysis_method.py", line 380, in do_analysis

    self.alignment, current_subset, tree_path)

  File "C:\partitionfinder\partitionfinder-paul_develop\partfinder\kmeans.py", line 83, in kmeans_split_subset

    str(tree_path))

  File "C:\partitionfinder\partitionfinder-paul_develop\partfinder\phyml.py", line 484, in gen_per_site_stats

    run_rates(command, report_errors=False)

  File "C:\partitionfinder\partitionfinder-paul_develop\partfinder\phyml.py", line 465, in run_rates

    stderr=subprocess.PIPE)

  File "C:\Anaconda2\lib\subprocess.py", line 710, in __init__

    errread, errwrite)

  File "C:\Anaconda2\lib\subprocess.py", line 958, in _execute_child

    startupinfo)

WindowsError: [Error 193] %1 is not a valid Win32 application

Any idea what may be causing this, or how to fix it?

Thanks

[John Soghigian]

Since my error sounded like a DLL mismatch or an issue with 32 bit Python/Anaconda installations, I uninstalled both, installed 64 bit versions, and have the same issue. I tried using the prerelease versions and now I get a totally different error:

Traceback (most recent call last):

  File "C:\partitionfinder-2.0.0-pre11\PartitionFinder.py", line 23, in <module>

    sys.exit(main.main("PartitionFinder", "DNA"))

  File "C:\partitionfinder-2.0.0-pre11\partfinder\main.py", line 399, in main

    run_analysis(cfg, options)

  File "C:\partitionfinder-2.0.0-pre11\partfinder\main.py", line 332, in run_analysis

    anal = method(cfg, options.force_restart, options.processes)

  File "C:\partitionfinder-2.0.0-pre11\partfinder\analysis.py", line 65, in __init__

    self.make_tree(cfg.user_tree_topology_path)

  File "C:\partitionfinder-2.0.0-pre11\partfinder\analysis.py", line 200, in make_tree

    the_config.cmdline_extras)

  File "C:\partitionfinder-2.0.0-pre11\partfinder\phyml.py", line 76, in make_branch_lengths

    util.dupfile(topology_path, tree_path)

  File "C:\partitionfinder-2.0.0-pre11\partfinder\util.py", line 94, in dupfile

    shutil.copyfile(src, dst)

  File "C:\Anaconda2\lib\shutil.py", line 82, in copyfile

    with open(src, 'rb') as fsrc:

IOError: [Errno 2] No such file or directory: '.\\analysis\\start_tree\\filtered_source.phy_phyml_tree'

Despite the fact that the file in question does indeed exist.  Same issue happens for other examples.  

I wonder if it is because I am invoking Anaconda's version of Python (similar to as described https://groups.google.com/forum/#!topic/partitionfinder/_2MEKUD_LKQ)...

[Brett Calcott]

Hi John,

A couple of questions that might help me figure out what is going on:

- What folder are you in when you run the command line.

- Where is the folder containing the analysis.

- What is your exact commandline.

Brett

--
You received this message because you are subscribed to the Google Groups "PartitionFinder" group.
To unsubscribe from this group and stop receiving emails from it, send an email to partitionfind...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[John Soghigian]

Hi Brett!

Sorry for not including that in my initial posting.  Details below, and a slight update.

I receive the error whether I am in the Partition Finder folder (C:/partitionfinder-2.0.0-pre11) or just after immediately launching my command prompt (if this is the case, I specify directory structure to the .py and .phy files).  I am using PF2.0 PRE11 in Windows 10.

I am calling Python from Anaconda, because for whatever reason, my python installation is not detecting Anaconda's packages and so just python is giving me errors about numpy not being installed (even though Anaconda recognizes it as present via conda list).  I resolved this issue by calling Anaconda's Python as per the thread I referenced above.

The command I am using is roughly (from memory, as I am not on my desktop that has PF):

C:\Anaconda2\Python.exe C:\partitionfinder-2.0.0-pre11\PartitionFinder.py C:\partitionfinder-2.0.0-pre11\examples\nucleotide\somefilename.phy

I say roughly because I have changed the third term to different data sets, in different locations, including storing the .phy and .cfg in the main PF2 folder, or different places that may have different administrative privileges (e.g. in a directory within my documents).  When directly within the PF directory I drop the file paths.  Regardless of location I tested, I receive the same error.

I checked the folders and the tree in question (filtered_source.phy_phyml_tree) is present. I moved it into the directory containing the analyses, modified the PartitionFinder.cfg file to include a user specified topology under alignment( via user_tree_topology = filtered_source.phy_phyml_tree; with that file stored in the same directory as the .cfg and .phy files), and the analysis started; I stupidly started a large analysis without confirming it would work via a single model, as I plan to do later today.  When I woke up in the morning, my desktop had gone to Sleep and my log contained this message:

INFO     | 2016-02-16 23:51:18,786 | config     | Loading configuration at '.\partition_finder.cfg'
INFO     | 2016-02-16 23:51:18,792 | config     | Setting 'alignment' to '74_spp_updated.phy'
INFO     | 2016-02-16 23:51:18,793 | config     | Setting 'branchlengths' to 'linked'
INFO     | 2016-02-16 23:51:18,795 | parser     | You set 'models' to: all
INFO     | 2016-02-16 23:51:18,805 | model_load | This analysis will use the following 56 models of molecular evolution
INFO     | 2016-02-16 23:51:18,806 | model_load | JC, K80, TRNEF, K81, TVMEF, TIMEF, SYM, F81, HKY, TRN, K81UF, TVM, TIM, GTR, JC+G, K80+G, TRNEF+G, K81+G, TVMEF+G, TIMEF+G, SYM+G, F81+G, HKY+G, TRN+G, K81UF+G, TVM+G, TIM+G, GTR+G, JC+I, K80+I, TRNEF+I, K81+I, TVMEF+I, TIMEF+I, SYM+I, F81+I, HKY+I, TRN+I, K81UF+I, TVM+I, TIM+I, GTR+I, JC+I+G, K80+I+G, TRNEF+I+G, K81+I+G, TVMEF+I+G, TIMEF+I+G, SYM+I+G, F81+I+G, HKY+I+G, TRN+I+G, K81UF+I+G, TVM+I+G, TIM+I+G, GTR+I+G
INFO     | 2016-02-16 23:51:18,815 | config     | Setting 'model_selection' to 'bic'
INFO     | 2016-02-16 23:51:18,819 | config     | Setting 'search' to 'greedy'
INFO     | 2016-02-16 23:51:18,821 | config     | ------------------------ BEGINNING NEW RUN -------------------------------
INFO     | 2016-02-16 23:51:18,822 | config     | Looking for alignment file '.\74_spp_updated.phy'...
INFO     | 2016-02-16 23:51:18,823 | config     | Looking for tree file '.\filtered_source.phy_phyml_tree'...
INFO     | 2016-02-16 23:51:18,825 | threadpool | Using 4 cpus
INFO     | 2016-02-16 23:51:18,831 | analysis   | Beginning Analysis
INFO     | 2016-02-16 23:51:18,890 | alignment  | Reading alignment file '.\74_spp_updated.phy'
INFO     | 2016-02-16 23:51:18,933 | analysis   | Starting tree will be estimated from the data.
INFO     | 2016-02-16 23:51:18,934 | analysis   | Using user supplied topology at .\filtered_source.phy_phyml_tree
INFO     | 2016-02-16 23:51:18,938 | phyml      | Estimating GTR+I+G branch lengths on tree
INFO     | 2016-02-16 23:54:41,025 | phyml      | Branchlength estimation finished
INFO     | 2016-02-16 23:54:41,025 | analysis_m | Performing Greedy Analysis
INFO     | 2016-02-16 23:54:41,025 | analysis_m |    *** Analysing starting scheme ***
ERROR    | 2016-02-17 08:02:03,265 | main       | A program that Partitionfinder uses failed. Output follows, in case it's helpful for finding the problem
ERROR    | 2016-02-17 08:02:03,265 | main       |
ERROR    | 2016-02-17 08:02:03,265 | main       | . The file '.\analysis\start_tree\filtered_source.phy_phyml_tree' doesn't exist. . Type any key to exit.

This is despite me including a user specified topology, and that file actually existing in the directory in question.  The error message appears to have only been printed once I woke the computer from sleep mode based on the time stamp.  I don't know why it was printed once I woke the computer up from sleep mode, so I'll be testing it on only a single model later to see if the problem persists.

Any insight would be appreciated.

John

[John Soghigian]

Hello Brett, others:

Sorry for posting back to back, but I have an update, including a new error as well as an understanding of what was causing my last one. 

Because I had legacy python versions and old Anaconda versions on my desktop at home, I decided to try this procedure at work.  I reinstalled following the manual, and while in Python's directory, typed python PartitionFinderProtein.py examples\aminoacid  ... I received the same error as in my second post. 

I then realized that the tree in C:\partitionfinder\examples\aminoacid\analysis\start_tree had a .txt extension.  I removed it, and re-ran the analysis.

PF2.0 proceeds to the analysis, but now has a new error:

INFO     | 2016-02-17 12:53:49,726 | Performing Greedy Analysis

INFO     | 2016-02-17 12:53:49,727 |    *** Analysing starting scheme ***

ERROR    | 2016-02-17 12:53:54,267 |       Failed to run models WAG+I+G, WAG+I+F

, DCMUT+I, DAYHOFF+G, DAYHOFF+F, DAYHOFF+I, VT+G+F, DCMUT+G, DCMUT+F, DCMUT+I+G,

 HIVB+G+F, CPREV+F, CPREV+G, LG+G+F, CPREV+I+G+F, MTART+I+F, DCMUT+I+F, WAG+G+F,

 HIVW+I+G, HIVW+I+F, WAG+I, LG+I+G, MTREV+G+F, DAYHOFF+I+G+F, WAG+G, JTT+G, MTAR

T, CPREV+G+F, MTART+I, MTREV+I+F, MTREV+I+G, JTT+G+F, HIVB, HIVB+I+F, HIVB+I+G,

BLOSUM62+I+G+F, MTART+F, DAYHOFF+G+F, RTREV+G, RTREV+F, JTT+I+F, JTT+I+G, HIVW,

MTREV, MTART+G, RTREV, CPREV+I, MTMAM+G+F, RTREV+I+G, RTREV+I+F, MTMAM+I+F, MTMA

M+I+G, WAG+I+G+F, JTT, MTREV+I, HIVW+I, HIVW+G, HIVW+F, MTREV+F, MTREV+G, RTREV+

I, DAYHOFF, JTT+F, BLOSUM62+I, VT+I+G, VT+I+F, BLOSUM62+G, BLOSUM62+F, MTMAM+I+G

+F, CPREV+I+F, CPREV+I+G, HIVB+G, WAG+F, RTREV+I+G+F, MTREV+I+G+F, HIVB+F, DAYHO

FF+I+F, WAG, HIVB+I, MTMAM+F, RTREV+G+F, MTART+I+G+F, DCMUT+I+G+F, JTT+I+G+F, LG

+I+G+F, BLOSUM62+I+G, BLOSUM62+I+F, MTMAM+I, DAYHOFF+I+G, HIVB+I+G+F, DCMUT, MTM

AM+G, LG+F, LG+G, LG, DCMUT+G+F, MTART+G+F, CPREV, MTMAM, LG+I, HIVW+G+F, VT, HI

VW+I+G+F, LG+I+F, JTT+I, VT+I, VT+G, VT+F, BLOSUM62, BLOSUM62+G+F, MTART+I+G, VT

+I+G+F; not sure why

ERROR    | 2016-02-17 12:53:54,273 | Failed to run. See previous errors.

I tried this with other examples and had the same issue (obviously using the right Python script).  I tried with file paths and without, same issue.  I tried this with fewer models (just one) and had the same error of "not sure why."

I noticed that the files within analysis/phylofiles had .txt extensions, but they were replaced with new .txt versions when I tried replacing the extensions.  Not sure if that is related.

~John

[Brett Calcott]

Hi,

Can you try a couple of things:

1. Run the examples that are included with partitionfinder (in the examples folder). I want to see if some very basic analyses work on your machine.

2. Try running your analysis with the option “—force-restart”.  I suspect the weird error is caused by left over stuff from old attempts to run it (but I’m not sure yet).

Brett

[Rob Lanfear]

Hi All,

A couple of things to add. 

1. It might be worth attempting to recompile PhyML for your windows Machine. The version in PF2 has not been updated yet (since PF2 isn't officially released) so I wonder if there's some incompatibility there. Brett, correct me if I'm wrong, but the version to compile is this one, right? https://github.com/brettc/phyml

2. A related thing you could try, which might help track down the bug, is to try the analysis with RAxML. You can get the latest raxml.exe file from here: 

https://github.com/stamatak/standard-RAxML/blob/master/WindowsExecutables_v8.2.4_Alternate/raxmlHPC-SSE3.exe

All you need to do is download it, rename it raxml.exe, and drop it into the /programs folder. If you're feeling fancy and are going to try using an ML starting tree, you will also need to get this version of raxml:

https://github.com/stamatak/standard-RAxML/blob/master/WindowsExecutables_v8.2.4_Alternate/raxmlHPC-PTHREADS-SSE3.exe

rename it raxml_pthreads.exe, and drop it into the /programs folder as well.

Thanks for you patience and help!

Rob

To unsubscribe from this group and stop receiving emails from it, send an email to partitionfinder+unsubscribe@googlegroups.com.

[Brett Calcott]

I’m not sure how easy it will be to build phyml. The code Rob pointed to is what I used last time I compiled it when I had access to a windows machine. You’ll need to install the cygwin tools to get it compiling, but I’m not sure how successful you will be.

Sorry — the developers here all use MacOSX.

Brett

[Rob Lanfear]

In which case the RAxML trick is the one to try first. If you still get the problem, that tells us the problem is ours and narrows down (a little bit) where in our code the problem is likely to be. If you don't get the problem, then it's likely down to our PhyML version, or the code we use to interact with it.

Rob

On Friday, 19 February 2016 09:48:22 UTC+11, Brett Calcott wrote:

I’m not sure how easy it will be to build phyml. The code Rob pointed to is what I used last time I compiled it when I had access to a windows machine. You’ll need to install the cygwin tools to get it compiling, but I’m not sure how successful you will be.

Sorry — the developers here all use MacOSX.

Brett

To unsubscribe from this group and stop receiving emails from it, send an email to partitionfinder+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "PartitionFinder" group.

To unsubscribe from this group and stop receiving emails from it, send an email to partitionfinder+unsubscribe@googlegroups.com.

[Rob Lanfear]

Hi John,

Looking back at all your errors so far, they all seem consistent with PhyML not working on your machine. From our end I think we need to double check that our error handling is working here (what should happen if PhyML doesn't work is that PF will tell you it didn't work, and give you PhyML's error message to help you out). But from your end there is a very simple thing you can do: try to use the phyml.exe file in the /programs folder, and see what happens. If it works on your machine, you should be able to just double click it and get the standard phyml options screen. If that doesn't happen, I think we have your problem cornered (for now). If PhyML doesn't work when you double click it, you need to compile a version that works on your machine.

Cheers,

Rob

Rob Lanfear
School of Biological Sciences,
Macquarie University,

Sydney

phone: +61 (0)2 9850 8204

www.robertlanfear.com

[John Soghigian]

Hello all,

On a fresh machine, I was able to use PF1.11 without any trouble. I am confident my initial PF1.11 issues were due to Python3 installations from prior to using PF, because on the new machine, installing Anaconda without any previous versions allowed me to use PF1.11 for my analysis.  But that doesn't address the PF2 issues on the same machine, so I'll keep exploring that when I have a bit more time.

Brett: I could not run the examples - the error I provide is actually from one of the examples.  I will try the force restart soon.

Rob: I was able to run PhyML just fine by clicking on the exe. I even completed a simple analysis on it without trouble.  So I don't think it is phyml.  I can use my compiled, recent version of phyml (version 20151215) if you think that would help.  

I will try running the analysis with RAxML later this week and report back, as well as try various suggestions.

Question: Should the tree file in the analysis\start_tree folder have a .txt extension?  PF2.0 is creating it with a .txt extension.  I know the program is looking for filtered_source.phy_phyml_tree without the txt extension based on my error in post 4 of this thread, which I resolved by removing the txt extension.   The phylofiles also have .txt extensions when the analysis fails; is it possible that is contributing to the error?

[Rob Lanfear]

Hi John,

Thanks for the update.

I'm not totally sure what files you expect in the start_tree folder on windows. However, one way to figure it out is to look at what PF1.1 is creating. PF2 should be the same, unless you specify the --ml-tree option. (And if you are specifying that, try it without that and see if it works!).

Cheers,

Rob