pathway_pipeline.py key error

[jayos...@gmail.com]

Hey all,

I am trying to run the pathway_pipeline.py and am encountering an issue (shown below). 

Command:

pathway_pipeline.py -i KO_metagenome_out/pred_metagenome_contrib.tsv.gz -o KEGG_pathways_out --no_regroup --map ../KEGG_pathways_to_KO.tsv 

Error:
Traceback (most recent call last):
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/bin/pathway_pipeline.py", line 283, in <module>
    main()
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/bin/pathway_pipeline.py", line 207, in main
    unstrat_abun_per_seq = pathway_pipeline(
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/lib/python3.9/site-packages/picrust2/pathway_pipeline.py", line 383, in pathway_pipeline
    path_out_raw = Parallel(n_jobs=proc)(delayed(contrib_format_pathway_levels)(
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/lib/python3.9/site-packages/joblib/parallel.py", line 1986, in __call__
    return output if self.return_generator else list(output)
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/lib/python3.9/site-packages/joblib/parallel.py", line 1914, in _get_sequential_output
    res = func(*args, **kwargs)
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/lib/python3.9/site-packages/picrust2/pathway_pipeline.py", line 862, in contrib_format_pathway_levels
    reaction_abun = unstrat_input[sample_id].to_dict(into=defaultdict(int))
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/lib/python3.9/site-packages/pandas/core/frame.py", line 4107, in __getitem__
    indexer = self.columns.get_loc(key)
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/lib/python3.9/site-packages/pandas/core/indexes/range.py", line 417, in get_loc
    raise KeyError(key)
KeyError: 'pathway_pipeline.py -i KO_metagenome_out/pred_metagenome_contrib.tsv.gz -o KEGG_pathways_out --no_regroup --map ../KEGG_pathways_to_KO.tsv
Traceback (most recent call last):
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/bin/pathway_pipeline.py", line 283, in <module>
    main()
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/bin/pathway_pipeline.py", line 207, in main
    unstrat_abun_per_seq = pathway_pipeline(
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/lib/python3.9/site-packages/picrust2/pathway_pipeline.py", line 383, in pathway_pipeline
    path_out_raw = Parallel(n_jobs=proc)(delayed(contrib_format_pathway_levels)(
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/lib/python3.9/site-packages/joblib/parallel.py", line 1986, in __call__
    return output if self.return_generator else list(output)
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/lib/python3.9/site-packages/joblib/parallel.py", line 1914, in _get_sequential_output
    res = func(*args, **kwargs)
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/lib/python3.9/site-packages/picrust2/pathway_pipeline.py", line 862, in contrib_format_pathway_levels
    reaction_abun = unstrat_input[sample_id].to_dict(into=defaultdict(int))
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/lib/python3.9/site-packages/pandas/core/frame.py", line 4107, in __getitem__
    indexer = self.columns.get_loc(key)
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/lib/python3.9/site-packages/pandas/core/indexes/range.py", line 417, in get_loc
    raise KeyError(key)
KeyError: 'JMF-2506-02-0001A'

The closest issue I could find was this one: https://groups.google.com/g/picrust-users/c/Hoc9nEOhy1o/m/1f4t_8k6EQAJ

It doesn't seem to have been resolved.

Is there any way to fix this?

Thank you!

[Robyn Wright]

Hello,

It looks like you are trying to run PICRUSt2 with a custom mapping file? Could you confirm whether PICRUSt2 works when you don't use this file? If it indeed does, could you send me this custom file please so I can try to work out what the issue might be? It appears as though some of the mapping names probably aren't matching. 

Thanks,

Robyn

[jayos...@gmail.com]

Hello,

That file was actually the KEGG_to_KO file from the picrust github. 

I have tried it again without the mapping file and I get the same issue. 

pathway_pipeline.py -i KO_metagenome_out/pred_metagenome_contrib.tsv.gz -o KEGG_pathways_out --no_regroup

Traceback (most recent call last):
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/bin/pathway_pipeline.py", line 283, in <module>
    main()
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/bin/pathway_pipeline.py", line 207, in main
    unstrat_abun_per_seq = pathway_pipeline(
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/lib/python3.9/site-packages/picrust2/pathway_pipeline.py", line 383, in pathway_pipeline
    path_out_raw = Parallel(n_jobs=proc)(delayed(contrib_format_pathway_levels)(
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/lib/python3.9/site-packages/joblib/parallel.py", line 1986, in __call__
    return output if self.return_generator else list(output)
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/lib/python3.9/site-packages/joblib/parallel.py", line 1914, in _get_sequential_output
    res = func(*args, **kwargs)
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/lib/python3.9/site-packages/picrust2/pathway_pipeline.py", line 862, in contrib_format_pathway_levels
    reaction_abun = unstrat_input[sample_id].to_dict(into=defaultdict(int))
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/lib/python3.9/site-packages/pandas/core/frame.py", line 4107, in __getitem__
    indexer = self.columns.get_loc(key)
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/lib/python3.9/site-packages/pandas/core/indexes/range.py", line 417, in get_loc
    raise KeyError(key)
KeyError: 'JMF-2506-02-0001A'

This issue doesn't happen if I use the "pred_metagenome_unstrat.tsv.gz" file.

Any ideas?

Thanks!

[Robyn Wright]

Hmm OK. Can you post your commands that you ran to get to this point? Was there a reason that you're running the commands separately rather than as part of the full pipeline? What is 'JMF-2506-02-0001A' - is this a sample name of yours or something? And are you able to run through everything with the test data here?

Also apologies in advance if my next reply is slow - I'm on vacation as of tomorrow for the next two weeks.

Best wishes,

Robyn

[Zheng]

Hello,

I am encountering the exact same issue when I tried to run:

pathway_pipeline.py -i KO_metagenome_out/pred_metagenome_contrib.tsv.gz -o KEGG_pathways_out --no_regroup --map KEGG_pathways_to_KO.tsv.

This command is copied from the FAQ page on picrust github, and the KEGG_pathways_to_KO.tsv is the file from default_files/pathway_mapfiles in the picrust repo.

I wonder if you were able to resolve this issue?

Thanks!

[Robyn Wright]

Hi there,

We are now moving everything from here over to Github. Could you please post this over there (PICRUSt2 Github) and fill in the bug report form with details of versions, etc?

Best wishes,

Robyn