pathway_pipeline.py key error

[jayos...@gmail.com]

Hey all,

I am trying to run the pathway_pipeline.py and am encountering an issue (shown below). 

Command:

pathway_pipeline.py -i KO_metagenome_out/pred_metagenome_contrib.tsv.gz -o KEGG_pathways_out --no_regroup --map ../KEGG_pathways_to_KO.tsv 

Error:
Traceback (most recent call last):
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/bin/pathway_pipeline.py", line 283, in <module>
    main()
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/bin/pathway_pipeline.py", line 207, in main
    unstrat_abun_per_seq = pathway_pipeline(
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/lib/python3.9/site-packages/picrust2/pathway_pipeline.py", line 383, in pathway_pipeline
    path_out_raw = Parallel(n_jobs=proc)(delayed(contrib_format_pathway_levels)(
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/lib/python3.9/site-packages/joblib/parallel.py", line 1986, in __call__
    return output if self.return_generator else list(output)
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/lib/python3.9/site-packages/joblib/parallel.py", line 1914, in _get_sequential_output
    res = func(*args, **kwargs)
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/lib/python3.9/site-packages/picrust2/pathway_pipeline.py", line 862, in contrib_format_pathway_levels
    reaction_abun = unstrat_input[sample_id].to_dict(into=defaultdict(int))
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/lib/python3.9/site-packages/pandas/core/frame.py", line 4107, in __getitem__
    indexer = self.columns.get_loc(key)
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/lib/python3.9/site-packages/pandas/core/indexes/range.py", line 417, in get_loc
    raise KeyError(key)
KeyError: 'pathway_pipeline.py -i KO_metagenome_out/pred_metagenome_contrib.tsv.gz -o KEGG_pathways_out --no_regroup --map ../KEGG_pathways_to_KO.tsv
Traceback (most recent call last):
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/bin/pathway_pipeline.py", line 283, in <module>
    main()
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/bin/pathway_pipeline.py", line 207, in main
    unstrat_abun_per_seq = pathway_pipeline(
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/lib/python3.9/site-packages/picrust2/pathway_pipeline.py", line 383, in pathway_pipeline
    path_out_raw = Parallel(n_jobs=proc)(delayed(contrib_format_pathway_levels)(
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/lib/python3.9/site-packages/joblib/parallel.py", line 1986, in __call__
    return output if self.return_generator else list(output)
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/lib/python3.9/site-packages/joblib/parallel.py", line 1914, in _get_sequential_output
    res = func(*args, **kwargs)
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/lib/python3.9/site-packages/picrust2/pathway_pipeline.py", line 862, in contrib_format_pathway_levels
    reaction_abun = unstrat_input[sample_id].to_dict(into=defaultdict(int))
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/lib/python3.9/site-packages/pandas/core/frame.py", line 4107, in __getitem__
    indexer = self.columns.get_loc(key)
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/lib/python3.9/site-packages/pandas/core/indexes/range.py", line 417, in get_loc
    raise KeyError(key)
KeyError: 'JMF-2506-02-0001A'

The closest issue I could find was this one: https://groups.google.com/g/picrust-users/c/Hoc9nEOhy1o/m/1f4t_8k6EQAJ

It doesn't seem to have been resolved.

Is there any way to fix this?

Thank you!

[Robyn Wright]

Hello,

It looks like you are trying to run PICRUSt2 with a custom mapping file? Could you confirm whether PICRUSt2 works when you don't use this file? If it indeed does, could you send me this custom file please so I can try to work out what the issue might be? It appears as though some of the mapping names probably aren't matching. 

Thanks,

Robyn

[jayos...@gmail.com]

Hello,

That file was actually the KEGG_to_KO file from the picrust github. 

I have tried it again without the mapping file and I get the same issue. 

pathway_pipeline.py -i KO_metagenome_out/pred_metagenome_contrib.tsv.gz -o KEGG_pathways_out --no_regroup

Traceback (most recent call last):
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/bin/pathway_pipeline.py", line 283, in <module>
    main()
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/bin/pathway_pipeline.py", line 207, in main
    unstrat_abun_per_seq = pathway_pipeline(
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/lib/python3.9/site-packages/picrust2/pathway_pipeline.py", line 383, in pathway_pipeline
    path_out_raw = Parallel(n_jobs=proc)(delayed(contrib_format_pathway_levels)(
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/lib/python3.9/site-packages/joblib/parallel.py", line 1986, in __call__
    return output if self.return_generator else list(output)
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/lib/python3.9/site-packages/joblib/parallel.py", line 1914, in _get_sequential_output
    res = func(*args, **kwargs)
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/lib/python3.9/site-packages/picrust2/pathway_pipeline.py", line 862, in contrib_format_pathway_levels
    reaction_abun = unstrat_input[sample_id].to_dict(into=defaultdict(int))
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/lib/python3.9/site-packages/pandas/core/frame.py", line 4107, in __getitem__
    indexer = self.columns.get_loc(key)
  File "/lisc/app/conda/miniforge3/envs/picrust2-2.6.2/lib/python3.9/site-packages/pandas/core/indexes/range.py", line 417, in get_loc
    raise KeyError(key)
KeyError: 'JMF-2506-02-0001A'

This issue doesn't happen if I use the "pred_metagenome_unstrat.tsv.gz" file.

Any ideas?

Thanks!

[Robyn Wright]

Hmm OK. Can you post your commands that you ran to get to this point? Was there a reason that you're running the commands separately rather than as part of the full pipeline? What is 'JMF-2506-02-0001A' - is this a sample name of yours or something? And are you able to run through everything with the test data here?

Also apologies in advance if my next reply is slow - I'm on vacation as of tomorrow for the next two weeks.

Best wishes,

Robyn