Error executing snakemake in PyPSA-Eur
542 views
Skip to first unread message
Swaechchha Dahal
Oct 5, 2023, 4:38:48 AM10/5/23
to pypsa
Hi,
I've encountered an error when attempting to run a snakemake command. Even after reinstalling pypsa-eur to ensure that everything was correctly set up, the error persisted. Here's the error message I received:
The only modification I made was to the countries in the config.default.
Would appreciate any guidance or insights on how to resolve this. Thank you in advance!
Best,
Swaechchha
Fabian Hofmann
Oct 5, 2023, 4:41:35 AM10/5/23
to py...@googlegroups.com
Hey Swaechchha,
could you send the full stack-trace (error message), please?
Best wishes
Fabian H
--
You received this message because you are subscribed to the Google Groups "pypsa" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pypsa+un...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/pypsa/3c412bfe-3fbb-4238-98b4-594930b71ca0n%40googlegroups.com.
-- Fabian Hofmann Postdoctoral Researcher Institute of Energy Technology Technische Universität Berlin http://fabianhofmann.org/ Group website: https://tub-ensys.github.io/
Swaechchha Dahal
Oct 9, 2023, 5:08:14 AM10/9/23
to pypsa
Hi Fabian,
I apologize if I wasn't clear in my initial post. When I run the tutorial with the command:
I apologize if I wasn't clear in my initial post. When I run the tutorial with the command:
snakemake -call results/test-elec/networks/elec_s_6_ec_lcopt_Co2L-24H.nc --configfile config/test/config.electricity.yaml
It executes without any issues. However, the error appears specifically when I modify the config file. Here's the detailed error message I received:
(pypsa-eur) sdaha@E-CND2191YQ3:~/pypsa-eur$ snakemake -call results/networks/elec.nc --configfile config/config.nordic.yaml
Config file config/config.yaml is extended by additional config specified via the command line.
localrules directive specifies rules that are not present in the Snakefile:
copy_conda_env
Building DAG of jobs...
MissingRuleException:
No rule to produce results/networks/elec.nc (if you use input functions make sure that they don't raise unexpected exceptions).
Config file config/config.yaml is extended by additional config specified via the command line.
localrules directive specifies rules that are not present in the Snakefile:
copy_conda_env
Building DAG of jobs...
MissingRuleException:
No rule to produce results/networks/elec.nc (if you use input functions make sure that they don't raise unexpected exceptions).
Thank you, and I truly appreciate any insights you can provide on this issue.
Best regards,
Swaechchha
Best regards,
Swaechchha
Johannes Hampp
Oct 10, 2023, 2:11:10 AM10/10/23
to Swaechchha Dahal, pypsa
Hi Swaechchha,
As the error message says:
```
Your target (`results/networks/elec.nc) in the snakemake call
```
```
does not seem supported by your configfile entries. Your configfile
entries would probably lead to a different filename/path.
Recommendation: Run
```
snakemake -call solve_elec_networks --configfile
config/config.nordic.yaml -n
```
which automatically fills in the correct file paths based on your
configfile with a dry-run (i.e. no files are changed).
Best,
Johannes
On 09/10/2023 11:08, Swaechchha Dahal wrote:
> Hi Fabian,
>
> I apologize if I wasn't clear in my initial post. When I run the
> tutorial with the command:
> /snakemake -call
> results/test-elec/networks/elec_s_6_ec_lcopt_Co2L-24H.nc --configfile
> config/test/config.electricity.yaml/
> /
> /
> /
> /(pypsa-eur) sdaha@E-CND2191YQ3:~/pypsa-eur$ snakemake -call
> /
> /
> /
>
> Group website:https://tub-ensys.github.io/ <https://tub-ensys.github.io/>
>
> ____
As the error message says:
```
No rule to produce results/networks/elec.nc (if you use input functions
make sure that they don't raise unexpected exceptions).
```
make sure that they don't raise unexpected exceptions).
Your target (`results/networks/elec.nc) in the snakemake call
```
```
does not seem supported by your configfile entries. Your configfile
entries would probably lead to a different filename/path.
Recommendation: Run
```
snakemake -call solve_elec_networks --configfile
config/config.nordic.yaml -n
```
which automatically fills in the correct file paths based on your
configfile with a dry-run (i.e. no files are changed).
Best,
Johannes
On 09/10/2023 11:08, Swaechchha Dahal wrote:
> Hi Fabian,
>
> I apologize if I wasn't clear in my initial post. When I run the
> tutorial with the command:
> results/test-elec/networks/elec_s_6_ec_lcopt_Co2L-24H.nc --configfile
> config/test/config.electricity.yaml/
> /
> /
> It executes without any issues. However, the error appears specifically
> when I modify the config file. Here's the detailed error message I
> received:/
> when I modify the config file. Here's the detailed error message I
> /
> /(pypsa-eur) sdaha@E-CND2191YQ3:~/pypsa-eur$ snakemake -call
> results/networks/elec.nc --configfile config/config.nordic.yaml
> Config file config/config.yaml is extended by additional config
> specified via the command line.
> localrules directive specifies rules that are not present in the Snakefile:
> copy_conda_env
>
> Building DAG of jobs...
> MissingRuleException:
> No rule to produce results/networks/elec.nc (if you use input functions
> make sure that they don't raise unexpected exceptions)./
> Config file config/config.yaml is extended by additional config
> specified via the command line.
> localrules directive specifies rules that are not present in the Snakefile:
> copy_conda_env
>
> Building DAG of jobs...
> MissingRuleException:
> No rule to produce results/networks/elec.nc (if you use input functions
> /
> /
> Thank you, and I truly appreciate any insights you can provide on this
> issue.
>
> Best regards,
> Swaechchha
>
>
>
>
>
> /
> issue.
>
> Best regards,
> Swaechchha
>
>
>
>
>
> /
> On Thursday, 5 October 2023 at 10:41:35 UTC+2 Fabian Hofmann wrote:
>
> __
>
>
> Hey Swaechchha,
>
>
> could you send the full stack-trace (error message), please?
>
>
> Best wishes
>
> Fabian H
>
>
> On 05.10.23 10:38, Swaechchha Dahal wrote:
>> Hi,
>>
>> I've encountered an error when attempting to run a snakemake
>> command. Even after reinstalling pypsa-eur to ensure that
>> everything was correctly set up, the error persisted. Here's the
>> error message I received:
> Hey Swaechchha,
>
>
> could you send the full stack-trace (error message), please?
>
>
> Best wishes
>
> Fabian H
>
>
> On 05.10.23 10:38, Swaechchha Dahal wrote:
>> Hi,
>>
>> I've encountered an error when attempting to run a snakemake
>> command. Even after reinstalling pypsa-eur to ensure that
>> everything was correctly set up, the error persisted. Here's the
>> error message I received:
>> Error.PNG
>>
>> The only modification I made was to the countries in the
>> config.default.
>>
>> Would appreciate any guidance or insights on how to resolve this.
>> Thank you in advance!
>>
>> Best,
>> Swaechchha
>> --
>> You received this message because you are subscribed to the Google
>> Groups "pypsa" group.
>> To unsubscribe from this group and stop receiving emails from it,
>> send an email to pypsa+un...@googlegroups.com.
>> To view this discussion on the web, visit
>> https://groups.google.com/d/msgid/pypsa/3c412bfe-3fbb-4238-98b4-594930b71ca0n%40googlegroups.com <https://groups.google.com/d/msgid/pypsa/3c412bfe-3fbb-4238-98b4-594930b71ca0n%40googlegroups.com?utm_medium=email&utm_source=footer>.
>>
>> The only modification I made was to the countries in the
>> config.default.
>>
>> Would appreciate any guidance or insights on how to resolve this.
>> Thank you in advance!
>>
>> Best,
>> Swaechchha
>> --
>> You received this message because you are subscribed to the Google
>> Groups "pypsa" group.
>> To unsubscribe from this group and stop receiving emails from it,
>> send an email to pypsa+un...@googlegroups.com.
>> To view this discussion on the web, visit
>
> --
> Fabian Hofmann
>
> Postdoctoral Researcher
> Institute of Energy Technology
> Technische Universität Berlin
> http://fabianhofmann.org/ <http://fabianhofmann.org/>
> --
> Fabian Hofmann
>
> Postdoctoral Researcher
> Institute of Energy Technology
> Technische Universität Berlin
>
> Group website:https://tub-ensys.github.io/ <https://tub-ensys.github.io/>
>
> ____
>
> --
> You received this message because you are subscribed to the Google
> Groups "pypsa" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to pypsa+un...@googlegroups.com
> <mailto:pypsa+un...@googlegroups.com>.
> --
> You received this message because you are subscribed to the Google
> Groups "pypsa" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to pypsa+un...@googlegroups.com
> To view this discussion on the web, visit
> https://groups.google.com/d/msgid/pypsa/ad3d73e5-1e6f-4f9b-8d0d-7605ebd05e36n%40googlegroups.com <https://groups.google.com/d/msgid/pypsa/ad3d73e5-1e6f-4f9b-8d0d-7605ebd05e36n%40googlegroups.com?utm_medium=email&utm_source=footer>.
Swaechchha Dahal
Oct 13, 2023, 2:26:06 AM10/13/23
to pypsa
Thank you, Johannes. I tried the command you suggested, and unfortunately, I'm encountering another error.
Error in rule build_ship_raster:
jobid: 17
input: data/shipdensity_global.zip, cutouts/europe-2013-sarah.nc, cutouts/europe-2013-era5.nc, cutouts/europe-2013-era5.nc, cutouts/europe-2013-era5.nc, cutouts/europe-2013-era5.nc
output: resources/shipdensity_raster.tif
log: logs/build_ship_raster.log (check log file(s) for error details)
conda-env: /home/sdaha/pypsa-eur/.snakemake/conda/ccbebbbab5ec647bc4ed52e844531a6e_
RuleException:
CalledProcessError in file /home/sdaha/pypsa-eur/rules/build_electricity.smk, line 206:
Command 'set -euo pipefail; /home/sdaha/miniconda3/envs/pypsa-eur/bin/python3.10 /home/sdaha/pypsa-eur/.snakemake/scripts/tmpc88p3lhk.build_ship_raster.py' returned non-zero exit status 1.
File "/home/sdaha/pypsa-eur/rules/build_electricity.smk", line 206, in __rule_build_ship_raster
File "/home/sdaha/miniconda3/envs/pypsa-eur/lib/python3.10/concurrent/futures/thread.py", line 58, in run
I have also attached the error log. I'd appreciate any insights you have on this.
jobid: 17
input: data/shipdensity_global.zip, cutouts/europe-2013-sarah.nc, cutouts/europe-2013-era5.nc, cutouts/europe-2013-era5.nc, cutouts/europe-2013-era5.nc, cutouts/europe-2013-era5.nc
output: resources/shipdensity_raster.tif
log: logs/build_ship_raster.log (check log file(s) for error details)
conda-env: /home/sdaha/pypsa-eur/.snakemake/conda/ccbebbbab5ec647bc4ed52e844531a6e_
RuleException:
CalledProcessError in file /home/sdaha/pypsa-eur/rules/build_electricity.smk, line 206:
Command 'set -euo pipefail; /home/sdaha/miniconda3/envs/pypsa-eur/bin/python3.10 /home/sdaha/pypsa-eur/.snakemake/scripts/tmpc88p3lhk.build_ship_raster.py' returned non-zero exit status 1.
File "/home/sdaha/pypsa-eur/rules/build_electricity.smk", line 206, in __rule_build_ship_raster
File "/home/sdaha/miniconda3/envs/pypsa-eur/lib/python3.10/concurrent/futures/thread.py", line 58, in run
I have also attached the error log. I'd appreciate any insights you have on this.
Johannes Hampp
Oct 16, 2023, 3:25:26 AM10/16/23
to Swaechchha Dahal, pypsa
Hi Swaechchha,
Did you check the log file `logs/build_ship_raster.log` which snakemake
is referring to for details on the error?
Best,
Johannes
On 13/10/2023 08:26, Swaechchha Dahal wrote:
> Thank you, Johannes. I tried the command you suggested, and
> unfortunately, I'm encountering another error.
>
> /Error in rule build_ship_raster:
> snakemake call
>
> ```
> snakemake -call results/networks/elec.nc <http://elec.nc> --configfile
> > results/networks/elec.nc <http://elec.nc> --configfile
> https://groups.google.com/d/msgid/pypsa/3c412bfe-3fbb-4238-98b4-594930b71ca0n%40googlegroups.com <https://groups.google.com/d/msgid/pypsa/3c412bfe-3fbb-4238-98b4-594930b71ca0n%40googlegroups.com> <https://groups.google.com/d/msgid/pypsa/3c412bfe-3fbb-4238-98b4-594930b71ca0n%40googlegroups.com?utm_medium=email&utm_source=footer <https://groups.google.com/d/msgid/pypsa/3c412bfe-3fbb-4238-98b4-594930b71ca0n%40googlegroups.com?utm_medium=email&utm_source=footer>>.
Did you check the log file `logs/build_ship_raster.log` which snakemake
is referring to for details on the error?
Best,
Johannes
On 13/10/2023 08:26, Swaechchha Dahal wrote:
> Thank you, Johannes. I tried the command you suggested, and
> unfortunately, I'm encountering another error.
>
> jobid: 17
> input: data/shipdensity_global.zip, cutouts/europe-2013-sarah.nc,
> cutouts/europe-2013-era5.nc, cutouts/europe-2013-era5.nc,
> cutouts/europe-2013-era5.nc, cutouts/europe-2013-era5.nc
> output: resources/shipdensity_raster.tif
> log: logs/build_ship_raster.log (check log file(s) for error details)
> conda-env:
> /home/sdaha/pypsa-eur/.snakemake/conda/ccbebbbab5ec647bc4ed52e844531a6e_
>
> RuleException:
> CalledProcessError in file
> /home/sdaha/pypsa-eur/rules/build_electricity.smk, line 206:
> Command 'set -euo pipefail;
> /home/sdaha/miniconda3/envs/pypsa-eur/bin/python3.10
> /home/sdaha/pypsa-eur/.snakemake/scripts/tmpc88p3lhk.build_ship_raster.py' returned non-zero exit status 1.
> File "/home/sdaha/pypsa-eur/rules/build_electricity.smk", line 206,
> in __rule_build_ship_raster
> File
> "/home/sdaha/miniconda3/envs/pypsa-eur/lib/python3.10/concurrent/futures/thread.py", line 58, in run/
> input: data/shipdensity_global.zip, cutouts/europe-2013-sarah.nc,
> cutouts/europe-2013-era5.nc, cutouts/europe-2013-era5.nc,
> cutouts/europe-2013-era5.nc, cutouts/europe-2013-era5.nc
> output: resources/shipdensity_raster.tif
> log: logs/build_ship_raster.log (check log file(s) for error details)
> conda-env:
> /home/sdaha/pypsa-eur/.snakemake/conda/ccbebbbab5ec647bc4ed52e844531a6e_
>
> RuleException:
> CalledProcessError in file
> /home/sdaha/pypsa-eur/rules/build_electricity.smk, line 206:
> Command 'set -euo pipefail;
> /home/sdaha/miniconda3/envs/pypsa-eur/bin/python3.10
> /home/sdaha/pypsa-eur/.snakemake/scripts/tmpc88p3lhk.build_ship_raster.py' returned non-zero exit status 1.
> File "/home/sdaha/pypsa-eur/rules/build_electricity.smk", line 206,
> in __rule_build_ship_raster
> File
>
> I have also attached the error log. I'd appreciate any insights you have
> on this.
>
> On Tuesday, 10 October 2023 at 11:56:10 UTC+5:45 Johannes wrote:
>
> Hi Swaechchha,
>
> As the error message says:
>
> ```
> No rule to produce results/networks/elec.nc <http://elec.nc> (if you
> I have also attached the error log. I'd appreciate any insights you have
> on this.
>
> On Tuesday, 10 October 2023 at 11:56:10 UTC+5:45 Johannes wrote:
>
> Hi Swaechchha,
>
> As the error message says:
>
> ```
> use input functions
> make sure that they don't raise unexpected exceptions).
> ```
>
> Your target (`results/networks/elec.nc <http://elec.nc>) in the
> make sure that they don't raise unexpected exceptions).
> ```
>
> snakemake call
>
> ```
> snakemake -call results/networks/elec.nc <http://elec.nc> --configfile
> config/config.nordic.yaml
> > Config file config/config.yaml is extended by additional config
> > specified via the command line.
> > localrules directive specifies rules that are not present in the
> Snakefile:
> > copy_conda_env
> >
> > Building DAG of jobs...
> > MissingRuleException:
> > No rule to produce results/networks/elec.nc <http://elec.nc> (if
> > Config file config/config.yaml is extended by additional config
> > specified via the command line.
> > localrules directive specifies rules that are not present in the
> Snakefile:
> > copy_conda_env
> >
> > Building DAG of jobs...
> > MissingRuleException:
> >
> > --
> > Fabian Hofmann
> >
> > Postdoctoral Researcher
> > Institute of Energy Technology
> > Technische Universität Berlin
> > http://fabianhofmann.org/ <http://fabianhofmann.org/>
> <http://fabianhofmann.org/ <http://fabianhofmann.org/>>
> >
> > Group website:https://tub-ensys.github.io/
> <https://tub-ensys.github.io/> <https://tub-ensys.github.io/
> > --
> > Fabian Hofmann
> >
> > Postdoctoral Researcher
> > Institute of Energy Technology
> > Technische Universität Berlin
> > http://fabianhofmann.org/ <http://fabianhofmann.org/>
> <http://fabianhofmann.org/ <http://fabianhofmann.org/>>
> >
> > Group website:https://tub-ensys.github.io/
> <https://tub-ensys.github.io/>>
> >
> > ____
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "pypsa" group.
> > To unsubscribe from this group and stop receiving emails from it,
> send
> > an email to pypsa+un...@googlegroups.com
> > <mailto:pypsa+un...@googlegroups.com>.
> > To view this discussion on the web, visit
> >
> https://groups.google.com/d/msgid/pypsa/ad3d73e5-1e6f-4f9b-8d0d-7605ebd05e36n%40googlegroups.com <https://groups.google.com/d/msgid/pypsa/ad3d73e5-1e6f-4f9b-8d0d-7605ebd05e36n%40googlegroups.com> <https://groups.google.com/d/msgid/pypsa/ad3d73e5-1e6f-4f9b-8d0d-7605ebd05e36n%40googlegroups.com?utm_medium=email&utm_source=footer <https://groups.google.com/d/msgid/pypsa/ad3d73e5-1e6f-4f9b-8d0d-7605ebd05e36n%40googlegroups.com?utm_medium=email&utm_source=footer>>.
> >
> > ____
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "pypsa" group.
> > To unsubscribe from this group and stop receiving emails from it,
> send
> > an email to pypsa+un...@googlegroups.com
> > <mailto:pypsa+un...@googlegroups.com>.
> > To view this discussion on the web, visit
> >
>
> --
> You received this message because you are subscribed to the Google
> Groups "pypsa" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to pypsa+un...@googlegroups.com
> <mailto:pypsa+un...@googlegroups.com>.
> To view this discussion on the web, visit
> https://groups.google.com/d/msgid/pypsa/7160c2de-7c06-4218-99be-73cef48e11dfn%40googlegroups.com <https://groups.google.com/d/msgid/pypsa/7160c2de-7c06-4218-99be-73cef48e11dfn%40googlegroups.com?utm_medium=email&utm_source=footer>.
> --
> You received this message because you are subscribed to the Google
> Groups "pypsa" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to pypsa+un...@googlegroups.com
> <mailto:pypsa+un...@googlegroups.com>.
> To view this discussion on the web, visit
Swaechchha Dahal
Oct 21, 2023, 7:52:32 AM10/21/23
to pypsa
Dear Johannes,
Thank you for your prompt response to my previous issue. I reinstalled everything and made another attempt, but I've run into a new error. Specifically, I'm encountering a "ValueError: negative column index found". I've attached a screenshot of the terminal output for your reference.
The relevant traceback points to a negative column index during some matrix operation, which I assume might be related to data indexing or referencing. I've made sure to use the most recent versions of all input files and followed the recommended setup and configuration steps.
Could you please provide guidance on how to address this issue? Any insights or potential solutions would be greatly appreciated.
Best regards,
Swaechchha Dahal
Thank you for your prompt response to my previous issue. I reinstalled everything and made another attempt, but I've run into a new error. Specifically, I'm encountering a "ValueError: negative column index found". I've attached a screenshot of the terminal output for your reference.
The relevant traceback points to a negative column index during some matrix operation, which I assume might be related to data indexing or referencing. I've made sure to use the most recent versions of all input files and followed the recommended setup and configuration steps.
Could you please provide guidance on how to address this issue? Any insights or potential solutions would be greatly appreciated.
Best regards,
Swaechchha Dahal
Reply all
Reply to author
Forward
0 new messages