Problem with final reports (pigx-rnaseq)

149 views
Skip to first unread message

Alisa Fuchs

unread,
Jun 24, 2019, 10:33:23 AM6/24/19
to pigx
Hi,

I am running the pigx-rnaseq pipeline and there is a problem with final reports:

----
label: run_deseq2
design formula:~ time + AnalysisGroup
Quitting from lines 155-200 (deseqReport.Rmd) 
Error in DESeqDataSet(se, design = design, ignoreRank) : 
  counts matrix should be numeric, currently it has mode: logical
Calls: runReport ... withVisible -> eval -> eval -> <Anonymous> -> DESeqDataSet

Execution halted
---

See also a log file attached for more details.

Not sure what to do.

Thanks for help,
Alisa



IC-VC_diff.report.star.log

Bora Uyar

unread,
Jun 24, 2019, 2:20:06 PM6/24/19
to Alisa Fuchs, pigx
Hi Alisa,

Could you please check the contents of the count file and colData file that was used to do the differential expression analysis?

countDataFile: "/fast/AG_Zinzen/Alisa/Analysis/results/preprocessed_data/counts_from_STAR.tsv"

I am guessing that the count file is empty. 

I recognized a bug with this recently and fixed it. I would recommend you to update the pigx pipeline to the latest version (0.0.7). 

The new version also comes with normalized bigwig files (you were looking for a solution to this in our last discussion), deseq normalized counts for all samples in one table, and possibility to configure the counting step my modifying the htseq-count tool's arguments in the settings.yaml file. 

Please check the latest pipeline documentation at http://bioinformatics.mdc-berlin.de/pigx_docs/pigx-rna-seq.html

Best,

Bora

--
You received this message because you are subscribed to the Google Groups "pigx" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pigx+uns...@googlegroups.com.
To post to this group, send email to pi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pigx/d105d38a-56de-4922-9903-2363e6bc777e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


--

_____________

Dr. Bora Uyar

Bioinformatics Scientist

Bioinformatics and Omics Data Science

Max Delbrueck Center (MDC) for Molecular Medicine

The Berlin Institute for Medical Systems Biology (BIMSB): 
Hannoversche Str. 28, 10115 Berlin 

web: http://bioinformatics.mdc-berlin.de/team.html#bora-uyar-phd
email: bora...@mdc-berlin.de
office tel: +49 30 9406 1545
mobile: +49 172 949 5680

Alisa Fuchs

unread,
Jun 25, 2019, 4:42:24 AM6/25/19
to pigx
Dear Bora,

Thank you very much for your help. 
However I think you overestimate my command line experience. I managed to find the 0.0.7 version of pigx-rnaseq, but the installation instruction page is down (https://guix.mdc-berlin.de/documentation.html#commands). So could you please lower your expectations and give me clear instructions on how to do it. 
It's been 5 weeks since I started to use pigs pipeline and I still did not manage to get full analysis done.

Best,
Alisa 

To unsubscribe from this group and stop receiving emails from it, send an email to pi...@googlegroups.com.

To post to this group, send email to pi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pigx/d105d38a-56de-4922-9903-2363e6bc777e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Bora Uyar

unread,
Jun 25, 2019, 5:03:29 AM6/25/19
to Alisa Fuchs, pigx
Hi Alisa,

You need to access the max cluster and get an interactive session. 
> ssh <username>@max-login.mdc-berlin.net
> qrsh

This is the command to install/update the package: 
> guixr package -i pigx-rnaseq

Bora


To unsubscribe from this group and stop receiving emails from it, send an email to pigx+uns...@googlegroups.com.

To post to this group, send email to pi...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Ricardo Wurmus

unread,
Jun 25, 2019, 5:07:26 AM6/25/19
to Alisa Fuchs, Bora Uyar, pigx
Hi Alisa,

Bora Uyar <borauy...@gmail.com> writes:

> You need to access the max cluster and get an interactive session.
>> ssh <username>@max-login.mdc-berlin.net
>> qrsh
>
> This is the command to install/update the package:
>> guixr package -i pigx-rnaseq

Optionally before that: run “guixr pull” to update Guix to the latest
version which includes pigx-rnaseq version 0.0.7.

--
Ricardo

Alisa Fuchs

unread,
Jun 25, 2019, 5:56:15 AM6/25/19
to pigx
Dear Bora, Dear Ricardo,

Thank you for fast reply.

Just to be clear, these are the instructions on pigx-rnaseq page:

If you want to install PiGx RNAseq from source, please make sure that all required dependencies (see "requirements.txt") are installed and then follow the common GNU build system steps after unpacking the latest release tarball:

./configure --prefix=/some/where
make install
If I run 
> guixr package -i pigx-rnaseq

I get:

The following package will be upgraded:

   pigx-rnaseq 0.0.5 -> 0.0.5 /gnu/store/m22fqf560vnla52kyiw67v5dxrb292jf-pigx-rnaseq-0.0.5


Only after  
> guixr pull 
> guixr package -i pigx-rnaseq

I get a 0.0.7 version:

The following package will be upgraded:

   pigx-rnaseq 0.0.5 -> 0.0.7 /gnu/store/vg6b5cba07nr3llwa8r5fnb3q0mnbbry-pigx-rnaseq-0.0.7




Maybe clearer instruction for users would help save time on both sides.

Thanks for your help,
Alisa

Ricardo Wurmus

unread,
Jun 25, 2019, 6:17:01 AM6/25/19
to Alisa Fuchs, pigx

Hi Alisa,

thanks for your feedback.

> Just to be clear, these are the instructions on pigx-rnaseq page:
>
> If you want to install PiGx RNAseq from source, please make sure that all
> required dependencies (see "requirements.txt") are installed and then
> follow the common GNU build system steps after unpacking the latest release
> tarball <https://github.com/BIMSBbioinfo/pigx_rnaseq/releases/latest>:
>
> ./configure --prefix=/some/where
> make install

Indeed. These are the general instructions to build the pipeline from
source.

A package manager like Guix does this internally (and in a suitable
environment) to provide you with a much better user experience. This is
why we mention the installation via Guix first in that section (and on
the main PiGx website at https://bioinformatics.mdc-berlin.de/pigx/).

--
Ricardo

Alisa Fuchs

unread,
Jun 26, 2019, 5:13:37 AM6/26/19
to pigx
Hi,

Thank you for instructions. I really appreciate the support, but every step which is trivial to you takes me days to figure out and it seems to take forever. 
Here is the next one: I updated the pigx-rnaseq version and restarted the pipeline. It finished some reports, but now it seems to be stuck at the step producing htseq_out.txt. The file is empty and nothing happens since yesterday.

Best,
Alisa

Bora Uyar

unread,
Jun 27, 2019, 3:51:01 PM6/27/19
to Alisa Fuchs, pigx
@rekado I realized that I didn't respond to the group before, so I am including Alisa's response to me here, which includes a new question. Maybe you can answer Alisa's last question better. It is about the locale settings. 

She gets these messages in logs:

perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LANG = "en_GB.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C")

or:
---
During startup - Warning messages:
1: Setting LC_CTYPE failed, using "C" 
2: Setting LC_COLLATE failed, using "C" 
3: Setting LC_TIME failed, using "C" 
4: Setting LC_MESSAGES failed, using "C" 
5: Setting LC_MONETARY failed, using "C" 
6: Setting LC_PAPER failed, using "C" 
7: Setting LC_MEASUREMENT failed, using "C" 


@Alisa Fuchs these messages don't cause any problem with the pipeline. It is just that there must be a bug I need to fix, but I have been sick the whole week, so I couldn't sit down to fix it. Fixing this issue with the pipeline will be the first thing once I start work again. Hopefully tomorrow. 

Best,

Bora


On Thu, Jun 27, 2019 at 9:49 AM Alisa Fuchs <alisa...@gmail.com> wrote:
Hi,

Thanks.
I get only salmon based reports. As you suggest I will continue with these results, but I would like to understand where the problem is. 
I sometimes get these warnings in the logs:

---
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LANG = "en_GB.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C")

or:
---
During startup - Warning messages:
1: Setting LC_CTYPE failed, using "C" 
2: Setting LC_COLLATE failed, using "C" 
3: Setting LC_TIME failed, using "C" 
4: Setting LC_MESSAGES failed, using "C" 
5: Setting LC_MONETARY failed, using "C" 
6: Setting LC_PAPER failed, using "C" 
7: Setting LC_MEASUREMENT failed, using "C" 

Can it be that my local settings are making problems for the pipeline?

Best,
Alisa

On Wed, Jun 26, 2019 at 11:49 AM Bora Uyar <borauy...@gmail.com> wrote:
Hi Alisa,

- Are the HTML reports only based on Salmon? (the html reports contain "salmon" or "star" in the file names) 

If so, I would think that the STAR alignments haven't finished yet or some of them may have failed. HTSEQ-count has to wait for 
all STAR alignments to finish. This is the situation we had in our last consultation meeting. I think some of your alignments failed due to memory restrictions. If your pipeline is still running without an error, I would guess that some alignments are taking longer to finish. 

If you are frustrated due to a deadline pressure, I would recommend going with salmon-based reports already. It is an equally valid approach. Most of the time people go for one or the other approach. However, we provide results based on two approaches in order to observe reproducibility when using different algorithms. 

But of course, if you want alignments, alignment-based read counts and genome coverage tracks, this problem needs to be solved. 

Best,

Bora




--
You received this message because you are subscribed to the Google Groups "pigx" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pigx+uns...@googlegroups.com.

To post to this group, send email to pi...@googlegroups.com.

Bora Uyar

unread,
Jun 28, 2019, 6:37:41 AM6/28/19
to Alisa Fuchs, pigx
Hi Alisa,

I can't reproduce this error with the latest guix and pigx-rnaseq version 0.0.7. 

And these are the testing results on a different system. Also the latest tests seem to pass. 

Ricardo briefly told me that the locale settings could have an impact on the behaviour of the pipeline. 
So, let's wait for him to comment on this issue. 

Best,

Bora

Bora Uyar

unread,
Jun 28, 2019, 9:59:32 AM6/28/19
to Alisa Fuchs, pigx
Hi Alisa,

Another thing to check (Ricardo pointed out) is if you have re-run the pipeline with the new version from scratch. 

Did you have some partial results with version 0.0.3 and try to finish the remaining results with version 0.0.7? If that's the case, it is definitely worth to re-run everything in a fresh output folder. 

Best,

Bora



 

Ricardo Wurmus

unread,
Jun 28, 2019, 10:24:06 AM6/28/19
to Alisa Fuchs, Bora Uyar, pigx

Hey Alisa,

> @rekado I realized that I didn't respond to the group before, so I am
> including Alisa's response to me here, which includes a new question. Maybe
> you can answer Alisa's last question better. It is about the locale
> settings.
>
> She gets these messages in logs:
>
> perl: warning: Setting locale failed.
> perl: warning: Please check that your locale settings:
> LANGUAGE = (unset),
> LC_ALL = (unset),
> LANG = "en_GB.UTF-8"
> are supported and installed on your system.
> perl: warning: Falling back to the standard locale ("C")
>
> or:
> ---
> During startup - Warning messages:
> 1: Setting LC_CTYPE failed, using "C"
> 2: Setting LC_COLLATE failed, using "C"
> 3: Setting LC_TIME failed, using "C"
> 4: Setting LC_MESSAGES failed, using "C"
> 5: Setting LC_MONETARY failed, using "C"
> 6: Setting LC_PAPER failed, using "C"
> 7: Setting LC_MEASUREMENT failed, using "C"

Generally, these things *shouldn’t* have any functional effect on the
behaviour of the tools. Locales specify e.g. how to sort strings, how
to format numbers and currencies, and are used to determine what
translations to use.

The warnings mean that locale data for British English (in UTF-8
encoding) were requested (via the LANG environment variable) but are not
found on your system, so the tools fall back to a reasonable
country-agnostic default locale (the C locale).

This shouldn’t be a problem, but the messages sure are ugly. This is a
side effect of using packages from Guix, which requires that a locales
package is installed by the user. On the MDC Max Cluster the locales
are installed globally, but if you run the pipeline elsewhere you may
have to do this:

guix install glibc-locales
export GUIX_LOCPATH=$HOME/.guix-profile/lib/locale

(You can add the “export” line to the bottom of your ~/.bashrc to have
this set up in all of your shell sessions.)

If you’re really curious about details see the Guix manual:

https://www.gnu.org/software/guix/manual/en/html_node/Application-Setup.html#Locales-1

While I don’t think this is the cause of your problems, I’m thinking
about a way to circumvent this annoying locales issue in future
releases.

--
Ricardo

Alisa Fuchs

unread,
Jul 1, 2019, 3:58:12 AM7/1/19
to pigx
Dera Bora,

I tried to finish the analysis I started with 0.0.3 version with the new 0.0.7 version, I will try to rerun it from scratch. 
In parallel I started a new analysis on a second data set with 0.0.7 version, and I still get only partial results. Some reports and htseq-count are still missing, I would be happy to show you the logs and results I got. In the meantime I am continuing my analysis with the salmon based output.

Best,
Alisa
...@Alisa Fuchs these messages don't cause any problem with the pipeline. It is just that there must be a bug I need to fix, but I have been sick the whole week, so I couldn't sit down to fix it. Fixing this issue with the pipeline will be the first thing once I start work again. Hopefully tomorrow. 

Best,

Bora


To unsubscribe from this group and stop receiving emails from it, send an email to pi...@googlegroups.com.

Alisa Fuchs

unread,
Jul 1, 2019, 8:58:19 AM7/1/19
to pigx
Dear Ricardo,

Thanks for looking into it. 
I don't care if the logs looks ugly, I was just trying to figure out why my analysis is getting stuck. 

Best,
Alisa

Bora Uyar

unread,
Jul 1, 2019, 9:58:31 AM7/1/19
to Alisa Fuchs, pigx
Hi Alisa & all,

I have been processing one of your bam files with htseq-count. The problem is the pipeline doesn't fail but it takes long to compute the results. 

Just processing 10 million reads took me half an hour. Your samples have on average ~ 5 GB (150 million reads per sample).  

24 samples each containing 150 million reads would take 8 days to compute at this speed, which is unacceptable. 

One factor could be that I am accessing /fast via our group server, which is much slower than accessing it via the cluster. 

The speed could also be improved if I designed the pipeline to run htseq-count jobs in parallel. 

This would improve the speed on the order of 10 fold, which would finish the jobs within 10 hours at this low speed.

I will come up with a faster solution for the next release and will let you know when it is available. 

Best,

Bora


--
You received this message because you are subscribed to the Google Groups "pigx" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pigx+uns...@googlegroups.com.
To post to this group, send email to pi...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Bora Uyar

unread,
Jul 4, 2019, 5:03:12 AM7/4/19
to Alisa Fuchs, pigx
Hi all,

I have published a new release for pigx-rnaseq. https://github.com/BIMSBbioinfo/pigx_rnaseq/releases/tag/v0.0.8

This one should run faster as the counting step is now distributable to multiple nodes and also I switched to using GenomicAlignments::summarizeOverlaps 
as it is much faster than htseq-count. 

It is not yet available on guix as I have just made the release. But, I believe Ricardo would push this on guix soon. 

Best,

Bora


Bora Uyar

unread,
Jul 4, 2019, 7:50:25 AM7/4/19
to Alisa Fuchs, pigx
Dear all,

Thanks to Ricardo, pigx-rnaseq 0.0.8 is now available via guix. 

You can update it on the cluster via:

> guixr pull
> guixr package -i pigx-rnaseq

Check out the section "counting" in the settings.yaml file by running pigx-rnaseq --init. 

By default, the reads are  counted for exons (grouped by gene ids) in a strand unspecific manner with 
a conservative counting mode ('Union'). 

Please let us know if you run into any issues. 

Best,

Bora




Alisa Fuchs

unread,
Jul 10, 2019, 2:13:16 PM7/10/19
to pigx
Dear Bora, 
Dear Ricardo,

Here is an update on the pigx-rnaseq 0.0.8, there are 3 issues:

1. I am running the pipeline on my full data set (48 samples). It finished trimming, mapping and most of the reports, except the most important one, which is supposed to combine all the data sets (48 samples). Can it be that it is too much data? All the other reports are finished, so it should not be a problem with odd characters in names. 
Both star and salmon logs for the report have the same massage:

<Quitting from lines 249-264 (deseqReport.Rmd) 
Error: Insufficient values in manual scale. 10 needed but only 8 provided.
In addition: Warning message:
In DESeqDataSet(se, design = design, ignoreRank) :
  some variables in design formula are characters, converting to factors

Execution halted>
(See full reports attached)

2. I do not get the norm_counts_deseq file (see log attached). 

3. The genome coverage files were not produced as well and I would really like to have them. Not sure why, all the mapping was complete and they should not depend on anything else.

I also attached the settings and sample sheet just in case.

Thanks for help,
Alisa

To unsubscribe from this group and stop receiving emails from it, send an email to pi...@googlegroups.com.
PvsN.report.star.log
norm_counts_deseq.log
PvsN.report.salmon.genes.log
settings_all.yaml
sample_sheet_all.csv

Bora Uyar

unread,
Jul 10, 2019, 2:46:33 PM7/10/19
to Alisa Fuchs, pigx
Hi Alisa,

- About your error in "norm_counts_deseq.log"
It is the same problem that Konsta reported before. You also have "-" characters in your samples. The script that imports the counts tables converts those characters to dots (this is the default behaviour of read.table function in R). However, I have fixed this in the latest version 0.0.9, but it is not yet released as a guix package. It should be available soon via guix once Ricardo is back from vacation. 

The last step that creates the bigwig files depend on this step to finish. The size factors obtained at this step are used to normalize the coverage tracks. 

- About the error in star reports: 

I need to look into this. It must be a discrete coloring issue. A feature that I added to make it color-blind friendly has caused the problem here. You have 10 distinct sample types and I have provided 8 possible discrete colors. It has to be fixed. I will try to fix this tomorrow. 

Bora

To unsubscribe from this group and stop receiving emails from it, send an email to pigx+uns...@googlegroups.com.

To post to this group, send email to pi...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Alisa Fuchs

unread,
Jul 11, 2019, 4:20:09 AM7/11/19
to pigx
Great. Thanks for the explanation. I got partial results, so I will wait for the next pigx-rnaseq version to complete the analysis.

Best,
Alisa

Bora Uyar

unread,
Jul 17, 2019, 6:58:22 AM7/17/19
to Alisa Fuchs, pigx
Hi Alisa,

The latest release of pigx-rnaseq (0.0.10) seems to be available on guix. Thanks to Ricardo. 

Could you please update your package and resume your analysis from where you left off? Please let us know if it works or not. 

Best,

Bora



To unsubscribe from this group and stop receiving emails from it, send an email to pigx+uns...@googlegroups.com.

To post to this group, send email to pi...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Konsta

unread,
Jul 17, 2019, 7:59:43 AM7/17/19
to pigx
Hi Bora,
Hi Ricardo,

and thanks for the new release.

I have the same issue as Alisa (fastq files' names starting with integers)
and just updated pigx-rnaseq to 0.0.10

(guixr pull; guixr package -i rnaseq)

submitting my script to run the pipeline
.#!/bin/bash
#$ -V
#$ -pe smp 4
#$ -l h_vmem=20G,os=centos6    
#$ -e /fast/AG_Zinzen/Konstantinos_Papadakis_onFAST/rnaSeqOfAlisData/results_1/
#$ -o /fast/AG_Zinzen/Konstantinos_Papadakis_onFAST/rnaSeqOfAlisData/results_1/

cd
/fast/AG_Zinzen/Konstantinos_Papadakis_onFAST/rnaSeqOfAlisData/
export PATH="/home/kpapadak/.guix-profile/bin${PATH:+:}$PATH"
pigx
-rnaseq -s usethesesettings_4.yaml samplesheet_usethis.csv


stops the pipeline immediately and
gives me the following outpout:
Cannot set bash as default shell because it is not available in your PATH. Falling back to sh.
Traceback (most recent call last):
  File "/home/kpapadak/.guix-profile/bin/pigx-rnaseq", line 40, in <module>
    from snakemake.utils import update_config
  File "/gnu/store/rgfibvq54m5kgm3dcz3cyz2b7hqc742c-snakemake-5.2.4/lib/python3.7/site-packages/snakemake/__init__.py", line 20, in <module>
    from snakemake.workflow import Workflow
  File "/gnu/store/rgfibvq54m5kgm3dcz3cyz2b7hqc742c-snakemake-5.2.4/lib/python3.7/site-packages/snakemake/workflow.py", line 23, in <module>
    from snakemake.shell import shell
  File "/gnu/store/rgfibvq54m5kgm3dcz3cyz2b7hqc742c-snakemake-5.2.4/lib/python3.7/site-packages/snakemake/shell.py", line 153, in <module>
    shell.executable(shell_exec)
  File "/gnu/store/rgfibvq54m5kgm3dcz3cyz2b7hqc742c-snakemake-5.2.4/lib/python3.7/site-packages/snakemake/shell.py", line 44, in executable
    if os.path.split(cmd)[-1] == "bash":
  File "/gnu/store/h8l1pby3cm6b4fxsfwwr65b4d1hyh6cs-python-3.7.0/lib/python3.7/posixpath.py", line 107, in split
    p = os.fspath(p)
TypeError: expected str, bytes or os.PathLike object, not NoneType

Best,
Konsta

Ricardo Wurmus

unread,
Jul 17, 2019, 3:58:10 PM7/17/19
to Konsta, pigx

Hi Konsta,

> .#!/bin/bash
> #$ -V
> #$ -pe smp 4
> #$ -l h_vmem=20G,os=centos6
> #$ -e
> /fast/AG_Zinzen/Konstantinos_Papadakis_onFAST/rnaSeqOfAlisData/results_1/
> #$ -o
> /fast/AG_Zinzen/Konstantinos_Papadakis_onFAST/rnaSeqOfAlisData/results_1/
>
> cd /fast/AG_Zinzen/Konstantinos_Papadakis_onFAST/rnaSeqOfAlisData/
> export PATH="/home/kpapadak/.guix-profile/bin${PATH:+:}$PATH"
> pigx-rnaseq -s usethesesettings_4.yaml samplesheet_usethis.csv
[…]
>> Cannot set bash as default shell because it is not available in your PATH.
>> Falling back to sh.

Huh, odd. Never seen this before. Snakemake wants to use Bash. What
an unnecessary error… :-/

What’s the value of $PATH? By default “/bin” should be on the PATH,
which contains “bash”, so I don’t see why Snakemake should complain.
You should also be able to work around that by running “guix install
bash”.

If that fixes it, I’ll begrudgingly add the “bash” package as a
dependency of the “snakemake” package in Guix. What a weird design
decision on their part…

>> File
>> "/gnu/store/rgfibvq54m5kgm3dcz3cyz2b7hqc742c-snakemake-5.2.4/lib/python3.7/site-packages/snakemake/shell.py",
>> line 44, in executable
>> if os.path.split(cmd)[-1] == "bash":
>> File
>> "/gnu/store/h8l1pby3cm6b4fxsfwwr65b4d1hyh6cs-python-3.7.0/lib/python3.7/posixpath.py",
>> line 107, in split
>> p = os.fspath(p)
>> TypeError: expected str, bytes or os.PathLike object, not NoneType

Frankly, though, this totally looks like a bug in Snakemake. They don’t
seem to be checking that the value they work on actually is a string…

--
Ricardo

Konsta

unread,
Jul 18, 2019, 4:21:34 AM7/18/19
to pigx
Hi Ricardo,

Huh, odd.  Never seen this before.  Snakemake wants to use Bash.  What
an unnecessary error… :-/

What’s the value of $PATH?  By default “/bin” should be on the PATH,
which contains “bash”, so I don’t see why Snakemake should complain.

/bin seems to be included in $PATH

$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin

 
You should also be able to work around that by running “guix install
bash”.

did guixr install bash
and the script is running for an hour now. it used to immediately produce the error

I'll let you know if it worked.

Thanks for looking into it,

Konsta

Konsta

unread,
Jul 18, 2019, 4:34:34 AM7/18/19
to pigx
stupid: that was my notebooks PATH


/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin

on the cluster

  echo $PATH
/home/kpapadak/.guix-profile/bin:/home/kpapadak/.guix-profile/bin:/usr/lib64/ccache:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin:/opt/mdc/shared/bin:/usr/local/bin:/opt/uge/bin/lx-amd64:/home/kpapadak/.local/bin:/home/kpapadak/bin

bin seems to be indeed absent.

since I installed bash with guix now and e script is running should I still

PATH=$PATH:/bin or PATH=/bin:$PATH ? 
or should I leave it as is ?

Best,
Konsta

Ricardo Wurmus

unread,
Jul 18, 2019, 5:30:37 AM7/18/19
to Konsta, pigx

Hi Konsta,

> since I installed bash with guix now and e script is running should I
> still
>
> PATH=$PATH:/bin or PATH=/bin:$PATH ?
> or should I leave it as is ?

Snakemake doesn’t seem to care about where it finds “bash” as long as it
is in one of the directories on PATH, so you can leave it as it is for
now.

I’m thinking about patching Snakemake to not look for “bash” at runtime
but to embed a reference to a particular variant of “bash” at build
time. It just seems really reckless to pick up whatever variant of
“bash” may be sitting in a PATH directory.

(Different versions of Bash have different bugs and features. You
either take a POSIX shell, and then “/bin/sh” should suffice or you add
a particular version of the shell to the dependencies. There is no
middle way.)

--
Ricardo

Konsta

unread,
Jul 18, 2019, 7:54:21 AM7/18/19
to pigx
Hi Ricardo,
Hi Bora,

after the last issue of snakemake requiring bash your newest release 0.0.10 run through and completed the analysis.

Thank you both very much for the great support,
Konsta

Alisa Fuchs

unread,
Jul 30, 2019, 4:53:11 AM7/30/19
to pigx
Dear Bora,
Dear Ricardo,

I finished the analysis with the latest version of pigx-rnaseq (0.0.10) and all the issues are solved, I got all the reports and bigwig files.

Thank you very much for all your help and work on the updates.

Best,
Alisa
Reply all
Reply to author
Forward
0 new messages