ERROR ~ Can't open cache DB

[Michał T. Lorenc]

Hello,

I would like to split FASTQ file into 100 chunks with the following code:

#!/usr/bin/env nextflow

params.query = "RSnSeQ_110Fruit-rm-clean-rmsmrtbell-gt-1k-unique.fastq"

params.chunk = 100

 Channel

        .fromPath(params.query)

        .splitFasta(by: params.chunk)

        .set { chunks }

During the run I have got the following error:

nextflow run splitFastq.nf

N E X T F L O W  ~  version 0.24.2

Launching `splitFastq.nf` [amazing_hoover] - revision: 872559dc0d

ERROR ~ Can't open cache DB: /lustre/work-lustre/Fruit_team/Fruit/contamination-free-pacbio/.nextflow/cache/6b7c1d06-32ff-46d8-b998-586a80df2dcf/db

 -- Check '.nextflow.log' file for details

.nextflow.log output

Jun-12 10:55:52.416 [main] DEBUG nextflow.cli.Launcher - $> /work/Fruit_team/miniconda2/bin/nextflow run splitFastq.nf

Jun-12 10:55:52.905 [main] INFO  nextflow.cli.CmdRun - N E X T F L O W  ~  version 0.24.2

Jun-12 10:55:52.933 [main] INFO  nextflow.cli.CmdRun - Launching `splitFastq.nf` [amazing_hoover] - revision: 872559dc0d

Jun-12 10:55:54.548 [main] DEBUG nextflow.Session - Session uuid: 6b7c1d06-32ff-46d8-b998-586a80df2dcf

Jun-12 10:55:54.548 [main] DEBUG nextflow.Session - Run name: amazing_hoover

Jun-12 10:55:54.551 [main] DEBUG nextflow.Session - Executor pool size: 32

Jun-12 10:55:54.685 [main] DEBUG nextflow.cli.CmdRun - 

  Version: 0.24.2 build 4271

  Modified: 05-04-2017 16:50 UTC (06-04-2017 02:50 AEDT)

  System: Linux 3.12.69-60.64.32-default

  Runtime: Groovy 2.4.10 on OpenJDK 64-Bit Server VM 1.8.0_102-b14

  Encoding: UTF-8 (UTF-8)

  Process: 86...@cl2n066.ib0.hpc.qut.edu.au [10.13.0.138]

  CPUs: 32 - Mem: 62.7 GB (545.4 MB) - Swap: 2 GB (984.3 MB)

Jun-12 10:55:54.878 [main] DEBUG nextflow.Session - Work-dir: /lustre/work-lustre/Fruit_team/Fruit/contamination-free-pacbio/work [lustre]

Jun-12 10:55:54.879 [main] DEBUG nextflow.Session - Script base path does not exist or is not a directory: /lustre/work-lustre/Fruit_team/Fruit/contamination-free-pacbio/bin

Jun-12 10:55:55.378 [main] ERROR nextflow.cli.Launcher - Can't open cache DB: /lustre/work-lustre/Fruit_team/Fruit/contamination-free-pacbio/.nextflow/cache/6b7c1d06-32ff-46d8-b998-586a80df2dcf/db

java.io.IOException: Can't open cache DB: /lustre/work-lustre/Fruit_team/Fruit/contamination-free-pacbio/.nextflow/cache/6b7c1d06-32ff-46d8-b998-586a80df2dcf/db

        at nextflow.CacheDB.openDb(CacheDB.groovy:117)

        at nextflow.CacheDB.open(CacheDB.groovy:128)

        at nextflow.Session.init(Session.groovy:290)

        at nextflow.script.ScriptRunner.execute(ScriptRunner.groovy:148)

        at nextflow.cli.CmdRun.run(CmdRun.groovy:223)

        at nextflow.cli.Launcher.run(Launcher.groovy:410)

        at nextflow.cli.Launcher.main(Launcher.groovy:564)

Caused by: java.io.IOException: Function not implemented

        at sun.nio.ch.FileDispatcherImpl.lock0(Native Method)

        at sun.nio.ch.FileDispatcherImpl.lock(FileDispatcherImpl.java:90)

        at sun.nio.ch.FileChannelImpl.tryLock(FileChannelImpl.java:1115)

        at java.nio.channels.FileChannel.tryLock(FileChannel.java:1155)

        at org.iq80.leveldb.impl.DbLock.<init>(DbLock.java:47)

        at org.iq80.leveldb.impl.DbImpl.<init>(DbImpl.java:167)

        at org.iq80.leveldb.impl.Iq80DBFactory.open(Iq80DBFactory.java:59)

        at nextflow.CacheDB.openDb(CacheDB.groovy:100)

        ... 6 common frames omitted

What did I miss?

Thank you in advance.

Best wishes,

Michal

[Paolo Di Tommaso]

Apparently your shared file system does not support file locks. You may need to concat your system admins for address this problem. 

As workaround you can launch your NF script in a local directory and specifying the shared work directory by using the `-w` command line option. For example:

nextflow run splitFastq.nf -w /lustre/work-lustre/Fruit_team/Fruit/contamination-free-pacbio/

Hope it helps. 

Cheers,

Paolo

  

--
You received this message because you are subscribed to the Google Groups "Nextflow" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nextflow+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/nextflow.
For more options, visit https://groups.google.com/d/optout.

[Michał T. Lorenc]

Hi,

It still does not work:

Jun-13 12:41:57.634 [main] DEBUG nextflow.cli.Launcher - $> /work/Fruit_team/miniconda2/bin/nextflow run splitFastq.nf -w /work/Fruit_team/Fruit/contamination-free-pacbio/tmp/

Jun-13 12:41:57.714 [main] INFO  nextflow.cli.CmdRun - N E X T F L O W  ~  version 0.24.2

Jun-13 12:41:57.722 [main] INFO  nextflow.cli.CmdRun - Launching `splitFastq.nf` [naughty_bassi] - revision: 872559dc0d

Jun-13 12:41:57.963 [main] DEBUG nextflow.Session - Session uuid: db6fbcb6-7e8b-4ab4-a2ad-d056fe9dcc05

Jun-13 12:41:57.963 [main] DEBUG nextflow.Session - Run name: naughty_bassi

Jun-13 12:41:57.964 [main] DEBUG nextflow.Session - Executor pool size: 48

Jun-13 12:41:57.976 [main] DEBUG nextflow.cli.CmdRun - 

  Version: 0.24.2 build 4271

  Modified: 05-04-2017 16:50 UTC (06-04-2017 02:50 AEDT)

  System: Linux 3.12.69-60.64.32-default

  Runtime: Groovy 2.4.10 on OpenJDK 64-Bit Server VM 1.8.0_102-b14

  Encoding: UTF-8 (UTF-8)

  Process: 19...@lyra03.ib0.hpc.qut.edu.au [10.13.0.3]

  CPUs: 48 - Mem: 125.8 GB (23.8 GB) - Swap: 2 GB (2 GB)

Jun-13 12:41:57.996 [main] DEBUG nextflow.Session - Work-dir: /work/Fruit_team/Fruit/contamination-free-pacbio/tmp [lustre]

Jun-13 12:41:57.996 [main] DEBUG nextflow.Session - Script base path does not exist or is not a directory: /lustre/work-lustre/Fruit_team/Fruit/contamination-free-pacbio/bin

Jun-13 12:41:58.089 [main] ERROR nextflow.cli.Launcher - Can't open cache DB: /lustre/work-lustre/Fruit_team/Fruit/contamination-free-pacbio/.nextflow/cache/db6fbcb6-7e8b-4ab4-a2ad-d056fe9dcc05/db

java.io.IOException: Can't open cache DB: /lustre/work-lustre/Fruit_team/Fruit/contamination-free-pacbio/.nextflow/cache/db6fbcb6-7e8b-4ab4-a2ad-d056fe9dcc05/db

        at nextflow.CacheDB.openDb(CacheDB.groovy:117)

        at nextflow.CacheDB.open(CacheDB.groovy:128)

        at nextflow.Session.init(Session.groovy:290)

        at nextflow.script.ScriptRunner.execute(ScriptRunner.groovy:148)

        at nextflow.cli.CmdRun.run(CmdRun.groovy:223)

        at nextflow.cli.Launcher.run(Launcher.groovy:410)

        at nextflow.cli.Launcher.main(Launcher.groovy:564)

Caused by: java.io.IOException: Function not implemented

        at sun.nio.ch.FileDispatcherImpl.lock0(Native Method)

        at sun.nio.ch.FileDispatcherImpl.lock(FileDispatcherImpl.java:90)

        at sun.nio.ch.FileChannelImpl.tryLock(FileChannelImpl.java:1115)

        at java.nio.channels.FileChannel.tryLock(FileChannel.java:1155)

        at org.iq80.leveldb.impl.DbLock.<init>(DbLock.java:47)

        at org.iq80.leveldb.impl.DbImpl.<init>(DbImpl.java:167)

        at org.iq80.leveldb.impl.Iq80DBFactory.open(Iq80DBFactory.java:59)

        at nextflow.CacheDB.openDb(CacheDB.groovy:100)

        ... 6 common frames omitted

What did I miss?

Thank you in advance.

Michal

[Paolo Di Tommaso]

That path is still on the lustre shared file system, you can see from this line in the log: 

nextflow.Session - Work-dir: /work/Fruit_team/Fruit/contamination-free-pacbio/tmp [lustre]

It looks a problem with the file system, I would suggest to concat your sysadmins. 

Cheers,
Paolo

 

--

[Marc Logghe]

I think it has to do with the way the lustre file system is mounted. Have you tried to mount using the flock option ?:

mount -t lustre -o flock ....

My $0.02

Cheers,

Marc

Op maandag 12 juni 2017 03:48:24 UTC+2 schreef Michał T. Lorenc:

[Michał T. Lorenc]

Hi all,

Thank you, the admins provided me with a non-lustre system.

Best wishes,

Michal

--
You received this message because you are subscribed to a topic in the Google Groups "Nextflow" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/nextflow/Sq4qz0b3IFw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to nextflow+u...@googlegroups.com.