no module named wfdb

626 views
Skip to first unread message

sven

unread,
Aug 30, 2018, 3:23:58 PM8/30/18
to physionet-challenges
Hey
I've been trying a lot with dependencies.txt but i always get the error:

import wfdb
ImportError: No module named 'wfdb'

The dependencies.txt contains: 
libwfdb10          =10.5.25~pre2-0~pn1
libwfdb-dev        =10.5.25~pre2-0~pn1
wfdb               =10.5.25~pre2-0~pn1
wfdb-app-toolbox   = 0.9.9+src-0~pn1

and the script is run by:
python3 run_my_classifier.py ./$RECORD

But the module can't be found. Any ideas?

Thanks!
Sven

Benjamin Moody

unread,
Aug 30, 2018, 4:51:39 PM8/30/18
to sven, physionet-challenges
You're looking for the WFDB Python Package, which is not the same
thing as WFDB despite having a similar name. Sorry for the confusion.


Also, pip is $^#&*@ broken and so is virtualenv, news at 11.


Here's what to do:

1. Download the package:
https://files.pythonhosted.org/packages/b2/96/c2200539fdf4f087e14d30ed62a66544b6f441196bcb8ecc7a29ec6503b9/wfdb-2.2.1.tar.gz
(why is the URL so long? let's not ask such questions.)

Include this file in the top level directory of your entry.

2. Add the following lines to your dependencies.txt:

python3-pip
python3-setuptools
python3-matplotlib
python3-sklearn
python3-pandas
python3-nose
python3-tk

(why the %$*$*# does this require tk? let's not ask those questions either.)

3. Add the following line to your setup.sh:

pip3 install --user --no-deps --no-binary :all: wfdb-2.2.1.tar.gz

Hope this helps!

jag...@gmx.de

unread,
Aug 31, 2018, 5:31:05 AM8/31/18
to physionet-challenges
Thank you very much!

I added everything, but could not really test it yet. The server is timing out ;)

jag...@gmx.de

unread,
Aug 31, 2018, 7:34:03 AM8/31/18
to physionet-challenges
Okay, now I wonder because there were multiple time outs if the dry run is stuck in pending or really running for about 5h? 



Am Donnerstag, 30. August 2018 22:51:39 UTC+2 schrieb Benjamin Moody:

kilin

unread,
Aug 31, 2018, 11:08:35 AM8/31/18
to physionet-challenges
Hey,
besides wfdb we also have an open-source library called biosppy which we added to the setup.sh in the same style as you told us to add the wfdb-package:

pip3 install --user --no-deps --no-binary :all: wfdb-2.2.1.tar.gz
pip3 install --user --no-deps --no-binary :all: bidict-0.17.2.zip
pip3 install --user --no-deps --no-binary :all: shortuuid-0.5.0.zip
pip3 install --user --no-deps --no-binary :all: six-1.11.0.zip
pip3 install --user --no-deps --no-binary :all: biosppy-0.6.1.zip

the middle three packages are further dependencies from biosppy. We added all the .zip files to the entry root directory of course.

Is that correct? We cannot test it since our dry-run it still pending for more than 10 hours now, I think there might be an error with the server? (P.S.: Our dry-run on the server doesn't have these dependencies yet, so we are pretty sure that it cannot be working?)

Thanks in advance!

Benjamin Moody

unread,
Aug 31, 2018, 11:18:54 AM8/31/18
to jag...@gmx.de, physionet-challenges
The error message is this:

Error: "./next.sh ./te0001" timed out after 28800 seconds (2.38e+10
instructions used)

followed by a bunch of python error messages.

What this means is that your entry was deadlocked - it was waiting for
something to happen that never happened. Eventually, the evaluation
script gave up waiting for it to exit.

I've just tested it, and your program runs for less than a minute
before freezing (which I can see because its CPU usage drops to zero.)

Perhaps the python backtraces will tell you something about why that
is happening.


On 8/31/18, jag...@gmx.de <jag...@gmx.de> wrote:
> Okay, now I wonder because there were multiple time outs if the dry run is
> stuck in pending or really running for about 5h?
>
>
>
> Am Donnerstag, 30. August 2018 22:51:39 UTC+2 schrieb Benjamin Moody:
>>
> --
> You received this message because you are subscribed to the Google Groups
> "physionet-challenges" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to physionet-challe...@googlegroups.com.
> Visit this group at https://groups.google.com/group/physionet-challenges.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/physionet-challenges/fc864a86-4c5d-40d2-a9ce-a9836e3e066d%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

kili...@gmail.com

unread,
Aug 31, 2018, 12:09:23 PM8/31/18
to physionet-challenges
Thanks, it seems like the scipy library is too old (we are using the findpeaks functions), we need version 1.1.0., so should we just manually add a newer version of scipy in our entry?

Benjamin Moody

unread,
Aug 31, 2018, 12:23:27 PM8/31/18
to kili...@gmail.com, physionet-challenges
If you can do that easily, that's what I'd recommend. I don't
remember if scipy is one of those really-annoying-to-install packages
or not. I'll have a look at building 1.1.0 but it may take a few
hours. :)

On 8/31/18, kili...@gmail.com <kili...@gmail.com> wrote:
> Thanks, it seems like the scipy library is too old (we are using the
> findpeaks functions), we need version 1.1.0., so should we just manually
> add a newer version of scipy in our entry?
>
> Am Freitag, 31. August 2018 17:18:54 UTC+2 schrieb Benjamin Moody:
>>
>> The error message is this:
>>
>> Error: "./next.sh ./te0001" timed out after 28800 seconds (2.38e+10
>> instructions used)
>>
>> followed by a bunch of python error messages.
>>
>> What this means is that your entry was deadlocked - it was waiting for
>> something to happen that never happened. Eventually, the evaluation
>> script gave up waiting for it to exit.
>>
>> I've just tested it, and your program runs for less than a minute
>> before freezing (which I can see because its CPU usage drops to zero.)
>>
>> Perhaps the python backtraces will tell you something about why that
>> is happening.
>>
>>
>> On 8/31/18, jag...@gmx.de <javascript:> <jag...@gmx.de <javascript:>>
>> > email to physionet-challe...@googlegroups.com
>> > <javascript:>.
>>
>> > Visit this group at
>> > https://groups.google.com/group/physionet-challenges.
>>
>> > To view this discussion on the web visit
>> >
>> https://groups.google.com/d/msgid/physionet-challenges/fc864a86-4c5d-40d2-a9ce-a9836e3e066d%40googlegroups.com.
>>
>>
>> > For more options, visit https://groups.google.com/d/optout.
>> >
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "physionet-challenges" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to physionet-challe...@googlegroups.com.
> Visit this group at https://groups.google.com/group/physionet-challenges.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/physionet-challenges/d9c08c8c-9fbc-4bad-9f26-0baa2d4f40c6%40googlegroups.com.

Benjamin Moody

unread,
Aug 31, 2018, 12:24:52 PM8/31/18
to kilin, physionet-challenges
In order to test this can you please point me to the exact URLs for
those zip files?
> --
> You received this message because you are subscribed to the Google Groups
> "physionet-challenges" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to physionet-challe...@googlegroups.com.
> Visit this group at https://groups.google.com/group/physionet-challenges.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/physionet-challenges/f6cbef76-864f-481a-a3dc-2ebc1445eaea%40googlegroups.com.

kili...@gmail.com

unread,
Aug 31, 2018, 12:48:01 PM8/31/18
to physionet-challenges
Thanks a lot Benjamin, all of the packages are from this page, always the *.tar.gz:

exact URLs:

some are just manually zipped

We just tried a dry-run with a manually added scipy zip but it didn't work... Run number 2029.

And again, thanks a lot... we are kind of confused now... :)

Am Freitag, 31. August 2018 18:24:52 UTC+2 schrieb Benjamin Moody:
In order to test this can you please point me to the exact URLs for
those zip files?

On 8/31/18, kilin <kili...@gmail.com> wrote:
> Hey,
> besides wfdb we also have an open-source library called biosppy which we
> added to the setup.sh in the same style as you told us to add the
> wfdb-package:
>
> pip3 install --user --no-deps --no-binary :all: wfdb-2.2.1.tar.gz
> pip3 install --user --no-deps --no-binary :all: bidict-0.17.2.zip
> pip3 install --user --no-deps --no-binary :all: shortuuid-0.5.0.zip
> pip3 install --user --no-deps --no-binary :all: six-1.11.0.zip
> pip3 install --user --no-deps --no-binary :all: biosppy-0.6.1.zip
>
> the middle three packages are further dependencies from biosppy. We added
> all the .zip files to the entry root directory of course.
>
> Is that correct? We cannot test it since our dry-run it still pending for
> more than 10 hours now, I think there might be an error with the server?
> (P.S.: Our dry-run on the server doesn't have these dependencies yet, so we
>
> are pretty sure that it cannot be working?)
>
> Thanks in advance!
>
> --
> You received this message because you are subscribed to the Google Groups
> "physionet-challenges" group.
> To unsubscribe from this group and stop receiving emails from it, send an

Benjamin Moody

unread,
Aug 31, 2018, 3:00:41 PM8/31/18
to kili...@gmail.com, physionet-challenges
You should be able to install these packages (except scipy) by running

pip3 install --user shortuuid-0.5.0.tar.gz
pip3 install --user bidict-0.17.2.tar.gz
pip3 install --user six-1.11.0.tar.gz
pip3 install --user --no-deps biosppy.zip

and also adding dependencies (in addition to what you already had):

python3-setuptools-scm
python3-pytest-runner

For scipy, add the following:

python3-scipy = 1.1.0-1~pn0

On 8/31/18, kili...@gmail.com <kili...@gmail.com> wrote:
> Thanks a lot Benjamin, all of the packages are from this page, always the
> *.tar.gz:
> https://pypi.org/project/scipy/#files
> https://pypi.org/project/bidict/#files
> https://pypi.org/project/six/#files
> https://pypi.org/project/shortuuid/#files
> biosppy: https://github.com/PIA-Group/BioSPPy.git
>
> exact URLs:
> https://files.pythonhosted.org/packages/79/a8/83a52f422b1c041758fcfa3ac4438fc92a14e3cbe2b36f97282d973aff55/bidict-0.17.2.tar.gz
> https://files.pythonhosted.org/packages/07/76/7e844757b9f3bf5ab9f951ccd3e4a8eed91ab8720b0aac8c2adcc2fdae9f/scipy-1.1.0.tar.gz
> https://files.pythonhosted.org/packages/16/d8/bc6316cf98419719bd59c91742194c111b6f2e85abac88e496adefaf7afe/six-1.11.0.tar.gz
> https://files.pythonhosted.org/packages/80/d7/2bfc9332e68d3e15ea97b9b1588b3899ad565120253d3fd71c8f7f13b4fe/shortuuid-0.5.0.tar.gz
> https://github.com/PIA-Group/BioSPPy/archive/212c3dcbdb1ec43b70ba7199deb5eb22bcb78fd0.zip
>
> some are just manually zipped
>
> We just tried a dry-run with a manually added scipy zip but it didn't
> work... Run number 2029.
>
> And again, thanks a lot... we are kind of confused now... :)
>
> Am Freitag, 31. August 2018 18:24:52 UTC+2 schrieb Benjamin Moody:
>>
>> In order to test this can you please point me to the exact URLs for
>> those zip files?
>>
>> > email to physionet-challe...@googlegroups.com
>> > <javascript:>.
>>
>> > Visit this group at
>> > https://groups.google.com/group/physionet-challenges.
>>
>> > To view this discussion on the web visit
>> >
>> https://groups.google.com/d/msgid/physionet-challenges/f6cbef76-864f-481a-a3dc-2ebc1445eaea%40googlegroups.com.
>>
>>
>> > For more options, visit https://groups.google.com/d/optout.
>> >
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "physionet-challenges" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to physionet-challe...@googlegroups.com.
> Visit this group at https://groups.google.com/group/physionet-challenges.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/physionet-challenges/95df995d-bc27-4842-8df6-6f2b335e0e4c%40googlegroups.com.

kili...@gmail.com

unread,
Aug 31, 2018, 4:32:00 PM8/31/18
to physionet-challenges

Thanks again, we tried installing scipy using the .whl file first which was working (however another error in our code), now we changed it to your code and tried running the dry-run again. Can you please check if it is stuck again? It's running for almost an hour again... Last time it was stuck it was because of scipy

Thanks!
>> > <javascript:>.
>>
>> > Visit this group at
>> > https://groups.google.com/group/physionet-challenges.
>>
>> > To view this discussion on the web visit
>> >
>> https://groups.google.com/d/msgid/physionet-challenges/f6cbef76-864f-481a-a3dc-2ebc1445eaea%40googlegroups.com.
>>
>>
>> > For more options, visit https://groups.google.com/d/optout.
>> >
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "physionet-challenges" group.
> To unsubscribe from this group and stop receiving emails from it, send an

Benjamin Moody

unread,
Aug 31, 2018, 6:08:16 PM8/31/18
to kili...@gmail.com, physionet-challenges
Indeed, it looks like it will freeze after ~ 13 minutes, after
displaying a series of errors, including the following:


Traceback (most recent call last):

File "/usr/lib/python3/dist-packages/tensorflow/python/ops/script_ops.py",
line 158, in __call__
ret = func(*args)

File "/usr/lib/python3/dist-packages/tensorflow/python/data/ops/dataset_ops.py",
line 410, in generator_py_func
values = next(generator_state.get_iterator(iterator_id))

File "/home/vmuser/entry_offline_package_dry_run/model.py", line 96, in gen
self._worker(self.batch_size)

File "/home/vmuser/entry_offline_package_dry_run/model.py", line 89,
in _worker
batch_size=batch_size, randomize=self._randomize)

File "/home/vmuser/entry_offline_package_dry_run/importer.py", line
132, in next_batch
signal = self.get_test(self.testing)

File "/home/vmuser/entry_offline_package_dry_run/importer.py", line
80, in get_test
record = wfdb.rdrecord(os.path.join(self._data_path, record_name))

File "/home/vmuser/.local/lib/python3.5/site-packages/wfdb/io/record.py",
line 1154, in rdrecord
record = rdheader(record_name, pb_dir=pb_dir, rd_segments=False)

File "/home/vmuser/.local/lib/python3.5/site-packages/wfdb/io/record.py",
line 991, in rdheader
dir_name, pb_dir)

File "/home/vmuser/.local/lib/python3.5/site-packages/wfdb/io/_header.py",
line 705, in _read_header_lines
with open(os.path.join(dir_name, file_name), 'r') as fp:

FileNotFoundError: [Errno 2] No such file or directory:
'/home/vmuser/entry_offline_package_dry_run/tr03-0166\\tr03-0166.hea'


Please note: I'm going to manually interrupt your current entry, so
that you won't have to wait for it to finish. (When I do this, it
won't report back a log message - it will just say "internal error" or
something like that.)

However, you need to fix your code so that it will abort in case of an
error, rather than freezing. Please do not submit another entry until
you have done that.



On 8/31/18, kili...@gmail.com <kili...@gmail.com> wrote:
>
> Thanks again, we tried installing scipy using the .whl file first which was
>
> working (however another error in our code), now we changed it to your code
>
> and tried running the dry-run again. Can you please check if it is stuck
> again? It's running for almost an hour again... Last time it was stuck it
> was because of scipy
>
> Thanks!
>
> Am Freitag, 31. August 2018 21:00:41 UTC+2 schrieb Benjamin Moody:
>>
>> You should be able to install these packages (except scipy) by running
>>
>> pip3 install --user shortuuid-0.5.0.tar.gz
>> pip3 install --user bidict-0.17.2.tar.gz
>> pip3 install --user six-1.11.0.tar.gz
>> pip3 install --user --no-deps biosppy.zip
>>
>> and also adding dependencies (in addition to what you already had):
>>
>> python3-setuptools-scm
>> python3-pytest-runner
>>
>> For scipy, add the following:
>>
>> python3-scipy = 1.1.0-1~pn0
>>
>> On 8/31/18, kili...@gmail.com <javascript:> <kili...@gmail.com
>> >> > email to physionet-challe...@googlegroups.com
>> <javascript:>
>> >> > <javascript:>.
>> >>
>> >> > Visit this group at
>> >> > https://groups.google.com/group/physionet-challenges.
>> >>
>> >> > To view this discussion on the web visit
>> >> >
>> >>
>> https://groups.google.com/d/msgid/physionet-challenges/f6cbef76-864f-481a-a3dc-2ebc1445eaea%40googlegroups.com.
>>
>>
>> >>
>> >>
>> >> > For more options, visit https://groups.google.com/d/optout.
>> >> >
>> >>
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> Groups
>> > "physionet-challenges" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> an
>> > email to physionet-challe...@googlegroups.com
>> > <javascript:>.
>>
>> > Visit this group at
>> > https://groups.google.com/group/physionet-challenges.
>>
>> > To view this discussion on the web visit
>> >
>> https://groups.google.com/d/msgid/physionet-challenges/95df995d-bc27-4842-8df6-6f2b335e0e4c%40googlegroups.com.
>>
>>
>> > For more options, visit https://groups.google.com/d/optout.
>> >
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "physionet-challenges" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to physionet-challe...@googlegroups.com.
> Visit this group at https://groups.google.com/group/physionet-challenges.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/physionet-challenges/76f97bf8-6e0d-41a3-bbf0-0bbf4182c99a%40googlegroups.com.
Message has been deleted

kili...@gmail.com

unread,
Sep 1, 2018, 3:25:44 AM9/1/18
to physionet-challenges
thanks Benjamin, the dry-run worked now, thank you for everything!
>> <javascript:>
>> >> > <javascript:>.
>> >>
>> >> > Visit this group at
>> >> > https://groups.google.com/group/physionet-challenges.
>> >>
>> >> > To view this discussion on the web visit
>> >> >
>> >>
>> https://groups.google.com/d/msgid/physionet-challenges/f6cbef76-864f-481a-a3dc-2ebc1445eaea%40googlegroups.com.
>>
>>
>> >>
>> >>
>> >> > For more options, visit https://groups.google.com/d/optout.
>> >> >
>> >>
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> Groups
>> > "physionet-challenges" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> an
>> > <javascript:>.
>>
>> > Visit this group at
>> > https://groups.google.com/group/physionet-challenges.
>>
>> > To view this discussion on the web visit
>> >
>> https://groups.google.com/d/msgid/physionet-challenges/95df995d-bc27-4842-8df6-6f2b335e0e4c%40googlegroups.com.
>>
>>
>> > For more options, visit https://groups.google.com/d/optout.
>> >
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "physionet-challenges" group.
> To unsubscribe from this group and stop receiving emails from it, send an
Reply all
Reply to author
Forward
0 new messages