Issue when using Fuel

[François Dupré]

Hello, I am currently trying to learn how to use Fuel.
When I run this piece of code :

#! usr/bin/python

from fuel.datasets import MNIST

mnist = MNIST("train")

I get the following error :

Traceback (most recent call last):
  File "test_fuel.py", line 5, in <module>
    mnist = MNIST("train")
  File "/home/francois/.local/lib/python2.7/site-packages/fuel/datasets/mnist.py", line 36, in __init__
    super(MNIST, self).__init__(self.data_path, which_set, **kwargs)
  File "/home/francois/.local/lib/python2.7/site-packages/fuel/datasets/hdf5.py", line 142, in __init__
    if which_set not in self.available_splits:
  File "/home/francois/.local/lib/python2.7/site-packages/fuel/datasets/hdf5.py", line 254, in available_splits
    return tuple(self.split_dict.keys())
  File "/home/francois/.local/lib/python2.7/site-packages/fuel/datasets/hdf5.py", line 237, in split_dict
    split_array = handle.attrs['split']
  File "_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (/tmp/pip_build_francois/h5py/h5py/_objects.c:2508)
  File "_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (/tmp/pip_build_francois/h5py/h5py/_objects.c:2461)
  File "/home/francois/.local/lib/python2.7/site-packages/h5py/_hl/attrs.py", line 52, in __getitem__
    attr = h5a.open(self._id, self._e(name))
  File "_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (/tmp/pip_build_francois/h5py/h5py/_objects.c:2508)
  File "_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (/tmp/pip_build_francois/h5py/h5py/_objects.c:2461)
  File "h5a.pyx", line 77, in h5py.h5a.open (/tmp/pip_build_francois/h5py/h5py/h5a.c:2107)
KeyError: "Can't open attribute (Can't locate attribute: 'split')"

Sorry if this question has already been asked somewhere else, but I couldn't find how to fix it anywhere else on the Internet.
My OS is Ubuntu.

If you need additional infos, do not hesitate.
Thank you.

[Vincent Dumoulin]

When was the last time you generated the HDF5 file with fuel-convert?

The interface has changed slightly in the last weeks, try running fuel-convert mnist again to see if it solves your problem.

[François Dupré]

When was the last time you generated the HDF5 file with fuel-convert?

I don't think I did.

The interface has changed slightly in the last weeks, try running fuel-convert mnist again to see if it solves your problem.

  1 #! usr/bin/python
  2
  3 import fuel
  4 from fuel.datasets import MNIST
  5
  6 mnist=MNIST("train")
  7 fuel-convert mnist

I get this error :

  File "test_fuel.py", line 7
    fuel-convert mnist
                     ^
SyntaxError: invalid syntax

[Bart van Merriënboer]

fuel-convert is a bash command that you should run in your terminal, not a Python command.

First set your FUEL_DATA_PATH (as described here) and then run the commands "cd $FUEL_DATA_PATH" (to go to the data folder) and then "fuel-download mnist" followed by the command "fuel-convert mnist" to download MNIST and convert it into a Fuel-compatible HDF5 format. After doing that, `MNIST("train")` should work in Python.

--
You received this message because you are subscribed to the Google Groups "fuel-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fuel-users+...@googlegroups.com.
To post to this group, send email to fuel-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fuel-users/f4513d17-4d17-493d-a022-71c677295cb3%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Vincent Dumoulin]

Here's a tutorial that explains it in more details:

https://github.com/mila-udem/fuel/blob/master/docs/built_in_datasets.rst

On Saturday, May 16, 2015 at 7:07:39 PM UTC-4, Bart van Merriënboer wrote:

fuel-convert is a bash command that you should run in your terminal, not a Python command.

First set your FUEL_DATA_PATH (as described here) and then run the commands "cd $FUEL_DATA_PATH" (to go to the data folder) and then "fuel-download mnist" followed by the command "fuel-convert mnist" to download MNIST and convert it into a Fuel-compatible HDF5 format. After doing that, `MNIST("train")` should work in Python.

On Sat, May 16, 2015 at 7:04 PM, François Dupré <franco...@gmail.com> wrote:

When was the last time you generated the HDF5 file with fuel-convert?

I don't think I did.


The interface has changed slightly in the last weeks, try running fuel-convert mnist again to see if it solves your problem.

  1 #! usr/bin/python
  2
  3 import fuel
  4 from fuel.datasets import MNIST
  5
  6 mnist=MNIST("train")
  7 fuel-convert mnist

I get this error :

  File "test_fuel.py", line 7
    fuel-convert mnist
                     ^
SyntaxError: invalid syntax

--
You received this message because you are subscribed to the Google Groups "fuel-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to fuel-users+unsubscribe@googlegroups.com.

To post to this group, send email to fuel-...@googlegroups.com.

[François Dupré]

fuel-convert is a bash command that you should run in your terminal, not a Python command.

First set your FUEL_DATA_PATH (as described here) and then run the commands "cd $FUEL_DATA_PATH" (to go to the data folder) and then "fuel-download mnist" followed by the command "fuel-convert mnist" to download MNIST and convert it into a Fuel-compatible HDF5 format. After doing that, `MNIST("train")` should work in Python.

francois@francois ~ % export FUEL_DATA_PATH=/home/your_data
francois@francois /home % cd $FUEL_DATA_PATH
francois@francois /home/your_data
 % fuel-download mnist
zsh: command not found: fuel-download

[François Dupré]

Here's a tutorial that explains it in more details:

https://github.com/mila-udem/fuel/blob/master/docs/built_in_datasets.rst

Thank you. I had already gone through these tutorials, but I can't manage to run fuel commands in the shell

[Bart van Merriënboer]

It looks like Fuel wasn't correctly installed. Just adding it to PYTHONPATH or being in the Fuel folder isn't enough, you'll need to run `python setup.py install` or `pip install -e .` so that the scripts get copied to the correct location.

(Otherwise, you can also run the script directly, using `bin/fuel-download` from the folder you cloned.)

--
You received this message because you are subscribed to the Google Groups "fuel-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to fuel-users+...@googlegroups.com.

To post to this group, send email to fuel-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/fuel-users/3e0afd55-9576-44ea-b0cb-6c01e0a6d9f5%40googlegroups.com.

[Vincent Dumoulin]

What was your installation method for Fuel?

[François Dupré]

It looks like Fuel wasn't correctly installed. Just adding it to PYTHONPATH or being in the Fuel folder isn't enough, you'll need to run `python setup.py install` or `pip install -e .` so that the scripts get copied to the correct location

francois@francois ~ % python setup.py install
python: can't open file 'setup.py': [Errno 2] No such file or directory
francois@francois ~ % pip install -e .
Directory '.' is not installable. File 'setup.py' not found.
Storing debug log for failure in /home/francois/.pip/pip.log

I haven't tried the second option yet.

[Bart van Merriënboer]

You need to run those commands from inside the folder that you cloned from GitHub, which contains a file called setup.py. Those errors suggest that you're not in the correct folder.

--

You received this message because you are subscribed to the Google Groups "fuel-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fuel-users+...@googlegroups.com.
To post to this group, send email to fuel-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/fuel-users/e4474a86-2d26-4d4b-b9a2-614ab09ae5fc%40googlegroups.com.

[François Dupré]

What was your installation method for Fuel?

I just followed this tutorial, http://fuel.readthedocs.org/en/latest/index.html#
If there were additional things to install, I don't think I did.

[François Dupré]

I think I hadn't cloned the file from GitHub. But, still :

francois@francois ~/blocks % ls
bin         CONTRIBUTING.rst  examples    req-scrutinizer.txt   setup.py
blocks         dist           LICENSE    req-travis-conda.txt  tests
blocks.egg-info  docs           README.rst    req-travis-pip.txt
build         doctests       req-rtd.txt    requirements.txt
francois@francois ~/blocks % fuel-download mnist
zsh: command not found: fuel-download

[François Dupré]

 Btw, I successfully ran python setup.py install in /home/francois/blocks

[François Dupré]

I managed to run the fuel commands, but a new error has come up :

francois@francois ~ % python tuto_fuel.py

Traceback (most recent call last):

  File "tuto_fuel.py", line 4, in <module>
    mnist = MNIST(which_set='train')
  File "/usr/local/lib/python2.7/dist-packages/fuel-0.0.1-py2.7.egg/fuel/datasets/mnist.py", line 36, in __init__
    super(MNIST, self).__init__(self.data_path, which_set, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/fuel-0.0.1-py2.7.egg/fuel/datasets/hdf5.py", line 142, in __init__
    if which_set not in self.available_splits:
  File "/usr/local/lib/python2.7/dist-packages/fuel-0.0.1-py2.7.egg/fuel/datasets/hdf5.py", line 254, in available_splits
    return tuple(self.split_dict.keys())
  File "/usr/local/lib/python2.7/dist-packages/fuel-0.0.1-py2.7.egg/fuel/datasets/hdf5.py", line 235, in split_dict
    self._out_of_memory_open()
  File "/usr/local/lib/python2.7/dist-packages/fuel-0.0.1-py2.7.egg/fuel/datasets/hdf5.py", line 304, in _out_of_memory_open
    name=self.path, mode="r", driver=self.driver)
  File "/home/francois/.local/lib/python2.7/site-packages/h5py/_hl/files.py", line 260, in __init__
    fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr)
  File "/home/francois/.local/lib/python2.7/site-packages/h5py/_hl/files.py", line 89, in make_fid
    fid = h5f.open(name, flags, fapl=fapl)

  File "_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (/tmp/pip_build_francois/h5py/h5py/_objects.c:2508)
  File "_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (/tmp/pip_build_francois/h5py/h5py/_objects.c:2461)

  File "h5f.pyx", line 76, in h5py.h5f.open (/tmp/pip_build_francois/h5py/h5py/h5f.c:1837)
IOError: Unable to open file (Unable to open file: name = 'home/your_data/mnist.hdf5', errno = 2, error message = 'no such file or directory', flags = 0, o_flags = 0)

What I fail to understand :

francois@francois ~ % cd /home/your_data
francois@francois /home/your_data
 % ls
fuel  fuel-master  mnist.hdf5

[François Dupré]

Btw, tuto_fuel.py still has the same content :

#! usr/bin/python

from fuel.datasets import

MNIST
mnist = MNIST(which_set='train')

[Bart van Merriënboer]

Did you set FUEL_DATA_PATH to home/... instead of /home/...?

--
You received this message because you are subscribed to the Google Groups "fuel-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fuel-users+...@googlegroups.com.
To post to this group, send email to fuel-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/fuel-users/a3cc378b-8cf0-41aa-a0be-db626d8d15ff%40googlegroups.com.

[François Dupré]

Le dimanche 17 mai 2015 17:49:37 UTC-4, Bart van Merriënboer a écrit :

Did you set FUEL_DATA_PATH to home/... instead of /home/...?

Indeed,

echo "data_path: /home/your_data" >> ~/.fuelrc

fixed the problem.

But, now I'm still stuck with

francois@francois ~ % python tuto_fuel.py
Traceback (most recent call last):
  File "tuto_fuel.py", line 4, in <module>
    mnist = MNIST(which_set='train')
  File "/usr/local/lib/python2.7/dist-packages/fuel-0.0.1-py2.7.egg/fuel/datasets/mnist.py", line 36, in __init__
    super(MNIST, self).__init__(self.data_path, which_set, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/fuel-0.0.1-py2.7.egg/fuel/datasets/hdf5.py", line 142, in __init__
    if which_set not in self.available_splits:
  File "/usr/local/lib/python2.7/dist-packages/fuel-0.0.1-py2.7.egg/fuel/datasets/hdf5.py", line 254, in available_splits
    return tuple(self.split_dict.keys())

  File "/usr/local/lib/python2.7/dist-packages/fuel-0.0.1-py2.7.egg/fuel/datasets/hdf5.py", line 237, in split_dict
    split_array = handle.attrs['split']

  File "_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (/tmp/pip_build_francois/h5py/h5py/_objects.c:2508)
  File "_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (/tmp/pip_build_francois/h5py/h5py/_objects.c:2461)

  File "/home/francois/.local/lib/python2.7/site-packages/h5py/_hl/attrs.py", line 52, in __getitem__
    attr = h5a.open(self._id, self._e(name))

  File "_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (/tmp/pip_build_francois/h5py/h5py/_objects.c:2508)
  File "_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (/tmp/pip_build_francois/h5py/h5py/_objects.c:2461)

[chendan...@gmail.com]

Have you figured out, I have the same error now.
在 2015年5月16日星期六 UTC-4下午5:37:58，François Dupré写道：

[Bart van Merriënboer]

This error was reported on a much older version of Fuel than is currently available. Please make sure you have an up to date installation of Fuel (i.e. from GitHub). If you still have this error, please show us the commands you used to download and convert MNIST, and the resulting error you get when trying to load the dataset, including a full traceback.

--
You received this message because you are subscribed to the Google Groups "fuel-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fuel-users+...@googlegroups.com.
To post to this group, send email to fuel-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/fuel-users/656321be-da64-481a-ad23-d347cf7dcc82%40googlegroups.com.