Error when building from git
Guillaume
First thanks for the great work. I am planning to use skimage for some
fluorescence microscopy analysis.
Today I installed from the git repository and an error occurred when
compiling:
File "/home/guillaume/Python/lib/scikits-image/skimage/util/
dtype.py", line 20, in <module>
in <module>
np.float16, np.float32, np.float64)
AttributeError: 'module' object has no attribute 'float16'
taking out 'float16' in the _supported_types tuple solves the issue
(at least it compiles), but I don't know whether it could have
consequences later on.
_supported_types = (np.uint8, np.uint16, np.uint32,
np.int8, np.int16, np.int32,
#np.float16, ### Commented this out...
np.float32, np.float64)
I am running Python 2.7.2+ with numpy 1.5.1 (that is what you get
with apt-get install python-numpy)
Thanks...
Stéfan van der Walt
On Fri, Feb 10, 2012 at 1:55 AM, Guillaume
<guil...@mitotic-machine.org> wrote:
> First thanks for the great work. I am planning to use skimage for some
> fluorescence microscopy analysis.
That's great! We'd love to see what you do once you've got some
working examples.
> Today I installed from the git repository and an error occurred when
> compiling:
>
> File "/home/guillaume/Python/lib/scikits-image/skimage/util/
> dtype.py", line 20, in <module>
> in <module>
> np.float16, np.float32, np.float64)
> AttributeError: 'module' object has no attribute 'float16'
The float16 data-type was introduced in NumPy 1.6, and while we
currently list NumPy 1.4 as a dependency, but we're going to have to
up that to 1.6 (I think some of it is being used in the data type
conversion code now, e.g.). I hope this isn't too much of an
inconvenience!
Regards
Stéfan
Guillaume
Also, I just ran into another problem with freeimage...
First, when I first tried to open a file with the freeimage plugin, it
was not found (I assumed it was bundled with skimage).
So I apt-get installed libfreeimage3.
but the error persisted (i.e. RuntimeError: Could not find the plugin
"freeimage" for imread.)
After some investigation, I found that I had to link /usr/lib/
libfreeimage.so.3 with /usr/lib/libfreeimage.so to pass the plugin
test:
sudo ln -s /usr/lib/libfreeimage.so.3 /usr/lib/libfreeimage.so
Then the test is ok. Maybe there should be some more flexibility in
the search for the shared object, in the _load_library(libname,
libdir) function? Putting a regexp somewhere?
Bye
Guillaume
> Hi Guillaume
>
> On Fri, Feb 10, 2012 at 1:55 AM, Guillaume
>
Stéfan van der Walt
<guil...@mitotic-machine.org> wrote:
> Also, I just ran into another problem with freeimage...
>
> After some investigation, I found that I had to link /usr/lib/
> libfreeimage.so.3 with /usr/lib/libfreeimage.so to pass the plugin
> test:
I just discovered this myself earlier this week. Zachary Pincus has
been working on better library location (on Windows mainly), so we should have
that sorted out soon. Would you mind creating an issue on GitHub in
the mean time, so that we don't lose track of it?
Thanks!
Stéfan
Zachary Pincus
Sorry for the trouble with the FreeImage loading -- I'm glad you got it straightened out. As Stéfan mentioned, we're trying to get this a lot smoother, and it should be soon! Please let me know if you have any other problems with FreeImage.
I should also mention (as you're doing microscopy and may need to get at image metadata from time to time, as I do) that I recently added that capability to the freeimage wrappers. See e.g.: https://github.com/scikits-image/scikits-image/blob/master/skimage/io/_plugins/test_freeimage.py
>> Also, I just ran into another problem with freeimage...
> [...]
>>
>> After some investigation, I found that I had to link /usr/lib/
>> libfreeimage.so.3 with /usr/lib/libfreeimage.so to pass the plugin
>> test:
>
> I just discovered this myself earlier this week. Zachary Pincus has
> been working on better library location (on Windows mainly), so we should have
> that sorted out soon. Would you mind creating an issue on GitHub in
> the mean time, so that we don't lose track of it?
>
https://github.com/scikits-image/scikits-image/pull/134
This should fix the above problem and a host of others.
A mechanism for auto-downloading a freeimage library on Win/Mac is still in progress... I'm not sure how to best integrate that with the distutils, but here's what I have for a start:
https://github.com/zachrahan/scikits-image/tree/freeimage-install-libs
https://github.com/zachrahan/freeimage-sharedlib
Zach
Guillaume Gay
Indeed metadata are important, thanks for pointing this out, I'll have a
look at it (though I find metamorph is not so good at recording good
metadata).
By the way, I have both .stk and .tif images, so I use Christoph Gohlke
tifffile.py for the .stk files. Is ther any chance this will be
integrated to skimage?
Guillaume
Zachary Pincus
>
> By the way, I have both .stk and .tif images, so I use Christoph Gohlke tifffile.py for the .stk files. Is ther any chance this will be integrated to skimage?
Wow! Somehow I was not familiar with Christoph's tifffile.py... that'll be very handy for these cursed LSM files I have laying around. (No help for the ZVIs, but I already have a silly zvi->ome-tiff workflow for them.)
Perhaps Christoph, who frequents this list, can weight in on whether it would be a good idea to wrap this in as a skimage IO plugin, or if it's best as a separate project...
Zach
Christoph Gohlke
A skimage io plugin for tifffile.py is straightforward (attached). The
module already implements imread, imsave, and imshow functions. Those
functions return/save/show n-dimensional numpy arrays (n>=2). I'm not
sure skimage can handle n>3?
Tifffile.py and tifffile.c are BSD licensed, so it is no problem to
include them with skimage. I don't think the module itself should be
maintained or supported as part of skimage.
Regarding those cursed, broken, undocumented or NDAed, microscopy file
formats:
I'm aware of two other projects that strive to read LSM files: pylibtiff
<http://code.google.com/p/pylibtiff/> and pylsm
<http://www.freesbi.ch/pylsm>.
How about a plugin for BioFormats
<http://loci.wisc.edu/software/bio-formats>, which supports many more
formats? The Cellprofiler project contains GPLed bindings
<https://svn.broadinstitute.org/CellProfiler/trunk/CellProfiler/bioformats/>.
Christoph
Emmanuelle Gouillart
> The float16 data-type was introduced in NumPy 1.6, and while we
> currently list NumPy 1.4 as a dependency, but we're going to have to
> up that to 1.6 (I think some of it is being used in the data type
> conversion code now, e.g.). I hope this isn't too much of an
> inconvenience!
isn't it possible to test which version of numpy is used, and remove
float16 from the list if numpy.__version__ < 1.6? It looks like a small
cost in order to be be able to support numpy 1.5. I can do the change if
there is no opposition to this.
The same thing that happened to Guillaume also happened to me as I was
rebuilding skimage 10 minutes ago, since I'm using the version of numpy
packaged by the most recent Ubuntu which is 1.5.
As I see things, it would be great if people could use skimage with the
dependencies provided by a recent version of Ubuntu, say the most recent
LTS, or the latest release of Ubuntu. We will lose many users if they
have to install a new numpy (which also means that they will have to
install non-packaged matplolib, and scipy). If you disagree with this
philosophy, please tell me :-).
My 2 cents,
Emmanuelle
Nelle Varoquaux
As I see things, it would be great if people could use skimage with the
dependencies provided by a recent version of Ubuntu, say the most recent
LTS, or the latest release of Ubuntu. We will lose many users if they
have to install a new numpy (which also means that they will have to
install non-packaged matplolib, and scipy). If you disagree with this
philosophy, please tell me :-).
Stéfan van der Walt
On Fri, Feb 10, 2012 at 10:58 AM, Emmanuelle Gouillart
<emmanuelle...@nsup.org> wrote:
> isn't it possible to test which version of numpy is used, and remove
> float16 from the list if numpy.__version__ < 1.6? It looks like a small
> cost in order to be be able to support numpy 1.5. I can do the change if
> there is no opposition to this.
If we can stay compatible with 1.5 with minor changes, that's probably
worth it. Can you also check the dtype conversion code to make sure
it all works? (Running the unit tests should be enough of an
indication)
> As I see things, it would be great if people could use skimage with the
> dependencies provided by a recent version of Ubuntu, say the most recent
> LTS, or the latest release of Ubuntu. We will lose many users if they
> have to install a new numpy (which also means that they will have to
> install non-packaged matplolib, and scipy). If you disagree with this
> philosophy, please tell me :-).
I understand the challenges of running on an older platform, but I
also wonder: if a user can recompile scikits-image, what's stopping
them from also building the latest (stable) numpy? I certainly have
the expectation that software should run with the available stable
versions of dependencies on the date of release. That said, any
benefit we can get with little effort is worth it, and there's no
reason to purposefully make it harder for users than need be. So
please, keep the PRs coming for backwards compatibility, as long as
they are not too disruptive.
Regards
Stéfan
Jérôme Kieffer
Hi Emmanuelle,
The latest Ubuntu LTS is 10.04 where numpy is v1.3
(http://packages.ubuntu.com/lucid/python-numpy)
The latest stable debian is squeeze where numpy is v1.4.1
(http://packages.debian.org/squeeze/python-numpy)
If you intent to package skimage for debian you should have a look at
what are the current versions in unstable (aka sid) and testing
(currently wheezy). If you keep compatibility with stable one could
even backport skimage to stable and make it available for current
distribution at virtually no-cost.
--
Jérôme Kieffer <jerome....@terre-adelie.org>
PS: We are currently deploying debian6 (squeeze) at work which should
be the reference until end 2013 (at least!) so putting dependency on a
too-recent numpy will prevent skimage to be deployed on the beamline.
Tony Yu
Hi Emmanuelle
If we can stay compatible with 1.5 with minor changes, that's probably
On Fri, Feb 10, 2012 at 10:58 AM, Emmanuelle Gouillart
<emmanuelle...@nsup.org> wrote:
> isn't it possible to test which version of numpy is used, and remove
> float16 from the list if numpy.__version__ < 1.6? It looks like a small
> cost in order to be be able to support numpy 1.5. I can do the change if
> there is no opposition to this.
worth it. Can you also check the dtype conversion code to make sure
it all works? (Running the unit tests should be enough of an
indication)
Stéfan van der Walt
> PS: We are currently deploying debian6 (squeeze) at work which should
> be the reference until end 2013 (at least!) so putting dependency on a
> too-recent numpy will prevent skimage to be deployed on the beamline.
I guess the point I was trying to make is that nobody expects SciPy 12
to work with NumPy 1.5, since they are released at different times.
If you install scikits-image 0.4, released when NumPy 1.5 was stable,
you'll get 1.5 compatibility. The problem only comes in when you
require the latest functionality--in which case upgrading the package
+ dependencies seems completely reasonable.
If we focus on getting scikits-image packaged in the different
distros, these problems should more-or-less go away, since people will
do "apt-get install python-skimage" and be happy with whichever
version is provided.
Stéfan
Emmanuelle Gouillart
> I'd be in favor of supporting numpy 1.5. Just note that the `convert`
> function uses `np.promote_types`, which is also new to numpy 1.6.
> -Tony
Thanks Tony, I figured this out when running the tests :-). So if we want
to support numpy 1.5, we'll have to write a conversion table to emulate
what np.promote_types does, is that correct?
Emmanuelle
Emmanuelle Gouillart
Hi St�fan,
> I understand the challenges of running on an older platform, but I
> also wonder: if a user can recompile scikits-image, what's stopping
> them from also building the latest (stable) numpy?
Well, it's just one more pain in the neck to build another package. And
if you have to build numpy, (I think) you also have to build your own
matplotlib, scipy, etc. Which is even a greater pain.
What's more, a large fraction of users want to use the packages installed
by their IT managers, who like to have one standard version of all
packages. People using Python(x,y) on Windows, for example, would have a
hard time building all the packages. I think the example given by
J�r�me is a good one: he works in a big institute where people might be
interested in using skimage, but users typically don't install their own
packages.
That said, we can of course tell people to install an older version
skimage. However, I have the impression that an important part of the
added value of new releases are in the algorithms/functions that the
release provides, rather than in using newer numpy functionalities.
This is by all means a biased point of view, since I've been more busy
with the former point.
> I certainly have
> the expectation that software should run with the available stable
> versions of dependencies on the date of release. That said, any
> benefit we can get with little effort is worth it, and there's no
> reason to purposefully make it harder for users than need be. So
> please, keep the PRs coming for backwards compatibility, as long as
> they are not too disruptive.
OK, I'll try to :-D.
Cheers,
Emmanuelle
Christoph Gohlke
Note that the original rewrite of the dtype._convert function had it's
own inline implementation of dtype promotion, e.g.
<https://github.com/cgohlke/scikits-image/commit/038db9d1abb684e22656af4a954c94109ac0f94f#L0R74>.
Christoph
Guillaume Gay
Thank you Christoph for the suggestion, I think I will use your
solution, which works fine for me.
Of course BioFormats is kind of the golden standard for those microscopy
formats but I don't feel very comfortable with the java/python bindings
(isn't it adding a java dependency, and making platform changes more
complex?).
Guillaume
Guillaume Gay
I think Stéfan is right: if I use the git repository for skimage, I
shall expect that it depends on the latest versions of the dependencies,
and not complain about lack of compatibility with the packaged version
of numpy, for example.
Yet ... somehow I felt I could use the latest version of skimage
(because there would be for example the latest algorithms implemented)
without having to reconsider my whole python environment, notably
because I am not 100% sure the rest of my code won't suffer. Said
otherwise numpy is at the root of lots of stuff I'm doing, and skimage
is closer to the tip. Pushing this a lot, you could say: the user shall
be ready to compile the latest linux kernel, as they are cloning skimage
from git....
I guess the equilibrium here is hard to catch.
Guillaume
Stéfan van der Walt
<emmanuelle...@nsup.org> wrote:
> What's more, a large fraction of users want to use the packages installed
> by their IT managers, who like to have one standard version of all
> packages. People using Python(x,y) on Windows, for example, would have a
> hard time building all the packages. I think the example given by
> interested in using skimage, but users typically don't install their own
> packages.
We are included in Python(x,y), for the record. The issue here is
only with people who want to use the latest (Git) version of skimage.
> That said, we can of course tell people to install an older version
> skimage. However, I have the impression that an important part of the
> added value of new releases are in the algorithms/functions that the
> release provides, rather than in using newer numpy functionalities.
> This is by all means a biased point of view, since I've been more busy
> with the former point.
That's a very valid point. And since it's so little effort to remain
1.5 compatible, there seems little reason not to.
I'm the guilty party that suggested the use of a numpy 1.6 call in the
converter functions--sorry about that! Chris has kindly made a PR to
address the issue.
So, let's talk about Matplotlib--can we use version 1 yet? If not, I
suggest that we at least copy over 'subplots', because this function
makes it so much easier to build clean examples.
Stéfan
Zachary Pincus
>
>
> Of course BioFormats is kind of the golden standard for those microscopy formats but I don't feel very comfortable with the java/python bindings (isn't it adding a java dependency, and making platform changes more complex?
I'll make a PR for the tifffile / scikits-plugin integration shortly.
As far as BioFormats and python/java, I think Ray has had the most experience working with this, if he wants to weigh in on that.
Zach
>
>
> Guillaume
>
>
>
> Le 10/02/2012 22:20, Christoph Gohlke a écrit :
>> On 2/10/2012 7:46 AM, Zachary Pincus wrote:
>>>> Indeed metadata are important, thanks for pointing this out, I'll have a look at it (though I find metamorph is not so good at recording good metadata).
>>>>
>>>> By the way, I have both .stk and .tif images, so I use Christoph Gohlke tifffile.py for the .stk files. Is ther any chance this will be integrated to skimage?
>>>
>>> Wow! Somehow I was not familiar with Christoph's tifffile.py... that'll be very handy for these cursed LSM files I have laying around. (No help for the ZVIs, but I already have a silly zvi->ome-tiff workflow for them.)
>>>
>>> Perhaps Christoph, who frequents this list, can weight in on whether it would be a good idea to wrap this in as a skimage IO plugin, or if it's best as a separate project...
>>>
>>> Zach
>>
>> A skimage io plugin for tifffile.py is straightforward (attached). The module already implements imread, imsave, and imshow functions. Those functions return/save/show n-dimensional numpy arrays (n>=2). I'm not sure skimage can handle n>3?
>>
>> Tifffile.py and tifffile.c are BSD licensed, so it is no problem to include them with skimage. I don't think the module itself should be maintained or supported as part of skimage.
>>
>>
>> Regarding those cursed, broken, undocumented or NDAed, microscopy file formats:
>>
>> I'm aware of two other projects that strive to read LSM files: pylibtiff <http://code.google.com/p/pylibtiff/> and pylsm <http://www.freesbi.ch/pylsm>.
>>
>> How about a plugin for BioFormats <http://loci.wisc.edu/software/bio-formats>, which supports many more formats? The Cellprofiler project contains GPLed bindings <https://svn.broadinstitute.org/CellProfiler/trunk/CellProfiler/bioformats/>.
>>
>> Christoph
>>
>>
>>
>
> <guillaume.vcf>
Emmanuelle Gouillart
> 1.5 compatible, there seems little reason not to.
> I'm the guilty party that suggested the use of a numpy 1.6 call in the
> converter functions--sorry about that! Chris has kindly made a PR to
> address the issue.
And it works like a charm, as I just tested with my "old" ;-) set-up
(numpy 1.5). Build and tests work fine, the doc builds well, just
perfect! I saw that you've already merged the PR, but I wanted to confirm
that it works fine for me. Thanks Chris, you rock!
> So, let's talk about Matplotlib--can we use version 1 yet? If not, I
> suggest that we at least copy over 'subplots', because this function
> makes it so much easier to build clean examples.
I think Matplotlib is a smaller issue, since it's only used to build the
doc (am I wrong?), that is, more a developer's than a user's job. And
Matplotlib 1.0.1 is packaged for the latest ubuntu. There is always the
risk that people download an example, try to execute it and are annoyed
by the error they get if they have an older version of matplotlib. If
people start complaining, then we might copy over matplotlib's subplots
and call skimage.subplots. But I think that this wouldn't discourage new
users, contrary to installation problems. Maybe a sentence in the user
guide on how to adapt examples for matplotlib < 1.0 could help, too. So
my opinion is, yes, we could use version 1 now.
Cheers,
Emmanuelle