How to distribute the binary FreeImage library

[Almar Klein]

I think we agree that in order to make this package work "out of the box" on all OS's, we need to somehow ship the FreeImage binary libs. At least on Windows and Mac. I'll start with writing down a kindof summary of the discussion we've had so far, and some random thoughts.

Zach has hosted the binaries here: https://github.com/zachrahan/freeimage-sharedlib

(On a side note, I suggest to put them somewhere on http (not https), because some users may have build Python without ssl (I found out that I did) ).

Zach also wrote some code to download these libs on the fly. The question is then: when do we download them. And what do we do when there is no internet connection at that time. Downloading at runtime is definetly a bad idea.

Zach: "The binaries aren't *that* big; one could just ship them all with the python source. Or one could make n different downloads for the different platforms. Not sure what's better / the more "pythonic" solution."

The binaries are about 2.5, 3.3 and 6.4 MB in size for win32, win64 and Mac, respectively. If we zip them, we get a total of 4MB. If we ship them all in one bundle, we could also zip the libs individually and only unzip the one that's valid for the users OS, but I don't think that's worth the effort.

Since we want to update the binaries as new versions of FreeImage come out, it's probably not good practice to put them in version control :)  But we could include them in the sdist zip/tar files for distribution. That means the binaries are downloaded at package-time. I that even a real word? I mean at the moment someone builds the zipfile for the new release.

Comments are welcome.

[Almar Klein]

I've got a setup script now that will download the library when executed (can be at build time or install time). So this is technically not a problem. We only need to decide what is the best way to do this. As I see it, there are three moments that the libs can be downloaded (and we can support multiple of these):

1. Download at runtime. By supporting this, someone can even check out the source in a directory that's on his PYTHONPATH and the libs will be downloaded the first time he/she imports imageio.
   
2. Download at install time. When using PIP or easy_install, this makes sense, because you can rely on an internet connection. (Still the hosting website can be down, so maybe we need a mirror?)
   
3. Dowload when packaging, and pack the libraries for *all* OS's in the sdist's zipfile. Or maybe create an sdist distribution per OS.
   

I like (1) and think we should support it if its easy to implement. But that's not enough, because you cannot rely on an internet connection. (2) is better, but may cause problems in some cases, for instance a user may download the zip-file and installs it later when there is no connection to the internet. Or is it acceptable to require an internet connection for installation?

Therefore I think we should definitely consider (3). Arguments in favor of packing all libraries in the distribution are simplicity and robustness.The only argument against of this approach is size. We can reduce this from 12MB to 4MB by zipping the individual libraries (unzipping can be done with 3 lines of Python code). But that still feels like a lot for a library that only reads and writes images.

What if we make one sdist distro per OS, and maybe one universal distribution that does not contain any libraries (so they're downloaded at install time). Do PIP and easy_install support this in any way?

Comments are welcome.

[Zach Pincus]

I know *nothing* about PIP/easy_install. Nor do I want to. So I can't help advise what's best there.

Most larger packages take the one-zipfile-per-OS route, which is reasonable, especially with a "download at install-time" no-library option. I don't think download at runtime really is necessary, and perhaps could cause plenty of headaches? Though, as above, I have no idea how download-at-install-time interacts with easy_install.

Zach

[edcjones]

I suggest that imageio should detect if freeimage is installed.  Also give the user some information about finding and installing freeimage.  This can be in the INSTALL file.

imageio and freeimage should be kept completely separated.  If imageio assumes the responsibility of installing freeimage, then bugs, suggestions, and complaints about freeimage will come to imageio and you will have to process them and pass them on to freeimage.  Users will tend not to think of imageio and freeimage as separate projects.

In the long run, complete separation will also simplify things for the packaging systems used by Debian, RedHat, etc.  imageio would be a package which depends on the package for freeimage.

[Almar Klein]

What do you mean by "freeimage"? Do you mean the FreeImage library, or do you mean a Python wrapper of it?

If you mean the latter, I dont see how splitting the package in two will help maintainability. And for the user it's not beneficial either. If we maintain a separate freeimage package, bugs of freeimage will still come to us, most likely on the imageio issue tracker in any case, and we still need to pass them on to the FreeImage people.

  Almar

[Almar Klein]

Yes, I think download at install time should be kind of the default. Providing one zipfile per OS seems reasonable. I think that putting all libs in a zipfile by default is a bad idea.

I like the idea of being able to install the libs at runtime, but I agree that it should not cost us too much trouble...

  Almar

[edcjones]

On Sunday, July 1, 2012 6:54:26 AM UTC-4, Almar Klein wrote:

What do you mean by "freeimage"? Do you mean the FreeImage library, or do you mean a Python wrapper of it?

The library.
 

[Zach Pincus]

The problem with not at least optionally distributing the FreeImage dll/dylib with imageio is that there are no pre-built win64 or OS X (32 or 64-bit) binaries available from the FreeImage project. And requiring win/os x users to build their own FreeImage from source is pretty inimical to the idea of a simple "low-dependency" python image IO library.

So I think it's important to at least provide pre-built binaries for these platforms, and some easy way to install them. 

[Almar Klein]

Hi,

As I already mentioned in another post, imageio is now available on PyPi. I uploaded an sdist distribtion that contains no libraries, these will be downloaded at install-time. For Windows users I also uploaded an installer that comes prepacked with both the 32bit and 64 bit binary. For other OS's there is currently not really a way to distribute binaries.

Imageio can also download the libraries at runtime; I like to develop on packages without having to install them. 

Some notes about freezing apps that use imageio: This does not work out of the box, because a program like cx_freeze does not know about the freeimage library. There is a discussion going in the cx_freeze mailing list to see if we can come up with an approach to let packages specify resources. In the mean time, a freeze script should now call a special function in imageio/freeze.py to copy the freeimage library along. This works for cx_freeze, I have no idea about other freezer programs though.

  Almar

--
You received this message because you are subscribed to the Google Groups "imageio" group.

To view this discussion on the web visit https://groups.google.com/d/msg/imageio/-/Q-j6xdHeh7cJ.

To post to this group, send email to ima...@googlegroups.com.
To unsubscribe from this group, send email to imageio+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/imageio?hl=en.