pure-ffi under MinGW

[Jiri Spitz]

Hi,

I have some hints for compiling pure-ffi under MinGW:

1. You can find working ffi for MinGW here:

http://ftp.gnome.org/pub/GNOME/binaries/win32/dependencies/libffi_3.0.6-1_win32.zip

http://ftp.gnome.org/pub/GNOME/binaries/win32/dependencies/libffi-dev_3.0.6-1_win32.zip

Notice that the include directory in libffi_3.0.6-1_win32.zip is
errorneously placed under bin so you have to move it to the proper place.

2. Make requires the following two more options for linking:
-lgmp -Wl,--enable-auto-import

3. Declarations from runtime.h, lines 910 (popen) and 911 (pclose)
conflict with stdio.h. (Rev. 637)

4. Make install guesses the destdir as .../MinGW/bin/pure.exe.

After dealing with these issues the interface compiles and the examples
work.

Great stuff!

Jiri

[Albert Graef]

Jiri Spitz wrote:
> I have some hints for compiling pure-ffi under MinGW:

Wonderful, many thanks. So I can probably release Pure 0.17 over the
weekend, with pure-ffi included in the Windows package. Toni has already
made a SUSE package of pure-ffi. Ryan, would you consider adding an OSX
port of pure-ffi? libffi is already in MacPorts, so that shouldn't be
very difficult.

Albert

--
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email: Dr.G...@t-online.de, a...@muwiinfa.geschichte.uni-mainz.de
WWW: http://www.musikinformatik.uni-mainz.de/ag

[Albert Graef]

Jiri Spitz wrote:
> 1. You can find working ffi for MinGW here:

I've added this to pure-mingw.zip now.

> 2. Make requires the following two more options for linking:
> -lgmp -Wl,--enable-auto-import

Fixed.

> 3. Declarations from runtime.h, lines 910 (popen) and 911 (pclose)
> conflict with stdio.h. (Rev. 637)

Fixed.

> 4. Make install guesses the destdir as .../MinGW/bin/pure.exe.

This works over here. Are you sure you ran this in msys shell? Maybe a
different 'which' program on the path?

Thanks,

[Ryan Schmidt]

On Jan 28, 2009, at 18:01, Albert Graef wrote:

> Ryan, would you consider adding an OSX
> port of pure-ffi? libffi is already in MacPorts, so that shouldn't be
> very difficult.

Sure, here's my work in progress:

http://trac.macports.org/ticket/18305

The problem is that pure-ffi wants to install its stuff into $
{prefix}/lib/pure but that is a symlink to ${prefix}/lib/pure-0.17
and MacPorts cannot install things into paths under symlinks. I would
need read up on exactly why that is to be able to explain it
properly, but it's unlikely to change in MacPorts. So my question is
whether pure's symlink is really necessary. I could modify pure-ffi
to install into ${prefix}/lib/pure-0.17 but then that binds it to the
version of pure. If I install pure 0.17, then pure-ffi, and then
update pure to 0.18, the pure-ffi stuff would be orphaned in the lib/
pure-0.17 directory. Can I just make lib/pure the real directory and
dispense with the symlink? Or if there must be something called lib/
pure-${version}, then can I make that the symlink pointing to lib/
pure? Are API/ABI-incompatible changes still being made in pure or
what is the idea behind the versioned library directory?

Other notes:

The README says "In particular, 'make install prefix=/usr' sets the
installation prefix" however the Makefile seems to respect "make
install DESTDIR=..." which MacPorts uses by default so I did not use
"make install prefix=..."

The Makefile uses "gcc" but I patched it to use "$(CC)" instead.

ffi.dylib doesn't build because of common symbols:

/usr/bin/gcc-4.0 -dynamiclib -o ffi.dylib ffi.c -I/mp/include -g -O2
-L/mp/lib -lffi -lpure -lgmp
ld: common symbols not allowed with MH_DYLIB output format with the -
multi_module option
/var/tmp//ccfdBB2f.o definition of common _ffi_type_string (size 16)

I added -fno-common to the CFLAGS to fix this. The Makefile did not
accept me setting CFLAGS in the environment; it required me to pass
CFLAGS as an argument to the make command.

[Albert Graef]

Ryan Schmidt wrote:
> I could modify pure-ffi to install into ${prefix}/lib/pure-0.17 but
> then that binds it to the version of pure.

You can't really avoid that, since a module built for Pure version x
won't in general be compatible with version y, so it needs to be
recompiled. Same with other PLs, e.g. Python also has a versioned
library directory and loads of modules, maybe you can take a look at how
that is handled in MacPorts.

But in any case I should add some variables to the Makefile which make
it easier to build against a specific Pure version.

For the time being, as a quick and dirty hack you could replace
{prefix}/lib/pure with `readlink {prefix}/lib/pure` in the Makefile
(note the backquotes). readlink(1) is available on OSX, right?

> Can I just make lib/pure the real directory and dispense with the
> symlink? Or if there must be something called lib/
> pure-${version}, then can I make that the symlink pointing to lib/
> pure?

No, that will both make it impossible to install different Pure versions
side by side.

> Are API/ABI-incompatible changes still being made in pure or
> what is the idea behind the versioned library directory?

Yes, and it's not just the runtime API, the library keeps changing as
well, so it is very desirable to be able to have different versions
installed on the same system.

> The README says "In particular, 'make install prefix=/usr' sets the
> installation prefix" however the Makefile seems to respect "make
> install DESTDIR=..." which MacPorts uses by default so I did not use
> "make install prefix=..."

prefix and DESTDIR are for different purposes. DESTDIR is the one to use
if you want to install into a staging directory. *Never* include the
DESTDIR in prefix instead, that will most surely break your package!

> The Makefile uses "gcc" but I patched it to use "$(CC)" instead.

Yeah, I should fix that.

> ffi.dylib doesn't build because of common symbols:

There should be only one ffi_type_string in ffi.o, and none in -lffi,
can you figure out where the other symbol comes from? Maybe the MacPorts
version of libffi is different?

> The Makefile did not
> accept me setting CFLAGS in the environment; it required me to pass
> CFLAGS as an argument to the make command.

I'll see what I can do about this. Would it make a big difference in
your build process?

Thanks,

[Jiri Spitz]

Albert Graef wrote:
> The Makefile uses "gcc" but I patched it to use "$(CC)" instead.
>
>
> Yeah, I should fix that.
>

CC has no default value in r726. Might there be platform dependent defaults?

Jiri

[Albert Graef]

Jiri Spitz wrote:
> CC has no default value in r726. Might there be platform dependent defaults?

According to the docs, CC defaults to cc in GNU make. At least it should
always be defined, as GNU make has built-in rules to compile C.

[Jiri Spitz]

Albert Graef wrote:
> 4. Make install guesses the destdir as .../MinGW/bin/pure.exe.
>
>
> This works over here. Are you sure you ran this in msys shell? Maybe a
> different 'which' program on the path?
>

$ which pure
/d/MinGW/bin/pure.exe

and hence

$ make install
test -d "/d/MinGW/bin/pure.exe/lib/pure" || mkdir -p
"/d/MinGW/bin/pure.exe/lib/pure"
mkdir: cannot create directory `/d/MinGW/bin/pure.exe': File exists
make: *** [install] Error 1

What is going wrong?

Jiri

[Ryan Schmidt]

On Feb 1, 2009, at 08:35, Albert Graef wrote:

> Ryan Schmidt wrote:
>
>> I could modify pure-ffi to install into ${prefix}/lib/pure-0.17 but
>> then that binds it to the version of pure.
>
> You can't really avoid that, since a module built for Pure version x
> won't in general be compatible with version y, so it needs to be
> recompiled. Same with other PLs, e.g. Python also has a versioned
> library directory and loads of modules, maybe you can take a look
> at how
> that is handled in MacPorts.

MacPorts has separate ports for each major version of Python
(python24, python25, python26, python30, etc.). And each module has a
port for each version of Python (py-setuptools, py25-setuptools, py26-
setuptools, etc.). When updating the port for a Python module, people
are constantly updating only the one for the version of Python they
use and forgetting about the other Python versions so things are
always out of sync. Most modules don't have ports for all versions of
Python so you probably can't even use the module you want until
you've made a copy of it for the version of Python you're using. It
is a royal mess that I never want to see again. :)

> But in any case I should add some variables to the Makefile which make
> it easier to build against a specific Pure version.
>
> For the time being, as a quick and dirty hack you could replace
> {prefix}/lib/pure with `readlink {prefix}/lib/pure` in the Makefile
> (note the backquotes). readlink(1) is available on OSX, right?

I can try this.

>> Can I just make lib/pure the real directory and dispense with the
>> symlink? Or if there must be something called lib/
>> pure-${version}, then can I make that the symlink pointing to lib/
>> pure?
>
> No, that will both make it impossible to install different Pure
> versions
> side by side.

That is never possible in MacPorts anyway.

>> Are API/ABI-incompatible changes still being made in pure or
>> what is the idea behind the versioned library directory?
>
> Yes, and it's not just the runtime API, the library keeps changing as
> well, so it is very desirable to be able to have different versions
> installed on the same system.

In MacPorts only one version of a port can be active at a time. I
don't know how we could allow multiple versions of pure to be active
at a time, since e.g. both pure 0.16 and 0.17 want to install a file
called ${prefix}/bin/pure. Which version would win?

>> The README says "In particular, 'make install prefix=/usr' sets the
>> installation prefix" however the Makefile seems to respect "make
>> install DESTDIR=..." which MacPorts uses by default so I did not use
>> "make install prefix=..."
>
> prefix and DESTDIR are for different purposes. DESTDIR is the one
> to use
> if you want to install into a staging directory.

Yes, that's what we do.

> *Never* include the
> DESTDIR in prefix instead, that will most surely break your package!

Many ports in MacPorts include the DESTDIR in the prefix at install
time, because the software in question does not support DESTDIR.

>> The Makefile uses "gcc" but I patched it to use "$(CC)" instead.
>
> Yeah, I should fix that.
>
>> ffi.dylib doesn't build because of common symbols:
>
> There should be only one ffi_type_string in ffi.o, and none in -lffi,
> can you figure out where the other symbol comes from?

How would I find this out?

> Maybe the MacPorts version of libffi is different?

It doesn't look like the portfile is doing anything out of the
ordinary. The version is 3.0.8.

http://trac.macports.org/browser/trunk/dports/devel/libffi/Portfile

>> The Makefile did not
>> accept me setting CFLAGS in the environment; it required me to pass
>> CFLAGS as an argument to the make command.
>
> I'll see what I can do about this. Would it make a big difference in
> your build process?

Not really, it just surprised me.

[Albert Graef]

Jiri Spitz wrote:
> $ which pure
> /d/MinGW/bin/pure.exe

It shouldn't print the .exe extension. What does 'which which' say?

Are you running msys 1.0.10 or 1.0.11?

> $ make install
> test -d "/d/MinGW/bin/pure.exe/lib/pure" || mkdir -p
> "/d/MinGW/bin/pure.exe/lib/pure"
> mkdir: cannot create directory `/d/MinGW/bin/pure.exe': File exists
> make: *** [install] Error 1
>
> What is going wrong?

Well, the extra .exe extension breaks the logic to determine the install
prefix. Here's the relevant line from the Makefile:

prefix = $(patsubst %/bin/pure,%,$(shell which pure 2>/dev/null))

If you replace %/bin/pure with %/bin/pure.exe, it should work as
advertized on your system. Or just add prefix=/whatever to your make
install command.

But, as I said, I really think that it's an issue with your 'which' program.

[Albert Graef]

Ryan Schmidt wrote:
> It is a royal mess that I never want to see again. :)

Oh dear. :) Well, that's because the maintainers are different. If you
maintain both the core package and the modules, you'll just rebuild the
modules when a new Pure version comes out (it should be easy to
automatize that with a little shell script), and everything will be fine.

> In MacPorts only one version of a port can be active at a time. I
> don't know how we could allow multiple versions of pure to be active
> at a time, since e.g. both pure 0.16 and 0.17 want to install a file
> called ${prefix}/bin/pure. Which version would win?

The version that gets installed last. That's the version you get when
running 'pure', to get a specific version you just type 'pure-x.y' instead.

That's the way it works with a source install and also with Toni's SUSE
rpms. This is very usual business, and is routinely done with packages
which allow different versions to be installed in parallel. Python is
one example (and most other scripting language interpreters), but also
any other software which anticipates the user's need to deal with
different legacy versions.

I don't say that you have to support this in MacPorts. That's entirely
up to you. But that doesn't mean that I shouldn't support it upstream.

> Many ports in MacPorts include the DESTDIR in the prefix at install
> time, because the software in question does not support DESTDIR.

This will usually work if the software in question doesn't use any
hardcoded paths, symlinks etc. derived from prefix. Otherwise you're
going to get a broken package. In that case you should ask the authors
to add DESTDIR support. It's the established way to handle staged
installs at least since the days of the good ol' X11 Imakefiles. Anyone
still remember these? ;-)

>> There should be only one ffi_type_string in ffi.o, and none in -lffi,
>> can you figure out where the other symbol comes from?
>
> How would I find this out?

Just an idea: Maybe the linker gets confused because it's linking
against -lffi while linking a shared module with the same basename
ffi.dylib? So that it links the module against itself?

If that's the case then renaming the shared module in the Makefile
(e.g., ffi$(DLL) -> pure-ffi$(DLL)) might help. I also do that on
Windoze to work around some limitations of the dynamic linker there,
which sometimes can't keep apart -lxyz and xyz.dll either. (Of course
then you also need to change the 'using "lib:ffi";' statement at the
beginning of ffi.pure to 'using "lib:pure-ffi";'.)

[Jiri Spitz]

Albert Graef wrote:
> Are you running msys 1.0.10 or 1.0.11?

"Which" from the package "msysCORE-1.0.11-20080826.tar.gz" is buggy.

Thanks,

Jiri