Undefined references when using swpld-ld.exe on Windows 10

[Sean Charles]

Hi,

I have tried very hard to solve this before posting. I found some other users with similar issues but nothing has resolved my issue so far. I am attempting to port an SDL2 interface and first I have to learn how to get a build chain working with something simple! I picked up from one of the other posts that this site is a good example:

    http://rosettacode.org/wiki/Call_a_foreign-language_function#Prolog

I used the code from the SWI docs (mylib) rather than their code but it shows the build process very clearly.

#include <SWI-Prolog.h>

#include <stdio.h>

static foreign_t

pl_say_hello(term_t to)

{ char *a;

  if ( PL_get_atom_chars(to, &a) )

  { MessageBox(NULL, a, "DLL test", MB_OK|MB_TASKMODAL);

    PL_succeed;

  }

  PL_fail;

}

install_t

install_mylib()

{ PL_register_foreign("say_hello", 1, pl_say_hello, 0);

  fprintf(stdout, "Installed say_hello/1\n");

}

It seems to be good and clear however the linker fails! I ran this from a command line session to double check that MinGW, SWI and the SWI Library folder where on PATH as I checked to determine the DLL search order here: https://msdn.microsoft.com/en-us/library/windows/desktop/ms682586(v=vs.85).aspx#search_order_for_desktop_applications

Initially I didn't have the /lib folder on the PATH which I hoped would fix it, so I added that and reran:

 ?- getenv("PATH",X), split_string(X, ";", ";", Y),print_term(Y,[]).

and in amongst the files I see:

  "C:\\MinGW\\bin",

  "C:\\Program Files\\swipl\\bin",

  "C:\\Program Files\\swipl\\lib",

Then I tried the build again with this command:

C:\Users\bp99\Documents\Prolog>swipl-ld -o mylib -v -shared mylib.c

        eval `swipl.exe --dump-runtime-variables`

                CC="gcc"

                PLBASE="c:/program files/swipl"

                PLARCH="x64-win64"

                PLLIBS=""

                PLLIB="-lswipl"

                PLCFLAGS=""

                PLLDFLAGS=""

                PLSOEXT="dll"

                PLTHREADS="yes"

        gcc -c   -D_REENTRANT -D__WINDOWS__ -D_WINDOWS -D__SWI_PROLOG__ -I"c:/program files/swipl/include" -o mylib.obj mylib.c

        gcc -o mylib.dll -shared  mylib.obj -L"c:/program files/swipl/lib" -lswipl

mylib.obj:mylib.c:(.text+0x14): undefined reference to `PL_get_atom_chars'

mylib.obj:mylib.c:(.text+0x76): undefined reference to `PL_register_foreign'

collect2.exe: error: ld returned 1 exit status

gcc returned code 1

        rm mylib.obj

*** swipl-ld exit status 1

I opened the definitions file from the c:\Progam Files\swipl\lib\libswipl.def and sure enough those two entry points are listed:

PL_get_atom_chars @80

PL_register_foreign @222

At one point I copied the contents of the lib older into my folder to see if it would magically find them but no change, my only observation is there appears to be no libswipl.dll ? Is that OK?? I am running 7.6.3, installed an hour ago.

So I am at a loss as to why it is not finding the references?!?!?! It's been many years since I tangled with Windows C/Linkers, mainly being OSX/Linux based and I can't tell what the problem might be.

[Jan Wielemaker]

On 12/01/2017 01:06 PM, Sean Charles wrote:
> Hi,
>
> I have tried very hard to solve this before posting. I found some other
> users with similar issues but nothing has resolved my issue so far. I am
> attempting to port an SDL2 interface and first I have to learn how to
> get a build chain working with something simple! I picked up from one of
> the other posts that this site is a good example:
>
>     http://rosettacode.org/wiki/Call_a_foreign-language_function#Prolog

> <http://rosettacode.org/wiki/Call_a_foreign-language_function#Prolog>
>
> <http://rosettacode.org/wiki/Call_a_foreign-language_function#Prolog>
> I
> <http://rosettacode.org/wiki/Call_a_foreign-language_function#Prolog> used

> the code from the SWI docs (mylib) rather than their code but it shows
> the build process very clearly.
>

> It seems to be good and clear however the linker fails! I ran this from
> a command line session to double check that MinGW, SWI and the SWI
> Library folder where on PATH as I checked to determine the DLL search
> order here:
> https://msdn.microsoft.com/en-us/library/windows/desktop/ms682586(v=vs.85).aspx#search_order_for_desktop_applications
>
> Initially I didn't have the /lib folder on the PATH which I hoped would
> fix it, so I added that and reran:
>
>  ?- getenv("PATH",X), split_string(X, ";", ";", Y),print_term(Y,[]).
>
> and in amongst the files I see:
>

> /  "C:\\MinGW\\bin",/
> /  "C:\\Program Files\\swipl\\bin",/
> /  "C:\\Program Files\\swipl\\lib",/

> see if it would magically find them but no change, *my only observation

> is there appears to be no libswipl.dll ? Is that OK?? I am running

> 7.6.3, installed an hour ago.*

libswipl.dll is in the `bin` directory. I don't think it requires the
.def file. I think MinGW should use lib/libswipl.dll.a. I wouldn't
be surprised if Windows needs some additional trick. Hopefully someone
has gotten this to work and will tell us ...

> So I am at a loss as to why it is not finding the references?!?!?! It's
> been many years since I tangled with Windows C/Linkers, mainly being
> OSX/Linux based and I can't tell what the problem might be.

It works on OSX and Linux. I'm sure it can somehow work on Windows as
well, but I don't have Windows around to test. Try adding -v to gcc to
see what happens.

Cheers --- Jan

>
>
> --
> You received this message because you are subscribed to the Google
> Groups "SWI-Prolog" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to swi-prolog+...@googlegroups.com
> <mailto:swi-prolog+...@googlegroups.com>.
> Visit this group at https://groups.google.com/group/swi-prolog.
> For more options, visit https://groups.google.com/d/optout.

[Sean Charles]

Jan,

I used "-v" to the swipl-ld command but do you mean adding verbose output to the GCC compiler? If that's the case I will try to do that and see what happens. I know there are some gcc flags that tell it to dump out include paths, library paths etc.

Will post back when I have some output.

Sean.

> <mailto:swi-prolog+unsub...@googlegroups.com>.

[Jan Wielemaker]

On 12/01/2017 03:10 PM, Sean Charles wrote:
> Jan,
>
> I used "-v" to the swipl-ld command but do you mean adding verbose
> output to the GCC compiler? If that's the case I will try to do that and
> see what happens. I know there are some gcc flags that tell it to dump
> out include paths, library paths etc.

Yes, just copy-paste-edit the swipl-ld.exe generated gcc command. P.s.
the `lib` directory does not need to be in %PATH% AFAIK.

--- Jan

> > an email to swi-prolog+...@googlegroups.com <javascript:>
> > <mailto:swi-prolog+...@googlegroups.com <javascript:>>.

> > Visit this group at https://groups.google.com/group/swi-prolog

> <https://groups.google.com/group/swi-prolog>.

> > For more options, visit https://groups.google.com/d/optout

> <https://groups.google.com/d/optout>.

>
> --
> You received this message because you are subscribed to the Google
> Groups "SWI-Prolog" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to swi-prolog+...@googlegroups.com

> <mailto:swi-prolog+...@googlegroups.com>.

[Sean Charles]

Ok, some progress... I am using MinGW system so I read:

    http://www.mingw.org/wiki/HOWTO_Specify_the_Location_of_Libraries_for_use_with_MinGW

This led me to perform this in my command shell:

set LIBRARY_PATH="c:/program files/swipl/lib"

I ran the previous swipl-ld.exe incantation and it built just fine! No more problems so it means that if one is to use MinGW then it's necessary to either explicitly set the library path OR to install the required libraries somewhere beneath the "unix like" tree structure that you will find under the MinGW installation root.

That was great, it solved my build problem however I now have another problem! HAHA

λ swipl

Welcome to SWI-Prolog (threaded, 64 bits, version 7.6.3)

SWI-Prolog comes with ABSOLUTELY NO WARRANTY. This is free software.

Please run ?- license. for legal details.

For online help and background, visit http://www.swi-prolog.org

For built-in help, use ?- help(Topic). or ?- apropos(Word).

1 ?- [mylib].

ERROR: c:/users/bp99/documents/prolog/mylib.pl:2:

        c:/users/bp99/documents/prolog/mylib.pl:2: Initialization goal raised exception:

        '$open_shared_object'/3: The specified module could not be found.

ERROR: Exported procedure mylib:say_hello/1 is not defined

true.

So...I now have a library but it seems to not be exporting its defined function... here is the mylib.pl file:

:- module(mylib, [say_hello/1]).

:- use_foreign_library(mylib).

And then running `nm` against the library I built gives this:

λ nm mylib

00000000 b .bss

00000000 d .data

00000000 i .drectve

00000000 r .eh_frame

00000000 r .rdata

00000000 r .rdata$zzz

00000000 t .text

00000050 T _install_mylib

         U _MessageBoxA@16

         U _PL_get_atom_chars

         U _PL_register_foreign

00000000 t _pl_say_hello

Again, not being as good at Windows development as I used to be (if ever!) I am not sure why the entry point isn't being found. At least I can now build things.

The function is called `pl_say_hello` in the C source, the compiler has named mangled it to `_pl_say_hello` and the foreign library setup call maps one to the other. And the Prolog code attempts to load the correct function so I really can't see what's wrong!

>      > <mailto:swi-prolog+unsub...@googlegroups.com <javascript:>>.

>      > Visit this group at https://groups.google.com/group/swi-prolog
>     <https://groups.google.com/group/swi-prolog>.
>      > For more options, visit https://groups.google.com/d/optout
>     <https://groups.google.com/d/optout>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "SWI-Prolog" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to swi-prolog+...@googlegroups.com

> <mailto:swi-prolog+unsub...@googlegroups.com>.

[Sean Charles]

I think I have been making a mistake! I noticed "-c" which means compile only... removing that put me straight back to square one!

λ swipl-ld -shared -v -o mylib mylib.c

        eval `swipl.exe --dump-runtime-variables`

                CC="gcc"

                PLBASE="c:/program files/swipl"

                PLARCH="x64-win64"

                PLLIBS=""

                PLLIB="-lswipl"

                PLCFLAGS=""

                PLLDFLAGS=""

                PLSOEXT="dll"

                PLTHREADS="yes"

        gcc -c   -D_REENTRANT -D__WINDOWS__ -D_WINDOWS -D__SWI_PROLOG__ -I"c:/program files/swipl/include" -o mylib.obj mylib.c

        gcc -o mylib.dll -shared  mylib.obj -L"c:/program files/swipl/lib" -lswipl

mylib.obj:mylib.c:(.text+0x14): undefined reference to `PL_get_atom_chars'

mylib.obj:mylib.c:(.text+0x76): undefined reference to `PL_register_foreign'

collect2.exe: error: ld returned 1 exit status

gcc returned code 1

        rm mylib.obj

*** swipl-ld exit status 1

Putting this stupidity to one side, and even with LIBRARY_PATH it seems still not to be finding the exported functions to resolve the linker phase and it's clear to see both the library and path being used in this case.

I am sure the answer is obviously simple to somebody, just not me, not yet!

:)

[Sean Charles]

[SOLVED]

Ok, there is no cure for stupidity is there? I will buy the entire warehouse...

Essentially, not yet being a hardended Windows+MinGW developer, I had installed THE 32 BIT VERSION and was then trying to build and link against, presumably, the wrong libraries and who knows what shenanigans were going on behind the scenes!

I uninstalled MinGW(32) and replaced it with the funky new 64 bit version, and guess what, it builds, loads and runs just fine!

Sigh....at least now I can get some SDL2 interoperability on the way...it feels a bit daunting as I've studied the plOpenGL package and I remember how much work it was doing the GNU Prolog version... the good news is that all of the code that I wrote for the GNU Prolog side of things will give me some practice in loading and transforming Prolog code into the correct SWI foreign declarations and so it should only be a matter of hacking the C code as well... I will see if I can be "clever" and use SWI prolog to manipulate the C source code as it's basically the same pattern: argument unmarshalling, call SDL2, return the response as success / fail etc.

Onwards!

:)

Thanks to everybody, and of course Jan! You are a busy man I'm sure!!!

[Sean Charles]

More of an update really and some pointers on how to move it forward...

I have now been able to compile, link and build a SWI extension that uses SDL2. I have created a few simple proof of concept calls that create a window and destroy a window. This is but the tip of the proverbial iceberg. I put in a lot of work doing the GNU Prolog binding and I now have to seriously sit down and look at some code reuse options to save some time and reduce errors.

The GNU Prolog code looks like this for the foreign module declarations:

:- foreign(sdl_Init_C(+positive), [fct_name(gp_SDL_Init)]).

:- foreign(sdl_Quit,              [fct_name(gp_SDL_Quit)]).

%% Window functions

:- foreign(sdl_CreateWindow_C(+codes, +integer, +integer, +positive, +positive, +integer, -positive), [fct_name(gp_SDL_CreateWindow)]).

:- foreign(sdl_DestroyWindow(+positive), [fct_name(gp_SDL_DestroyWindow)]).

:- foreign(sdl_SetWindowTitle(+positive, +codes), [fct_name(gp_SDL_SetWindowTitle)]).

    :

    :

I am 100% confident that even with my limited skills I will be able to read those terms and then create the equivalent SWI module declarations as well as create the PL_register_foreign calls as well.

It'll take me some time but I am looking forward to increasing my Prolog skills on all fronts. The thought of being able to have both OpenGL and SDL2 with SWI is too good to pass up, as SDL2 allows you to create an OpenGL context as well so a large part of plOpenGL would instantly be useful.

At this point I would ask if anybody has any high-falutin' meta-level tips or pointers that would help me translate the above or things to read that are relevant to term transformation / translation, I know that predicates exist for that.

Thanks again for everybodies help in getting this far!

Sean.

[Jan Wielemaker]

Well, there is a lot of stuff out there. The gprolog binding reminds me
a bit of the Quintus and SICStus bindings. This is supported by the
library(qpforeign). This library accepts a similar specification and
creates C code for the wrappers. That is one. Second is that years
ago I wrote a wrapper generator for OpenGL. That was, as I recall,
based on reading the OpenGL headers. If you are interested (or
anyone else), I'll look around and put it on Github.

Finally, I'm investigating the options for a dynamic calling interface
that is around in various forms in many scripting languages. I'm not
really sure where this is heading. Just calling functions with scalar
arguments and return values is easy, but how to deal with pointers and
structures is a completely different story. So far I've been looking
at dyncall and cinvoke. The biggest issue seems obtaining type info.
cinvoke lets you dynamically build structures, but you need to know
the basic C types of the fields. Modern libraries typically hide those
details behind layers of abstractions to deal with their portability
requirements. Roughly, there seem to be two routes. One is to parse
the C code and the other seems to demand the library built with debug
info and inspect the debug info to derive the API.

If someone has pointers to other solutions worthy exploring or is
willing to help, please share.

Cheers --- Jan

> <http://rosettacode.org/wiki/Call_a_foreign-language_function#Prolog>
>
> <http://rosettacode.org/wiki/Call_a_foreign-language_function#Prolog>
> I

> <http://rosettacode.org/wiki/Call_a_foreign-language_function#Prolog> used

> /  "C:\\MinGW\\bin",/

> /  "C:\\Program Files\\swipl\\bin",/
> /  "C:\\Program Files\\swipl\\lib",/

> *my only observation is there appears to be no libswipl.dll
> ? Is that OK?? I am running 7.6.3, installed an hour ago.*

>
>
> So I am at a loss as to why it is not finding the
> references?!?!?! It's been many years since I tangled with
> Windows C/Linkers, mainly being OSX/Linux based and I can't
> tell what the problem might be.
>
>

> --
> You received this message because you are subscribed to the Google
> Groups "SWI-Prolog" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to swi-prolog+...@googlegroups.com

> <mailto:swi-prolog+...@googlegroups.com>.