Mingw64/msys2 linking issues

[Stephen Woodbridge]

I have mingw64 and msys2 installed on Windows 10 64bit. I have been able to install most everything using pacman. I created a sources directory like:

$ ls /sources
dlfcn-win32                libspatialite-4.3.0a.tar.gz   spatialite-tools-4.3.0
librasterlite-1.1g         spatialite_gis-1.0.0c.tar.gz  spatialite-tools-4.3.0.tar.gz
librasterlite-1.1g.tar.gz  spatialite_gui-1.7.1.tar.gz
libspatialite-4.3.0a       spatialite-docs

and want to build from source so I can eventually compile my own C/C++ application that uses these libraries.

Issue 1: compiling libspatialite

cd libspatialite-4.3.0a
./configure --target=mingw64
make
<lots of output>
libtool: link: gcc -I./../src/headers -g -O2 -o .libs/demo1.exe demo1.o  -L../src /sources/libspatialite-4.3.0a/src/.libs/libspatialite.a -L/mingw64/lib -LC:/building/msys64/mingw64/lib -lpthread -ldl -lxml2 -llzma -lws2_32 -lfreexl -liconv -lproj -lz -lsqlite3 -lgeos_c
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/6.1.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -ldl
collect2.exe: error: ld returned 1 exit status

I can resolve this via:

 git clone https://github.com/dlfcn-win32/dlfcn-win32.git
 cd dlfcn-win32/
 ./configure
 make
 make install
 ln -s /mingw/lib/libdl.a /mingw64/lib/libdl.a

cd ../libspatialite-4.3.0a
make  ### now suceeds
make install

But I'm not sure if the is the correct thing to do.

Issue 2: compiling librasterlite

cd librasterlite-1.1g/
./configure
make
/bin/sh ../libtool --tag=CC   --mode=link gcc  -g -O2 -version-info 2:0:0 -no-undefined  -o librasterlite.la -rpath /mingw64/lib rasterlite_io.lo rasterlite_image.lo rasterlite_aux.lo rasterlite_quantize.lo rasterlite_gif.lo rasterlite_png.lo rasterlite_jpeg.lo rasterlite_tiff.lo rasterlite_version.lo rasterlite.lo -L/mingw64/lib -lspatialite -liconv -lz -lm -L/mingw64/lib -lpng16 -lz -lgeotiff -ltiff -ljpeg -lspatialite -lproj -lgeotiff -lproj -ltiff -ljpeg -lz

*** Warning: This system can not link to static lib archive /mingw64/lib/libspatialite.la.
*** I have the capability to make that library automatically link in when
*** you link to this library.  But I can only do this if you have a
*** shared version of the library, which you do not appear to have.

libtool: link: rm -fr  .libs/librasterlite.dll.a
libtool: link: gcc -shared  .libs/rasterlite_io.o .libs/rasterlite_image.o .libs/rasterlite_aux.o .libs/rasterlite_quantize.o .libs/rasterlite_gif.o .libs/rasterlite_png.o .libs/rasterlite_jpeg.o .libs/rasterlite_tiff.o .libs/rasterlite_version.o .libs/rasterlite.o   -L/mingw64/lib -liconv -lpng16 -L=C:/building/msys64/mingw64/lib -lgeotiff -lproj -ltiff -ljpeg -lz  -O2   -o .libs/librasterlite-2.dll -Wl,--enable-auto-image-base -Xlinker --out-implib -Xlinker .libs/librasterlite.dll.a
.libs/rasterlite.o: In function `fetch_resolutions':
C:\Users\woodbri\projects\sources\librasterlite-1.1g\lib/rasterlite.c:180: undefined reference to `sqlite3_prepare_v2'
C:\Users\woodbri\projects\sources\librasterlite-1.1g\lib/rasterlite.c:188: undefined reference to `sqlite3_reset'
<lots more similar errors to these>

I'm not sure how to resolve this. I presume that this is probably related to the libdl issue and the fact the libspatialite library should have been statically linked with libsqlite so all these entry points would be include in the library.

Not sure how to fix these.

[mj10777]

libdl is a common requirement for most libraries that require Dynamic-Library support

- sqlite3 also requires this

https://en.wikipedia.org/wiki/Dynamic_loading

Your above solution looks correct

- but I would have assumed that mingw would of have offered a standard solution for this

Issue 2: compiling librasterlite

cd librasterlite-1.1g/
./configure
make
/bin/sh ../libtool --tag=CC   --mode=link gcc  -g -O2 -version-info 2:0:0 -no-undefined  -o librasterlite.la -rpath /mingw64/lib rasterlite_io.lo rasterlite_image.lo rasterlite_aux.lo rasterlite_quantize.lo rasterlite_gif.lo rasterlite_png.lo rasterlite_jpeg.lo rasterlite_tiff.lo rasterlite_version.lo rasterlite.lo -L/mingw64/lib -lspatialite -liconv -lz -lm -L/mingw64/lib -lpng16 -lz -lgeotiff -ltiff -ljpeg -lspatialite -lproj -lgeotiff -lproj -ltiff -ljpeg -lz

*** Warning: This system can not link to static lib archive /mingw64/lib/libspatialite.la.
*** I have the capability to make that library automatically link in when
*** you link to this library.  But I can only do this if you have a
*** shared version of the library, which you do not appear to have.

libtool: link: rm -fr  .libs/librasterlite.dll.a
libtool: link: gcc -shared  .libs/rasterlite_io.o .libs/rasterlite_image.o .libs/rasterlite_aux.o .libs/rasterlite_quantize.o .libs/rasterlite_gif.o .libs/rasterlite_png.o .libs/rasterlite_jpeg.o .libs/rasterlite_tiff.o .libs/rasterlite_version.o .libs/rasterlite.o   -L/mingw64/lib -liconv -lpng16 -L=C:/building/msys64/mingw64/lib -lgeotiff -lproj -ltiff -ljpeg -lz  -O2   -o .libs/librasterlite-2.dll -Wl,--enable-auto-image-base -Xlinker --out-implib -Xlinker .libs/librasterlite.dll.a
.libs/rasterlite.o: In function `fetch_resolutions':
C:\Users\woodbri\projects\sources\librasterlite-1.1g\lib/rasterlite.c:180: undefined reference to `sqlite3_prepare_v2'
C:\Users\woodbri\projects\sources\librasterlite-1.1g\lib/rasterlite.c:188: undefined reference to `sqlite3_reset'
<lots more similar errors to these>

I'm not sure how to resolve this. I presume that this is probably related to the libdl issue and the fact the libspatialite library should have been statically linked with libsqlite so all these entry points would be include in the library.

I am not sure on this, but I believe building as a static library is no longer not the main objective

- it was on very old versions of spatialite (v 2.*), where spatialite often included that latest sqlite3 source in the project itself

-- but is no longer so as implied by the message:

But I can only do this if you have a *** shared version of the library, which you do not appear to have.

Rasterlite1 (as apposed to RasterLite2, which is still in development), which you are using, was developed for spatialite 2,3 up to 4.1

- but has been deprecated since  2014-06-07 

Adding a '-lsqlite3' may help here

- where it would then (using libdl) dynamically load sqlite3

I assume you have worked through the instruction found at: 

http://www.gaia-gis.it/gaia-sins/mingw64_how_to.html

which should also help in getting RasterLite2 compiled.

Mark

[Stephen Woodbridge]

Mark,

Yes, I have read mingw64_how_to.html do and been reading lots of other
docs on the site. There is a lot to absorb! Anyway, I missed the fact I
needed to use librasterlite2, in part because it was not linked to the
main page, or I just missed it. It compiles and installs ok - yeah!

One of the things I'm trying to do is build based on install all of the
dependencies that I can via pacman. I'm not sure if this is a good or
bad thing to do long term. Since my goal is to build a standalone
tool(s) that is easy to deploy, when I'm done I would like to be able to
package up just the exe and dll's need and not have to have users
install msys or mingw at all.

I'm still working my way through things. I normally work on Linux so
msys is kind of new to me. I used cygwin years ago, and have ported code
to msys environment that someone set up in the past, but never had to
set it up myself and figure out how to deploy it.

Thanks again for your quick response and help.

-Steve

> <http://librasterlite.la> -rpath /mingw64/lib rasterlite_io.lo

> rasterlite_image.lo rasterlite_aux.lo rasterlite_quantize.lo
> rasterlite_gif.lo rasterlite_png.lo rasterlite_jpeg.lo
> rasterlite_tiff.lo rasterlite_version.lo rasterlite.lo
> -L/mingw64/lib -lspatialite -liconv -lz -lm -L/mingw64/lib -lpng16
> -lz -lgeotiff -ltiff -ljpeg -lspatialite -lproj -lgeotiff -lproj
> -ltiff -ljpeg -lz
>
> *** Warning: This system can not link to static lib archive

> /mingw64/lib/libspatialite.la <http://libspatialite.la>.

> library,which you donotappear to have.

> |
>
> Rasterlite1 (as apposed to RasterLite2, which is still in
> development), which you are using, was developed for spatialite 2,3 up
> to 4.1
> - but has been deprecated since 2014-06-07
>
> Adding a '-lsqlite3' may help here
> - where it would then (using libdl) dynamically load sqlite3
>
> I assume you have worked through the instruction found at:
> http://www.gaia-gis.it/gaia-sins/mingw64_how_to.html
>
> which should also help in getting RasterLite2 compiled.
>
> Mark
>
>
>
> Not sure how to fix these.
>

> --
> You received this message because you are subscribed to a topic in the
> Google Groups "SpatiaLite Users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/spatialite-users/vWDE2fTWy6o/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> spatialite-use...@googlegroups.com
> <mailto:spatialite-use...@googlegroups.com>.
> To post to this group, send email to spatiali...@googlegroups.com
> <mailto:spatiali...@googlegroups.com>.
> Visit this group at https://groups.google.com/group/spatialite-users.
> For more options, visit https://groups.google.com/d/optout.



---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

[mj10777]

On Thursday, 28 July 2016 06:29:33 UTC+2, Stephen Woodbridge wrote:

Mark,

Yes, I have read mingw64_how_to.html do and been reading lots of other
docs on the site. There is a lot to absorb! Anyway, I missed the fact I
needed to use librasterlite2, in part because it was not linked to the
main page, or I just missed it. It compiles and installs ok - yeah!

One of the things I'm trying to do is build based on install all of the
dependencies that I can via pacman. I'm not sure if this is a good or
bad thing to do long term.

That will be difficult

- since you are using linux, try:

sudo ldd /usr/local/rl2sniff

That will return a list of libraries that are loaded when it runs

- 71 (among them 'libdl.so.2') [for spatialite: 21]

This will probably be close to the list of things you will need to deploy ...

Mark

> <mailto:spatialite-users+unsub...@googlegroups.com>.

[a.fu...@lqt.it]

On Thu, 28 Jul 2016 00:29:29 -0400, Stephen Woodbridge wrote:
> Mark,
>
> Yes, I have read mingw64_how_to.html do and been reading lots of
> other docs on the site. There is a lot to absorb! Anyway, I missed
> the
> fact I needed to use librasterlite2, in part because it was not
> linked
> to the main page, or I just missed it. It compiles and installs ok -
> yeah!
>
> One of the things I'm trying to do is build based on install all of
> the dependencies that I can via pacman. I'm not sure if this is a
> good
> or bad thing to do long term. Since my goal is to build a standalone
> tool(s) that is easy to deploy, when I'm done I would like to be able
> to package up just the exe and dll's need and not have to have users
> install msys or mingw at all.
>
> I'm still working my way through things. I normally work on Linux so
> msys is kind of new to me. I used cygwin years ago, and have ported
> code to msys environment that someone set up in the past, but never
> had to set it up myself and figure out how to deploy it.
>

Hi Stephen,

all ./configure scripts are usually intended for Linux or for
other *nix derivatives, as e.g. Mac OS X.

Windows is a completely unrelated platform presenting many strong
differences: just to say, it's not fully POSIX compliant.
so if you wish to build your own binaries on Windows using the
canonical ./configure approach you surely need to use some special
tool acting as a "bridge" between *nix and Windows:

- cygwin is a complete abstraction layer implementing full
POSIX compatibility and supporting all *nix standard
commands (ls, vi, make, gcc and so on).
its main disadvantage is that any binary built on cygwin
will then require full cygwin support at run time, so you
necessarily have to install the whole cygwin on any PC
where you intend to deploy your own software.
unhappily cygwin is rather complex and heavy, so this
one is not a viable solution.

- MinGW / MSYS are a simpler and more effective cygwin
derivative; they don't rely on full POSIX, they rely
instead on native Windows API whenever is possible.
so any binary built using MinGW / MSYS can directly
run on the standard Windows run-time and will not
require installing any other "intermediate" component
as in the case of cygwin.

short conclusion: building on cygwin exactly is the
same as building on any other *nix.
but building on MinGW+MSYS still continue to be
building on an "exotic" non-POSIX platform presenting
many possible pitfalls and hidden traps.

just to enumerate the most obvious:
- in many cases POSIX APIs are not supported on Windows,
or are supported but in a slightly different way
(different names, different arguments and so on)
- in other cases Windows has its own native APIs that
are completely different from POSIX: this is the case
e.g. of WinSockets, of the Threading API and so on.
just to say, your problems about libldl are quickly
explained once you know that Windows has its own
native APIs for dynamically loading DLLs, so there
is no need at all to use -ldl on Windows.
- last but not least: on Linux the most obvious
option is building your own packages using as
far as possible dynamic linkage.
at the opposite on Windows a very common option
is building self-contained packages fully based
on static linkage, so to avoid the infamous
"DLL hell".

as a general rule, you can never expect to build on
MinGW / MSYS in the same identical way you'll usually
follow on Linux, because very often specifying some
fancy ./configure setting will be strictly required.

SpatiaLite actively supports a fully detailed and
frequently updated how-to guide about building Windows
binaries using MinGW/MSYS, in both 32 and 64 versions:
http://www.gaia-gis.it/gaia-sins/mingw_how_to.html
http://www.gaia-gis.it/gaia-sins/mingw64_how_to.html

anyway all MinGW/MSYS recipes are expected to successfully
work only if you verbatim follow the instructions from
the very beginning to the end.

I notice that you are currently using many pre-built
packages distributed by pacman.
I don't think this could be a good idea, because this
way you'll lost full control about the build workflow
as a whole including the most fine grained details.
and it could possibly contribute to inadvertently do
some trivial mistake, as the following one:

> cd libspatialite-4.3.0a
> ./configure --target=mingw64
> make

> ld.exe: cannot find -ldl
>

if you carefully check the how-to guide about MinGW-64
you'll discover that the expected argument is
"--target=mingw32", even when using the 64 bit compiler;
this will effectively trigger the Windows native API
support, and will discard any reference to -ldl
"--target=mingw64" simply is an unrecognized option,
and will have absolutely no action.

bye Sandro