Problems linking to different versions of an ABI (ICU)

[Luc Sarzyniec]

I'm writing a wrapper to the ICU library and I have an issue to link my Crystal lib to different versions of the shared objects (as many wrapper does since it's hard to have the same version of a lib on every OS/distro).

The main issue: in many distributions of the library, a suffix containing the version number is added to the symbols making different versions of the ABI 100% incompatible.

I tried to sum it up and proposed some solutions in this issue.

Does anyone have an idea on how to proceed / can anyone help me on this ?

[Kostya M]

+1. I also want binding to libicu and have the same problem with method versions.

In ruby ffi binding used dlsym to find correct method names, and then generate ffi binding related on this names.

I think crystal can do something similar, when install shard, it can find .so file, and using dlsym find methods names, then generate binding.

about distributing self icu lib, i think this is bad, because it quite big.

--
You received this message because you are subscribed to the Google Groups "Crystal" group.
To unsubscribe from this group and stop receiving emails from it, send an email to crystal-lang+unsubscribe@googlegroups.com.
To post to this group, send email to crysta...@googlegroups.com.
Visit this group at https://groups.google.com/group/crystal-lang.
To view this discussion on the web visit https://groups.google.com/d/msgid/crystal-lang/0fd1c62b-c955-4938-8d30-31126ac9ad92%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Kostya M]

https://github.com/fantasticfears/ffi-icu/blob/master/lib/ffi-icu/lib.rb#L84

[Kostya M]

Another way, crystal should support macro in lib definitions, 

so we can write crystal script which detect prefixes by dlsym, which executed with macro `run` at compile time, and generate prefixes for functions.

[Kostya M]

btw this is already works:

generate_prefix.cr

```crystal

puts 1

```

main.cr

```crystal

{% begin %}

{% prefix = run("./generate_prefix") %}

lib Bla

  fun bla = bla_{{prefix}}()

end

{% end%}

Bla.bla

```

so need only write generate_prefix.cr to detect correct version of methods

[Chris Hobbs]

I would suggest either writing stable bindings in C and binding those, or using macros if the C macro magic is sufficiently simple. If the names are suffixed with a version, doesn’t that mean that each version will be incompatible? (and thus you need different bindings for each case)

To unsubscribe from this group and stop receiving emails from it, send an email to crystal-lang...@googlegroups.com.

To post to this group, send email to crysta...@googlegroups.com.
Visit this group at https://groups.google.com/group/crystal-lang.

To view this discussion on the web visit https://groups.google.com/d/msgid/crystal-lang/CACAwW2jL7WR4DrL2EFXowdniEfuh1%2B%3Drn1my8Knoo52q7vfwEA%40mail.gmail.com.

​

[Luc Sarzyniec]

Wonderful ! I forgot about the run() keyword, I think something like this can definitely solve the issue.

I think it will be possible to play around with dlsym() to look for the right writing of the suffix as it's done in the ffi-icu binding you linked before.

The last thing I can't figure out: how to get the lib's version ... If pkg-config is installed it can be done using pkg-config --modversion but if it's not, I don't know.

Thank you very much for your help ! I'll try to come with something asap.

https://github.com/fantasticfears/ffi-icu/blob/master/lib/ffi-icu/lib.rb#L84

To unsubscribe from this group and stop receiving emails from it, send an email to crystal-lang...@googlegroups.com.

[Luc Sarzyniec]

I think that writing a C binding to wrap the thing up can work but wouldn't it have performances cost ? (+ there is more than 2k functions to wrap).

I don't think that different versions are incompatible, there may be some small changes in the API between the versions but you have to face this issue every time you write a "generic" binding.

The suffix is added by an option at compile time and this is not really visible when you are using the C lib since there is also magic macros to hide that ...

Still I didn't figure out yet why most distributions of the lib are compiled using this option (see 1, 2, 3).

On Thursday, April 27, 2017 at 7:44:57 PM UTC+2, RX14 wrote:

I would suggest either writing stable bindings in C and binding those, or using macros if the C macro magic is sufficiently simple. If the names are suffixed with a version, doesn’t that mean that each version will be incompatible? (and thus you need different bindings for each case)

On 27/04/17 14:27, Kostya M wrote:

+1. I also want binding to libicu and have the same problem with method versions.

In ruby ffi binding used dlsym to find correct method names, and then generate ffi binding related on this names.

I think crystal can do something similar, when install shard, it can find .so file, and using dlsym find methods names, then generate binding.

about distributing self icu lib, i think this is bad, because it quite big.

2017-04-26 17:17 GMT+03:00 Luc Sarzyniec <de...@olbat.net>:

I'm writing a wrapper to the ICU library and I have an issue to link my Crystal lib to different versions of the shared objects (as many wrapper does since it's hard to have the same version of a lib on every OS/distro).

The main issue: in many distributions of the library, a suffix containing the version number is added to the symbols making different versions of the ABI 100% incompatible.

I tried to sum it up and proposed some solutions in this issue.

Does anyone have an idea on how to proceed / can anyone help me on this ?

--
You received this message because you are subscribed to the Google Groups "Crystal" group.

To unsubscribe from this group and stop receiving emails from it, send an email to crystal-lang...@googlegroups.com.
To post to this group, send email to crysta...@googlegroups.com.
Visit this group at https://groups.google.com/group/crystal-lang.
To view this discussion on the web visit https://groups.google.com/d/msgid/crystal-lang/0fd1c62b-c955-4938-8d30-31126ac9ad92%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

​

[Kostya M]

icu-ffi find .so library in the paths for systems, and seems extract version from filename

To unsubscribe from this group and stop receiving emails from it, send an email to crystal-lang+unsubscribe@googlegroups.com.

To post to this group, send email to crysta...@googlegroups.com.
Visit this group at https://groups.google.com/group/crystal-lang.

To view this discussion on the web visit https://groups.google.com/d/msgid/crystal-lang/39089a8e-fa25-4d7c-b6ab-b1b5b760f481%40googlegroups.com.

[Luc Sarzyniec]

Okay, but, erm ... I'm not sure I want to do something like this :)

Here is what I've done so far: https://gist.github.com/olbat/4a5d6d474efd54e14fbac317b3ae8819 .

To find the version out, I first try to use the "icuinfo" tool that's packaged with the lib, then I try with pkg-config, do you have other ideas ?

(Any advice on the code are welcome, I'm a Crystal newbie)

[Kostya M]

i think using xml parser in run script is too heavy, it can be done by  split or grep. 

also this is not worked on osx, icuinfo command not exist, pkgconfig no return.

on ubuntu 12.04 it returns:

icuinfo -v

<ICUINFO>

International Components for Unicode for C/C++

 Copyright (C) 2011, International Business Machines Corporation and others. All Rights Reserved.

Compiled-Version: 4.8.1.1

Runtime-Version: 4.8.1.1

Compiled-Unicode-Version: 6.0

Runtime-Unicode-Version: 6.0

Platform: U_LINUX

Build: x86_64-pc-linux-gnu

C compiler: gcc

C++ compiler: g++

ICUDATA: icudt48l

Data Directory:

ICU Initialization returned: U_ZERO_ERROR

Default locale: ru_RU

CLDR-Version: 2.0.1

Default converter: UTF-8

Default TZ: Europe/Moscow

TZ data version: 2011k

Plugin file is: /usr/lib/icu/icuplugins48.txt

</ICUINFO>

but, ffi-icu nice install on ubuntu and OSX, so i think need to do the same as it does.

To unsubscribe from this group and stop receiving emails from it, send an email to crystal-lang+unsubscribe@googlegroups.com.

To post to this group, send email to crysta...@googlegroups.com.
Visit this group at https://groups.google.com/group/crystal-lang.

To view this discussion on the web visit https://groups.google.com/d/msgid/crystal-lang/2d81705a-3a98-4d78-ba68-b2439590d2dd%40googlegroups.com.

[Luc Sarzyniec]

On Friday, April 28, 2017 at 11:18:53 AM UTC+2, Kostya M wrote:

i think using xml parser in run script is too heavy, it can be done by  split or grep.

It's maybe a little overkill yes but since it's run at compile time, do we really care ?

Btw since old versions (< 50 ?) of icuinfo does not print the info in XML, I think I'll parse the string as you suggested.

 also this is not worked on osx, icuinfo command not exist, pkgconfig no return.

Sad, I should have tried on OSX, thank you for testing :)

(do you /does someone know an OSX VM image including development tools ?)

 

but, ffi-icu nice install on ubuntu and OSX, so i think need to do the same as it does.

Okay, I'm not really fan of their way to find out the version of ICU so I'll see if I can find another way to get it ...

Still, I'm curious about the way you'll implement it, can you share once you'll get something working ?

Another time, thank you for your help !

[Kostya M]

https://gist.github.com/kostya/cf691c9ac0a3dc4056102091f71f7f4e

here i just implement what ffi-icu does:

osx:

files - ["/usr/lib/libicucore.dylib"]

suffix - ""

ubuntu 12.04:

files - ["/usr/lib/libicui18n.so.48", "/usr/lib/libicutu.so.48"]

suffix - "_48"

ubuntu 16.04:

files - ["/usr/lib/x86_64-linux-gnu/libicui18n.so.55", "/usr/lib/x86_64-linux-gnu/libicutu.so.55"]

suffix - "_55"

To unsubscribe from this group and stop receiving emails from it, send an email to crystal-lang+unsubscribe@googlegroups.com.

To post to this group, send email to crysta...@googlegroups.com.
Visit this group at https://groups.google.com/group/crystal-lang.

To view this discussion on the web visit https://groups.google.com/d/msgid/crystal-lang/7a1c23c5-eb39-43a6-9135-01ed04f3a209%40googlegroups.com.

[Luc Sarzyniec]

Ty for your code/help ! I'll try to find some way to improve my piece of code and post something here if I can, use your's if I cannot :)

I was thinking, do you plan to create a generic ICU binding or you just wrote a something to be used in another project ?

If you want to create a generic binding, we should maybe concentrate our efforts ? (the lib is huuuuge)

[Kostya M]

i not going to write full binding, i just write for BreakIterator which i needed: https://github.com/kostya/icu

To unsubscribe from this group and stop receiving emails from it, send an email to crystal-lang+unsubscribe@googlegroups.com.

To post to this group, send email to crysta...@googlegroups.com.
Visit this group at https://groups.google.com/group/crystal-lang.

To view this discussion on the web visit https://groups.google.com/d/msgid/crystal-lang/c58cb003-9406-4ac2-bbe2-bab09cd141d0%40googlegroups.com.

[Luc Sarzyniec]

Okay !

I've finished working on my script, I updated the gist if needed.

If you're interested, the full implementation of the fix (usage of the script to get the lib's suffix from run() in a macro, generation of macros for suffixes, selection of the code do enable/disable depending on the version of the lib, etc...) is available in this PR.

The last issue to solve: understand why the binding does not work on macOS but it's not really related to this discussion ...

Again, thanks everybody :-)