unixODBC and ClickHouse ODBC driver in Fedora25

[kvra...@gmail.com]

Hi guys,

Do anybody try clickhouse-odbc driver in Linux (Fedora 25 is intering more for me) ?

I'm trying to build clickhouse-odbc driver on Fedora25. For some historical reasons we use RedHat distro. [ClickHouse server works as docker image in Ubuntu]

ODBC driver is important for me because I use R Studio Connect for data analysis. R Studio Connect should work on both Fedora/Centos and Windows.

So I'm working on clickhouse-odbc building on Fedora.

First of all project was not compiled on F25. It didn't see odbc library.

it was because link.txt file was :

'

/bin/c++ -fPIC -std=c++14 -O2 -g -fvisibility-inlines-hidden -O2 -g -DNDEBUG  -Wl,-Bstatic,--whole-archive -static-libgcc -static-libstdc++ -Wl,--no-whole-archive -shared -Wl,-soname,clickhouse-odbc.so -o clickhouse-odbc.so CMakeFiles/clickhouse-odbc.dir/attr.cpp.o CMakeFiles/clickhouse-odbc.dir/config.cpp.o CMakeFiles/clickhouse-odbc.dir/connection.cpp.o CMakeFiles/clickhouse-odbc.dir/diagnostics.cpp.o CMakeFiles/clickhouse-odbc.dir/handles.cpp.o CMakeFiles/clickhouse-odbc.dir/info.cpp.o CMakeFiles/clickhouse-odbc.dir/odbc.cpp.o CMakeFiles/clickhouse-odbc.dir/result_set.cpp.o CMakeFiles/clickhouse-odbc.dir/statement.cpp.o CMakeFiles/clickhouse-odbc.dir/type_parser.cpp.o ../lib/libPocoNet.a ../lib/libPocoFoundation.a -lodbc

'

Then I moved -lodbc before static libraries:

'

/bin/c++ -fPIC -std=c++14 -lodbc -O2 -g -fvisibility-inlines-hidden -O2 -g -DNDEBUG  -Wl,-Bstatic,--whole-archive -static-libgcc -static-libstdc++ -Wl,--no-whole-archive -shared -Wl,-soname,clickhouse-odbc.so -o clickhouse-odbc.so CMakeFiles/clickhouse-odbc.dir/attr.cpp.o CMakeFiles/clickhouse-odbc.dir/config.cpp.o CMakeFiles/clickhouse-odbc.dir/connection.cpp.o CMakeFiles/clickhouse-odbc.dir/diagnostics.cpp.o CMakeFiles/clickhouse-odbc.dir/handles.cpp.o CMakeFiles/clickhouse-odbc.dir/info.cpp.o CMakeFiles/clickhouse-odbc.dir/odbc.cpp.o CMakeFiles/clickhouse-odbc.dir/result_set.cpp.o CMakeFiles/clickhouse-odbc.dir/statement.cpp.o CMakeFiles/clickhouse-odbc.dir/type_parser.cpp.o ../lib/libPocoNet.a ../lib/libPocoFoundation.a

'

And 'clickhouse-odbc.so' file was created.

According with recomendation in github repo I created file .odbc.ini

'

[ClickHouse]

Driver = /home/kirill/Projects/clickhouse-odbc/build/driver/clickhouse-odbc.so

Description = ClickHouse driver

DATABASE = default

SERVER = 172.17.0.2

PORT = 8123

FRAMED = 0

'

And run:

$ isql -v ClickHouse

But isql fails .... :(

Running this command under 'strace' (strace -o strace.log isql -v ClickHouse) I see:

..... 

execve("/bin/isql", ["isql", "-v", "ClickHouse"], 0x7ffc75da6b20 /* 44 vars */) = 0

brk(NULL)                               = 0x5560f759a000

mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7faf36d07000

access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)

open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3

fstat(3, {st_mode=S_IFREG|0644, st_size=137368, ...}) = 0

mmap(NULL, 137368, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7faf36ce5000

close(3)                                = 0

open("/lib64/libodbc.so.2", O_RDONLY|O_CLOEXEC) = 3

read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0 i\0\0\0\0\0\0"..., 832) = 832

fstat(3, {st_mode=S_IFREG|0755, st_size=439728, ...}) = 0

-----

And here I see /lib64/libodbc.so.2 loaded properly

-----

open("/tmp/clickhouse-odbc-stderr", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3

dup3(3, 2, 0)                           = 2

close(3)                                = 0

writev(2, [{iov_base="isql", iov_len=4}, {iov_base=": ", iov_len=2}, {iov_base="symbol lookup error", iov_len=19}, {iov_base=": ", iov_len=2}, {iov_base="/home/kirill/Projects/clickhouse"..., iov_len=69}, {iov_base=": ", iov_len=2}, {iov_base="undefined symbol: SQLGetPrivateP"..., iov_len=44}, {iov_base="", iov_len=0}, {iov_base="", iov_len=0}, {iov_base="\n", iov_len=1}], 10) = 143

exit_group(127)                         = ?

+++ exited with 127 +++

But here I see some file is created in /tmp and isql stops with issue.

......

File /tmp/clickhouse-odbc-stderr contains string:

isql: symbol lookup error: /home/kirill/Projects/clickhouse-odbc/build/driver/clickhouse-odbc.so: undefined symbol: SQLGetPrivateProfileString

Then I see symbols in clickhouse-odbc.so and in libodbc.so.2.0.0:

$ strings clickhouse-odbc.so | grep SQLGetPrivateProfileString

SQLGetPrivateProfileString

SQLGetPrivateProfileString

SQLGetPrivateProfileString

And 

$ strings /lib64/libodbc.so.2.0.0 | grep SQLGetPrivateProfileString

SQLGetPrivateProfileString.c

Then I see ld dependencies of clickhouse-odbc.so:

$ ldd clickhouse-odbc.so 

linux-vdso.so.1 (0x00007fff7fcc8000)

libodbc.so.2 => /lib64/libodbc.so.2 (0x00007fbf890ca000)

libm.so.6 => /lib64/libm.so.6 (0x00007fbf88dc1000)

libc.so.6 => /lib64/libc.so.6 (0x00007fbf889fb000)

/lib64/ld-linux-x86-64.so.2 (0x000055e386852000)

libltdl.so.7 => /lib64/libltdl.so.7 (0x00007fbf887f1000)

libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fbf885d3000)

libdl.so.2 => /lib64/libdl.so.2 (0x00007fbf883cd000)

Just to be sure I see files formats:

For libodbc:

$ file /lib64/libodbc.so.2

/lib64/libodbc.so.2: symbolic link to libodbc.so.2.0.0

$ file /lib64/libodbc.so.2.0.0 

/lib64/libodbc.so.2.0.0: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=22d5e0e71157f6ccb55af14ecf06bd3598849ae6, stripped

[kirill@g710 driver]$ 

For clickhouse-odbc:

$ file clickhouse-odbc.so 

clickhouse-odbc.so: ELF 64-bit LSB shared object, x86-64, version 1 (GNU/Linux), dynamically linked, BuildID[sha1]=e98c5602241d5a3f643e41579d8b76d47bacacec, not stripped

.... have no idea any more why it happens ... 

I'd appreciate any sugession why ld doesn't see function ...

[kvra...@gmail.com]

It works on Fedora 25!

But there are some tips.

1) Package unixODBC has to be build as static.

2) Change FindODBC and add path to your own built odbc before standard paths (better to set it first).

3) File link.txt is generated automatically but its single line ends at <path-to-odbc>/libodbc.so. You have to change extention 'so' to 'a'

( my particular link.txt is :

/bin/c++ -fPIC  -std=c++14 -O2 -g -fvisibility-inlines-hidden -O2 -g -DNDEBUG  -Wl,-Bstatic,--whole-archive -static-libgcc -static-libstdc++ -Wl,--no-whole-archive -shared -Wl,-soname,clickhouse-odbc.so -o clickhouse-odbc.so CMakeFiles/clickhouse-odbc.dir/attr.cpp.o CMakeFiles/clickhouse-odbc.dir/config.cpp.o CMakeFiles/clickhouse-odbc.dir/connection.cpp.o CMakeFiles/clickhouse-odbc.dir/diagnostics.cpp.o CMakeFiles/clickhouse-odbc.dir/handles.cpp.o CMakeFiles/clickhouse-odbc.dir/info.cpp.o CMakeFiles/clickhouse-odbc.dir/odbc.cpp.o CMakeFiles/clickhouse-odbc.dir/result_set.cpp.o CMakeFiles/clickhouse-odbc.dir/statement.cpp.o CMakeFiles/clickhouse-odbc.dir/type_parser.cpp.o -Wl,-rpath,/usr/local/lib ../lib/libPocoNet.a ../lib/libPocoFoundation.a /usr/local/lib/libodbc.a

)

I don't think it's production way but it works.

P.S. Thanks everyone who at least tried to read and understand my issue :).

понедельник, 19 июня 2017 г., 18:34:37 UTC+3 пользователь kvra...@gmail.com написал: