How to set Tcl's library path?

[Georgios Petasis]

Hi all,

I am trying to switch from ActiveTcl to another distribution (I chose
BAWT). For some reason (probably a patch?) ActiveTcl was getting its
library correctly, no matter where it was installed.

Plain Tcl seems that it cannot do that. By looking at libtcl.so, it
seems that the installation paths are embedded, and Tcl uses these to
define where the library is located and how auto_path is defined.
Looking at the sources, showed very little options. Either a few env
variables (like TCL_LIBRARY & TCLLIBPATH, both must be defined despite
the manual says TCL_LIBRARY should not be defined).

But the problem is what happens if you cannot define these variables.
Like you are using Apache Rivet, and everything gets started by the
operating system, and these variables are not defined.

Looking at the activestate's shared library, I get the impression that
they have used for this kind of stuff a big string, and then somehow
"patch" the binary with the real path string...

Is there a solution other than trying to hack my operating system to
define these env variables so that they are visible when the services start?

George

[Gerald Lester]

George,

You don't have to define them if and only if you put Tcl in the place it
was built to expect them.

Yes, AFAK the AS install patched the binary.

As to how to get them defined for a service -- what OS are you running?

--
+----------------------------------------------------------------------+
| Gerald W. Lester, President, KNG Consulting LLC |
| Email: Gerald...@kng-consulting.net |
+----------------------------------------------------------------------+

[Georgios Petasis]

Fedora 25.

The command to set them is:

systemctl edit <service name>

It creates an override file in /etc/systemd/system/service.d/, where you
can add things like:

[Service]
Environment="TCLLIBPATH=/opt/BAWT/Tcl/lib"
Environment="TCL_LIBRARY=/opt/BAWT/Tcl/lib/tcl8.7"

But it shouldn't be so difficult :-)

(Adding them to /etc/environment does not work).

Regards,

George

[Rich]

Georgios Petasis <peta...@yahoo.gr> wrote:
> ???? 19/9/2017 16:47, ? Gerald Lester ??????:

>> As to how to get them defined for a service -- what OS are you running?
>
> Fedora 25.

Does Fedora 25 not ship with an already installed Tcl?

[Georgios Petasis]

Yes, 8.6.4, but not with the packages I need. I always prefer to install
a newer, batteries included version in /opt.

George

[Rich]

Georgios Petasis <peta...@yahoo.gr> wrote:
> ???? 19/9/2017 18:27, ? Rich ??????:

>> Georgios Petasis <peta...@yahoo.gr> wrote:
>>> ???? 19/9/2017 16:47, ? Gerald Lester ??????:
>>>> As to how to get them defined for a service -- what OS are you running?
>>>
>>> Fedora 25.
>>
>> Does Fedora 25 not ship with an already installed Tcl?
>>
> Yes, 8.6.4, but not with the packages I need.

Can you not just compile (assuming they need to be compiled) the
packages you need against the system installed Tcl?

> I always prefer to install a newer, batteries included version in
> /opt.

Well, if by "batteries included" you also mean "pre-compiled binaries"
then there lies part of the trouble. The ELF so format stores paths to
where the so files are supposed to be stored, and putting them
somewhere else creates no amount of trouble in working around those
hard coded paths.

If you want a new Tcl, plus packages, have you tried beginning by
simply compiling Tcl by telling configure it is in opt/???? and then
installing it into /opt/????. That should set all the ld paths so Tcl
and Tk know that their .so's are in /opt/????.

[Georgios Petasis]

Στις 19/9/2017 19:09, ο Rich έγραψε:
> Georgios Petasis <peta...@yahoo.gr> wrote:
>> ???? 19/9/2017 18:27, ? Rich ??????:
>>> Georgios Petasis <peta...@yahoo.gr> wrote:
>>>> ???? 19/9/2017 16:47, ? Gerald Lester ??????:
>>>>> As to how to get them defined for a service -- what OS are you running?
>>>>
>>>> Fedora 25.
>>>
>>> Does Fedora 25 not ship with an already installed Tcl?
>>>
>> Yes, 8.6.4, but not with the packages I need.
>
> Can you not just compile (assuming they need to be compiled) the
> packages you need against the system installed Tcl?

No, as this will write things in places that are managed with operating
system packages.

>
>> I always prefer to install a newer, batteries included version in
>> /opt.
>
> Well, if by "batteries included" you also mean "pre-compiled binaries"
> then there lies part of the trouble. The ELF so format stores paths to
> where the so files are supposed to be stored, and putting them
> somewhere else creates no amount of trouble in working around those
> hard coded paths.

This is not the case with tcl (and not the problem I am facing).
Setting LD_LIBRARY_PATH can solve there rpath issues.

The problem is finding the tcl library, as in tcl init.tcl.
Not finding libtcl8.7.so.

>
> If you want a new Tcl, plus packages, have you tried beginning by
> simply compiling Tcl by telling configure it is in opt/???? and then
> installing it into /opt/????. That should set all the ld paths so Tcl
> and Tk know that their .so's are in /opt/????.
>

The discussion is about the tcl scripts of the tcl library. Where
packages are loaded. Which is stored in static C variables inside
libtcl8.7.so.

Compiling for /opt will work, but I don't want to do this. I want to
build in another location, test, and then move to /opt.

George

[Gerald Lester]

On 09/19/2017 11:41 AM, Georgios Petasis wrote:
>...

>>
>> If you want a new Tcl, plus packages, have you tried beginning by
>> simply compiling Tcl by telling configure it is in opt/????  and then
>> installing it into /opt/????.  That should set all the ld paths so Tcl
>> and Tk know that their .so's are in /opt/????.
>>
>
> The discussion is about the tcl scripts of the tcl library. Where
> packages are loaded. Which is stored in static C variables inside
> libtcl8.7.so.
>
> Compiling for /opt will work, but I don't want to do this. I want to
> build in another location, test, and then move to /opt.

Compile/build for the libraries to be in some long name.

Then patch the binaries with where you want them (i.e. do the same thing
as AS did).

[Christian Gollwitzer]

Am 19.09.17 um 21:51 schrieb Gerald Lester:

> On 09/19/2017 11:41 AM, Georgios Petasis wrote:
>> ...
>>>
>>> If you want a new Tcl, plus packages, have you tried beginning by
>>> simply compiling Tcl by telling configure it is in opt/????  and then
>>> installing it into /opt/????.  That should set all the ld paths so Tcl
>>> and Tk know that their .so's are in /opt/????.
>>>
>>
>> The discussion is about the tcl scripts of the tcl library. Where
>> packages are loaded. Which is stored in static C variables inside
>> libtcl8.7.so.
>>
>> Compiling for /opt will work, but I don't want to do this. I want to
>> build in another location, test, and then move to /opt.
>
> Compile/build for the libraries to be in some long name.
>
> Then patch the binaries with where you want them (i.e. do the same thing
> as AS did).

I am surprised that it is actually THAT archaic. Patching the binary
sounds like a very crude hack. I was always thinking that
TclFindExecutable is there to find the library relative to the tclsh
executable, regardless of the path. OTOH I am usually using a tclkit,
which does not have this problem.

Christian

[Christian Gollwitzer]

Am 20.09.17 um 08:03 schrieb Christian Gollwitzer:

Looking at Tcl_Init in tclInterp.c, it indeed tries to derive a suitable
location from the path of the executable. There is an embedded script:

" lappend scripts {\n"
"set parentDir [file dirname [file dirname [info nameofexecutable]]]\n"
"set grandParentDir [file dirname $parentDir]\n"
"file join $parentDir lib tcl[info tclversion]} \\\n"
" {file join $grandParentDir lib tcl[info tclversion]} \\\n"
" {file join $parentDir library} \\\n"

.... which continues with more options

So, basically, if there is a suitable init.tcl in the vicinity of the
executable, it should be found. This does not work for embedding Tcl,
because the dir is relative to libtcl*.so, which might be in a different
place than the exe. There the doc tells the embedding app devleoper to
propose a path via the tcl_library variable.

Christian

[Georgios Petasis]

No matter what Tcl_Init does, it does not really work. Because the
location of the library is *hardcoded* in libtcl.so. And the code is
clear: If TCL_LIBRARY is not defined, the hardcoded value is used.

Which ends "info library" to return a false position, unless you define
TCL_LIBRARY. It is very easy to see that it does not work:

[petasis@server ~]$ which tclsh
/opt/BAWT/Tcl/bin/tclsh
[petasis@server ~]$ tclsh
% info library
/opt/BAWT/Tcl/lib/tcl8.7
% [petasis@server ~]$ unset TCL_LIBRARY
[petasis@server ~]$ tclsh
% info library
/home/tcl/bawt/BawtBuild/Linux/x64/Release/Install/Tcl/lib/tcl8.7
%

Patching the libtcl.so (as ActiveState did), actually solves all
problems. And the embedding one.

I think there is no alternative, and the build scripts should allow
control over that. Not build in an fake large path, but allow through
options to modify what is placed in these static variables.

George

[Christian Gollwitzer]

Am 20.09.17 um 08:58 schrieb Georgios Petasis:

> No matter what Tcl_Init does, it does not really work. Because the
> location of the library is *hardcoded* in libtcl.so. And the code is
> clear: If TCL_LIBRARY is not defined, the hardcoded value is used.
>
> Which ends "info library" to return a false position, unless you define
> TCL_LIBRARY. It is very easy to see that it does not work:
>
> [petasis@server ~]$ which tclsh
> /opt/BAWT/Tcl/bin/tclsh
> [petasis@server ~]$ tclsh
> % info library
> /opt/BAWT/Tcl/lib/tcl8.7
> % [petasis@server ~]$ unset TCL_LIBRARY
> [petasis@server ~]$ tclsh
> % info library
> /home/tcl/bawt/BawtBuild/Linux/x64/Release/Install/Tcl/lib/tcl8.7
> %

which means, to me, that the code in Tcl_Init is broken. As I read it,
it should first look into ../lib/tcl8.7 from the path of the executable
*before* checking the hardcoded path. I suggest that is worth a ticket.

Christian