Reasons why --enable-threads shouldn't be on by default

180 views
Skip to first unread message

George Peter Staplin

unread,
Dec 10, 2007, 3:25:06 AM12/10/07
to
After much research I want to share my concerns over the ideas and
conflicting goals some of the core team has with threads, and why I am
against having --enable-threads by default.


We have the following goals/constraints:

1. The core team wants extensions that use fork() like Expect, TclX,
etc. to work in a threaded core.

2. The core team wants the ability to work with existing libraries that
don't necessarily use the Tcl pthread interfaces (based on what I
understood KBK to say).

3. QNX and possibly others, disable fork() syscalls when pthreads are
used. According to the documentation for QNX if you use fork() and have
more than 1 thread created, an ENOSYS will result. The alternative to
fork() + exec*() is spawn(). This means that QNX would be an
odd-one-out in the Tcl world, and TclX and Expect wouldn't work anyway,
unless a user built with --disable-threads.

Constraint 1 is possible with some platforms using pthread_atfork(), but
it requires keeping a record for every mutex, spinlock, etc. used by
Tcl. However, constraint 2 eliminates this possibility.

I now suggest you read what David Butenhof has to say about
pthread_atfork(). He is a recognized expert with pthreads, an author of
a pthread book, and helped form the pthread standard.
http://groups.google.com/group/comp.programming.threads/msg/3a43122820983fde

He has also written other articles about how to possibly implement the
solution to 1. But given what he has written, "threads by default" may
not be as appealing as some have thought.


I believe the following:
A) threaded builds should be specifically marked, and use a
library/executable suffix.
B) package distributors should probably shy away from threaded builds
of Tcl, unless absolutely needed.


In a somewhat related chain of thoughts, I'm concerned about the
implementation of Tcl's threading. In particular it bothers me that
thread-specific data (TSD) is stored in a bunch of hash tables, so
everytime a C function needs TSD, it has to find the hash table, do a
hash table lookup, and return the result. This is not ideal. The
reason I believe this was chosen is that many threading implementations,
such as pthreads, and Win32, have a limited, fixed number of TSD slots.
A proper fix in my opinion is to use one or 2 structures per-thread, and
restructure Tcl to keep its data in those structures. This should
result in better performance, and not be detrimental in the general case
where you have 1 thread.


What are your thoughts on all of this?


George

tom.rmadilo

unread,
Dec 10, 2007, 9:59:48 AM12/10/07
to
On Dec 10, 12:25 am, George Peter Staplin

<georgepsSPAMME...@xmission.com> wrote:
> After much research I want to share my concerns over the ideas and
> conflicting goals some of the core team has with threads, and why I am
> against having --enable-threads by default.
>
> We have the following goals/constraints:
>
> 1. The core team wants extensions that use fork() like Expect, TclX,
> etc. to work in a threaded core.
>
> 2. The core team wants the ability to work with existing libraries that
> don't necessarily use the Tcl pthread interfaces (based on what I
> understood KBK to say).
>
> 3. QNX and possibly others, disable fork() syscalls when pthreads are
> used. According to the documentation for QNX if you use fork() and have
> more than 1 thread created, an ENOSYS will result. The alternative to
> fork() + exec*() is spawn(). This means that QNX would be an
> odd-one-out in the Tcl world, and TclX and Expect wouldn't work anyway,
> unless a user built with --disable-threads.
>
> Constraint 1 is possible with some platforms using pthread_atfork(), but
> it requires keeping a record for every mutex, spinlock, etc. used by
> Tcl. However, constraint 2 eliminates this possibility.
>
> I now suggest you read what David Butenhof has to say about
> pthread_atfork(). He is a recognized expert with pthreads, an author of
> a pthread book, and helped form the pthread standard.http://groups.google.com/group/comp.programming.threads/msg/3a4312282...

>
> He has also written other articles about how to possibly implement the
> solution to 1. But given what he has written, "threads by default" may
> not be as appealing as some have thought.
>
> I believe the following:
> A) threaded builds should be specifically marked, and use a
> library/executable suffix.
> B) package distributors should probably shy away from threaded builds
> of Tcl, unless absolutely needed.
>
> In a somewhat related chain of thoughts, I'm concerned about the
> implementation of Tcl's threading. In particular it bothers me that
> thread-specific data (TSD) is stored in a bunch of hash tables, so
> everytime a C function needs TSD, it has to find the hash table, do a
> hash table lookup, and return the result. This is not ideal. The
> reason I believe this was chosen is that many threading implementations,
> such as pthreads, and Win32, have a limited, fixed number of TSD slots.
> A proper fix in my opinion is to use one or 2 structures per-thread, and
> restructure Tcl to keep its data in those structures. This should
> result in better performance, and not be detrimental in the general case
> where you have 1 thread.
>
> What are your thoughts on all of this?
>
> George

My initial thought is that if you are using threads you should know
what you are doing, and that may require building your Tcl from
scratch anyway. I never use the stock Tcl unless I just need a quick
command line to test out something. But I also have a pre-written
configure command line as a file which I just copy around when I need
to build Tcl.

But I use AOLserver pthreads. I don't know what the difference is with
Tcl threads, but they are related. AOLserver works pretty well with
tsd, I've never seen or heard of any problem with that. Since
AOLserver was developed over many years, specifically for performance,
it is hard to believe that there is any issue with threads. Tests or
more pointers to alternative code would still be very interesting.

I've done tests with both thread types combined:

http://junom.com/document/aolserver/apache-bench/nsbgwrite/logs/

Instead of package distributors shying away from anything I recommend
testing, and more testing.

Donald G Porter

unread,
Dec 10, 2007, 10:59:32 AM12/10/07
to
George Peter Staplin wrote:
> After much research I want to share my concerns over the ideas and
> conflicting goals some of the core team has with threads, and why I am
> against having --enable-threads by default.

Just to add a few observations relevant to this discussion...

Tcl/Tk 8.5.0 are days away from release. The source distributions
do *not* configure for --enable-threads by default on unix systems.
We're not going to change that in a few days, and then compatibilty
concerns will likely keep us fixed at a --disable-threads default
through the 8.5.* series of source code releases.

So, while the discussion is important, it is not urgent. We missed
the boat to make this particular change in Tcl 8.5.

That said, there are many repackagers and redistributors of Tcl/Tk
that do create binaries from the --enable-threads configurations.
Debian Linux is one. No matter what the default configuration is,
we need to make both configurations works as best we can.

--
| Don Porter Mathematical and Computational Sciences Division |
| donald...@nist.gov Information Technology Laboratory |
| http://math.nist.gov/~DPorter/ NIST |
|______________________________________________________________________|

Kevin Kenny

unread,
Dec 10, 2007, 11:00:40 AM12/10/07
to
George Peter Staplin wrote:
[many useful thoughts about --enable-threads being the default]

George paints a bleak picture of a "box with no inside:" a set of
constraints that cannot be satisfied. Alas, he's probably entirely
right. Posix has threads that are Broken By Design, or at least ones
that fail to interact well with other Posix features, most notably
fork{} and signals. And the different users of Tcl have different
objectives, some of which depend on threads and some of which
interact badly with threads. I quite agree with George that
the current state of the world demands both threaded and non-threaded
builds.

Oh, before I dive in any farther, I had better get one basic item
out of the way: I'm talking about Unix builds here. Windows builds
have been threaded-by-default for years. Windows lacks fork(), and
its ideas of asynchronous signals are completely different from Posix.
Threads are less problematic there. Moreover, a good many Windows
API's are blocking and simply cannot be civilised without the use
of threads to manage them.

On Unix, the interaction between threads and fork() is the real
killer. Signal delivery is secondary, since Tcl doesn't do a competent
job of that to begin with. (That's an issue for another day.)
Let's, then, take a closer look at fork().

Expect and BLT both do their own forking rather than use [exec]
or Tcl_CreatePipeline. They do this for very good reasons: it has
become quite clear that [exec] is rather a botch. This immediate
problem can't be fixed until a better alternative is available,
so a first project might be for someone to develop one. [blt_bgexec]
might be a good starting point. The developed alternative should
use posix_spawn() in preference to fork()/exec(), in order to
make sure that a threaded build on QNX doesn't completely take
away the ability to launch subprocesses.

TclX also exports fork()-without-exec() at script level, primarily
to support programmers who want to write forking server processes.
It's not clear to me that we want to encourage this usage in a
multithreaded world. I'd be more or less comfortable with disabling
this functionality in a threaded build, and it is unusual enough
that it's far from clear that losing it is a strong argument
against making a threaded build the default. (It is a strong
argument for keeping non-threaded builds available.)

George also raises (and ascribes to me) a red herring regarding
existing libraries that spin threads and manage concurrency without
going through Tcl. I know that TclBlend does this, because it has
an entire Java Virtual Machine under the hood. When it comes to
whether other extensions also do it, I'm arguing from ignorance.
It appears imprudent to break any of them gratuitously, so I'm
suspicious of the solution of "acquire all mutexes before forking,
release them afterward;" it seems to be every bit as dangerous as
the pthread_atfork that George despises (and cites Butenhof).
Migrating at least those uses of fork/exec that we control over
to using spawn() instead strikes me as a more useful approach,
at least on the platforms where spawn() is available.

A summary of all this might be:

1. We need non-threaded builds for existing code that uses fork().
2. We need threaded builds for those that need threads.
3. We need to fix Tcl_CreatePipeline() so that the two don't
fight over the normal, bread-and-butter ways to launch subprocesses.
4. Threads seem to be in the ascendancy, while forking subprocesses
(particularly, fork-without-exec) is on the decline. At some
point it begins to make sense to switch the default from
non-threaded to threaded. No choice of that time will please
everyone. My experience is that the TCT comes under fire far
more often for being overconservative than for abandoning
legacy users.

There is no right answer here. Either we break existing code that
uses fork(), or we fail to support developers who need threads to
exploit their multicore machines fully. The design of Posix seems
to leave us with no clear water between that Scylla and that
Charybdis.

George's question about TSD management is really a separate issue from
the rest of this. Since this posting is far too long already, I'll
address that one separately.

--
73 de ke9tv/2, Kevin

Donal K. Fellows

unread,
Dec 10, 2007, 11:44:47 AM12/10/07
to
Kevin Kenny wrote:
> TclX also exports fork()-without-exec() at script level, primarily
> to support programmers who want to write forking server processes.
> It's not clear to me that we want to encourage this usage in a
> multithreaded world. I'd be more or less comfortable with disabling
> this functionality in a threaded build, and it is unusual enough
> that it's far from clear that losing it is a strong argument
> against making a threaded build the default. (It is a strong
> argument for keeping non-threaded builds available.)

Having thought about this a bit, it seems to me that a reasonable
alternative to this would be to offer a 'tcl-daemon' program that does
the forking disconnect before it hands control to a Tcl script. After
all, this sort of approach is needed anyway on Windows, where services
are even more different from normal programs than on Unix. But the
advantage of doing this is that the "bad" arbitrary-fork is banished
to a place where the number of threads and locks is under strict
control...

Donal.

Darren New

unread,
Dec 10, 2007, 4:42:41 PM12/10/07
to
Kevin Kenny wrote:
> 1. We need non-threaded builds for existing code that uses fork().

Is it --enable-threads that breaks fork(), or is it actually using
threads? It would seem the best of both worlds to say "use threads, or
use fork(), either will work, but don't use both. Is it the use of
mutexes, or the actual launching of a thread that breaks things?

--
Darren New / San Diego, CA, USA (PST)
It's not feature creep if you put it
at the end and adjust the release date.

Alexandre Ferrieux

unread,
Dec 10, 2007, 5:35:44 PM12/10/07
to
On Dec 10, 10:42 pm, Darren New <d...@san.rr.com> wrote:
> Kevin Kenny wrote:
> > 1. We need non-threaded builds for existing code that uses fork().
>
> Is it --enable-threads that breaks fork(), or is it actually using
> threads? It would seem the best of both worlds to say "use threads, or
> use fork(), either will work, but don't use both. Is it the use of
> mutexes, or the actual launching of a thread that breaks things?

IIRC, in --enable-threads mode the Tcl notifier intialization
procedure already creates extra threads before you can move a
finger... Sigh...

-Alex

davidn...@gmail.com

unread,
Dec 11, 2007, 3:26:01 AM12/11/07
to
On Dec 10, 11:35 pm, Alexandre Ferrieux <alexandre.ferri...@gmail.com>
wrote:

Yes, that's correct.

davidw@fortrock:~$ tclsh
%

davidw@fortrock:~$ ps -L auxwOs | grep tclsh
davidw 8034 8034 0.0 2 0.0 11272 1516 pts/6 Sl+ 09:23
0:00 tclsh
davidw 8034 8035 0.0 2 0.0 11272 1516 pts/6 Sl+ 09:23
0:00 tclsh

Reply all
Reply to author
Forward
0 new messages