Tcl/Tk 8.2 in the works
Paul Gardiner
an
8.2 release of Tcl/Tk in the near future.
Tcl 8.1 added many major new features such as internationalization and
thread safety. There has been one patch release since then (8.1.1) and
we
had intended to make a second sometime in July or August. However, there
are a few new features we want to introduce and, rather than add
functionality in patch releases, we intend that the next release of
Tcl/Tk will be called 8.2. This release will contain everything that was
going to go into 8.1.2 and there will be no separate 8.1.2 release. Nor
are we planning to do an 8.0.6 release.
The new features in 8.2 will include:
-the Trf patch for stacked I/O channels that is needed for extensions to
the channel system - such as the SSL extension
-the DDE patch that provides the poke command
-some new procedures for manipulating strings that offer substantial
performance improvements
-a first implementation of some TEA-related changes
-the Tcl and Tk test harnesses as loadable packages
The new features notwithstanding, the main thrust of the release will be
bug fixes. We're currently going through the backlog of contributed
patches. If you have new bug fixes that you think should be included in
8.2, please submit them to http://www.scriptics.com/support/bugForm.html
before Friday, July 23rd.
We plan to make a beta of 8.2 available in early July with the final
version to follow in August.
Paul
Jan Nijtmans
> The new features notwithstanding, the main thrust of the release will be
> bug fixes. We're currently going through the backlog of contributed
> patches.
How about other enhancements such as 1575, 1540, 2040, 2059, 2113 (and
a few more), which are also required for some extensions (Img, TclKit,
Tkspline ....)? Do they make any chance to be accepted, finally?
--
Jan Nijtmans, CMG Arnhem B.V.
email: Jan.Ni...@wxs.nl (private)
Jan.Ni...@cmg.nl (work)
url: http://home.wxs.nl/~nijtmans/
Phil
Any remote chance of slave interps defaulting to a threaded
model?
Paul Gardiner <pa...@scriptics.com> wrote:
>
>The new features in 8.2 will include:
>-the Trf patch for stacked I/O channels that is needed for extensions to
>the channel system - such as the SSL extension
>-the DDE patch that provides the poke command
>-some new procedures for manipulating strings that offer substantial
>performance improvements
>-a first implementation of some TEA-related changes
>-the Tcl and Tk test harnesses as loadable packages
--
Phil Ehrens <peh...@ligo.caltech.edu>| Fun stuff:
The LIGO Laboratory, MS 18-34 | http://www.ralphmag.org
California Institute of Technology | http://www.yellow5.com
1200 East California Blvd. | ftp://ftp.no.pgpi.com/pub/pgp
Pasadena, CA 91125 USA | http://slashdot.org
Phone:(626)395-8518 Fax:(626)793-9744 | http://freshmeat.net
Paul Duffin
>
> Can you tell us about what is happening in the thread arena?
> Any remote chance of slave interps defaulting to a threaded
> model?
>
No, no, no, no, no. Changing this sort of default behaviour will break
scripts. They should however look at getting feedback on the Tcl
threading interface.
> Paul Gardiner <pa...@scriptics.com> wrote:
> >
> >The new features in 8.2 will include:
> >-the Trf patch for stacked I/O channels that is needed for extensions to
> >the channel system - such as the SSL extension
> >-the DDE patch that provides the poke command
> >-some new procedures for manipulating strings that offer substantial
> >performance improvements
> >-a first implementation of some TEA-related changes
> >-the Tcl and Tk test harnesses as loadable packages
>
--
Paul Duffin
DT/6000 Development Email: pdu...@hursley.ibm.com
IBM UK Laboratories Ltd., Hursley Park nr. Winchester
Internal: 7-246880 International: +44 1962-816880
Phil
>Phil wrote:
>>
>> Can you tell us about what is happening in the thread arena?
>> Any remote chance of slave interps defaulting to a threaded
>> model?
>>
>
>No, no, no, no, no. Changing this sort of default behaviour will break
>scripts. They should however look at getting feedback on the Tcl
>threading interface.
Surprise Paul, Tcl8.1 already broke scripts. That doesn't seem to
be a problem for Scriptics. Nevertheless, I will concede, having it
be an option would be just as nice ;-)
Phil
Scott Redman
Paul Duffin wrote:
>
> Phil wrote:
> >
> > Can you tell us about what is happening in the thread arena?
> > Any remote chance of slave interps defaulting to a threaded
> > model?
> >
>
> No, no, no, no, no. Changing this sort of default behaviour will break
> scripts. They should however look at getting feedback on the Tcl
> threading interface.
Paul is right. However, instead of having "slave" interps, the
testthread code (which may eventually be a public function called
"thread") creates interps in separate threads as you are wanting.
Changing the behavior of slave interps is a bad idea, but
creating new threads with their own master interps seems to be
a reasonable solution.
Paul, can you elaborate on "getting feedback", do you mean
something like "testthread $thread result"? Or maybe a fileevent-like
mechanism? Just want to get your thoughts on the subject (and
anyone else's).
-- Scott
>
<snip>
Phil
Scott Redman <red...@scriptics.com> wrote:
>
>Paul, can you elaborate on "getting feedback", do you mean
>something like "testthread $thread result"? Or maybe a fileevent-like
>mechanism? Just want to get your thoughts on the subject (and
>anyone else's).
--
lvi...@cas.org
According to Scott Redman <red...@scriptics.com>:
:Paul Duffin wrote:
:> scripts. They should however look at getting feedback on the Tcl
:> threading interface.
:Paul, can you elaborate on "getting feedback", do you mean
I read Paul's remark to mean "Scriptics should look at getting feedback
from the Tcl community on future directions for the Tcl threading interface.
--
<URL: mailto:lvi...@cas.org> Quote: Saving the world before bedtime.
<*> O- <URL: http://www.purl.org/NET/lvirden/>
Unless explicitly stated to the contrary, nothing in this posting
should be construed as representing my employer's opinions.
Scott Redman
"...feedback from the Tcl...".
In any case, any ideas regarding changing the threading
interface in testthread? One suggestion was to get
the stdout/stdin/stderr of the thread hooked up as a
channel that the parent could read (fileevent). I also
think you need the return status of the thread as well.
-- Scott
Gerald W. Lester
Paul Duffin wrote:
>
> Phil wrote:
> >
> > Can you tell us about what is happening in the thread arena?
> > Any remote chance of slave interps defaulting to a threaded
> > model?
> >
>
> No, no, no, no, no. Changing this sort of default behaviour will break
> scripts. They should however look at getting feedback on the Tcl
> threading interface.
Paul is right. However, instead of having "slave" interps, the
testthread code (which may eventually be a public function called
"thread") creates interps in separate threads as you are wanting.
Changing the behavior of slave interps is a bad idea, but
creating new threads with their own master interps seems to be
a reasonable solution.
I agree, don't change the default behavior.
But instead of adding a new command, "thread", to create an interp in a thread why not add a "-thread" option to interp create? Less command namespace pollution.
Paul, can you elaborate on "getting feedback", do you mean
something like "testthread $thread result"? Or maybe a fileevent-like
mechanism? Just want to get your thoughts on the subject (and
anyone else's).
Actually I'd vote to have a command named bind that would take an "object" (file, widget, threaded-interp) an event and a script to call when that event happened. Add in event query as a new event subcommand and things start to look real nice. Maybe even through in a timer command (oneshot and repeating) to create timer objects (e.g. replace after) and get everything event driven under one interface.
Of course, you still support the old commands, they just don't pick up any new features that get added.
--
+--------------------------------+---------------------------------------+
| Gerald W. Lester
| "The man who fights for his ideals is |
| gerald...@mtolive-lcms.org | the man who is alive."
-- Cervantes |
|
Webmaster for http://www.mtolive-lcms.org
|
+--------------------------------+---------------------------------------+
Alexandre Ferrieux
>
> any ideas regarding changing the threading
> interface in testthread?
As already stated, and acknowledged by Brent, I vote against [thread
wait] !
> One suggestion was to get
> the stdout/stdin/stderr of the thread hooked up as a
> channel that the parent could read (fileevent).
Why the stdout/stdin/stderr, why not normally named inter-thread
channels ? An excellent candidate for that is Andreas's memchan, which
AFAIK already works perfectly with fileevents. All this of course, only
to handle high-bandwidth inter-thread communication. For slower, small
messages, the [thread send] is perfect.
Another neat thing would (again) be a TclX-like [pipe] command to create
a ready-to-use pipe. Of course this would be useful outside threading
concerns, but in the specific context of redirections, there is a
frequent situation where it is needed:
Thread 1 launches [exec cmd...]
Thread 2 blockingly reads the process's stdout
Thread 3 blockingly reads the process's stderr
Of course the [pipe] command would also solve the issue in fully
event-driven mode:
pipe p1r p1w
pipe p2r p2w
open "|cmd >@ $p1w 2>@ $p2w" w
fileevent $p1r readable ...
fileevent $p2r readable ...
But the threaded version would be cool too. Notice that we'd need a
primitive to hand a channel over to another thread:
pipe p1r p1w
pipe p2r p2w
set tp1r [thread channelproxy $tid1 $p1r]
set tp2r [thread channelproxy $tid2 $p2r]
exec cmd >@ $p1w 2>@ $p2w
Now another question: what's the plan for synchronization primitives
(mutexes, semaphores...) ?
-Alex
Paul Duffin
>
> Paul Duffin wrote:
> >
> > Phil wrote:
> > >
> > > Can you tell us about what is happening in the thread arena?
> > > Any remote chance of slave interps defaulting to a threaded
> > > model?
> > >
> >
> > No, no, no, no, no. Changing this sort of default behaviour will break
> > scripts. They should however look at getting feedback on the Tcl
> > threading interface.
>
> Paul is right. However, instead of having "slave" interps, the
> testthread code (which may eventually be a public function called
> "thread") creates interps in separate threads as you are wanting.
> Changing the behavior of slave interps is a bad idea, but
> creating new threads with their own master interps seems to be
> a reasonable solution.
>
> something like "testthread $thread result"? Or maybe a fileevent-like
> mechanism? Just want to get your thoughts on the subject (and
> anyone else's).
>
By feedback I meant just what you are asking for, i.e. what people
think about the interface, what they want, what they need, etc.
As I have not used threads I am not the one to ask.
Don Libes
> Scott Redman wrote:
> > any ideas regarding changing the threading
> > interface in testthread?
>
> As already stated, and acknowledged by Brent, I vote against [thread
> wait] !
>
> > One suggestion was to get
> > the stdout/stdin/stderr of the thread hooked up as a
> > channel that the parent could read (fileevent).
>
> Why the stdout/stdin/stderr, why not normally named inter-thread
> channels ? An excellent candidate for that is Andreas's memchan, which
> AFAIK already works perfectly with fileevents. All this of course, only
> to handle high-bandwidth inter-thread communication. For slower, small
> messages, the [thread send] is perfect.
>
> Another neat thing would (again) be a TclX-like [pipe] command to create
> a ready-to-use pipe. Of course this would be useful outside threading
> concerns, but in the specific context of redirections, there is a
> frequent situation where it is needed:
Please don't use real pipes. They've got all sorts of problems. For
example, on some systems, select doesn't support pipe (so fileevent
doesn't work). They're limited size. Etc.
Don
Donal K. Fellows
Alexandre Ferrieux <alexandre...@cnet.francetelecom.fr> wrote:
> Scott Redman wrote:
>> any ideas regarding changing the threading interface in testthread?
>
> Now another question: what's the plan for synchronization primitives
> (mutexes, semaphores...) ?
Don't you only need those where you have shared data to be protected?
Without shared data, you can implement equivalent stuff through
thread-local resources and communication channels...
Just because C and Java do it one way doesn't mean that we must follow
the same path. Especially given the fact that an interpreter can only
exist in a single thread.
Donal.
--
Donal K. Fellows http://www.cs.man.ac.uk/~fellowsd/ fell...@cs.man.ac.uk
-- The small advantage of not having California being part of my country would
be overweighed by having California as a heavily-armed rabid weasel on our
borders. -- David Parsons <o r c @ p e l l . p o r t l a n d . o r . u s>
Paul Duffin
>
> In article <3779D8...@cnet.francetelecom.fr>,
> Alexandre Ferrieux <alexandre...@cnet.francetelecom.fr> wrote:
> > Scott Redman wrote:
> >> any ideas regarding changing the threading interface in testthread?
> [...]
> >
> > Now another question: what's the plan for synchronization primitives
> > (mutexes, semaphores...) ?
>
> Don't you only need those where you have shared data to be protected?
> Without shared data, you can implement equivalent stuff through
> thread-local resources and communication channels...
>
> Just because C and Java do it one way doesn't mean that we must follow
> the same path. Especially given the fact that an interpreter can only
> exist in a single thread.
>
It would be nice to be able to have a "shared memory" equivalent between
threads. Obviously this would need something like that.
Alexandre Ferrieux
>
> In article <3779D8...@cnet.francetelecom.fr>,
> Alexandre Ferrieux <alexandre...@cnet.francetelecom.fr> wrote:
> > Scott Redman wrote:
> >> any ideas regarding changing the threading interface in testthread?
> [...]
> >
> > Now another question: what's the plan for synchronization primitives
> > (mutexes, semaphores...) ?
>
> Don't you only need those where you have shared data to be protected?
> Without shared data, you can implement equivalent stuff through
> thread-local resources and communication channels...
Oooops - you're fully right.
Must have been the heat <blush>.
> Just because C and Java do it one way doesn't mean that we must follow
> the same path. Especially given the fact that an interpreter can only
> exist in a single thread.
Yes. This is related to something I've realized only recently with the
Tcl threading model: a purposeful, strong isolations of threads. At the
beginning, I believed that "at most one thread by interp" was a dirty
hack to hide bad reentrancy in the core. Now I have come to understand
that instead, it pushes forward a new (as compared to C, as you
mentioned) and also very "Tclish" style: basically, Tcl threads are
nearly as isolated as *processes*, which is nice because it means all
the modularity and atomic-testing we want, without the fork/exec and
address space switch overhead ! Another interesting aspect is with
extensions: since an extension exists only in interps with which it has
been initialized, a properly written (though in a thread-aware fashion)
extension should not be taken by surprise by a multi-threaded Tcl
program...
-Alex
lvi...@cas.org
According to Donal K. Fellows <fell...@cs.man.ac.uk>:
:> Yes. This is related to something I've realized only recently with
:> the Tcl threading model: a purposeful, strong isolations of
:This style of concurrency is actually much closer to what is
:understood in the Formal Semantics world. The key is that it is
Anyone familar with the concepts of thread programming of this style
interested in putting up a Wiki page on the topic?
tdun...@my-deja.com
Alexandre Ferrieux
<alexandre...@cnet.francetelecom.fr> wrote:
> Why the stdout/stdin/stderr, why not normally
named inter-thread
> channels ? An excellent candidate for that is
Andreas's memchan, which
> AFAIK already works perfectly with fileevents.
All this of course, only
> to handle high-bandwidth inter-thread
communication. For slower, small
> messages, the [thread send] is perfect.
All filevent-based communication mechanisms would
have the serious fault that no complex data
structures could be passed this way without losing
everything but the string representation.
Since I use customized TCL objects quite
extensively, and since the objects are relatively
expensive to convert this would be quite painful
to me.
What I would much prefer is to allow threads to
pass selected values back to the parent thread. I
believe (but have no time to prove) that if
{Incr,Decr}RefCount are made thread safe, then
this mechanism could be reasonably easy to work
with. One place that there might be trouble is in
the typical idiom where a test of the form:
if (Tcl_isShared(...)) {
duplicate ...
}
else {
use private copy
}
Given that only the owner can give away a
variable, it seems that this construct would
actually not result in a dangerous race condition.
The only defect would be occasional superfluous
copies which result when one thread determines
that an object is shared just before another
thread stops referring to it. The inverse race
cannot occur since objects with only one reference
can only be referenced in one thread.
Sent via Deja.com http://www.deja.com/
Share what you know. Learn what you don't.
tdun...@my-deja.com
communication now that TCL has real data structures inside of Tcl_Obj's.
What I would prefer is to be able to pass arbitrary TCL values back to
the master interpreter. This can be done relatively safely since TCL
objects with a reference count > 1 are read-only.
Even the customary idiom:
if (...is_shared(obj)) {
duplicate obj
}
else {
munge directly
}
is safe since the race condition fails safely. If an object is shared
by two threads, then it will be duplicated before modification. If
an object is recently deallocated by one thread and another thread
has already noticed that the object is shared, then an extra copy is
created but no failure occurs.
Note also that any race conditions in DecrRefCount and IncrRefCount
can probably be made fail-safe as well. Even serializing all calls to
Incr/DecrRefCount would probably be acceptable in terms of performance
(although I would prefer to only synchronize on calls affecting shared
objects).
My expectation is that Tcl_Obj's will gain currency as an extension
mechanism. Thus, picking a thread communication primitive which
encourages and supports Tcl_Obj's is an important thing to do.
Using a byte serial channel is a bad option.
In article <s6avhc5...@nist.gov>,
Don Libes <li...@nist.gov> wrote:
> > Why the stdout/stdin/stderr, why not normally named inter-thread
> > channels ? An excellent candidate for that is Andreas's memchan,
which
> > AFAIK already works perfectly with fileevents. All this of course,
only
> > to handle high-bandwidth inter-thread communication. For slower,
small
> > messages, the [thread send] is perfect.
> Please don't use real pipes. They've got all sorts of problems. For
> example, on some systems, select doesn't support pipe (so fileevent
> doesn't work). They're limited size. Etc.
Sent via Deja.com http://www.deja.com/
sanjay...@my-deja.com
> Yes. This is related to something I've realized only recently with
the
the
> beginning, I believed that "at most one thread by interp" was a dirty
> hack to hide bad reentrancy in the core. Now I have come to
understand
> that instead, it pushes forward a new (as compared to C, as you
> mentioned) and also very "Tclish" style: basically, Tcl threads are
> nearly as isolated as *processes*, which is nice because it means all
> the modularity and atomic-testing we want, without the fork/exec and
> address space switch overhead ! Another interesting aspect is with
> extensions: since an extension exists only in interps with which it
> has been initialized, a properly written (though in a thread-aware
> fashion) extension should not be taken by surprise by a
> multi-threaded Tcl program...
Any extension written in Tcl/Tk will work fine in threading
environment. However, extensions written in C (e.g. in Windows as DLL
files) will have to be aware of different threads. This would require
that if DLL has any globals or static vars then they should locked
before use.
- Sanjay
Paul Duffin
>
> In fact, any byte serial channel would be a big lose for thread
> communication now that TCL has real data structures inside of Tcl_Obj's.
>
> What I would prefer is to be able to pass arbitrary TCL values back to
> the master interpreter. This can be done relatively safely since TCL
> objects with a reference count > 1 are read-only.
>
> Even the customary idiom:
>
> if (...is_shared(obj)) {
> duplicate obj
> }
> else {
> munge directly
> }
>
> is safe since the race condition fails safely. If an object is shared
> by two threads, then it will be duplicated before modification. If
> an object is recently deallocated by one thread and another thread
> has already noticed that the object is shared, then an extra copy is
> created but no failure occurs.
Thread1 owns object1 whose refCount is 1.
Thread1 stores it somewhere thread2 can get it, refCount is 2.
Thread2 removes it, refCount is still 2.
If either thread tries to modify it then it would be duplicated.
>
> Note also that any race conditions in DecrRefCount and IncrRefCount
> can probably be made fail-safe as well. Even serializing all calls to
Not sure about that.
> Incr/DecrRefCount would probably be acceptable in terms of performance
It would require a change from a macro to a function call.
> (although I would prefer to only synchronize on calls affecting shared
> objects).
>
Not sure how that would work either.
> My expectation is that Tcl_Obj's will gain currency as an extension
> mechanism. Thus, picking a thread communication primitive which
Your expectations are correct. See
http://purl.oclc.org/net/pduffin/home/feather/feather.htm
> encourages and supports Tcl_Obj's is an important thing to do.
>
I think so too. I am thinking of having some sort of shared memory
object, or a queue which handles locking.
However the big problem with passing objects between threads is the
internal representation. It is almost as if each object type needs a
flag to say whether it is "thread safe" or not.
> Using a byte serial channel is a bad option.
>
It is not a 'bad' option it should just not be the only option.
> In article <s6avhc5...@nist.gov>,
> Don Libes <li...@nist.gov> wrote:
>
> > > Why the stdout/stdin/stderr, why not normally named inter-thread
> > > channels ? An excellent candidate for that is Andreas's memchan, which
> > > AFAIK already works perfectly with fileevents. All this of course, only
> > > to handle high-bandwidth inter-thread communication. For slower, small
> > > messages, the [thread send] is perfect.
>
> > Please don't use real pipes. They've got all sorts of problems. For
> > example, on some systems, select doesn't support pipe (so fileevent
> > doesn't work). They're limited size. Etc.
>
> Sent via Deja.com http://www.deja.com/
> Share what you know. Learn what you don't.
--
Donal K. Fellows
> In fact, any byte serial channel would be a big lose for thread
> communication now that TCL has real data structures inside of Tcl_Obj's.
>
> What I would prefer is to be able to pass arbitrary TCL values back to
> the master interpreter. This can be done relatively safely since TCL
> objects with a reference count > 1 are read-only.
It would be quite acceptable (from my PoV at least) to provide
ordinary serial byte-channels between threads in the core, and then
add whole-value passing as a separate beastie (in a standard
extension?) Especially as this scheme gives you inter-thread comms
simply (strings are enough for many applications) and puts off the
more complex case[*] of general object passing until it has actually
been written... :^)
Donal.
[* Hacking together something that mostly works is easy. Hammering
out all the bugs and memory leaks (as is needed for anything in the
core; we like well-engineered stuff) is much more work. ]
Donal K. Fellows
> Any extension written in Tcl/Tk will work fine in threading
> environment. However, extensions written in C (e.g. in Windows as
> DLL files) will have to be aware of different threads. This would
> require that if DLL has any globals or static vars then they should
> locked before use.
Any extension written in C whose varying data is only ever at most
interp-global or stack-local (all global/static values must be
constants) will be automatically thread-safe in Tcl (if it is compiled
with the correct options; but lib/tclConfig.sh will have those options
in if you need them...)
Creating interp-global data is really quite easy with the
Tcl_SetAssocData(3) call (if you can't just use the clientData
argument to the commands in question, that is...)
Donal.