fork vs [open |process]
graeme....@gmail.com
threadenabled TCL (i.e. most binary distributions).
What about opening a process with [open |tclsh myscipt.tcl]? Is it
safe (it has to be, surely)? How does it work at lower levels?
Also, according to this discussion, it would not matter much in terms
of memory consumption if I fork exec new processes:
http://archives.postgresql.org/pgsql-hackers/1998-01/msg00677.php.
Does it follow that using [open |tclsh myscript.tcl] would not waste
memory?
Also the problems with fork could do with being more prominently
mentioned. I added to the page on the fork page on the wiki as a
start. IMHO the expect man page should also have a warning that fork
will be problematic with most tcl binaries.
My Linux distro (Mandriva) has expect it its repos and so thought it
would work with the tcl from the repos (logical enough, surely), but
the latter has threads enabled (that is what tcl_platform(threaded)
means, right?). I do not think I am likely to be the only person to
make this mistake...
graeme....@gmail.com
<graeme.piete...@gmail.com> wrote:
--snip--
> Also, according to this discussion, it would not matter much in terms
> of memory consumption if I fork exec new processes:http://archives.postgresql.org/pgsql-hackers/1998-01/msg00677.php.
> Does it follow that using [open |tclsh myscript.tcl] would not waste
> memory?
OK, I have answered part of this question for myself. TCL is compiled
as a shared library, and most of the bloat of my script when running
is more shared libraries, so all that will no be wasted. I am guessing
that other things (like some shared data) will be needlessly
duplicated. Am I right?
Incidentally, this does lead on from a question I asked a few days
ago. I have started a new thread because the topic has changed.
Cameron Laird
graeme....@gmail.com <graeme....@gmail.com> wrote:
>I know that [fork] (from TclX or Expect) has problems with
>threadenabled TCL (i.e. most binary distributions).
>
>What about opening a process with [open |tclsh myscipt.tcl]? Is it
>safe (it has to be, surely)? How does it work at lower levels?
.
.
[open] indeed is entirely safe for use, even in thread-enabled Tcl.
[open |]'s low-level implementation is platform-dependent, in
rather sophisticated ways. On the other hand, the source code has
won awards for its clarity, and you might enjoy reading through it
for yourself.
graeme....@gmail.com
> In article <a8a8b5b3-2a99-4a2e-afea-dbec93b03...@2g2000hsn.googlegroups.com>,graeme.piete...@gmail.com <graeme.piete...@gmail.com> wrote:
> >I know that [fork] (from TclX or Expect) has problems with
> >threadenabled TCL (i.e. most binary distributions).
>
> >What about opening a process with [open |tclsh myscipt.tcl]? Is it
> >safe (it has to be, surely)? How does it work at lower levels?
>
> .
> .
> .
> [open] indeed is entirely safe for use, even in thread-enabled Tcl.
Thanks.
>
> [open |]'s low-level implementation is platform-dependent, in
> rather sophisticated ways. On the other hand, the source code has
> won awards for its clarity, and you might enjoy reading through it
> for yourself.
I have heard that it is very clear code, and as far as I can see it
is.
On the other hand, I do not know much C, so looking at any large body
of C is not easy for me (I know this from experience). That said I am
willing to learn and looking at the code is good advice which I will
take. I am finding it difficult to know where to start though. Are
there any newbie guides to the Tcl source?
Neil Madden
[...]
>
> On the other hand, I do not know much C, so looking at any large body
> of C is not easy for me (I know this from experience). That said I am
> willing to learn and looking at the code is good advice which I will
> take. I am finding it difficult to know where to start though. Are
> there any newbie guides to the Tcl source?
Not that I know of. It still changes quite a bit from release to
release. Various bits are documented to some extent on the wiki
(http://wiki.tcl.tk/) -- search for "Category Internals".
The actual source code layout is fairly straightforward -- most C code
is in generic/ with platform-specific stuff in win/ unix/ and macosx/.
The library/ folder contains parts of the implementation that are Tcl
scripts (e.g. parts of [clock]), and the init.tcl which is loaded into
new interpreters. And so on.
The best place to start if you are new to the source is to get used to
how Tcl extensions are written: how Tcl represents values (Tcl_Obj
structure, and Tcl_ObjType), how commands are invoked (Tcl_ObjCmdProc),
and so on. Several Tcl books cover this, and the wiki has plenty of
information on this. In the source, you can look at sample commands such
as Tcl_IncrObjCmd in generic/tclCmdIL.c, or e.g. look at how lists are
implemented in generic/tclListObj.c. Details of how [open "|..."] is
handled is platform-specific, so e.g. for unix systems this is
unix/tclUnixPipe.c where TclpCreateProcess does a fork()/execvp() sequence.
If you want to get the heart of the interpreter, then look at
generic/tclBasic.c, and in particular at the implementation of
TclEvalObjvInternal (TEOVI as it is affectionately known). I'd advise
grabbing a version of the source code from early 8.5 or even 8.4 branch,
as TEOVI in HEAD (8.6) is rather more complex due to recent enhancements
(indeed, it seems to have disappeared now, so look at Tcl_EvalObjv).
After that, there is the byte-compiler in tclComp*.{h,c}, namespaces in
tclNamesp.c, etc.
Enjoy!
-- Neil
Cameron Laird
graeme....@gmail.com <graeme....@gmail.com> wrote:
.
.
>Also, according to this discussion, it would not matter much in terms
>of memory consumption if I fork exec new processes:
>http://archives.postgresql.org/pgsql-hackers/1998-01/msg00677.php.
>Does it follow that using [open |tclsh myscript.tcl] would not waste
>memory?
.
.
Interesting question. The answer, as is often the case,
is, "yes and no".
The details are rather complex. The short version is:
anything that has the functionality you want is ENTIRELY
less wasteful of memory than an incorrect application;
feel free to use any Tcl resources necessary to get your
program running as you want.
Next: Windows ... has issues.
Cameron Laird
graeme....@gmail.com <graeme....@gmail.com> wrote:
.
.
.
>mentioned. I added to the page on the fork page on the wiki as a
>start. IMHO the expect man page should also have a warning that fork
>will be problematic with most tcl binaries.
>
>My Linux distro (Mandriva) has expect it its repos and so thought it
>would work with the tcl from the repos (logical enough, surely), but
>the latter has threads enabled (that is what tcl_platform(threaded)
>means, right?). I do not think I am likely to be the only person to
>make this mistake...
I agree: <URL: http://wiki.tcl.tk/1967 > should have mentioned
a long time ago the conflict with thread safety. There are
debates here that have been goin gon for a decade, and I'm a
little surprised we haven't captured more of it than is first
apparent in the Wiki. In any case, these turn out to be hard
problems, and we've sometimes had difficulty communicating them
to distribution maintainers.
Cameron Laird
graeme....@gmail.com <graeme....@gmail.com> wrote:
>On Aug 4, 3:48 pm, "graeme.piete...@gmail.com"
><graeme.piete...@gmail.com> wrote:
>--snip--
>> Also, according to this discussion, it would not matter much in terms
>> of memory consumption if I fork exec new
>processes:http://archives.postgresql.org/pgsql-hackers/1998-01/msg00677.php.
>> Does it follow that using [open |tclsh myscript.tcl] would not waste
>> memory?
>--snip--
>
>OK, I have answered part of this question for myself. TCL is compiled
>as a shared library, and most of the bloat of my script when running
>is more shared libraries, so all that will no be wasted. I am guessing
>that other things (like some shared data) will be needlessly
>duplicated. Am I right?
.
.
I've lost track of the question. I think the main
answer is that, if proliferating Tcl processes helps
you achieve the functionality you're after, it's go-
ing to bother few Tcl experts. On the other hand,
are you aware of [source]? There are several dif-
ferent ways for processes to share code.
graeme....@gmail.com
> In article <3a41b422-c361-4e2c-b3a8-d9c0f69d6...@k13g2000hse.googlegroups.com>,
.
> I think the main
> answer is that, if proliferating Tcl processes helps
> you achieve the functionality you're after, it's go-
> ing to bother few Tcl experts. On the other hand,
> are you aware of [source]? There are several dif-
> ferent ways for processes to share code.
Thanks for all your replies, thanks to Neil too.
I do know about [source]. In this case I do need multiple processes
(concurrency even when one process is busy for a while, resource
limits etc.).
I have added a bug to the Mandriva bug tracker about the fork/threads
issue: I do not think they have anything that depends on Tcl being
threaded, while they do have expect in the repos, so I think they
should be compiling without threads. In the meantime I compliled Tcl
without threads myself.