Using TCL as an embedded interpreter ?
John P. Looney
I've to write a simple server that will most likely have to be
*very* extensible in the future. I would like to present each
connecting client with a text parser, running in it's own thread.
Then I thought of how AOLserver's "Admin port" works. You can
telnet in, give a username/password, and type TCL commands in
directly. This would be perfect for what I'd in mind, though AOLserver
is a tad complex to dive straight into.
Does anyone have any pointers to a simple server that's implemented
like this, that I could cannabalize/learn from ?
Kate
--
"The fool must be beaten with a stick, for an intelligent person
the merest hint is sufficient" -- Zen Master Greg
Scott Redman
web server. It's Tcl-only, and you can configure the port that it listens
to. You can also add a "domain" handler that generates pages and does not
need .html files on disk.
Look for it at http://dev.scriptics.com (fyi, http://www.scriptics.com, our
other website, is running on TclHttpd).
-- Scott
"John P. Looney" <jploon...@online.ie> wrote in message
news:38E87E5E...@online.ie...
Cameron Laird
Scott Redman <red...@nospam.scriptics.com> wrote:
>Assuming that you keep the Tcl event loop alive, you can use the TclHttpd
>web server. It's Tcl-only, and you can configure the port that it listens
>to. You can also add a "domain" handler that generates pages and does not
>need .html files on disk.
>
>Look for it at http://dev.scriptics.com (fyi, http://www.scriptics.com, our
>other website, is running on TclHttpd).
>
>-- Scott
>
>"John P. Looney" <jploon...@online.ie> wrote in message
>news:38E87E5E...@online.ie...
>>
>> I've to write a simple server that will most likely have to be
>> *very* extensible in the future. I would like to present each
>> connecting client with a text parser, running in it's own thread.
>>
>> Then I thought of how AOLserver's "Admin port" works. You can
>> telnet in, give a username/password, and type TCL commands in
>> directly. This would be perfect for what I'd in mind, though AOLserver
>> is a tad complex to dive straight into.
>>
>> Does anyone have any pointers to a simple server that's implemented
>> like this, that I could cannabalize/learn from ?
>>
>> Kate
.
.
Scott, it occurs to me that Kate (John P. Looney?) might mean
"simple server" in a much different sense. Lots of us certainly
have written tiny little application servers that interpret Tcl
or some subset and wire the result back to clients. "Interact-
ing with other programs", chapter 7 of the M&M book <URL:http://
starbase.neosoft.com/~claird/comp.lang.tcl/tcl_books.html#effective>,
lays out such an example quite clearly.
--
Cameron Laird <cla...@NeoSoft.com>
Business: http://www.Phaseit.net
Personal: http://starbase.neosoft.com/~claird/home.html
John P. Looney
> Scott, it occurs to me that Kate (John P. Looney?) might mean
> "simple server" in a much different sense. Lots of us certainly
> have written tiny little application servers that interpret Tcl
> or some subset and wire the result back to clients. "Interact-
> ing with other programs", chapter 7 of the M&M book <URL:http://
> starbase.neosoft.com/~claird/comp.lang.tcl/tcl_books.html#effective>,
> lays out such an example quite clearly.
That's the one. I was hoping to write the "little server" in C,
calling something like Tcl_Main, and redirecting stdin/out to a
network socket, or a little more sensible, like doing the dirty
work of creating a tcl_interp myself, and calling Tcl_Eval myself,
on every line it gets in.
I was sorta hoping I could get away with out buying another book,
but if there it's the only place to get example C code, I suppose I
can shell out the few quid for the book!
Cameron Laird
John P. Looney <jploon...@online.ie> wrote:
>Cameron Laird wrote:
>> Scott, it occurs to me that Kate (John P. Looney?) might mean
>> "simple server" in a much different sense. Lots of us certainly
>> have written tiny little application servers that interpret Tcl
>> or some subset and wire the result back to clients. "Interact-
>> ing with other programs", chapter 7 of the M&M book <URL:http://
>> starbase.neosoft.com/~claird/comp.lang.tcl/tcl_books.html#effective>,
>> lays out such an example quite clearly.
>
> That's the one. I was hoping to write the "little server" in C,
>calling something like Tcl_Main, and redirecting stdin/out to a
>network socket, or a little more sensible, like doing the dirty
>work of creating a tcl_interp myself, and calling Tcl_Eval myself,
>on every line it gets in.
>
> I was sorta hoping I could get away with out buying another book,
>but if there it's the only place to get example C code, I suppose I
>can shell out the few quid for the book!
.
.
I think we're confusing each other. The *Effective
...* Chapter 7 shows how to write slick little clients
and servers in Tcl. If you want "example C code" ...
well, I can only conclude I've completely misapprehended
your goal.
If you favor the negative goal of not buying books, we
can make lots of other recommendations. My suggestion:
you tell us again what you're after.
John P. Looney
> > That's the one. I was hoping to write the "little server" in C,
> >calling something like Tcl_Main, and redirecting stdin/out to a
> >network socket, or a little more sensible, like doing the dirty
> >work of creating a tcl_interp myself, and calling Tcl_Eval myself,
> >on every line it gets in.
> >
> > I was sorta hoping I could get away with out buying another book,
> >but if there it's the only place to get example C code, I suppose I
> >can shell out the few quid for the book!
> .
> .
> .
> I think we're confusing each other. The *Effective
> ...* Chapter 7 shows how to write slick little clients
> and servers in Tcl. If you want "example C code" ...
> well, I can only conclude I've completely misapprehended
> your goal.
>
> If you favor the negative goal of not buying books, we
> can make lots of other recommendations. My suggestion:
> you tell us again what you're after.
Right, first off, I'll tell you want I've done so far, and then
what I want to do.
I've a threaded C program, that starts off a thread, which listens
for incoming connections on a arbitary port. When it gets a connection,
it starts off a "new_client" thread, which reads from, and writes to
the socket. At the moment, it's just printing out what's being read,
and doesn't write anything.
What I want to do now is create an instance of a TCL interpreter in
each "new_client" thread, where anything read from that port is sent to
the TCL interpreter, and what would otherwise be the interpreters output
is sent to the network port.
The idea being that you could do:
% telnet host 1239
set a 5
set b 6
expr $a + $b
11
exit
%
See ?
Cameron Laird
.
.
> Right, first off, I'll tell you want I've done so far, and then
>what I want to do.
>
> I've a threaded C program, that starts off a thread, which listens
>for incoming connections on a arbitary port. When it gets a connection,
>it starts off a "new_client" thread, which reads from, and writes to
>the socket. At the moment, it's just printing out what's being read,
>and doesn't write anything.
>
> What I want to do now is create an instance of a TCL interpreter in
>each "new_client" thread, where anything read from that port is sent to
>the TCL interpreter, and what would otherwise be the interpreters output
>is sent to the network port.
>
> The idea being that you could do:
>
>% telnet host 1239
>set a 5
>set b 6
>expr $a + $b
>11
>exit
>%
.
.
Oh.
I'll repeat: the M&M book shows a nice small example
of this in pure Tcl, and there are several other in-
stances around; it's a useful construction, and is
rebuilt at least monthly, from appearances.
You want something different, though. You already
have a client-server infrastructure in place, and you
just want to embed a Tcl interpreter within a thread
of your server. In one sense, that's even easier; all
you have to do is read up on embedding, and everything
else is already done.
There are abundant descriptions of embedding an inter-
preter in a thread; <URL:http://
starbase.neosoft.com/~claird/comp.lang.tcl/HowToC.html>
barely opens the subject.
Is there anything else you need?
John P. Looney
> You want something different, though. You already
> have a client-server infrastructure in place, and you
> just want to embed a Tcl interpreter within a thread
> of your server. In one sense, that's even easier; all
> you have to do is read up on embedding, and everything
> else is already done.
Exactly. I was stunned at how easy it was to get going!
> There are abundant descriptions of embedding an inter-
> preter in a thread; <URL:http://
> starbase.neosoft.com/~claird/comp.lang.tcl/HowToC.html>
> barely opens the subject.
>
> Is there anything else you need?
Well, I can't find any good tutorial on the subject on the web.
I've now gotten to the stage that I can telnet into the server,
and it creates an interpreter for each connection, and Tcl_Eval
seems to be working fine (though I've not taught it to redirect
stdout to the network, instead of the terminal - but I can almost
smell victory, I'm so close), though a "These are the functions
that you will find useful, and these are some things to watch
out for" is what I'm looking for now. I took one look at the
man page section, and went "Eeeek!". There are a lot of them!
John P. Looney
> Is there anything else you need?
Oh - I've worked out (I think) how to bind a network socket to
TCL's stdout/stderr;
tcl_sock = Tcl_MakeTcpClientChannel((ClientData) (&socket));
Tcl_RegisterChannel(interp, tcl_sock);
Tcl_SetStdChannel(tcl_sock,TCL_STDOUT );
Tcl_SetStdChannel(tcl_sock,TCL_STDERR );
But now, when I read from the socket, with read(), I get an "illegal
seek". Once a file descriptor is registered, as a Tcl_Channel am I
a Bad Monkey for going behind it's back, and using read() ?
The idea was that I wanted TCL's output to go straight to the
network port, but that I'd have some control over when stuff was
read, and then could pass it to TCL with Tcl_Eval later...
Cameron Laird
>> You want something different, though. You already
>> have a client-server infrastructure in place, and you
>> just want to embed a Tcl interpreter within a thread
>> of your server. In one sense, that's even easier; all
.
.
>I've now gotten to the stage that I can telnet into the server,
>and it creates an interpreter for each connection, and Tcl_Eval
>seems to be working fine (though I've not taught it to redirect
>stdout to the network, instead of the terminal - but I can almost
>smell victory, I'm so close), though a "These are the functions
>that you will find useful, and these are some things to watch
>out for" is what I'm looking for now. I took one look at the
.
.
Don't work so hard. You don't need to learn the
C API for the Tcl library; if you're happy with
your output, just return it as a value from
Tcl_Eval(), rather than [puts]ing it. Then your
existing C-coded server passes it back through
the wire, and you're done.
Paul Duffin
>
> Cameron Laird wrote:
> > Is there anything else you need?
>
> Oh - I've worked out (I think) how to bind a network socket to
> TCL's stdout/stderr;
>
> tcl_sock = Tcl_MakeTcpClientChannel((ClientData) (&socket));
> Tcl_RegisterChannel(interp, tcl_sock);
> Tcl_SetStdChannel(tcl_sock,TCL_STDOUT );
> Tcl_SetStdChannel(tcl_sock,TCL_STDERR );
>
Why do you want to catch stdout / stderr ?
Could you give an example of what Tcl script you want to run which
requires this redirection.
This method will reduce the amount of control that you have over how the
socket is used and could cause you some major problems.
> But now, when I read from the socket, with read(), I get an "illegal
> seek". Once a file descriptor is registered, as a Tcl_Channel am I
> a Bad Monkey for going behind it's back, and using read() ?
>
Certainly not recommended. Just like mixing fread and read, or fwrite
and write.
> The idea was that I wanted TCL's output to go straight to the
> network port, but that I'd have some control over when stuff was
> read, and then could pass it to TCL with Tcl_Eval later...
>
Why is that necessary ?
Once you have registered the socket as a channel in the interpreter you
just need to provide the Tcl code with access to the channel name and
then the rest of your code can be in Tcl.
If this does not fit in then use Tcl_Read to read what is on the socket
and call Tcl_Eval. You should then send the interpreter's result back
to the caller.
John P. Looney
>
> "John P. Looney" wrote:
> >
> > Cameron Laird wrote:
> > > Is there anything else you need?
> >
> > Oh - I've worked out (I think) how to bind a network socket to
> > TCL's stdout/stderr;
> >
> > tcl_sock = Tcl_MakeTcpClientChannel((ClientData) (&socket));
> > Tcl_RegisterChannel(interp, tcl_sock);
> > Tcl_SetStdChannel(tcl_sock,TCL_STDOUT );
> > Tcl_SetStdChannel(tcl_sock,TCL_STDERR );
> >
>
> Why do you want to catch stdout / stderr ?
>
> Could you give an example of what Tcl script you want to run which
> requires this redirection.
Well, it's not that I'm running a TCL script. The idea is that clients
would connect over the network, and write TCL to the server - that's how
the client/server would communicate. To put stuff to the server, you use
set or run a command with args that are saved on the server, and to get
data back, you run a command that would either return the data, or print
it to the screen. I've a feeling that writing what Tcl_Interp->result
would be safer (though how do you tell clients the TCL they gave
resulted
in a syntax error ?
> > The idea was that I wanted TCL's output to go straight to the
> > network port, but that I'd have some control over when stuff was
> > read, and then could pass it to TCL with Tcl_Eval later...
> >
>
> Why is that necessary ?
>
> Once you have registered the socket as a channel in the interpreter you
> just need to provide the Tcl code with access to the channel name and
> then the rest of your code can be in Tcl.
Well, I'd intended that the clients would write TCL code to the server.
It would have loads of specific TCL procs written in TCL and C that they
could call.
> If this does not fit in then use Tcl_Read to read what is on the socket
> and call Tcl_Eval. You should then send the interpreter's result back
> to the caller.
I tried that, and came up with this, which I think should work, but
dumps
core at "Tcl_Read()";
The new client gets passed a pointer to a "client" struct, and the
"sock" bit of the struct contains a working, socket.
typedef struct {
pthread_t thread;
struct sockaddr *net_addr;
int net_addr_len;
int sock;
Tcl_Channel tcl_sock;
} client;
void *new_client(void *myp)
{
static char buffer[255];
Tcl_Interp *interp = Tcl_CreateInterp();
client *me = (client *)myp;
ssize_t bytes_read;
me->tcl_sock = Tcl_MakeTcpClientChannel((void *) (&me->sock));
Tcl_RegisterChannel(interp, me->tcl_sock);
Tcl_SetStdChannel(me->tcl_sock,TCL_STDOUT );
Tcl_SetStdChannel(me->tcl_sock,TCL_STDERR );
Tcl_SetStdChannel(me->tcl_sock,TCL_STDIN);
Tcl_Write(me->tcl_sock, "% ", 2); // Nothing comes out the socket
fprintf(stderr, "New client, on %d\n",me->sock); // The server does
print this out
while ((bytes_read=Tcl_Read(me->tcl_sock,buffer,250))) { // Core
dump
buffer[bytes_read]=0;
Tcl_Eval(interp,buffer);
if( interp->result != NULL) {
Tcl_Write(me->tcl_sock,interp->result,strlen(interp->result));
Tcl_Write(me->tcl_sock,"\n", 1);
}
Tcl_Write(me->tcl_sock, "% ", 3);
}
fprintf(stderr, "Ending client on %d\n",me->sock);
close(me->sock);
Tcl_UnregisterChannel(interp, me->tcl_sock);
Tcl_Close(interp,me->tcl_sock);
Tcl_DeleteInterp(interp);
pthread_exit(NULL);
return NULL;
Paul Duffin
>
> Paul Duffin wrote:
> >
> > "John P. Looney" wrote:
> > >
> > > Cameron Laird wrote:
> > > > Is there anything else you need?
> > >
> > > Oh - I've worked out (I think) how to bind a network socket to
> > > TCL's stdout/stderr;
> > >
> > > tcl_sock = Tcl_MakeTcpClientChannel((ClientData) (&socket));
> > > Tcl_RegisterChannel(interp, tcl_sock);
> > > Tcl_SetStdChannel(tcl_sock,TCL_STDOUT );
> > > Tcl_SetStdChannel(tcl_sock,TCL_STDERR );
> > >
> >
> > Why do you want to catch stdout / stderr ?
> >
> > Could you give an example of what Tcl script you want to run which
> > requires this redirection.
>
> Well, it's not that I'm running a TCL script. The idea is that clients
> would connect over the network, and write TCL to the server - that's how
> the client/server would communicate. To put stuff to the server, you use
> set or run a command with args that are saved on the server, and to get
> data back, you run a command that would either return the data, or print
> it to the screen. I've a feeling that writing what Tcl_Interp->result
> would be safer (though how do you tell clients the TCL they gave
> resulted
> in a syntax error ?
>
The way I do this is that I have a proc called remoteeval which does [eval]
on the remote machine. The remote machine catches any errors and either
returns the result back to the client or returns the error plus the
errorInfo and errorCode. The remoteeval function either returns the result
or rethrows the error. This means that the remoteeval function behaves just
like eval.
> > > The idea was that I wanted TCL's output to go straight to the
> > > network port, but that I'd have some control over when stuff was
> > > read, and then could pass it to TCL with Tcl_Eval later...
> > >
> >
> > Why is that necessary ?
> >
> > Once you have registered the socket as a channel in the interpreter you
> > just need to provide the Tcl code with access to the channel name and
> > then the rest of your code can be in Tcl.
>
> Well, I'd intended that the clients would write TCL code to the server.
> It would have loads of specific TCL procs written in TCL and C that they
> could call.
>
The clients could only send Tcl, they could not send C code, unless you
intended to automatically compiler, link and load it..... ;-)
The functionality that you want can easily be written in Tcl, there are
certainly reasons you may not want to, such as needing to use the socket for
something else, or not using the Tcl event loop.
What are you trying to embed this code in ?
What you want is a Tcl RPC mechanism so maybe Tcl-DP would do what you want.
> > If this does not fit in then use Tcl_Read to read what is on the socket
> > and call Tcl_Eval. You should then send the interpreter's result back
> > to the caller.
>
> I tried that, and came up with this, which I think should work, but
> dumps
> core at "Tcl_Read()";
>
> The new client gets passed a pointer to a "client" struct, and the
> "sock" bit of the struct contains a working, socket.
>
> typedef struct {
> pthread_t thread;
> struct sockaddr *net_addr;
> int net_addr_len;
> int sock;
> Tcl_Channel tcl_sock;
> } client;
>
> void *new_client(void *myp)
> {
> static char buffer[255];
> Tcl_Interp *interp = Tcl_CreateInterp();
> client *me = (client *)myp;
> ssize_t bytes_read;
>
> me->tcl_sock = Tcl_MakeTcpClientChannel((void *) (&me->sock));
> Tcl_RegisterChannel(interp, me->tcl_sock);
Don't redirect these channels as that gives the Tcl script access to your
cocket which is not a good idea.
> Tcl_SetStdChannel(me->tcl_sock,TCL_STDOUT );
> Tcl_SetStdChannel(me->tcl_sock,TCL_STDERR );
> Tcl_SetStdChannel(me->tcl_sock,TCL_STDIN);
>
> Tcl_Write(me->tcl_sock, "% ", 2); // Nothing comes out the socket
You need to flush this out.
> fprintf(stderr, "New client, on %d\n",me->sock); // The server does print this out
> while ((bytes_read=Tcl_Read(me->tcl_sock,buffer,250))) { // Core dump
We need more information on the core dump.
You need to decide how you are detecting the end of the script, is it by having a
header which contains a length, or is it anytime you receive a complete script, or
is it whenever you get end of line, .... ?
> buffer[bytes_read]=0;
> Tcl_Eval(interp,buffer);
> if( interp->result != NULL) {
>
>
> Tcl_Write(me->tcl_sock,interp->result,strlen(interp->result));
> Tcl_Write(me->tcl_sock,"\n", 1);
> }
> Tcl_Write(me->tcl_sock, "% ", 3);
> }
>
> fprintf(stderr, "Ending client on %d\n",me->sock);
>
> close(me->sock);
> Tcl_UnregisterChannel(interp, me->tcl_sock);
> Tcl_Close(interp,me->tcl_sock);
> Tcl_DeleteInterp(interp);
> pthread_exit(NULL);
> return NULL;
> }
>
You seem to be making this much harder than is strictly necessary.
John P. Looney
> The way I do this is that I have a proc called remoteeval which does [eval]
> on the remote machine. The remote machine catches any errors and either
> returns the result back to the client or returns the error plus the
> errorInfo and errorCode. The remoteeval function either returns the result
> or rethrows the error. This means that the remoteeval function behaves just
> like eval.
Oh right. That's more for *just* remote execution though. I don't want
the
clients to have any TCL stuff there at all. It's all on the server.
> > Well, I'd intended that the clients would write TCL code to the server.
> > It would have loads of specific TCL procs written in TCL and C that they
> > could call.
> The clients could only send Tcl, they could not send C code, unless you
> intended to automatically compiler, link and load it..... ;-)
Oh, of course. The idea is that in another thread, I've some code that
sets up
and maintains some pretty complex data structures (it's a game server,
so the
internals of the game are stored there). The idea would be that I'd have
a few
basic TCL procs implemented in C to get at them, and then all the
manipulation
that the clients would do would be done in TCL.
> The functionality that you want can easily be written in Tcl, there are
> certainly reasons you may not want to, such as needing to use the socket for
> something else, or not using the Tcl event loop.
It's avoiding the TCL event loop I'm mainly doing this for. The idea
being that
thread handling the client's data can be working away, and periodically
running
select() on the socket, to see if there is anything there, if there is,
get it
with Tcl_Read(), and process it with Tcl_Eval.
> What are you trying to embed this code in ?
It's a game server. The idea being that I'm experimenting with the idea
of using
TCL as a data transport. It's probably silly, but I want to play with
it, see what
I can do, before scrappying it, and using something simple. <grin>
> What you want is a Tcl RPC mechanism so maybe Tcl-DP would do what you want.
Nah. There isn't any TCL on the clients. For instance, to authenticate
someone,
they would put "login username password" to their socket, the server
would read it,
run a proc called "login" on the server, and process stuff.
> > me->tcl_sock = Tcl_MakeTcpClientChannel((void *) (&me->sock));
> > Tcl_RegisterChannel(interp, me->tcl_sock);
> Don't redirect these channels as that gives the Tcl script access to your
> cocket which is not a good idea.
Oh. Right. So, when I run "puts", how do I send it to the socket ? If
there
is some sort of error, how can the client not get it, if it's not spat
back at it ?
Should I just not *ever* use puts, just "set" then ?
> > Tcl_Write(me->tcl_sock, "% ", 2); // Nothing comes out the socket
> You need to flush this out.
D'oh. I was using plain write() before hand, and it didn't need to be
flushed.
> > fprintf(stderr, "New client, on %d\n",me->sock); // The server does print this out
> > while ((bytes_read=Tcl_Read(me->tcl_sock,buffer,250))) { // Core dump
> We need more information on the core dump.
I had a look there. It's just seg faulting, not leaving a core dump. I
wonder is that
because it's using threads...
> You need to decide how you are detecting the end of the script, is it by having a
> header which contains a length, or is it anytime you receive a complete script, or
> is it whenever you get end of line, .... ?
I was going to go for an end of line. Seeing as Tcl_Read seems to go
for that anyway.
> You seem to be making this much harder than is strictly necessary.
It's because I'm not really sure how it should work, and I'm
learning...
Kate
John P. Looney
> You seem to be making this much harder than is strictly necessary.
I've reverted to a much more primitive:
void *new_client(void *myp)
{
static char buffer[255];
Tcl_Interp *interp = Tcl_CreateInterp();
client *me = (client *)myp;
ssize_t bytes_read;
int tcl_result;
net_write_line("% ", 2, me->sock,0);
fprintf(stderr, "New client, on %d\n",me->sock);
while ((bytes_read=net_read_line(buffer,250,me->sock ))) {
buffer[bytes_read]=0;
tcl_result=Tcl_Eval(interp,buffer);
if(tcl_result) {
snprintf(buffer, 50, "Error, code %d:", tcl_result);
net_write_line(buffer, strlen(buffer), me->sock,0);
}
if( interp->result != NULL) {
net_write_line(interp->result,strlen(interp->result),me->sock,1);
}
net_write_line("% ", 2, me->sock,0);
}
fprintf(stderr, "Ending client on %d\n",me->sock);
close(me->sock);
pthread_exit(NULL);
return NULL;
}
It works fine, so I'll keep it that way ;)
Paul Duffin
main reason has been that the information about what you are trying to do has
been supplied in little pieces and with each new piece your requirements have
changed. To begin with you wanted a telnet style session into which you could
log and use Tcl as your shell instead of bsh / ksh etc. and now it turns out
to be a games engine whose clients do not actually use Tcl and you only want
to use Tcl's syntax as the protocol for your clients to access the server.
You will find that you will get much more help if you give as much information
as possible up front because not only does this let us see what you are
trying to do it will also let us see what you want to do. Often there are
better ways to do both.
> Oh right. That's more for *just* remote execution though. I don't want
> the clients to have any TCL stuff there at all. It's all on the server.
Why not ?
Why not use Tk for your GUI (assuming you have one) ?
You will find that it is much easier to write your server and your client
if you use Tcl on both sides.
>
> Oh, of course. The idea is that in another thread, I've some code that
> sets up and maintains some pretty complex data structures (it's a game server,
> so the internals of the game are stored there). The idea would be that I'd have
> a few basic TCL procs implemented in C to get at them, and then all the
> manipulation that the clients would do would be done in TCL.
>
> It's avoiding the TCL event loop I'm mainly doing this for. The idea being that
> thread handling the client's data can be working away, and periodically running
> select() on the socket, to see if there is anything there, if there is, get it
> with Tcl_Read(), and process it with Tcl_Eval.
>
What other things is this thread doing ?
Why can it not just be done using the Tcl event loop anyway ?
e.g.
while (1) {
DoStuffForClient ();
Tcl_DoOneEvent ();
}
If you do it this way you do not have to worry about using select etc and can
use Tcl to do all the socket code for you using [fileevent] etc.
> > What are you trying to embed this code in ?
>
> It's a game server. The idea being that I'm experimenting with the idea of using
> TCL as a data transport. It's probably silly, but I want to play with it, see what
> I can do, before scrappying it, and using something simple. <grin>
>
For security reasons you should probably not just evaluate what your client is
sending you, you should check that it is one of the allowed procs first and then
evaluate it.
What format does your client want the results back in ?
Byron Servies
> > You seem to be making this much harder than is strictly necessary.
>
> I've reverted to a much more primitive:
Hi,
After reading this thread, I would still recommend you browse through the
Effective Tcl/Tk Programming book mentioned in the first couple of posts.
Chapter 7 has some really excellent examples of using tcl in client/
server applications and using tcl scripts as a protocol. I have used
these techniques successfully in a number of pure tcl, tcl to perl, and
embedded tcl applications.
<http://cseng.aw.com/bookdetail.qry?ISBN=0-201-63474-0&ptype=0>
An hour at the bookstore coffee shop would probably clear up what I
perceive to be a bit on confusion in your design; or at least give you
some alternative implementation ideas. Then again, I may just not
understand.
Using tcl to expand applications is one of the things it is really good
at, and it has been much simpler for me than this thread indicates it has
been for you.
Good luck,
Byron
Sent via Deja.com http://www.deja.com/
Before you buy.
John P. Looney
>
> I have found this conversation remarkably frustrating and I think that the
> main reason has been that the information about what you are trying to do has
> been supplied in little pieces and with each new piece your requirements have
> changed. To begin with you wanted a telnet style session into which you could
> log and use Tcl as your shell instead of bsh / ksh etc. and now it turns out
> to be a games engine whose clients do not actually use Tcl and you only want
> to use Tcl's syntax as the protocol for your clients to access the server.
Sorry about that. I was just trying to keep the information as succinct
as
possible, without waffling on for ages about what I'm trying to do.
What I wanted, to start with was a sort of network shell. Where
programs could
connect to a port, and be given the equvilant of tclsh. This is for a
game server
I'm writing.
I'll try and give you the "whole picture" this time.
The idea is that a server starts up, and starts a network thread that
listens on
a port for connections. When it gets one, it starts a new thread, which
has it's
own TCL interpreter. It accepts commands from the client, evaling them.
When a
client wants to update the player's status, it would send something
like;
player turn 90
to turn the player 90 degrees. It would then allow commands like;
info players names
to print a formatted list of players on the server.
I want people to be able to write their own TCL Macros, like
proc check_health { players } {
foreach { player } $players
if { [ info player $player health ] < 10 } {
message $me(name) "$player is down to 10 health"
}
}
}
And, then they could do something like
set team
cron 30 check_health [ info team $(me)team names ]
etc. Also, I could have TCL modules on the server for everyone to use.
Because
it's scriptable, it should make it easy to have AI players and the like
also.
> > Oh right. That's more for *just* remote execution though. I don't want
> > the clients to have any TCL stuff there at all. It's all on the server.
> Why not ?
> Why not use Tk for your GUI (assuming you have one) ?
Well, there is no reason why not I suppose, but that's for the people
writing
the clients to decide. I should really have said "I don't want the
clients to have
to have any TCL"...
> What other things is this thread doing ?
> Why can it not just be done using the Tcl event loop anyway ?
> e.g.
> while (1) {
>
> DoStuffForClient ();
>
> Tcl_DoOneEvent ();
>
> }
>
> If you do it this way you do not have to worry about using select etc and can
> use Tcl to do all the socket code for you using [fileevent] etc.
Oh right. I didn't know there was a "DoOneEvent()" function. Will that
allow me
to set timer events too ? (/me starts looking for his TCL book)
> > > What are you trying to embed this code in ?
> >
> > It's a game server. The idea being that I'm experimenting with the idea of using
> > TCL as a data transport. It's probably silly, but I want to play with it, see what
> > I can do, before scrappying it, and using something simple. <grin>
> For security reasons you should probably not just evaluate what your client is
> sending you, you should check that it is one of the allowed procs first and then
> evaluate it.
Well, I'll be using a crippled version of libtcl.a - removing stuff
like "exec" and
the like, for clients. I'll have to go through all the commands, and
decide "Do I
really need this" first. I suppose letting clients run commands on a
server is
usually asking for trouble.
> What format does your client want the results back in ?
Just ascii. It'll make debugging & stuff much simpler, as well as
making Perl/TCL
clients easier to write.
Cameron Laird
.
.
> Well, I'll be using a crippled version of libtcl.a - removing stuff
>like "exec" and
>the like, for clients. I'll have to go through all the commands, and
>decide "Do I
>really need this" first. I suppose letting clients run commands on a
>server is
>usually asking for trouble.
.
.
No, you do *not* "have to go through all the commands ..."
Others have been here before. Tcl supports a "safe inter-
preter" construct, which makes it easy for you to specify
the capabilities you want to expose to clients. Read `man
interp`.
John P. Looney
> > Well, I'll be using a crippled version of libtcl.a - removing stuff
> >like "exec" and
> >the like, for clients. I'll have to go through all the commands, and
> >decide "Do I
> >really need this" first. I suppose letting clients run commands on a
> >server is
> >usually asking for trouble.
> Others have been here before. Tcl supports a "safe inter-
> preter" construct, which makes it easy for you to specify
> the capabilities you want to expose to clients. Read `man
> interp`.
Good good. I'd a look at that man page - it seems that you use
the interp command to make sub-interpreters (slaves). At the
moment, the interpreters I'm setting up source an init.tcl file,
which loads any packages they need, etc. The
interp create -safe sandbox
command seems to do the trick...but from then on, when calling
"Tcl_Eval(...)", I'd have to run a command "command" as
interp eval sandbox "command"
yes ? Is there a way of making that safe interpreter the top
level one ? Something similar to the "exec" command in shell
scripts ?
Also, I tryed to load a package into a safe interpreter, but
encountered resistance:
% set in [interp create -safe one]
% interp expose $in load
% interp eval $in load ~/code/xenocide/server/tcl-sql/sql.so
can't use package in a safe interpreter: no Sql_SafeInit procedure
%
I would like to load packages, during the interpreter
initialisation, and then hide the load command, subsequently.
Cameron Laird
>encountered resistance:
>
>% set in [interp create -safe one]
>% interp expose $in load
>% interp eval $in load ~/code/xenocide/server/tcl-sql/sql.so
>can't use package in a safe interpreter: no Sql_SafeInit procedure
>%
>
> I would like to load packages, during the interpreter
>initialisation, and then hide the load command, subsequently.
.
.
Right. The idiomatic approach to this is to load a
package into the master interpreter, and then expose
as much or as little as you want of the package to
the slaves. At the expense of more file I/O, you
also can have the master teach the slaves how to do
their own loads--keep in mind that, whenever you en-
counter a
can't ... in a safe interpreter ...
you have the choice to define a specific proc for
that interpreter that performs that action.
Mark Harrison
>> You seem to be making this much harder than is strictly necessary.
You've got the basic idea right, but you've got some problems...
for example, if somebody sends you a string "exec rm *".
Fortunately, that can be fixed with just a page or two of code.
You need to create a "safe" interpreter which is responsible
for evaluating your program-specific commands, and which has
all the dangerous commands disabled.
Mark.
BTW, I agree with everybody who mentioned ch. 7 of the M&M book,
but I'm probably biased. ;-)
--
Mark Harrison ma...@usai.asiainfo.com
AsiaInfo Computer Networks http://www.markharrison.net
Beijing / Santa Clara http://usai.asiainfo.com:8080
todd...@my-deja.com
I am trying to find a good embedded scripting
language for use in my company's products. In
all of my searching, I can't seem to find enough
information to make the effort anything less than
a nightmare.
What I want to do:
1. Embed Tcl into a system which is based on an
RTOS. This RTOS follows a process model
something like "one process, many threads". That
is, each task shares the same heap, but have
individual stacks.
The scripting would pull together the entire
application, allowing me to configure it using
TCL.
2. Have multiple Tcl scripts running
simultaneously. Remember, all tasks share the
same heap but have individual stacks. Thus,
interpreter structures can be allocated per-task,
and the pointer to them can be kept on each
task's stack.
3. I have an unusual file system, so I can't use
anything like: "run script from file". I must be
able to read a line of the script (that is, ANY
statement, procedure declaration, expression,
etc.) from my file system, then evaluate that
line by a C call. Standard fopen,
fread/write/fgets, etc will not work here.
4. Tcl needs to use moderate resources. I have a
total of about 4 Mb of RAM to run from, but my
application uses approx 1MB (stacks, heap, etc)
5. I just want to be able to develop scripts on a
PC, download them to storage on the device, then
execute them by sending each line to the
interpreter. Tcl needs to then execute the
script.
Frustrations:
1. No one seems to have a concise listing of
which files (regardless of Tcl version) I need to
be able to run a minimal Tcl interpreter. (I
can't use sockets, etc.)
2. There also doesn't seem to be much reference
material for doing so...
Suggestions would be greatly appreciated!!
lvi...@cas.org
According to <todd...@my-deja.com>:
:1. No one seems to have a concise listing of
:which files (regardless of Tcl version) I need to
:be able to run a minimal Tcl interpreter. (I
:can't use sockets, etc.)
Tcl isn't designed in this way - instead of being designed in an
'ala carte' fashion, where one gets to pick and choose from a variety
of featers, Tcl is written in a fashion where all features are spread
all through all files.
:2. There also doesn't seem to be much reference
:material for doing so...
Doing 'so'? If you mean stripping out features, the way people have done
that in the path is going back, back, back in time and picking up ancient
versions (with all the bugs and missing features that implies) and moving
on from there...
--
<URL: http://dev.scriptics.com/>
<URL: mailto:lvi...@cas.org> <URL: http://www.purl.org/NET/lvirden/>
Unless explicitly stated to the contrary, nothing in this posting
should be construed as representing my employer's opinions.
Paul Duffin
> which files (regardless of Tcl version) I need to
> be able to run a minimal Tcl interpreter. (I
> can't use sockets, etc.)
>
The trouble with this is that the Tcl code is not really modularised in that
way and embedded systems are so different in terms of the trade offs they
make that it would be difficult to do in such a way as to satisfy everyone.
I would recommend that you simply take the latest code, look in tclBasic.c
at which commands you want to make available and which ones you don't and
then conditionally compile out those ones you don't want. You will then need
to go through the rest of the code conditionally compiling out those functions
which are no longer needed. You will also need to make sure that any
functions which Tcl relies on from libc are available, if not you will have
to emulate them yourself.
> 2. There also doesn't seem to be much reference
> material for doing so...
>
The trouble is that this sort of thing is very difficult to document unless
you actually do it, and embedded systems vary so much that what works for one
may not work for another.
> Suggestions would be greatly appreciated!!
>
You seem to have quite a lot of memory so you may not need to use a really
old version of Tcl. You could probably use version 8.0 with some success.
All I can say is allocate a lot of time for it, do it and then provide some
documentation on how you did it. Maybe you can organise a mailing list for
those people interested in embedding Tcl. If you have any suggestions on how
the structure of Tcl code could be changed to improve the process then send
them to Scriptics.
Tim Wilson
Paul> todd...@my-deja.com wrote:
>> I am trying to find a good embedded scripting language for use
>> in my company's products. [...] What I want to do:
>> 1. Embed Tcl into a system which is based on an RTOS.
I've done something like this a couple of times, once for Nucleus RTOS
and once for ThreadX RTOS.
My objective was to give the embedded application a simple
command-line for the purpose of automated tested by a connected PC.
>> 1. No one seems to have a concise listing of which files
>> (regardless of Tcl version) I need to be able to run a minimal
>> Tcl interpreter. (I can't use sockets, etc.)
My strategy was to delete all the OS-related stuff and all the
built-in commands, and then work iteratively until it linked. I then
have a bare command interpreter that uses nothing more than a few libc
routines (about 2000 non-comment lines). I then added back [info
commands] and my special-purpose commands.
I also removed two-thirds of the substitution logic: $ and [] are now
ordinary characters. The reason for this is that the test scripts run
(in Tcl) on my PC host, so I only send fully-substituted commands to
the embedded system. The host Tcl sends [info commands] to the
embedded system at the start and defines local proxies for all the
embedded commands, so it seems like your test scripts are running on
the PC (complete with loops, variables, nested commands, file system,
etc), but the embedded commands are actually executed at the bottom
level on the attached embedded system.
Paul> I would recommend that you simply take the latest code, look
Paul> in tclBasic.c at which commands you want to make available
Paul> and which ones you don't and then conditionally compile out
Paul> those ones you don't want.
I started from Tcl 7.4p3, to avoid the complexity of the object system
and byte-compiler.
Tim