Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

TCL Parser

76 views
Skip to first unread message

Aaron Deno

unread,
May 24, 2001, 2:06:55 PM5/24/01
to
Does anyone know where I can get my hands on lex- and yacc-like files for
writing my own TCL interpreter? If not, is the grammar spelled out
anywhere? I'm looking for something along the lines of Backus-Naur format.

Thanks in advance,
-Aaron


Gerald W. Lester

unread,
May 24, 2001, 4:16:54 PM5/24/01
to
Aaron Deno wrote:

> Does anyone know where I can get my hands on lex- and yacc-like files for
> writing my own TCL interpreter? If not, is the grammar spelled out
> anywhere? I'm looking for something along the lines of Backus-Naur format.

The Tcl interpreter source is free, readily available and extremely stable. Why
do you want to write your own?

--
**************************************************************************
* Gerald W. Lester | Voice: +1 (504)-889-2784 *
* Computerized Processes Unlimited, Inc. | FAX: +1 (504)-889-2799 *
* 4200 S. I-10 Service Rd., Suite 205 | E-Mail: Gerald...@cpu.com *
* Metairie, LA 70001 | Web: http://www.cpu.com *
**************************************************************************


Gerald.Lester.vcf

Aaron Deno

unread,
May 24, 2001, 5:38:16 PM5/24/01
to
As I mentioned, I'm looking for the full grammar including every TCL
procedure and its parameters spelled out in a standard (preferably BNF) way.
Why I need to write my own interpreter is a moot point. Believe me, I have
my reasons. If I could get my hands on the lex and yacc files, my task
would be greatly simplified.

-Aaron


"Robert Heller" <hel...@deepsoft.com> wrote in message
news:99073249...@cache.news.support.nl...
> "Aaron Deno" <aaronan...@yahoo.com>,
> In a message on Thu, 24 May 2001 11:06:55 -0700, wrote :
>
> "D> Does anyone know where I can get my hands on lex- and yacc-like files
for
> "D> writing my own TCL interpreter? If not, is the grammar spelled out
> "D> anywhere? I'm looking for something along the lines of Backus-Naur
format.
>
> Tcl has no 'grammar' as such. Just a *simple* syntax using these
> characters: {} " [] $ whitespace and \
>
> "D>
> "D> Thanks in advance,
> "D> -Aaron
> "D>
> "D>
> "D>
>
>
>
>
>
>
>
> --
> \/
> Robert Heller ||InterNet: hel...@cs.umass.edu
> http://vis-www.cs.umass.edu/~heller || hel...@deepsoft.com
> http://www.deepsoft.com /\FidoNet: 1:321/153


Pat McGuire

unread,
May 24, 2001, 6:23:17 PM5/24/01
to
I don't know why there is so much 'bristling' going on when a person asks a simple
question. Yes, Tcl is a wonderful language for doing a lot of things but believe
me, it can't do everything. So whether Aaron's reasons are for academic purposes
(as a assignment for example) or if its for what I needed a year or so ago; a
language that I can 'extend' beyond the capability of standard issue Tcl; is really
not very important. What is important is that a person needs a little help finding
some documentation and that's all that matters.
As for helping you Aaron, I never did find a BNF for the Tcl language; I ended up
'layering' my own extensions using lex and yacc and adding them to the base
language through the Tcl/C API. If you do find the technical documentation I would
sure like to see it.

Thanks,
Pat McGuire

Robert Heller wrote:

> "Aaron Deno" <aaronan...@yahoo.com>,
> In a message on Thu, 24 May 2001 11:06:55 -0700, wrote :
>

> "D> Does anyone know where I can get my hands on lex- and yacc-like files for
> "D> writing my own TCL interpreter? If not, is the grammar spelled out
> "D> anywhere? I'm looking for something along the lines of Backus-Naur format.

Cameron Laird

unread,
May 24, 2001, 6:40:06 PM5/24/01
to
In article <9ejv09$8...@news.or.intel.com>,

Aaron Deno <aaronan...@yahoo.com> wrote:
>As I mentioned, I'm looking for the full grammar including every TCL
>procedure and its parameters spelled out in a standard (preferably BNF) way.
>Why I need to write my own interpreter is a moot point. Believe me, I have
>my reasons. If I could get my hands on the lex and yacc files, my task
>would be greatly simplified.
.
.
.
BNF isn't natural for Tcl. Several of us have
threatened to shoehorn it into that form, but
never done it. The question certainly comes up
often enough ...

<URL: http://dev.scriptics.com/man/tcl8.4/TclCmd/Tcl.htm >
summarizes Tcl's syntax.

Tcl's commands are responsible for their own
syntax. <URL: http://mini.net/tcl/1019.html >
might interest you. The commands are decoupled
syntactically from the language proper, so your
question sounds funny.

I want to make sure we're being clear with you:
there *are* no Lex and Yacc sources for Tcl.
While there are several parsers for Tcl around,
it's roughly true that no one bothers maintaining
any of them except the one for the standard
distribution. It's always easiest simply to
work with it.

Would it satisfy you to write a Tcl parser in
Tcl? That's easy enough.
--

Cameron Laird <cla...@NeoSoft.com>
Business: http://www.Phaseit.net
Personal: http://starbase.neosoft.com/~claird/home.html

Pat McGuire

unread,
May 24, 2001, 6:29:33 PM5/24/01
to
I don't know why there is so much 'bristling' going on when a person asks a simple
question. Yes, Tcl is a wonderful language for doing a lot of things but believe
me, it can't do everything. So whether Aaron's reasons are for academic purposes
(as a assignment for example) or if its for what I needed a year or so ago; a
language that I can 'extend' beyond the capability of standard issue Tcl; is really

not very important. What is important is that a person needs a little help finding

some documentation and that's all that matters.
As for helping you Aaron, I never did find a BNF for the Tcl language; I ended up
'layering' my own extensions using lex and yacc and adding them to the base
language through the Tcl/C API. If you do find the technical documentation I would

sure like to see it.

Thanks,
Pat McGuire

Cameron Laird

unread,
May 24, 2001, 6:44:40 PM5/24/01
to
In article <3B0D7B27...@zootweb.com>,

Pat McGuire <pmcg...@zootweb.com> wrote:
>I don't know why there is so much 'bristling' going on when a person asks a simple
>question. Yes, Tcl is a wonderful language for doing a lot of things but believe
.
.
.

>not very important. What is important is that a person needs a little help finding
>some documentation and that's all that matters.
>As for helping you Aaron, I never did find a BNF for the Tcl language; I ended up
>'layering' my own extensions using lex and yacc and adding them to the base
>language through the Tcl/C API. If you do find the technical documentation I would
>sure like to see it.
.
.
.
I want to be clear: there is no "technical documentation"
for Tcl that includes its BNF grammar. I write this with
absolutely no hostility. It's absolutely fine to ask for
it; the answer, though, is that Tcl is different from other
languages, and the question makes little sense for Tcl.
This is the most helpful reply I know to make.

Jon Guyer

unread,
May 24, 2001, 6:54:33 PM5/24/01
to
In article <3B0D7B27...@zootweb.com>, Pat McGuire
<pmcg...@zootweb.com> wrote:

> I don't know why there is so much 'bristling' going on when a person
> asks a simple question.

Um... it would be helpful if you had quoted a message that actually
involved any "bristling".

--
Jonathan E. Guyer

<http://www.his.com/jguyer/>

Aaron Deno

unread,
May 24, 2001, 6:55:54 PM5/24/01
to
Thanks for your response! Those web pages look like VERY helpful
information.

Can you give me an example of TCL not fitting into BNF? I really only have
to implement a subset of TCL commands to acheive my goal, but I would need
to understand what you're talking about to evaluate whether I can get enough
functionality out of my interpreter.

-Aaron

"Cameron Laird" <cla...@starbase.neosoft.com> wrote in message
news:7AB89181657BA892.E58076B4...@lp.airnews.net...

Andreas Kupries

unread,
May 24, 2001, 5:34:18 PM5/24/01
to

"Aaron Deno" <aaronan...@yahoo.com> writes:

> Does anyone know where I can get my hands on lex- and yacc-like files for
> writing my own TCL interpreter? If not, is the grammar spelled out
> anywhere? I'm looking for something along the lines of Backus-Naur format.

Sorry, we don't have the grammer in BNF. See the Tcl.n manpage coming
with the source distribution for a description of the whole grammar.

Note that the tclpro project on sourceforge provides the tclparser
extension which makes the internal parser of the tcl core available to
tcl scripts as well.

--
Sincerely,
Andreas Kupries <a.ku...@westend.com>
Developer @ <http://www.activestate.com/>
Private <http://www.purl.org/NET/akupries/>
-------------------------------------------------------------------------------

Robert Heller

unread,
May 24, 2001, 7:55:54 PM5/24/01
to
"Aaron Deno" <aaronan...@yahoo.com>,
In a message on Thu, 24 May 2001 14:38:16 -0700, wrote :

"D> As I mentioned, I'm looking for the full grammar including every TCL
"D> procedure and its parameters spelled out in a standard (preferably BNF) way.
"D> Why I need to write my own interpreter is a moot point. Believe me, I have
"D> my reasons. If I could get my hands on the lex and yacc files, my task
"D> would be greatly simplified.
"D>
"D> -Aaron

Tcl is like LISP -- it effectly does not have a grammar. There are no
reserved words, etc. Tcl, like LISP is a prefix language. Yes, the
expr *function* uses infix notation, as does the if and while
*functions*. That is right, if and while (and proc) are just plain old
*functions*. It is possible to, for example, replace all of these
functions with other functions that work very differently.

There are no lex or yacc sources in the Tcl system (unless expr, if, and
while use something). There is probably not even lex sources for the
base Tcl reader code. And there most certainly does not really exist a
BNF notation of any of the Tcl system, AFAIK.

The Tcl 'interpreter' is a truly trivial one in many ways -- it is like
a LISP 'interpreter': the LISP eval and the Tcl eval are probably much
alike and neither look like what you are thinking of as an
'interpreter'. They take the 'car' (first element) of a list, do a
function lookup, then pass the rest of the list to the function found.
And that is all. The *rest* of the tclsh executable contains the C
coded functions behind all of the core functions, plus the system
interface stuff (I/O interface, event loop games, etc.).

Nobody is being hostile or anything, just stating facts.

From most 'traditional' programmer points of view (people coming from a
C, C++, Pascal, FORTRAN, Java, BASIC, Algol, Ada, etc. compiled
language world), Tcl and LISP are rather 'strange' languages -- neither
use a conventional type compiler -- LISP 'compilers' don't look anything
like the sort of compiler one would use for a language like C, C++,
Pascal, FORTRAN, Java, BASIC, Algol, Ada, etc. -- a LISP compiler for
example would never build a 'syntax tree', since a LISP expression is
already a *kind* if 'syntax tree'. So is a Tcl 'statement'. The Tcl
'byte compiler' probably looks much like a typical LISP compiler and
NOT like a sort of compiler one would write for C, C++, Pascal,
FORTRAN, Java, BASIC, Algol, Ada, etc. In other words, Tcl and LISP
don't fit the idea of 'compiler' that is typically taught in compiler
design classes.

In Tcl (like LISP), each function is responsible for its own 'syntax'
and any function can be re-defined. There are no reserved words and no
hard grammar rules. There is only the 'reader' syntax, which is a
simple set of rules about balanced and properly nested uses of {} [] () "
with the use of \ for escaping and $ for substitution and the use of
whitespace and semicolon as a *optional* EOS marker (as an alternative
to newline).

"D>
"D>
"D> "Robert Heller" <hel...@deepsoft.com> wrote in message
"D> news:99073249...@cache.news.support.nl...
"D> > "Aaron Deno" <aaronan...@yahoo.com>,


"D> > In a message on Thu, 24 May 2001 11:06:55 -0700, wrote :
"D> >

"D> > "D> Does anyone know where I can get my hands on lex- and yacc-like files
"D> for
"D> > "D> writing my own TCL interpreter? If not, is the grammar spelled out
"D> > "D> anywhere? I'm looking for something along the lines of Backus-Naur
"D> format.
"D> >
"D> > Tcl has no 'grammar' as such. Just a *simple* syntax using these
"D> > characters: {} " [] $ whitespace and \
"D> >
"D> > "D>


"D> > "D> Thanks in advance,

"D> > "D> -Aaron
"D> > "D>
"D> > "D>
"D> > "D>
"D> >
"D> >
"D> >
"D> >
"D> >
"D> >
"D> >
"D> > --
"D> > \/
"D> > Robert Heller ||InterNet: hel...@cs.umass.edu
"D> > http://vis-www.cs.umass.edu/~heller || hel...@deepsoft.com
"D> > http://www.deepsoft.com /\FidoNet: 1:321/153

Aaron Deno

unread,
May 24, 2001, 8:08:29 PM5/24/01
to
Ah, I understand.

Thanks, everyone, for your input. I see that I will have to approach
writing my interpreter much differently than I had originally anticipated,
but I think the information you have all given me will allow me to do it.

Thanks again!
-Aaron

"Robert Heller" <hel...@deepsoft.com> wrote in message

news:99074923...@cache.news.support.nl...

Don Porter

unread,
May 24, 2001, 10:29:12 PM5/24/01
to
Robert Heller <hel...@deepsoft.com> wrote:
> There are no lex or yacc sources in the Tcl system (unless expr, if, and
> while use something).

No, but there is generic/tclGetDate.y that defines the grammar used
by [clock scan] to make sense of various date and time formats.

--
| Don Porter Mathematical and Computational Sciences Division |
| donald...@nist.gov Information Technology Laboratory |
| http://math.nist.gov/~DPorter/ NIST |
|______________________________________________________________________|

David Gravereaux

unread,
May 25, 2001, 1:46:28 AM5/25/01
to
Andreas Kupries <a.ku...@westend.com> wrote:

>Note that the tclpro project on sourceforge provides the tclparser
>extension which makes the internal parser of the tcl core available to
>tcl scripts as well.

Yes, my thought exactly. I was scanning through the relies to see if someone
had mentioned this. If you want to parse Tcl code, use Tcl itself (the C
api)... Sounds reasonable to me.
--
David Gravereaux <davy...@pobox.com>

D. J. Hagberg

unread,
May 25, 2001, 1:40:33 AM5/25/01
to
In article <9ek3hu$b...@news.or.intel.com>, "Aaron says...

>Can you give me an example of TCL not fitting into BNF? I really only have
>to implement a subset of TCL commands to acheive my goal, but I would need
>to understand what you're talking about to evaluate whether I can get enough
>functionality out of my interpreter.

Sure. Take the incr command for instance in non-BNF man-page
grammar:

incr varname ?increment?

But say some programmer for some kooky reason wants to add
a -repeat switch to say how many times to perform the
increment (yes, this is a very silly example). The script
writer simply redefines the command:

rename incr _my_incr
proc incr {args} {
set argc [llength $args]
set cmd [list _my_incr]
if {$argc == 3 || $argc == 4} {
if {[string compare -repeat [lindex $args 0]]} {
error "Usage: incr ?-repeat count? varname ?increment?"
}
set repeatcnt [lindex $args 1]
lappend cmd [lindex $args 2]
if {$argc == 4} {lappend cmd [lindex $args 3]}
while {$repeatcnt > 0} {
incr $repeatcnt -1
uplevel 1 $cmd
}
} elseif {$argc == 2} {
lappend cmd [lindex $args 0] [lindex $args 1]
uplevel 1 $cmd
} elseif {$argc == 1} {
lappend cmd [lindex $args 1]
uplevel 1 $cmd
} else {
error "Usage: incr ?-repeat count? varname ?increment?"
}
}

So all of a sudden *at runtime* the grammar for the incr
command has changed. This is rather contrived, but it is
done, and not infrequently.

-=- D. J.

Donal K. Fellows

unread,
May 25, 2001, 5:55:03 AM5/25/01
to
Don Porter wrote:
> Robert Heller <hel...@deepsoft.com> wrote:
>> There are no lex or yacc sources in the Tcl system (unless expr, if, and
>> while use something).
>
> No, but there is generic/tclGetDate.y that defines the grammar used
> by [clock scan] to make sense of various date and time formats.

'Course, that's not exactly code to be all that proud of, given its
fairly-well-known problems (even before considering i18n issues...)

Donal.
--
Donal K. Fellows http://www.cs.man.ac.uk/~fellowsd/ fell...@cs.man.ac.uk
-- If somebody in a suit or a uniform can sit you down and have you believe
every word he says, you've just surrendered your license to be a thinking
human being and traded it in for a zombie suit. -- mh...@prince.carleton.ca

Donal K. Fellows

unread,
May 25, 2001, 5:50:14 AM5/25/01
to
Aaron Deno wrote:
> Can you give me an example of TCL not fitting into BNF? I really only have
> to implement a subset of TCL commands to acheive my goal, but I would need
> to understand what you're talking about to evaluate whether I can get enough
> functionality out of my interpreter.

You could fit Tcl *as normally written* into a BNF grammar, so long as that
BNF grammar was updateable at runtime and your lexer was sophisticated
enough (i.e. something remarkably like the standard Tcl parser!)

The problem with a standard static grammar for Tcl stems from the fact that
the language isn't static (technically, I believe it belongs to a different
class of languages - the context-sensitive languages - that cannot be
properly parsed by anything less than a Turing machine, and BNF-based
grammars are just not capable of recognising such things.) An example
might be a Tcl script that calls [package require Tk] and suddenly becomes
extended with a whole bunch of new commands with non-trivial syntaxes like:

button .b -text "foo bar" -command {puts boo}

Worse still, you can even imagine Tcl scripts that contain fragments from
some other language (e.g. (most?) database extensions let you write SQL,
and that's a completely different language.)

Still, you might be able to do something with an interesting subset of the
language.

Donal K. Fellows

unread,
May 25, 2001, 6:03:14 AM5/25/01
to
Pat McGuire wrote:
> I don't know why there is so much 'bristling' going on when a person asks
> a simple question. Yes, Tcl is a wonderful language for doing a lot of
> things but believe me, it can't do everything. So whether Aaron's
> reasons are for academic purposes (as a assignment for example) or if its
> for what I needed a year or so ago; a language that I can 'extend' beyond
> the capability of standard issue Tcl; is really not very important. What
> is important is that a person needs a little help finding some
> documentation and that's all that matters.

I don't see much "bristling". Tcl just doesn't have a (useful) BNF grammar;
you might be able to write something to handle the sorts of parsing
described in the Tcl(n) manpage, but that wouldn't tell you very much info,
particularly of the kinds that most people fiddle around with such grammars
to find out.

BNF has been a very successful way of building and describing grammars of
machine languages, but it isn't the only way of doing things. :^)

morgan mair fheal

unread,
May 25, 2001, 6:19:15 AM5/25/01
to
>The problem with a standard static grammar for Tcl stems from the fact that
>the language isn't static (technically, I believe it belongs to a different
>class of languages - the context-sensitive languages - that cannot be
>properly parsed by anything less than a Turing machine, and BNF-based
>grammars are just not capable of recognising such things.) An example
>might be a Tcl script that calls [package require Tk] and suddenly becomes
>extended with a whole bunch of new commands with non-trivial syntaxes like:

all nontrivial languages have a type 1 or 0 grammar but few language
definitions give anything formally beyond a type 2 or 3 grammar

so you can say tcl contains a very simple type 2 and then explain rest of
it with a type 0 grammar or verbosity or pointing to parser

the difference with c is how much you can load on type 2 grammar and how
much you have to define elsewise

Neil Madden

unread,
May 25, 2001, 7:36:58 AM5/25/01
to
Umm... what can't Tcl do? I always thought it was Turing-complete and
could do no more or less than any other complete language. Am I wrong to
think this? (genuine question - no 'bristling').
Also, what do you mean by a version of Tcl that you can 'extend'. Tcl is
the most extensible language I've ever come across, allowing you to
completely redefine the whole thing at runtime if you wished. Extensions
to Tcl can define whatever grammatical format they want. Could you
clarify?

--
--------------------------------------------------------------
Neil Madden. |
Personal: | Computer Science:
ne...@tallniel.co.uk | nem...@cs.nott.ac.uk
http://www.tallniel.co.uk | http://www.cs.nott.ac.uk/~nem00u
--------------------------------------------------------------

ldup...@videotron.ca

unread,
May 25, 2001, 8:49:39 AM5/25/01
to
In article <3B0E2DE2...@cs.man.ac.uk>, "Donal K. Fellows"
<fell...@cs.man.ac.uk> wrote:

> Pat McGuire wrote:
>> I don't know why there is so much 'bristling' going on when a person
>> asks a simple question.
>

> I don't see much "bristling".

Maybe this is the "bristle":

The Tcl interpreter source is free, readily available and extremely stable. Why
do you want to write your own?

L

Bob Techentin

unread,
May 25, 2001, 10:09:33 AM5/25/01
to
Pat McGuire wrote:
>
> "Gerald W. Lester" wrote:
>
> > Aaron Deno wrote:
> >
> > > Does anyone know where I can get my hands on lex- and yacc-like
> > > files for writing my own TCL interpreter? If not, is the grammar
> > > spelled out anywhere? I'm looking for something along the lines
> > > of Backus-Naur format.
> >
> > The Tcl interpreter source is free, readily available and
> > extremely stable. Why do you want to write your own?
>
> I don't know why there is so much 'bristling' going on when a person
> asks a simple question.

No animosity here, Pat. Most of the answers on this thread are pretty
straight forward answers. (Even if the answer is, "There is no such
thing.")

Gerald's statement is also perfectly reasonable. If your homework
assignment is to write a Tcl interpreter, then you've got to write one.
But if you're thinking of using Tcl or embedding it into your
application, then you're better off just grabbing the official sources
and using them.

There is, however, some confusion about free software licenses. I've
worked with people that wrote their own version of Tcl for a commercial
product because they didn't want any licensing complications. They had
heard about the "viral" nature of the GNU GPL, and didn't want to have
anything to do with it. But Tcl is NOT covered by the GPL.

The fact is, there are no license complications with Tcl. You can do
whatever you want with Tcl. Use it. Change it. Share it. Put it in a
commercial product. World domination. Whatever. It just works. It
doesn't make much business sense to create a new Tcl when you can just
use the existing stuff however you like.

Bob
--
Bob Techentin techenti...@mayo.edu
Mayo Foundation (507) 538-5495
200 First St. SW FAX (507) 284-9171
Rochester MN, 55901 USA http://www.mayo.edu/sppdg/

Cameron Laird

unread,
May 25, 2001, 10:36:44 AM5/25/01
to
In article <7AB89181657BA892.E58076B4...@lp.airnews.net>,

some geezer wrote:
> .
> .
> .
>BNF isn't natural for Tcl. Several of us have
.
.
.
"Unnatural parsing"--isn't that one of the actions
the US Federal government recently criminalized?

Kevin Kenny

unread,
May 25, 2001, 10:33:00 AM5/25/01
to
Don Porter wrote:
>
> Robert Heller <hel...@deepsoft.com> wrote:
> > There are no lex or yacc sources in the Tcl system (unless expr, if, and
> > while use something).
>
> No, but there is generic/tclGetDate.y that defines the grammar used
> by [clock scan] to make sense of various date and time formats.

And the maintainer of *that* is trying to get rid of it. 8^)

--

Aaron Deno

unread,
May 25, 2001, 11:43:12 AM5/25/01
to
> Umm... what can't Tcl do? I always thought it was Turing-complete and
> could do no more or less than any other complete language. Am I wrong to
> think this? (genuine question - no 'bristling').
Many new industry-standard EDA design tools use TCL as their input source.
i.e., to give commands to the tool, you write TCL scripts. Well, in this
case, straight TCL isn't enough. One nameless tool I am involved with had
to define some extra data types that TCL doesn't handle. On top of all of
this, I'm not try to RUN the TCL code, I'm trying to interpret it (and I
don't mean the compiler/interpreter definition of 'interpret'). I need to
be able to pick apart the code and rewrite small parts of it. This has to
be done in a script.

Jeffrey Hobbs

unread,
May 25, 2001, 12:06:59 PM5/25/01
to
Aaron Deno wrote:
...

> Many new industry-standard EDA design tools use TCL as their input source.
> i.e., to give commands to the tool, you write TCL scripts. Well, in this
> case, straight TCL isn't enough. One nameless tool I am involved with had
> to define some extra data types that TCL doesn't handle. On top of all of
> this, I'm not try to RUN the TCL code, I'm trying to interpret it (and I
> don't mean the compiler/interpreter definition of 'interpret'). I need to
> be able to pick apart the code and rewrite small parts of it. This has to
> be done in a script.

This is common enough. Since you've fit it into Tcl, the truly best
answer is to rely on Tcl itself to parse and understand this. If
you could provide a small example of the changes, others might be
able to recommend best and/or easiest techniques for you. Tcl
requires Tcl to parse itself...

--
Jeff Hobbs The Tcl Guy
Senior Developer http://www.ActiveState.com/
Tcl Support and Productivity Solutions

lvi...@yahoo.com

unread,
May 25, 2001, 12:49:57 PM5/25/01
to

According to Pat McGuire <pmcg...@zootweb.com>:
:I don't know why there is so much 'bristling' going on when a person
:asks a simple
:question.

:"Gerald W. Lester" wrote:
:> The Tcl interpreter source is free, readily available and extremely


:stable. Why
:> do you want to write your own?

:>

Pat, I have to agree - why did you bristle so much when Gerald asked the
simple question "Why do you want to write your own"? I mean, the first
thing I always try to do when asked a question is understand the _REAL_
problem the user is trying to solve. When people post questions which
show they had not yet encountered the fact that "Tcl is not based on a
BNF based parser, and there are no commonly available sources for
such things", people like to find out what problem is trying to be solved.

--
--
"See, he's not just anyone ... he's my son." Mark Schultz
<URL: mailto:lvi...@cas.org> <URL: http://www.purl.org/NET/lvirden/>
Even if explicitly stated to the contrary, nothing in this posting

lvi...@yahoo.com

unread,
May 25, 2001, 12:46:32 PM5/25/01
to

According to Aaron Deno <aaronan...@yahoo.com>:
:Does anyone know where I can get my hands on lex- and yacc-like files for

:writing my own TCL interpreter? If not, is the grammar spelled out
:anywhere? I'm looking for something along the lines of Backus-Naur format.

Tcl's grammer is defined in straight C code, not in lex and yacc. About
8 years ago someone wrote some lex and yacc a portion of the tcl
parser. If you can find some really old (say 1993) archives of clt
you might be able to locate those postings.

Neil Madden

unread,
May 25, 2001, 12:58:31 PM5/25/01
to

Hmm. But wouldn't the Tcl core still be the best place to start? The
source code is freely available, with a very open license, so you could
just hack it to meet your needs rather than starting from scratch.

Chang LI

unread,
May 25, 2001, 1:40:17 PM5/25/01
to
And somebody may prove that Tcl is not fit in the BNF.

Chang LI
Neatware

Larry Smith

unread,
May 25, 2001, 1:44:24 PM5/25/01
to
ldup...@videotron.ca wrote:

> Maybe this is the "bristle":
>
> The Tcl interpreter source is free, readily available and extremely stable. Why
> do you want to write your own?

I guess I can see why someone would take it as such.
One problem with the current implementation is that
it is not re-entrant - longjmps over the interpreter
to code that calls the interpreter again core dump.

--
.-. .-. .---. .---. .-..-. | Do not mistake my cynicism
| |__ / | \| |-< | |-< > / | for dispair for deep inside
`----'`-^-'`-'`-'`-'`-' `-' | me is an optimist forever
http://www.smith-house.org/ | asking, "Why not?"

Aaron Deno

unread,
May 25, 2001, 2:38:44 PM5/25/01
to
I'll take a look at it.


"Neil Madden" <nem...@cs.nott.ac.uk> wrote in message
news:3B0E8F37...@cs.nott.ac.uk...

Andreas Kupries

unread,
May 25, 2001, 2:23:08 PM5/25/01
to

"Aaron Deno" <aaronan...@yahoo.com> writes:

> As I mentioned, I'm looking for the full grammar including every TCL

> procedure and its parameters spelled out in a standard (preferably
> BNF) way.

Well, in that case you will have to read the manpages of the commands
you need as their syntax is described in these, albeit not in BNF.

You might also want to take a look at the "procheck" application
(based upon the "tclchecker" library), both again part of the "tclpro"
project at sourceforge. This is a static checker written in tcl and
using the "tclparser" extension I had mentioned before.

Checkers like this are of course limited due to the dynamic nature of
the language (-> rename, as already pointed out by others), but if you
don't use these features a static syntax can be written. Of course,
just using [proc] to define new commands is enough to extend the
language and thus the syntax :)

Andreas Kupries

unread,
May 25, 2001, 5:38:08 PM5/25/01
to

"Aaron Deno" <aaronan...@yahoo.com> writes:

> On top of all of this, I'm not try to RUN the TCL code, I'm trying
> to interpret it (and I don't mean the compiler/interpreter
> definition of 'interpret'). I need to be able to pick apart the
> code and rewrite small parts of it. This has to be done in a
> script.

I repeat myself here, but this definitely points to
procheck/tclchecker/tclparser as a starting point for you.

In other words, look how the tclchecker is implemented and rework it
so that it doesn't check the code but modifies it as defined by your
project specifications. Or write something new (maybe easier) using
the code of tclchecker as a guide.

morgan mair fheal

unread,
May 26, 2001, 12:36:42 AM5/26/01
to
In article
<E457C1635211A18B.20E333E6...@lp.airnews.net>,
cla...@starbase.neosoft.com (Cameron Laird) wrote:

>In article <7AB89181657BA892.E58076B4...@lp.airnews.net>,
>some geezer wrote:
>> .
>> .
>> .
>>BNF isn't natural for Tcl. Several of us have
> .
> .
> .
>"Unnatural parsing"--isn't that one of the actions
>the US Federal government recently criminalized?

backus was from the ibm of the bad old days (think microsoft but not so
warm and fuzzy)

i dont know where peter naur comes from

Mikael Sylvest

unread,
May 27, 2001, 4:12:24 AM5/27/01
to
warm and fuzzy)
>
> i dont know where peter naur comes from

From Denmark :-) see http://www.naur.com/

Richard.Suchenwirth

unread,
May 28, 2001, 5:31:36 AM5/28/01
to
> * From: David Gravereaux <davy...@pobox.com>

You can also build a Tcl parser in pure Tcl:
eval $script
parses and executes the contents of string 'script'. Hmm.. you didn't
want it to execute? Then just change the behavior of the problem
commands, both from "standard Tcl" and your specific extensions:
...
proc exit args {puts "was told to exit, but won't"}
proc exec args {puts "in real app, would have executed $args"}
proc explode args {puts "not exploding"}
...
Now the script is still executed, but does no harm.

Instead of the 'puts', you could insert all the code you like. Just make
sure your diverted procs match the originals in argument structure - but
to just use 'args' is bullet-proof in every case.
--
Schoene Gruesse/best regards, Richard Suchenwirth - +49-7531-86 2703
Siemens Dematic AG, PA RC D2, Buecklestr.1-5, 78467 Konstanz,Germany
Personal opinions expressed only unless explicitly stated otherwise.

Cameron Laird

unread,
May 28, 2001, 10:29:08 AM5/28/01
to
In article <mair_fheal-25...@c8.ppp.tsoft.com>,
morgan mair fheal <mair_...@www.yahoo.com> wrote:
.
.
.

>i dont know where peter naur comes from

Danish secretary of Algol 60 committee. There's
some controversy about "Backus-Naur Form" vs.
"Backus Normal Form", with Naur accused by differ-
ent people of both excessive modesty and excessive
immodesty.

Donal K. Fellows

unread,
May 28, 2001, 7:36:19 AM5/28/01
to
In article <3B121AF8...@kst.siemens.de>, Richard.Suchenwirth
<Richard.S...@kst.siemens.de> writes

>You can also build a Tcl parser in pure Tcl:
> eval $script
>parses and executes the contents of string 'script'. Hmm.. you didn't
>want it to execute? Then just change the behavior of the problem
>commands, both from "standard Tcl" and your specific extensions:
> ...
> proc exit args {puts "was told to exit, but won't"}
> proc exec args {puts "in real app, would have executed $args"}
> proc explode args {puts "not exploding"}
> ...
>Now the script is still executed, but does no harm.
>
>Instead of the 'puts', you could insert all the code you like. Just make
>sure your diverted procs match the originals in argument structure - but
>to just use 'args' is bullet-proof in every case.

The easiest way of doing the above is to use a separate interpreter.
The following code illustrates how to do it, but is just a *little* bit
on the extreme side (it doesn't write to variables or produce sensible
results, which can cause some problems...)

interp create helper
foreach cmd [helper eval {info commands}] {
helper hide $cmd
helper alias $cmd printinfo $cmd
}
proc printinfo args {
puts "INVOKED:$args"
}
helper eval $script

If you wanted to add execution as well, you'd modify the definition of
printinfo. I leave that as an exercise...

Donal.
--
Donal K. Fellows (at home)
--
FOOLED you! Absorb EGO SHATTERING impulse rays, polyester poltroon!!
(WARNING: There is precisely one error in this message.)

Donal K. Fellows

unread,
May 28, 2001, 7:21:31 AM5/28/01
to
In article <3B0E99F8...@smith-house.org>, Larry Smith
<la...@smith-house.org> writes

>I guess I can see why someone would take it as such.
>One problem with the current implementation is that
>it is not re-entrant - longjmps over the interpreter
>to code that calls the interpreter again core dump.

Arguably that's a feature! :^) :^)

(The real problem with longjmp() is that the intervening code has no
chance to clear up its temporaries; properly written code will let you
sling an error out through return-value signalling though, something
which is mapped at the Tcl level as its errors and other exceptional
conditions.)

Donal.
--

Someone Else

unread,
May 28, 2001, 5:57:42 PM5/28/01
to
cla...@starbase.neosoft.com (Cameron Laird) wrote:

:In article <mair_fheal-25...@c8.ppp.tsoft.com>,


:morgan mair fheal <mair_...@www.yahoo.com> wrote:
: .
: .
: .
:>i dont know where peter naur comes from
:
:Danish secretary of Algol 60 committee. There's
:some controversy about "Backus-Naur Form" vs.
:"Backus Normal Form", with Naur accused by differ-
:ent people of both excessive modesty and excessive
:immodesty.

When I first encountered computers (back in the '60s), it was "Backus
Normal Form". Somewhere along the line, it did indeed change.

Donal K. Fellows

unread,
May 31, 2001, 10:35:29 AM5/31/01
to
morgan mair fheal wrote:
> all nontrivial languages have a type 1 or 0 grammar but few language
> definitions give anything formally beyond a type 2 or 3 grammar
>
> so you can say tcl contains a very simple type 2 and then explain rest of
> it with a type 0 grammar or verbosity or pointing to parser
>
> the difference with c is how much you can load on type 2 grammar and how
> much you have to define elsewise

Following the discussion round here recently, I'd say that the *useful*
description of Tcl as most commonly used is probably in terms of a Type 1
grammar, though a fair number of its practitioners make strong use of the
fact that the line between code and data is thin-to-nonexistant and may
well produce programs that are Type 0. An interesting case to consider
is the [subst] command...

-- The small advantage of not having California being part of my country would
be overweighed by having California as a heavily-armed rabid weasel on our
borders. -- David Parsons <o r c @ p e l l . p o r t l a n d . o r . u s>

0 new messages