ooRexx Tokenizer?
John
a compiler? I know IBM's old Object REXX had a compiler and I would think
that if the ooRexx Tokenizer was the same as a compiler it would be called
so. Am I correct in assuming that the tokenizer just converts the source
into some sort of "pre-interpreted" format (but not technically compiled),
to save the processor from doing this interpretation at invocation time?
Rick McGuire
it into a more easily interpreted form. The Object Rexx one does quite
a bit more. I generally refer to it as a "translator" rather than a
"tokenizer", but the term tokenization has hung around for historical
reasons. However, the program is still fundamentally interpreted rather
than executed from generated native code, so this definitely does not
qualify as a compiler. I've toyed with the idea of writing a
Just-In-Time compiler for Object Rexx, but those tend to be a pain to
port to new platforms, so it's pretty low on my priority list.
Rick
John
"Rick McGuire" <objec...@gmail.com> wrote in message
news:_%tLf.1064$KQ...@fe02.lga...
Rick, so is it just sort of, like, removing extraneous whitespace, comments
and things like that to tell the interpreter what's really important in
order to save the interpreter from having to do that at runtime, etc.? I'm
relatively new to programming and have used compilers before, but have not
heard of tokenizers...
Rick McGuire
For example, the instruction is generally decoded into an instruction
type code and a series of binary tokens that identify what the token is
(Rexx "symbol", quoted literal, operator, etc).
The Object Rexx translator uses a completely different approach,
creating a "self-executing" parse tree with a lot of information
determined at translation time rather than dynamically at runtime. For
example, builtin functions and function calls are all resolved at
translation time, so there's no searching required at runtime to figure
out where to go.
Rick
Lee Peedin
In addition to what Rick stated, rexxc is also an excellant way to
"pre-check" your code before placing it on-line. We edit most all our
code under ooRexx on Windows even if it is Classic Rexx code that will
run under OS/2. By running rexxc against the code, one can easily
find missing "parts" that will not pop up under Classic Rexx until
that portion of the code is executed.
However, my main use for it is to "hide" my source code from my
clients. I distribute the "tokenized" versions of my programs. One
thing to be aware of - the tokenized version will be considerably
larger than the raw code. Example: one of my program's size is
483,668 in raw code, but 1,579,786 tokenized.
Lee
John
"Lee Peedin" <lpeedinD...@nc.rr.com> wrote in message
news:qfluv1lpi3cmpiknl...@4ax.com...
Rick, Lee,
Thanks for all the answers, that clears it up.
John
In the OORexx Programming Guide, it says that the runtime can be distributed
via tokenized execs. Does this mean that if I tokenize a script anyone can
run it by just typing the scriptname on the commandline even if they don't
have OORexx itself installed? I assume I'd have to copy over subfacilities
like RxQueue, etc., if the scripts need those?
I remember making someone a script once with Reginald, but I had to copy the
'reginald.dll' into their path in order to run it, and I'm hoping to avoid
having to do that, if possible.
If this is not the case, would I just need to distribute certain .DLLs with
my scripts?
Rick McGuire
but you still need the full ooRexx installed to run it. This is the
rexxc process that allows him to distribute obfuscated programs. \
Rick
Lee Peedin
John,
The IBM release of Object Rexx included a "Runtime" version that would
only execute tokenized code. That is no longer necessary with ooRexx
since the interpreter is now FREE. That portion of the documentation
has been removed and will not be in future releases of ooRexx.
Lee