Sealed as Source Annotation
Mark K. Gardner
I would like to start a philosophical discussion of the merits /
demerits of directly annotating the source code to seal classes (or
make them final) versus having the annotations be external to the
source. I am cross-posting to Dylan and Java newsgroups as those two
languages are the only ones that I am aware of that allow annotations
which declare that a class will not be extended.
As background, I am a long time lurker to Dylan (having obtained
Apple's printed DRM within a short time after announcement). I have
played with Thomas and Mindy and have built d2c but have not
programmed anything useful. Indeed I have done almost no programming
in Dylan at all. I have programmed in Java extensively and thus my
experience has been exclusively with Java's final class annotation.
Question: Why are internal (i.e., direct source) annotations better
than external annotations?
My experience with Java has indicated that internal annotations are a
problem. By declaring a class final so that it cannot be modified, we
are presupposing that the class is complete and needs no modification
or extension. Invariably, however, the class is not complete for all
uses and yet it cannot extended because it is final. The programmer's
only recourse is to duplicate the functionality of the finalized class
and then to extend it. This is clearly wasteful of the programmer's
time, as well as system resources. (As an example, there have been
discussions concerning the need to declaring Java's String class final
for efficiency reasons and the attendent limit on extensibility.)
It appears to me that declaring the class to be sealed outside the
source would allow extension where appropriate without "code reuse by
copying". Additionally, having the annotations external to the source
naturally leads to allowing specific instances to be sealed while
leaving others unsealed. Finally, the need for annotations should
decrease as compilers become better at determining for themselves what
should be sealed and what should not. From my perspective, external
annotations appear to complement smart compilers better than do
internal annotations. (By way of analogy, most modern C compilers
ignore the "register" keyword and determine for themselves which
variables should be kept in registers unless a compilation flag is
set.)
On the other hand, external annotations increase the complexity of the
build process by virtue of the fact that some of the information
necessary to determine the semantics of the source is not contained
within the source. In spite of this, I am leaning toward external
annotations over internal ones.
What are the counter arguments in favor of internal annotations?
Mark
--
Mark K. Gardner
RADIANT Team
Network Engineering, CIC-5
Los Alamos National Laboratory
P.O. Box 1663, M.S. D451
Los Alamos, NM 87545
Email: m...@lanl.gov
Phone: 1-505-665-4953
--
Bruce Hoult
wrote:
> I would like to start a philosophical discussion of the merits /
> demerits of directly annotating the source code to seal classes (or
> make them final) versus having the annotations be external to the
> source. I am cross-posting to Dylan and Java newsgroups as those two
> languages are the only ones that I am aware of that allow annotations
> which declare that a class will not be extended.
You will know, but I expect most Java programmers won't, that a sealed
class in Dylan *can* be extended, but only within the same compile unit.
> Question: Why are internal (i.e., direct source) annotations better
> than external annotations?
I think you have to ask yourself: "what is the purpose of having these
source annotations".
They might serve merely as a form of documentation of the programmer's
intent, or they might be to enable better/faster code to be generated.
In the case of Dylan, a large part of the reason for having "sealing" is
to enable more static, faster, code to be generated. In order to do
this, the compiler needs to know about the sealing while it is compiling
any code that uses that class. The greatest effect tends to come while
compiling code that is in the same compile unit as the class itself,
which means that the sealing has to happen in either the source file
containing the class, or else perhaps in the source file which contains
the import/export information for that module.
> My experience with Java has indicated that internal annotations are a
> problem. By declaring a class final so that it cannot be modified, we
> are presupposing that the class is complete and needs no modification
> or extension. Invariably, however, the class is not complete for all
> uses and yet it cannot extended because it is final. The programmer's
> only recourse is to duplicate the functionality of the finalized class
> and then to extend it. This is clearly wasteful of the programmer's
> time, as well as system resources. (As an example, there have been
> discussions concerning the need to declaring Java's String class final
> for efficiency reasons and the attendent limit on extensibility.)
Dylan avoids this problem to a large extent by allowing subclasses to be
declared in the same compile unit with the sealed class. So it is not
that the class has *no* subclasses, but that the list of possible
subclasses is fixed and known to the compiler (and in many cases is
empty).
Thus Dylan programmers only have this problem when they want to extend
someone *else's* sealed class, and in particular, if they want to extend
a sealed class declared in the runtime library, such as <integer> or
<single-float> or <byte-string>. Of course in Java there is no question
of being able to extend "int" or "float" because they aren't classes in
the first place.
There is no real way to get around the fact that you can't extend
<integer>. There are just too many places in compiled code that the
compiler takes advantage of the fact that <integer> values are of fixed
size, are immutable, and have no subclasses thus enabling the compiler
to use a smaller and faster unboxed representation -- in Java terms, it
is these properties of <integer> that allows the Dylan compiler to
automatically use "int" instead of "Integer" in many places in the
compiled code, and automatically and transparently switch between them
on assignment, passing function arguments, putting them into collections
etc.
What Dylan *does* do, is to provide unsealed base classes for all the
sealed classes in the runtime library. So <integer> is a subclass of
<number> and <byte-string> is a subclass of <string>. The user is free
to create new subclasses of <number> and <string>. If you write code
that has variables and function arguments declared as <number> and
<string> then you can pass objects of user-defined classes into them,
but your code will be slower than if you use <integer> and <byte-string>.
> It appears to me that declaring the class to be sealed outside the
> source would allow extension where appropriate without "code reuse by
> copying". Additionally, having the annotations external to the source
> naturally leads to allowing specific instances to be sealed while
> leaving others unsealed.
I'm sorry. I don't understand your intent. Do you propose that
particular *instances* of some class might be sealed/final, but that
other objects of that class will not be? How would that work? What
would it mean? As I understand it, sealed/final is an attribute of a
class, not of an object. [bindings can also be final (called "constant"
in Dylan), but that's an entirely different thing]
> Finally, the need for annotations should
> decrease as compilers become better at determining for themselves what
> should be sealed and what should not. From my perspective, external
> annotations appear to complement smart compilers better than do
> internal annotations. (By way of analogy, most modern C compilers
> ignore the "register" keyword and determine for themselves which
> variables should be kept in registers unless a compilation flag is
> set.)
I don't think that's a reasonable comparison. "register" is a very
local property. (Can you make a global variable "register"?)
In Dylan, at least, sealing is mostly useful because of libraries and
separate compilation [1]. If you compile the whole program at once then
the compiler can see all the classes and sealing declarations are (in
theory) unnecessary because the compiler can automatically seal
*everything*. When you compile a library, however, the compiler has no
way of knowing whether users of the library might want to extend classes
exported by the library, so a programmer declaration -- visible to the
compiler at the time the library is compiled -- is the only choice.
-- Bruce
[1] in Dylan it is also possible to create new classes at runtime, by
calling make(<class>, ...). Most programs don't do this, and most
programs are written such that the compiler can prove that they don't do
it.
Maury Markowitz
news:bruce-171C44....@news.akl.ihug.co.nz...
> You will know, but I expect most Java programmers won't, that a sealed
> class in Dylan *can* be extended, but only within the same compile unit.
As one that is similarily newbie to Dylan as the original poster, what
does htis "mean" in terms I may be more familiar with? Does "compile unit"
map onto "package" well?
> In the case of Dylan, a large part of the reason for having "sealing" is
> to enable more static, faster, code to be generated.
Why is this? I understand a little of the reasoning in Java, but much of
this is foreign to my Obj-C experience - the only language/runtime I feel I
have any basic grasp on it's inner workings. This information does what to
the compiler's output exactly? Does it increase dispatch times?
> Thus Dylan programmers only have this problem when they want to extend
> someone *else's* sealed class
Ahhh. Thus the basis for TOM, where _everything_ can be extended upon,
even compiled libs.
> is these properties of <integer> that allows the Dylan compiler to
> automatically use "int" instead of "Integer" in many places in the
> compiled code, and automatically and transparently switch between them
> on assignment, passing function arguments, putting them into collections
> etc.
Ahhh.
> What Dylan *does* do, is to provide unsealed base classes for all the
> sealed classes in the runtime library. So <integer> is a subclass of
> <number> and <byte-string> is a subclass of <string>. The user is free
> to create new subclasses of <number> and <string>. If you write code
> that has variables and function arguments declared as <number> and
> <string> then you can pass objects of user-defined classes into them,
> but your code will be slower than if you use <integer> and <byte-string>.
While this can often address the problem on the code side, it doesn't help
the tools side much. For instance, what if you have a IDE tool that
generates something using a byte-string, but you need to add to it? Or does
Dylan address this?
Maury
P T Withington
[...]
>> My experience with Java has indicated that internal annotations are a
>> problem. By declaring a class final so that it cannot be modified, we
>> are presupposing that the class is complete and needs no modification
>> or extension. Invariably, however, the class is not complete for all
>> uses and yet it cannot extended because it is final. The programmer's
>> only recourse is to duplicate the functionality of the finalized class
>> and then to extend it. This is clearly wasteful of the programmer's
>> time, as well as system resources. (As an example, there have been
>> discussions concerning the need to declaring Java's String class final
>> for efficiency reasons and the attendent limit on extensibility.)
>
> Dylan avoids this problem to a large extent by allowing subclasses to be
> declared in the same compile unit with the sealed class. So it is not
> that the class has *no* subclasses, but that the list of possible
> subclasses is fixed and known to the compiler (and in many cases is
> empty).
Also, Dylan permits open subclasses of sealed classes. This is a variation
on having a sealed subclass with an open base class. In either case, a
sophisticated compiler could use "copy down" to create both efficient
methods for the sealed domain and flexible methods for the open domain of
the class.
[...]
>> It appears to me that declaring the class to be sealed outside the
>> source would allow extension where appropriate without "code reuse by
>> copying". Additionally, having the annotations external to the source
>> naturally leads to allowing specific instances to be sealed while
>> leaving others unsealed.
>
> I'm sorry. I don't understand your intent. Do you propose that
> particular *instances* of some class might be sealed/final, but that
> other objects of that class will not be? How would that work? What
> would it mean? As I understand it, sealed/final is an attribute of a
> class, not of an object. [bindings can also be final (called "constant"
> in Dylan), but that's an entirely different thing]
It sounds like he might be asking that there be a link-time optimization
that could seal the class when I don't need to extend it, but otherwise
leave the class open?
I think I disagree with this idea, because I think part of the design (and
contract) of a class is whether you expect it to be extended or not. A
class that you design to be open requires much more engineering than a
sealed one.
P T Withington
wrote:
>> In the case of Dylan, a large part of the reason for having "sealing" is
>> to enable more static, faster, code to be generated.
>
> Why is this? I understand a little of the reasoning in Java, but much of
> this is foreign to my Obj-C experience - the only language/runtime I feel I
> have any basic grasp on it's inner workings. This information does what to
> the compiler's output exactly? Does it increase dispatch times?
As an extreme example:
define class <foo> () end;
define method frob(x :: <foo>) ... end;
define variable y :: <foo> ...;
frob(y);
will not do any dispatching at all. Because classes and methods are sealed
by default, and the compiler knows y is a <foo> and it knows there is only
one method for the generic function frob that applies to <foo>s, it can
compile the generic function call directly into a call to the method.
The reasoning in the compiler is more complex for classes with multiple
inheritance, open subclasses, generics with multiple dispatched arguments,
etc., but this gives you an idea of what optimization can be done with
sealed declarations (or conversely, the cost of an open declaration).
[Shameless plug: there is more information on this subject in the
Performance chapter of _Dylan_Programming_, available at
http://www.fun-o.com/products/books.phtml#dp.]
Maury Markowitz
news:B594BB23.8427%p...@callitrope.com...
> I think I disagree with this idea, because I think part of the design (and
> contract) of a class is whether you expect it to be extended or not. A
> class that you design to be open requires much more engineering than a
> sealed one.
While this might be true, it seems to put the crat before the horse. This
feature was (apparently) added to allow the compiler to do things, not for
the user to hide things.
Maury
Maury Markowitz
> will not do any dispatching at all. Because classes and methods are
sealed
> by default, and the compiler knows y is a <foo> and it knows there is only
> one method for the generic function frob that applies to <foo>s, it can
> compile the generic function call directly into a call to the method.
Ahhh.
But this being the case, I assume that "categories" can be applied only to
"open" things right? If so, that basically removes the entire basis for
them, which is one of the most wonderful tools I've used in any language.
Maury
Bruce Hoult
<maury@remove_this.sympatico.ca.invalid> wrote:
> "Bruce Hoult" <br...@hoult.org> wrote in message
> news:bruce-171C44....@news.akl.ihug.co.nz...
> > You will know, but I expect most Java programmers won't, that a sealed
> > class in Dylan *can* be extended, but only within the same compile
> > unit.
>
> As one that is similarily newbie to Dylan as the original poster, what
> does htis "mean" in terms I may be more familiar with? Does "compile
> unit" map onto "package" well?
"Compile unit". Literally, whatever unit of source code you compile at
one time, so that the compiler can see all the definitions at once and
make optomizations between mutually-dependent things.
In Dylan, a "library" consists of a number of source files, all of which
are read at the same time by the compiler (ok, it parses them all, and
then optomizes them all, and then generates code for them all) and which
produce e.g. a single DLL.
> > In the case of Dylan, a large part of the reason for having "sealing"
> > is to enable more static, faster, code to be generated.
>
> Why is this? I understand a little of the reasoning in Java, but much
> of this is foreign to my Obj-C experience - the only language/runtime
> I feel I have any basic grasp on it's inner workings. This
> information does what to the compiler's output exactly? Does it
> increase dispatch times?
Why? Because without it Dylan would run as slowly as Java while with it
Dylan runs as fast as C.
> > What Dylan *does* do, is to provide unsealed base classes for all the
> > sealed classes in the runtime library. So <integer> is a subclass of
> > <number> and <byte-string> is a subclass of <string>. The user is free
> > to create new subclasses of <number> and <string>. If you write code
> > that has variables and function arguments declared as <number> and
> > <string> then you can pass objects of user-defined classes into them,
> > but your code will be slower than if you use <integer> and
> > <byte-string>.
>
> While this can often address the problem on the code side, it doesn't
> help the tools side much. For instance, what if you have a IDE tool that
> generates something using a byte-string, but you need to add to it?
Add to it in what way?
You can't subclass <byte-string>, but you can of course add new methods
to it.
-- Bruce
P T Withington
wrote:
> "P T Withington" <p...@callitrope.com> wrote in message
> news:B594BB23.8427%p...@callitrope.com...
>> I think I disagree with this idea, because I think part of the design (and
>> contract) of a class is whether you expect it to be extended or not. A
>> class that you design to be open requires much more engineering than a
>> sealed one.
>
> While this might be true, it seems to put the crat before the horse. This
> feature was (apparently) added to allow the compiler to do things, not for
> the user to hide things.
I don't think it a matter of cart and horse. I think the concepts go
hand-in-hand. A philosophy behind the design of Dylan is that you should
pay only for what you use. Thus, the default is that classes are not
extensible and they are not externally visible. When either of these
features is needed, they must be requested. The explicit request alerts the
designer both to the additional runtime cost and the need for additional
engineering.
Mark K. Gardner
external annotations, I am seeking to understand the pros and cons in
abstract terms. As Dylan and Java are the only languages I am at all
familar with that contain the sealing/final class concept, I am asking
in that context. I seek enlightenment not a change to either language.
On Fri, 14 Jul 2000 12:31:06 +1200, Bruce Hoult <br...@hoult.org> wrote:
>You will know, but I expect most Java programmers won't, that a sealed
>class in Dylan *can* be extended, but only within the same compile unit.
Actually, I did not know this. As I said in my previous post, all I
know about Dylan came from reading (in 1992) Apple's 1992 book "Dylan:
an Object-Oriented Dynamic Language". (I have largely been a sporadic
lurker to Dylan since around 1994.) In Apple's book, the only mention
of sealing concerned sealing classes (pg 94):
seal /class/ => /class/
seal makes the /class/ read-only and, in addition, new subclasses
cannot be created from the /class/.
Thus marking a class as sealed *did* prevent one from subclassing. I
should read the latest DRM for a more up to date understanding. ;-)
>In the case of Dylan, a large part of the reason for having "sealing" is
>to enable more static, faster, code to be generated. In order to do
>this, the compiler needs to know about the sealing while it is compiling
>any code that uses that class. The greatest effect tends to come while
>compiling code that is in the same compile unit as the class itself,
>which means that the sealing has to happen in either the source file
>containing the class, or else perhaps in the source file which contains
>the import/export information for that module.
I will concede that separate compilation along with internal
annotations may yield better code optimization than without the
annotations but I am, as of yet, still undecided about the need for
separate compilation [1] and it isn't apparent to me that external
annotations won't achieve the same effect. Not having internal
annotations would allow reuse of the source in more projects than
would be possible with internal annotations due to not prematurely
declaring a class sealed.
[1] There are indications that separate compilation may be unnecessary
with today's machine resources. As one data point, the SmallEiffel
compiler does whole program source compilation in a fraction of the
time of other Eiffel compilers and seems to provide excellent code
optimization. From what I can tell, the speed of compilation is the
result of building an in-core representation of all the source files
before beginning compilation. Since Eiffel is a semantically rich
language, the SmallEiffel compiler may be representative of what could
be done with other languages if the separate compilation model is
abandoned.
BTW, complete source compilation allows the maximum amount of
information to be available so that a compiler can eliminate most of
the unnecessary dynamicism [2]. External annotations, supplied as part
of the nearly ubiquitous "project IDE approach" perhaps, could take
care of the rest without preventing the classes from being reused.
[2] The Concurrent Aggregates (CA) compiler of the Illinois Concert
system <http://www-csag.ucsd.edu/projects/concert.html> used whole
program analysis, concrete type inference, flow analysis, and call
site specific class/methond cloning to obtain performance comparable
to that obtained with C or Fortran from a fully dynamic language
without any annotations! (Actually, there were a few type annotations
for the methods of "primitive types", which were actually defined in
CA rather than assembly or C, but there were none for user defined
types.) The CA compiler was a research tool written in Lisp and was
not particularly fast or efficient. I believe many of the techniques
could be implemented in a production compiler without sacrificing
compilation speed if that was the goal. Alas, the project has been
completed (i.e., the students have all graduated) so I'm not sure we
will ever know for sure if such a compiler can be made faster.
The purpose of the above discussion is to point out that one does not
necessarily to have to separate compilation nor is it necessary to
have internal annotations to achieve good performance. As I believe
both the existing Dylan compilers, Gwydion and FD, assume separate
compilation, the point is moot as far as Dylan is concerned. The same
can be said for Java.
>> My experience with Java has indicated that internal annotations are a
>> problem. By declaring a class final so that it cannot be modified, we
>> are presupposing that the class is complete and needs no modification
>> or extension. Invariably, however, the class is not complete for all
>> uses and yet it cannot extended because it is final. The programmer's
>> only recourse is to duplicate the functionality of the finalized class
>> and then to extend it.
>
>Dylan avoids this problem to a large extent by allowing subclasses to be
>declared in the same compile unit with the sealed class. So it is not
>that the class has *no* subclasses, but that the list of possible
>subclasses is fixed and known to the compiler (and in many cases is
>empty).
>
>Thus Dylan programmers only have this problem when they want to extend
>someone *else's* sealed class, and in particular, if they want to extend
>a sealed class declared in the runtime library, such as <integer> or
><single-float> or <byte-string>. Of course in Java there is no question
>of being able to extend "int" or "float" because they aren't classes in
>the first place.
This is precisely when I have had difficulty with internal
annotations. If the writer of the library (package in Java or module
in the Dylan context) did not forsee a need to extend a class, I am
unable to reuse those bits that will work unchanged and to redefine
those bits that do not. (And sometimes, code that one writes must also
be treated as if someone else wrote it because changing it would
affect other projects which also use the code.) While it is possible
to remove internal annotation from the source and recompile, compilers
that assume internal annotations will now have less information to
perform optimizations. On the other hand, compilers that assume
external annotations can be given those annotations. Thus it appears
to me that external annotations allow better reusability and good
performance.
>> Additionally, having the annotations external to the source
>> naturally leads to allowing specific instances to be sealed while
>> leaving others unsealed.
>
>I'm sorry. I don't understand your intent. Do you propose that
>particular *instances* of some class might be sealed/final, but that
>other objects of that class will not be? How would that work? What
>would it mean? As I understand it, sealed/final is an attribute of a
>class, not of an object. [bindings can also be final (called "constant"
>in Dylan), but that's an entirely different thing]
My thinking on this point is still a bit abstract. Thinking back to my
experience with Concurrent Aggregates (see [2]), site specific
optimizations were very important in obtaining "maximum performance".
I was applying the experience out of context to what could be done if
annotations were external. It would be unwise to read more into it
than this.
>In Dylan, at least, sealing is mostly useful because of libraries and
>separate compilation [1]. If you compile the whole program at once then
>the compiler can see all the classes and sealing declarations are (in
>theory) unnecessary because the compiler can automatically seal
>*everything*. When you compile a library, however, the compiler has no
>way of knowing whether users of the library might want to extend classes
>exported by the library, so a programmer declaration -- visible to the
>compiler at the time the library is compiled -- is the only choice.
>
>[1] in Dylan it is also possible to create new classes at runtime, by
>calling make(<class>, ...). Most programs don't do this, and most
>programs are written such that the compiler can prove that they don't do
>it.
As I said earlier, I am not sure whether or not separate compilation
is sufficiently important to fix that dimension and thereby constrain
other choices in the program development search space. Even assuming
the need for separate compilation (in order to ship binary libraries,
for example), that still does not imply that annotations need be
internal. External annnotations would also suffice. (They might be
less convenient, however.)
Mark (who is awaiting your though provoking comments)
Mark K. Gardner
>I think that separate compilation is *vastly* overused. But it's still
>necessary, no matter *how* fast computers get. Do you want to have to
>compile all of Windows/MacOS/Linux/??? every time you compile "hello
>world", just because you're using the console I/O calls? Do you even
>want to have to recompile your SQL database every time you change a
>program that uses it?
I get your drift. It would be a pain to have to compile libc with
every application.
BTW, I would call Windows/MacOS/Linux part of the executable's run
time system. ;-)
Mark
Bruce Hoult
wrote:
> On Fri, 14 Jul 2000 12:31:06 +1200, Bruce Hoult <br...@hoult.org> wrote:
> >You will know, but I expect most Java programmers won't, that a sealed
> >class in Dylan *can* be extended, but only within the same compile unit.
>
> Actually, I did not know this. As I said in my previous post, all I
> know about Dylan came from reading (in 1992) Apple's 1992 book "Dylan:
> an Object-Oriented Dynamic Language". (I have largely been a sporadic
> lurker to Dylan since around 1994.) In Apple's book, the only mention
> of sealing concerned sealing classes (pg 94):
>
> seal /class/ => /class/
>
> seal makes the /class/ read-only and, in addition, new subclasses
> cannot be created from the /class/.
>
> Thus marking a class as sealed *did* prevent one from subclassing. I
> should read the latest DRM for a more up to date understanding. ;-)
Yes, a lot has changed since then. For example, sealing was originally
an operation, while it is now a declaration.
> >In the case of Dylan, a large part of the reason for having "sealing" is
> >to enable more static, faster, code to be generated. In order to do
> >this, the compiler needs to know about the sealing while it is compiling
> >any code that uses that class. The greatest effect tends to come while
> >compiling code that is in the same compile unit as the class itself,
> >which means that the sealing has to happen in either the source file
> >containing the class, or else perhaps in the source file which contains
> >the import/export information for that module.
>
> I will concede that separate compilation along with internal
> annotations may yield better code optimization than without the
> annotations but I am, as of yet, still undecided about the need for
> separate compilation [1]
I think that separate compilation is *vastly* overused. But it's still
necessary, no matter *how* fast computers get. Do you want to have to
compile all of Windows/MacOS/Linux/??? every time you compile "hello
world", just because you're using the console I/O calls? Do you even
want to have to recompile your SQL database every time you change a
program that uses it?
For many smaller libraries, I tend to agree that the process of
importing and linking and so forth can take as long as or longer than
simply compiling it each time. In fact this might not even be dependent
on computer speed, as systems ranging from the original Dartmouth BASIC
to the WATFOR and WATBOL (I think) compilers on the PDP-11 to Turbo
Pascal on CP/M were all fast systems that didn't do separate
compilation. Hell, Turbo Pascal even "recompiled" the program in order
to map an error location in the executable back to the source location!
> [2] The Concurrent Aggregates (CA) compiler of the Illinois Concert
> system <http://www-csag.ucsd.edu/projects/concert.html> used whole
> program analysis, concrete type inference, flow analysis, and call
> site specific class/methond cloning
This is something that is intended to be in Dylan. I believe that
Functional Developer implements a programmer-hinted version of method
cloning, and the Gwydion Dylan folks have in the (recent) past been
discussing the best way to approach it.
-- Bruce
Steve Rowley
From: P T Withington <p...@callitrope.com>
Subject: Re: Sealed as Source Annotation
Maury Markowitz maury@remove_this.sympatico.ca.invalid on 2000-07-14 12:45 wrote:
>> In the case of Dylan, a large part of the reason for having "sealing" is
>> to enable more static, faster, code to be generated.
>
> this is foreign to my Obj-C experience - the only language/runtime I feel I
> have any basic grasp on it's inner workings. This information does what to
> the compiler's output exactly? Does it increase dispatch times?
define class <foo> () end;
define method frob(x :: <foo>) ... end;
define variable y :: <foo> ...;
frob(y);
by default, and the compiler knows y is a <foo> and it knows there is only
one method for the generic function frob that applies to <foo>s, it can
compile the generic function call directly into a call to the method.
Even more: if frob is not exported from its module and is not used anywhere else
in the library, then the compiler knows there's only 1 call site, and can INLINE
the method body. Inlining in a language like Dylan can sometimes actually lead
to space REDUCTION (counter-intuitively), since that means method bodies can be
specialized to particular arguments, not just argument types. Very weird, very
useful.
(Now, if only you'd declared y to be constant, maybe it could be compile-time
evaluated, too... :)
Hugh Greene
from doesn't carry it. The following is getting pretty Dylan-specific in
any case but feel free, Mark, to forward it to c.l.j if you think they'd
want to read it.]
On 17 Jul 2000, Mark K. Gardner wrote:
> To more directly state my purpose in asking about internal vs.
> external annotations, I am seeking to understand the pros and cons in
> abstract terms. ...
Yes, I got this point :-) I may yet go back and reply to some of your
earlier messages but in any case I wanted to point out something an XML
wizard pointed out to me once. Maybe it'll be obvious to everyone but it
was a minor revalation to me at the time :-)
If your langauge allows both inline (internal) and stand-off (external)
markup, and you have a good enough (read-write) processor for it (and the
syntax is reasonably tractable, I suppose), then inline and stand-off
markup can be isomorphic. The XML(/SGML) community's terms for converting
between the two are "knitting" and "unknitting", IIRC. [Hmm ... some joke
about "knit one, Perl two" is lurking there, I'm sure ;-)]
This is ignoring the fact that running your source through some mutant
version of "lint" all the time in order to understand it may be
unacceptable -- but hey, Dylan would like to live in a world of wonderful
development environments, no ;-?
It also gets much more complicated, I'm sure, if you have the source and
annotations (text and markup) stored (at a given point in time, not
necessarily always) in stand-off form, and you change one but not the
other. Keeping track of what the "knitted" version looks like and means
could stretch your brain, if not your tools. And it gets even worse if
you have several different (and possibly not mutually exclusive) sets of
markup.
[BTW, that last is partly what this mechanism is for. In XML and SGML,
marked-up ranges have to nest -- they can't overlap. But if you want to
mark up different sets of information, the text ranges covered by each
set's tags might overlap. So you may have to use some linking convention
(like the W3C's XLink) to point at start/end points of each range for a
given semantic set from a separate file. Hope that makes sense -- I *am*
typing this at nearly midnight so it might not ;-)]
> On Fri, 14 Jul 2000 12:31:06 +1200, Bruce Hoult <br...@hoult.org> wrote:
> >...
>
> ... [In] Apple's 1992 book "Dylan: an Object-Oriented Dynamic Language".
> ... the only mention of sealing concerned sealing classes (pg 94):
>
> seal /class/ => /class/
>
> seal makes the /class/ read-only and, in addition, new subclasses
> cannot be created from the /class/.
BTW, it seems to me that Dylan already has (at least) three kinds of
annotation which may be
- external only: module/library exports (A recent thread mentioned
that the language designers did try to get an internal(-only?)
syntax/mechanism for this.)
- internal only: class sealing
- internal or external: domain sealing (Indeed, you can seal domains
over types which are solely from imported libraries, although this is
bad practice if your code is to be reused, as you don't know that some
other also-reused library won't do the same, which would be an error.
It's fine in not-for-reuse, "EXE-targeted" libraries, though, I
think.)
I wonder whether or not there are good semantic reasons for these cases,
or other reasons (good or bad), such as "syntactic simplicity", "we didn't
have time in the language design" etc.
> ... I am, as of yet, still undecided about the need for separate
> compilation [1] and it isn't apparent to me that external
> annotations won't achieve the same effect. ...
>
> [1] There are indications that separate compilation may be unnecessary
> with today's machine resources. As one data point [... SmallEiffel]
As a counter-point, we might want to get our lovely dynamic langauges into
resource-constrained devices such as set-top boxes, mobile 'phones etc.
So it might be nice to have the *option* to do *either*.
Another conceivable reason (dunno if it's a good one) for wanting separate
compilation is the ability to deliver a reusable "component" to someone
else, without source code. Note only does this prevent them changing your
code (or reading it to generate "bug-compatible" client code :-), it
may make it much harder for them to copy your implementation, if you're
supplying that component commercially. There may be various other,
non-performance-related reasons. As a veteran fence-sitter [I should show
you my war wounds -- no, maybe I should just get some sleep %-)] I'd like,
as I said, to have the option of either.
I suppose you might also usefully combine separate compilation and
external annotations to deliver components which can be used in a number
of ways with little or no extra compilation/linking/etc.
> The purpose of the above discussion is to point out that one does not
> necessarily to have to separate compilation nor is it necessary to
> have internal annotations to achieve good performance. As I believe
> both the existing Dylan compilers, Gwydion and FD, assume separate
> compilation, the point is moot as far as Dylan is concerned. The same
> can be said for Java.
I believe Fun-D allows whole-program compilation, although it may not
expose a (G)UI for it, or have a complete model of how to do it well.
I also know that at one point we talked about being able to impose
project-specific boundaries on which client libraries to recompile when
you recompile the top-level library of your project. You can think of
this as using delivered components which consist of multiple libraries.
This is intended to solve the "recompile Linux when recompiling
hello-world" problem mentioned elsewhere :-)
> >...
> >Thus Dylan programmers only have this problem when they want to extend
> >someone *else's* sealed class [whose source they can't recompile, for
> >some reason]
>
> This is precisely when I have had difficulty with internal
> annotations. If the writer of the library (package in Java or module
> in the Dylan context) did not forsee a need to extend a class, I am
> unable to reuse those bits that will work unchanged and to redefine
> those bits that do not. ...
It might well be that there are other ways to trade off efficiency against
extensibility and I'm glad to hear that people are researching them.
> >> Additionally, having the annotations external to the source
> >> naturally leads to allowing specific instances to be sealed while
> >> leaving others unsealed.
> >
> >I'm sorry. I don't understand your intent. Do you propose that
> >particular *instances* of some class might be sealed/final, but that
> >other objects of that class will not be? How would that work? ...
>
> My thinking on this point is still a bit abstract. Thinking back to my
> experience with Concurrent Aggregates (see [2]), site specific
> optimizations were very important in obtaining "maximum performance".
> ...
Ah, so you mean "specific instances" of function calls, not of classes,
right? (The latter might make sense in languages where you can, say, add
slots to individual instances -- which you could do in the Dylan described
in the Apple book you mention). If so, I can see why Bruce was confused.
> ...
> Mark (who is awaiting your though provoking comments)
I hope your thoughts are provoked; now I just need something
sleep-provoking ...
Good--- uh, morning :-)
Hugh
Jason Trenouth
wrote:
> If your langauge allows both inline (internal) and stand-off (external)
> markup, and you have a good enough (read-write) processor for it (and the
> syntax is reasonably tractable, I suppose), then inline and stand-off
> markup can be isomorphic. The XML(/SGML) community's terms for converting
> between the two are "knitting" and "unknitting", IIRC. [Hmm ... some joke
> about "knit one, Perl two" is lurking there, I'm sure ;-)]
>
> This is ignoring the fact that running your source through some mutant
> version of "lint" all the time in order to understand it may be
> unacceptable -- but hey, Dylan would like to live in a world of wonderful
> development environments, no ;-?
Heh. This sounds like it is related to 'literate programming' where one
maintains a single coherent document containing the many parts of a unit and
'unknits' it to compile source, produce documentation etc. No doubt some part
of the literate programming community is working with XML.
__Jason
hou...@acm.org
> Heh. This sounds like it is related to 'literate programming' where one
> maintains a single coherent document containing the many parts of a unit and
> 'unknits' it to compile source, produce documentation etc. No doubt some part
> of the literate programming community is working with XML.
Several people are, including myself. See
http://members.home.com/housel/projects.htm for some references and my
take on the problem.
Cheers,
-Peter S. Housel- hou...@acm.org http://members.home.com/housel/