composite libraries (re-exporting all exports)
Ramana
library? are there any plans to include it in a future Scheme, or has
someone implemented it in R6RS somehow? are the good reasons to
disallow it?
here's the issue again - suppose I want to make a library like (rnrs):
In R6RS this is what I do:
(library (mylib)
(export about a hundred different identifiers)
(import (mylib base) (mylib arithmetic) (mylib colors) (mylib
elephants) (mylib balloons)))
and to use it I can just (import (mylib)) and I get colors and
elephants and everything. But note I had to type about a hundred
different identifiers.
One major maintanence problem is what happens when I add or remove
exports from the contentful libraries (colors, balloons, etc.). In one
of my programs I did (import (mylib base) (mylib colors)), and then
when I add a definition of red to the colors library, I can see it in
the program. But in another one of my programs I just did (import
mylib) because I knew mylib contains both colors and base, but since I
forgot to update the export list in the composite library (as well as
the export list in the colors library itself), I don't get to see red
in this program.
Requiring library writers to list all the exports in a composite
library discourages composite libraries. Is that the intention?
If composite libraries are bad, why do we have (rnrs)?
If listing all the identifiers on import or export is good, why are we
allowed to (import (rnrs)) and get all the identifiers without listing
them?
I think composite libraries are a great idea because they let you
avoid listing huge numbers of identifiers, and they let you
restructure a bunch of libraries someone else wrote to suit your
common usage better. An (export everything) export form would make
writing composite libraries much easier.
Abdulaziz Ghuloum
> so what is the status of the ability to export all definitions from a
> library? are there any plans to include it in a future Scheme, or has
> someone implemented it in R6RS somehow?
It cannot be implemented portably in R6RS.
> Requiring library writers to list all the exports in a composite
> library discourages composite libraries. Is that the intention?
I don't think so. It's went more like this: instead of people
thinking about these rather important issues, they went on and
on bickering about mutable strings and unicode, nil and #f, and
why we call programs script, and other silly issues. Libraries,
which were the single most important addition to the Scheme
standard, did not receive the attention they deserve in the
r6rs-discuss mailing list.
> I think composite libraries are a great idea because they let you
> avoid listing huge numbers of identifiers, and they let you
> restructure a bunch of libraries someone else wrote to suit your
> common usage better. An (export everything) export form would make
> writing composite libraries much easier.
Agreed. The syntax of libraries in R6RS is too restrictive.
All imports and exports have to be typed, literally, in the
library form. You cannot generate imports, exports, or other
libraries dynamically which is a shame. HOWEVER, Scheme
implementations can provide additional functionality that
complement R6RS without contradicting it. In this particular
issue that you bring up, Ikarus has its own (ikarus) library
which exports, among other things, an "import" keyword which
allows you to insert library imports not only at the library
top-level but also in any definition context. This is good
because it allows you to write macros that generate your
imports for you instead of having to type it all by hand.
Today, and since you brought this up, I have added two little
things to the (ikarus) library: (1) there is a procedure called
environment-symbols that takes an environment and returns the
names of all identifiers exported from that environment, and
(2) an internal (export ---) form that frees the ability to
export identifiers from the (library ---) form.
Together, environment-symbols and export allow you to define
composite libraries yourself, as well as do a bunch of other
tricks (globbing for example).
(library (rnrs-big)
(export) ;;; below
(import (ikarus))
(define-syntax import-and-reexport-all-from
(lambda (x)
(syntax-case x ()
[(ctxt impspec* ...)
(with-syntax ([(id* ...)
(datum->syntax #'ctxt
(environment-symbols
(apply environment
(syntax->datum #'(impspec* ...)))))])
#'(begin
(import impspec* ...)
(export id* ...)))])))
(import-and-reexport-all-from
(rnrs)
(rnrs eval)
(rnrs mutable-pairs)
(rnrs mutable-strings)
(rnrs r5rs)))
Aziz,,,
leppie
wrote:
Cool :)
Does this mean export can be placed anywhere a definition is expected
and the identifier will be exported as part of the containing library?
Cheers
leppie
Abdulaziz Ghuloum
> Does this mean export can be placed anywhere a definition is expected
> and the identifier will be exported as part of the containing library?
Yes. Well, (export x* ...) found in the library's top-level will cause
x* to be exported, but (export x* ...) in an internal definition inside
a library is ignored.
Minor correction to the code I posted above (in order to avoid conflicts
and to properly handle renaming, etc.):
(library (rnrs-big)
(export) ;;; below
(import (ikarus))
(define-syntax import-and-reexport-all-from
(lambda (x)
(syntax-case x ()
[(ctxt impspec* ...)
(cons #'begin
(map
(lambda (lib)
(let ([sym* (environment-symbols
(environment (syntax->datum lib)))])
(with-syntax ([lib lib]
[(id* ...) (datum->syntax #'ctxt sym*)]
[(t* ...) (generate-temporaries sym*)])
#'(begin
(import (rename lib (id* t*) ...))
(export (rename (t* id*) ...))))))
#'(impspec* ...)))])))
leppie
wrote:
Thanks, I meant library top-level :)
Cheers
leppie
PS: I already use something like: (environment-symbols (interaction-
environment)) in IronScheme's REPL, which is very handy for code
completion :)
Shiro Kawai
wrote:
> > Requiring library writers to list all the exports in a composite
> > library discourages composite libraries. Is that the intention?
>
> I don't think so. It's went more like this: instead of people
> thinking about these rather important issues, they went on and
> on bickering about mutable strings and unicode, nil and #f, and
> why we call programs script, and other silly issues. Libraries,
> which were the single most important addition to the Scheme
> standard, did not receive the attention they deserve in the
> r6rs-discuss mailing list.
Yeah it was full of bikeshed. For the record, though, I did find
an email from one of the editors that stated they dropped the feature
to keep the library syntax simple, when I was considering to make a
formal comment on it. Now I can't find the email, however, so this
may have been my misunderstanding or something.
Abdulaziz Ghuloum
> Yeah it was full of bikeshed. For the record, though, I did find
> an email from one of the editors that stated they dropped the feature
> to keep the library syntax simple, when I was considering to make a
> formal comment on it. Now I can't find the email, however, so this
> may have been my misunderstanding or something.
I thought I made a formal comment about dropping the phase
specification
from the export form. (glad that went through, otherwise, we would've
had one hell of a mess trying to make sense of exporting in one phase
an identifier that was imported for a different phase) Can't seem to
find it now. My comment about dropping the phasing specification from
the import clause was partly accepted though.
Another comment was about dropping the indirect-export clause by
making *all* library definitions implicitly exported. (This may have
been at the time when parts of R6RS were published as SRFIs, so, it
may not have been r6rs-discuss). That greatly simplifies writing
macros
that expand to references to private variables. Glad that was
adopted.
Aziz,,,
Ramana
library - thank you Aziz!
Does anyone know whether these facilities for composite libraries
might be included in R7RS? Is there discussion happening already to
compile a set of improvements to the library system (perhaps a
hierarchy of improvements - backwards-compatible, difficult for users,
difficult for implementors, difficult for everyone but better in the
end) that we can discuss incorporating when the next revision of the
standard comes around? (when might that be, by the way?)
Abdulaziz Ghuloum
> I'm very happy to see the functionality available in the (ikarus)
> library - thank you Aziz!
Sure thing.
> Does anyone know whether these facilities for composite libraries
> might be included in R7RS? Is there discussion happening already to
> compile a set of improvements to the library system (perhaps a
> hierarchy of improvements - backwards-compatible, difficult for users,
> difficult for implementors, difficult for everyone but better in the
> end) that we can discuss incorporating when the next revision of the
> standard comes around? (when might that be, by the way?)
I am not aware of any such discussion taking place, though the
RnRS authors may be scheming about it somewhere.
Aziz,,,
Ramana
> RnRS authors may be scheming about it somewhere.
who are the RnRS authors? is there a set for all n? if they vary, how
are they chosen?
Shiro Kawai
I found what I saw. There was a short exchange about exporting most
identifiers from a library at once. The mail says the feature
is dropped for simplification.
(follow the "references" link to get the entire context.")
http://srfi.schemers.org/srfi-83/mail-archive/msg00026.html
It may be reasonable to submit a SRFI for something like
import-and-reexport-from, so that other Scheme implementations agree
on the interface (and raise the possibility for its adoption to R7RS.)
The feature can be implemented as a preprocessor that processes the
extended library definition and generates R6RS-compliant library
defintion,
I think.
--shiro
andreu...@yahoo.com
wrote:
> Agreed. The syntax of libraries in R6RS is too restrictive.
> All imports and exports have to be typed, literally, in the
> library form. You cannot generate imports, exports, or other
> libraries dynamically which is a shame. HOWEVER, Scheme
> implementations can provide additional functionality that
> complement R6RS without contradicting it. In this particular
> issue that you bring up, Ikarus has its own (ikarus) library
> which exports, among other things, an "import" keyword which
> allows you to insert library imports not only at the library
> top-level but also in any definition context. This is good
> because it allows you to write macros that generate your
> imports for you instead of having to type it all by hand.
This would admittedly have been useful, but in their defense, I do not
blame the R6RS editors for leaving it out. If they had included it,
they would have had to explain the hygienic context of imported
identifiers, which (in one interpretation) requires understanding the
nontrivial meaning of datum->syntax. They would also have had to
explain that what is imported are identifiers and what is exported are
symbols (I think, since presumably two different but symbolically
equal identifiers would not clash as imports but would clash as
exports). And so on...
I'm not saying the problem cannot be solved in a reasonable way, just
that it may have added a little more complication to the
specification than one might initially think.
Andre
Ramana
> explain that what is imported are identifiers and what is exported are
> symbols (I think, since presumably two different but symbolically
> equal identifiers would not clash as imports but would clash as
> exports). And so on...
I believe both exports and imports are bindings, and you can't have
multiple bindings for the same identifier in either case. Importing
the same name twice with different bindings would be a problem unless
you renamed one of them. Importing the same name with the same binding
twice is fine. Exporting the same name with different bindings is not
allowed. Exporting the same name twice with the same binding is fine.
Does that make sense?
Ray Dillinger
> I believe both exports and imports are bindings, and you can't have
> multiple bindings for the same identifier in either case. Importing
> the same name twice with different bindings would be a problem unless
> you renamed one of them. Importing the same name with the same binding
> twice is fine. Exporting the same name with different bindings is not
> allowed. Exporting the same name twice with the same binding is fine.
> Does that make sense?
Somewhat, but only in a single-namespace world.
When names are lexically scoped, as in scheme, it seems
reasonable to me to import the same name with different
bindings, if you do it in different scopes.
The difficulty of actually doing so is probably why the R6RS
library export and import forms are so restricted as to keep
out of lexical scopes, even though the standard still makes
noise about (I paraphrase here) removing the restrictions
that make complex workarounds seem necessary.
I would expect an "import" clause at top level to behave as
you say, but I'd also expect an "import" clause inside a let
or lambda form to be able to shadow identifiers imported at
a higher lexical level with different values.
On the export side, things are different. A set of bindings
imported from a library should contain no name conflicts.
But is a library limited to exporting a single set of bindings?
Let's say we have a math library and it exports several sets
of bindings -- for argument's sake, 'quaternions, 'statistics,
'transcendental, 'randomnums, 'calculus, and 'accounting.
Each of these sets of bindings is exported from a lexical
location in the library code where all the exported values for
that set are visible. And if 'statistics and 'accounting both
contain something called 'make-entry, with different bindings,
it's incumbent on you to avoid importing both into the same
lexical scope.
Bear
Ramana
> reasonable to me to import the same name with different
> bindings, if you do it in different scopes.
>
> The difficulty of actually doing so is probably why the R6RS
> library export and import forms are so restricted as to keep
> out of lexical scopes, even though the standard still makes
> noise about (I paraphrase here) removing the restrictions
> that make complex workarounds seem necessary.
>
> I would expect an "import" clause at top level to behave as
> you say, but I'd also expect an "import" clause inside a let
> or lambda form to be able to shadow identifiers imported at
> a higher lexical level with different values.
>
> On the export side, things are different. A set of bindings
> imported from a library should contain no name conflicts.
> But is a library limited to exporting a single set of bindings?
> Let's say we have a math library and it exports several sets
> of bindings -- for argument's sake, 'quaternions, 'statistics,
> 'transcendental, 'randomnums, 'calculus, and 'accounting.
> Each of these sets of bindings is exported from a lexical
> location in the library code where all the exported values for
> that set are visible. And if 'statistics and 'accounting both
> contain something called 'make-entry, with different bindings,
> it's incumbent on you to avoid importing both into the same
> lexical scope.
So we both know R6RS does not have scoped or multiple imports or
exports. However they both sound like useful functionalities that we
should discuss and implement for the next version of the standard. I
believe (ikarus) exports (the R6RS way) 'import' which can then be
used locally. Does anyone know whether it behaves the way Bear
described? And what about packages of exports - any implementations,
or difficulties with such a concept?
Pascal Costanza
The nasty thing about "computed" imports and exports are the potential
nameclashes. Importing all identifiers from several modules can create
problems, especially when modules evolve over time and, for example,
deprecate functionality by removing identifiers, or add new ones. You
can thus create conflicts in unrelated third-party modules that you
aren't even aware of.
The only language that solved this issue at the root is Oberon, where
such nameclashes simply cannot occur. I strongly recommend to take a
look at that solution.
Pascal
--
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/
Aaron W. Hsu
>So we both know R6RS does not have scoped or multiple imports or
>exports. However they both sound like useful functionalities that we
>should discuss and implement for the next version of the standard. I
>believe (ikarus) exports (the R6RS way) 'import' which can then be
>used locally. Does anyone know whether it behaves the way Bear
>described? And what about packages of exports - any implementations,
>or difficulties with such a concept?
As someone has pointed out to me, there are some people who don't think
that these features have to come from the library form, and I admit that
I lean a little in that direction too.
Modules and Libraries can be considered two different things that can
co-exist quite well together. Rather than trying to complicate the
standard by addding in all these extra features like scoping, which
might not make sense for libraries, why not just let the implementations
develop module systems for that, which many already have?
Basically, why not keep the library form useful for only doing what it
was meant to do, which is to facilitate portable code sharing between
implementations. Rather than trying to make it more "featureful,"
let's remove complications from it. Meta levels would be a good place
to start.
On the other hand, I do agree that having the ability to create
composite libraries more easily would be a nice thing, so I would be in
favor of adding the option to include library names in the export form,
which would mean that all the library forms of that library would be
exported. This could be done simply enough.
Aaron Hsu
--
+++++++++++++++ ((lambda (x) (x x)) (lambda (x) (x x))) +++++++++++++++
Email: <arc...@sacrideo.us> | WWW: <http://www.sacrideo.us>
Scheme Programming is subtle; subtlety can be hard.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Abdulaziz Ghuloum
> So we both know R6RS does not have scoped or multiple imports or
> exports. However they both sound like useful functionalities that we
> should discuss and implement for the next version of the standard. I
> believe (ikarus) exports (the R6RS way) 'import' which can then be
> used locally. Does anyone know whether it behaves the way Bear
> described? And what about packages of exports - any implementations,
> or difficulties with such a concept?
Ikarus provides libraries, which are basically separate compilation
units. They are top-level and not scoped. They only import other
libraries, via the library's (import ---) subform, and such imports
define the language in which the library is written. The library's
exports are a subset of the identifiers that are bound in the library's
top-level (either by definition or by import). The exports can be
anything that can be defined: variables, macros, enumeration types,
record types, modules, etc. Library names look like lists of symbols,
and all library names must be globally unique.
Ikarus also provides modules (refer to Oscar Waddell's dissertation
for details). Modules are not separate compilation units: they are
compiled as part of the program in which the modules are defined.
Modules are lexically scoped: they inherit and contribute to the
scope of the context in which they are defined. A modules, like
a library, can export anything that's defined in the module body
including variables, macros, record types, and other modules. The
name of a module is an identifier (not a list) and therefore hygienic
macros can insert module definitions and module imports without fear
of name clashes.
A library can define and export multiple modules, each of which has
its own set of exports like in the following example:
(library (L)
(export a M0 M1)
(import (ikarus))
(define a 'a-of-L)
(module M0 (a)
(define a 'a-of-M0))
(module M1 (a M2)
(module M2 (a)
(define a 'a-of-M2))
(define a 'a-of-M1)))
#!scheme-script
(import (ikarus) (L))
(pretty-print
(list
a
(let () (import M0) a)
(let () (import M1) a)
(let () (import M1) (import M2) a)))
;;; prints (a-of-L a-of-M0 a-of-M1 a-of-M2)
There is no technical difficulty in mixing and matching libraries
and modules.
Aziz,,,
leppie
wrote:
Thanks again :)
I can now I have a (ikarus) compatibility library in IronScheme :-)
Cheers
leppie
Ramana
> co-exist quite well together. Rather than trying to complicate the
> standard by addding in all these extra features like scoping, which
> might not make sense for libraries, why not just let the implementations
> develop module systems for that, which many already have?
>
> Basically, why not keep the library form useful for only doing what it
> was meant to do, which is to facilitate portable code sharing between
> implementations. Rather than trying to make it more "featureful,"
> let's remove complications from it. Meta levels would be a good place
> to start.
I agree with you that modules and libraries together would be a fine
state of affairs, since they can serve different purposes. And I
happen to like the module systems of both Chez Scheme and Ikarus quite
a lot. However, the problem is that any library containing modules
won't be portable as source code. (Isn't that right? Would it possibly
be portable as compiled code? Probably not....) So the minimal
functionality we need for fine control over lexical scoping, whether
it be provided by libraries or modules, (and which is not available in
the R6RS library system), needs to appear in the standard to be
portable.
I also think we need to discuss opening either the library system, or
a new module system (probably based on an existing implementation) up
to the macro system (i.e. so macros can generate them - I think that
might already be the case with the two module systems I mentioned).
What would be the drawback of removing (for ...) meta level syntax
from the next revised report? Backwards compatibility? That should
really be an easy removal...
samth
Actually, the ability to specify meta-levels is a feature that at
least some of us think is important. For example, see Matthew Flatt's
paper 'Composable and Compilable Macros' (ICFP 2002), which presents
the PLT Scheme module system, and Ryan Culpepper's paper 'Advanced
Macrology and the Implementation of Typed Scheme' (Scheme workshop
2007), which details some advanced uses of the system.
In general, the interaction of modules, macros and libraries presents
hard technical problems, about which the Scheme community has not
reached anything resembling consensus. That's one important reason
why R6RS did not specify something more featureful.
sam th