The Birth and Death of Units

[Alexis King]

Hello,

Racket’s units predate its module system. Indeed, it would seem that
units were originally intended to *be* Racket’s module system, but
evidently, that did not work out. My understanding is their replacement
was due in large part to the desire for a more predictable macro
compilation model, so Racket’s phased module system is by necessity far
more statically bound than units were ever designed to be (if anything,
units seem to have been designed with the express purpose of dynamic
linking, which is hard to reconcile with compile-time macroexpansion).

Recently, however, I stumbled upon the paper in which the unit system
was initially presented, “Units: Cool Modules for HOT Languages” by
Flatt and Felleisen, and it outlines some interesting, reasonably
compelling motivations for dynamic linking of first-class modules.
However (though my memory may be imperfect here), I think I remember
Matthias mentioning at some point that units were one of the areas where
Racket “got it wrong”, and evidently whole-program construction via
units did not pan out. I’m curious about this, since units are still
distributed with Racket, so they clearly weren’t so wrong that they
needed to be relegated to the dustbin, and in theory they still have all
the interesting properties motivated by the original paper. Despite
that, units seem essentially unused in Racket except in a few small
places to allow modules with circular dependencies, not really taking
advantage of any of units’ additional expressive power.

My question to the list is therefore this: are units truly “dead”, or
do they have redeeming qualities? Are there some situations where units
might be preferable to a solution involving, say, first-class functions
or the object system? If there are useful applications for units, why
are they essentially unused in modern Racket programming? Are units
theoretically useful but lacking in some implementation property that
makes them practically viable? If so, is that practical flaw inherent
to the concept of dynamically-linked, first-class modules, or is it
merely a deficiency in Racket’s implementation?

Apologies for the meandering set of questions, but I am quite curious
to know more about this particular abstraction from the people who
experienced it, since its exploration long predates my exposure to
Racket.

Alexis

[Matthias Felleisen]

Units were the wrong approach to everyday modularity.They were, and they are, the correct solution to design problems such as those explained in our original paper on the idea.

Units suffer from a large notational overhead for everyday use. The second edition, due to Owens and Flatt, does a bit better with the introduction of 'linking inference’ but it was too little too ate. The other disadvantage of units concerns syntactic abstraction. You really want to parameterize modules over their implementation language. Shriram generalized units to unit/lang in his dissertation, but this proved too difficult to implement. Matthew proposed to make modules first-order, as opposed to first-class values, and this worked out well. I still use units on rare occasions, for example to mimic complex testing contexts and to deploy the same module-level abstraction _twice_ in the same program. These situations are rare, however.

Historical note. ML programmers got it similarly wrong when they pushed for “fully functorized” style. Once their compilation manager came online, they abandoned this style too and simply allowed the linker to connect structs as specified. I happened to be with these MLers for a year, which inspired me for the unit-like abstraction when we launched Racket/PLT Scheme. Functors are needed in the same situations as units but had less expressive power (recursive/dynamic linking).

— Matthias

> --
> You received this message because you are subscribed to the Google Groups "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to racket-users...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Alexis King]

Thank you for this explanation.

I consider myself largely an outsider looking in on module systems, but
I often hear SML and OCaml programmers complaining, wondering when
Haskell is going to “get a real module system”. I’d like to do right
thing in Hackett if I can, but I don’t really know what the right thing
is, since I have written precious little ML, and what I have written is
certainly not enough to get a feel for how modules affect large program
construction. It’s unfortunately difficult (perhaps impossible?) to get
an understanding of the usefulness of an abstraction without using it
oneself, so maybe I ought to just sit down and write some ML programs.

Alexis

> On Jan 21, 2018, at 1:26 PM, Matthias Felleisen

[Matthias Felleisen]

I think MLers complain about two things here:

— ML’s module system is basically a language that works at the type level. I think OldCamlers mean this mostly.
— ML’s module system is also a typed lambda calculus whose basic values are structs, aka, atomic modules. You can get a sense of this with units.

Haskell lacks both. There is a paper with Harper and Dreyer and a Haskell person that tries to connect Haskell with ML at the type level, but not comprehensively like the above.

Yes, it is difficult to evaluate an abstraction without having worked with them. Not even implementing them helps :-) Let me recommend an exercise:

work thru the Sendai paper with an SML that finally provides recursive functors.
https://www2.ccs.neu.edu/racket/pubs/#tacs94-cf
(This is the paper that inspired Units. I had coded everything in sml/nj when I figured out that you couldn’t tie the knot with functors. So I rewrote all of it with Scheme.)

— Matthias

[Jon Zeppieri]

There's also Backpack, which adds something similar to ML's module system* to Haskell, except at the package level: https://plv.mpi-sws.org/backpack/backpack-paper.pdf

* It's more similar to Rossberg & Dreyer's MixML than it is to Standard ML or OCaml's module systems.

> To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscribe@googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Racket Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscribe@googlegroups.com.

[Anthony Carrico]

Alexis, have you seen this?:

https://people.mpi-sws.org/~rossberg/1ml/

"ML is two languages in one: there is the core, with types and
expressions, and there are modules, with signatures, structures and
functors. Modules form a separate, higher-order functional language on
top of the core. There are both practical and technical reasons for this
stratification; yet, it creates substantial duplication in syntax and
semantics, and it reduces expressiveness. For example, selecting a
module cannot be made a dynamic decision. Language extensions allowing
modules to be packaged up as first-class values have been proposed and
implemented in different variations. However, they remedy expressiveness
only to some extent, are syntactically cumbersome, and do not alleviate
redundancy."

"We propose a redesign of ML in which modules are truly first-class
values, and core and module layer are unified into one language. In this
"1ML", functions, functors, and even type constructors are one and the
same construct; likewise, no distinction is made between structures,
records, or tuples. Or viewed the other way round, everything is just
("a mode of use of") modules. Yet, 1ML does not require dependent types,
and its type structure is expressible in terms of plain System Fω, in a
minor variation of our F-ing modules approach. We introduce both an
explicitly typed version of 1ML, and an extension with
Damas/Milner-style implicit quantification. Type inference for this
language is not complete, but, we argue, not substantially worse than
for Standard ML.

"An alternative view is that 1ML is a user-friendly surface syntax for
System Fω that allows combining term and type abstraction in a more
compositional manner than the bare calculus."

--
Anthony Carrico

[ben.rudgers]

To me, one of the redeeming qualities of units in Racket is that they make "the tent" bigger. Modules may usually be the right way in Racket, but the grace of Racket is that modules are not "The one true way" because the Racket community tends to eschew the OTW concept when it comes to computer languages (while tending to embrace it when it comes to teaching languages).

One way of putting it is that by linking Racket to ML, Units enrich the Racket ecosystem. Conceptually, Units provides an outbound pointer rather than an internal/self-referential one. Units provide greater diversity in the way I can think about programming and allow me to explore their abstractions within the Racket ecosystem in the same way Racklog does. If nothing else the Units documentation is good reading. Racket would be poorer without Units...but I don't have to maintain them.

Ben

[Paulo Matos]

Hello,

Let me revive this by adding a data point on the usage of units.
I have always thought in my mind units as allowing you to have a
`plugin` system where you can plugin additional functionality at runtime.

However, I had never really gotten my hands dirty. I decided to have a
got under the following premises:

1. I have a framework that allows one to run a user algorithm chosen at
runtime.
2. This framework is sent to a client as a binary, therefore he won't
have the original sources.
3. The user is free to implement a new algorithm, and the framework will
plug it in at runtime, run his algo and exit.

I don't think, but I am happy to be shown wrong, you can do this
_without_ units.

I have written a use case here:
https://github.com/LinkiTools/unit-use-case

The idea is to do the following:
1. Clone
2. raco exe -o run-algo framework/main.rkt
3. remove all other files from framework such that only run-algo remains
4. cd framework
5. `./run-algo -u fact 3` => 6
6. `./run-algo -u double 5` => 10
7. You could at this point distribute framework and public-sigs and get
clients without access to internal framework sources to create its own
algos. (I note that I have found that the bytecode lives inside the
executable, and therefore any keen client would possibly be able to
reverse engineer but I don't think that's an issue)

This doesn't quite work yet because for some reason it says algo-base%
in fact.rkt is not defined even though it's part of an import signature
(framework^). Does anyone understand why this is the case?

Also, Check Syntax crashes on main.rkt (but that doesn't block this from
working) -- Issue #1991.

Happy to have any comments on my use case for units.

Cheers,

Paulo Matos

--
Paulo Matos