Racket-on-Chez status

[Matthew Flatt]

The Racket-on-Chez experiment is going mostly as expected. It moved a
while ago to a "racket7" repo:

https://github.com/racket/racket7

At this point, on non-Windows platforms, `raco setup` running on Chez
Scheme can get well into the set of installed packages before things go
too wrong. It's slow and uses a lot of memory, but many things have to
go right --- including filesystem access, I/O, delimited control,
thread and event scheduling, hash tables and weak references, writing
and reading code, number parsing, and initial FFI support --- to get
that far.

The "slow and uses a lot of memory" part is in compiling code and
loading compiled code. Programs tend to run faster and with less
additional memory after they are compiled and loaded. Before trying the
implementation and taking any performance results at all seriously, be
sure to see the performance notes in "racket/src/cs/README.txt".

The Racket macro expander currently runs slightly faster in Chez
Scheme. I think it should be even faster in Chez Scheme, so there's
still more to sort out there.

Single-precision floats, extended-precision floats, incremental GC, and
memory accounting are still off the table, for now. I think the other
big gaps are all closed, but there could be surprises. I'm currently
trying to get the Racket test suites to run, and that's going to be a
long process of debugging and filling in corners --- lots of small gaps
that add up.

Other work in progress:

- Gustavo is working on a compiler pass in Chez Scheme to eliminate
redundant run-time checks. Eliminating those checks turns out to be
especially important to remove the overhead of supporting chaperones
and impersonators (even for programs that don't use chaperones and
impersonators).

- Sarah is working on futures.


Here is a list of things that I know need to be done. I think enough
pieces are now in place that anyone who wants to contribute could pick
a bullet, coordinate through Slack on the "linklet" channel, and make
some progress. See the top-level "README.md" in "racket7" for
pointers.

- The printer is only partly implemented. I'll probably have to
implement more of it to make progress on the test suites.

- The FFI adapter doesn't yet support foreign functions. The next
step probably involves using Chez Scheme's FII's to get to
dlopen(), etc., to make an library-loading API that works the same
as Racket's. Racket's FFI is more dynamic than Chez Scheme's FFI
for constructing function types, but I think it may be ok for
Racket to use the `$`-prefixed primitive operations that Chez
Scheme uses to implement its FFI.

- Most I/O functionality is implemented, but it's slow for operations
like `read-byte` or `read-char`. Maybe it's a matter of creating
fast paths for simple cases.

- Unicode support at the level of character classifications and
string conversions needs to be sorted out. I don't know how well
Racket and Chez Scheme line up already, so this may be easy or
difficult.

- The path-parsing functions are incomplete, especially for Windows.

- Subprocesses are not yet implemented. This functionality belongs in
the "io" layer, and it's probably straightforward, since rktio and
the thread-scheduling layers are in place.

- File locks: ditto.

- Environment variables: ditto.

- TCP and UDP: ditto, but much larger.

- Adding `replace-evt` requires a light (hopefully) refactoring of
the current `sync` implementation.

- Evt and channel chaperones: probably easy, but there could be
surprises.

- Calling impersonated/chaperoned procedures was slow the last time I
checked. Probably the implementation needs faster paths for simple
cases.

- There's a basic bridge in place to convert Chez scheme errors into
Racket errors, but more work is needed to make error messages more
consistent.

- Places... I don't know.

- We should add a lambda-lifting pass to Chez Scheme, reflecting
Racket's different performance model for function definitions within
functions. A pass is currently in "schemify/lift.rkt", but it would
be better as a post-cp0 pass, and then the "core.sls" implementation
might benefit from the pass.

- We'll need cross-linklet optimization. I don't know how this will
go, but the existing cross-library optimization support in Chez
Scheme (see also Andy Keep's dissertation) is probably the starting
point.

- Wrapper executables need to be set up to replace `racket`, etc. I
don't know whether it's better for the wrapper to find `scheme` and
launch it or better for the wrapper to statically link to the Chez
Scheme implementation (as a replacement for the `scheme`
executable). The latter is probably necessary on Windows, where
there's no execve(). This task could tun out to be more of a
build-system problem than anything.

- Building and running doesn't work at all on Windows, yet.

[Juan Francisco Cantero Hurtado]

On 25/07/17 17:35, Matthew Flatt wrote:
[...]

>
> - Wrapper executables need to be set up to replace `racket`, etc. I
> don't know whether it's better for the wrapper to find `scheme` and
> launch it or better for the wrapper to statically link to the Chez
> Scheme implementation (as a replacement for the `scheme`
> executable). The latter is probably necessary on Windows, where
> there's no execve(). This task could tun out to be more of a
> build-system problem than anything.

Just a little comment.

I don't know what is the plan for the scripts in $HOME/.racket/*/bin,
but the scripts should call the interpreter in /usr. I hope you don't
want to install real binaries in $HOME for racket-minimal + raco
packages (like drracket).

On OpenBSD, we mark the racket binaries as "wxneeded" and the /usr/local
partition is mounted as "wxallowed". If you move the executable to
$HOME, it will not work because /home is not mounted as "wxallowed".



--
Juan Francisco Cantero Hurtado http://juanfra.info