Racket CS release plan

[Matthew Flatt]

With the improvements in the upcoming v7.8 release, Racket CS provides
all of the functionality of Racket BC (the current default
implementation). Also, end-to-end performance measurements
increasingly favor Racket CS. For example, Racket CS now builds a
distribution slightly faster and in slightly less memory than Racket
BC:

https://build-plot.racket-lang.org/

https://build-plot-cs.racket-lang.org/

It's time to consider shifting the Racket release to use Racket CS as
the default --- while Racket BC will remain an option for a long time
to come.

We (the release managers) propose the following rule to trigger the
switch to Racket CS in the default distribution:

Between this release and the next, if Racket CS testing and use
uncover no bugs that are more serious than ones typically discovered
for Racket BC, then Racket CS becomes the default for the next
release.

At the earliest, the switch would happen with the release *after* the
soon-to-be-released v7.8. Our expectation is that the switch would
happen with the next release (in October), but we'll see how that
expectation lines up with reality.

This rule is somewhat subjective, in that "more serious" and "typical"
are in the eye of the beholder, but we keep a close eye on the results
of the pkg-build service as well as Racket tests. We'll also count
performance problems as bugs, so Racket CS must not have substantial
known performance problems relative to Racket BC.

Jay, John, Matthew, Matthias, Robby, Ryan, Sam, and Vincent
(the release managers)

[wanp...@gmail.com]

I noticed that the size of the CS version is 244% compare to BS version. Wondering why it became so large. Does that mean Chez Scheme runtime/vm 100 MB larger than the original one?

Racket Mac OS X

      64-bit Intel 116.7 MB SHA1: 521b5a264afcfb3f390afacc682987268f650a25

Racket CS Mac OS X

      64-bit Intel 285.8 MB SHA1: 060f311fc6621c5797a62f98b743499fa4277793

https://pre-release.racket-lang.org/

Matthew Flatt 在 2020年7月30日 星期四下午8:46:42 [UTC+8] 的信中寫道：

[George Neuner]

On Fri, 31 Jul 2020 20:20:05 -0700 (PDT),
"wanp...@gmail.com"

<wanp...@gmail.com> wrote:

>I noticed that the size of the CS version is 244% compare to BS
>version. Wondering why it became so large. Does that mean Chez Scheme
>runtime/vm 100 MB larger than the original one?
>
>Racket Mac OS X
> 64-bit Intel 116.7 MB SHA1: 521b5a264afcfb3f390afacc682987268f650a25
>
>Racket CS Mac OS X
> 64-bit Intel 285.8 MB SHA1: 060f311fc6621c5797a62f98b743499fa4277793
>
>https://pre-release.racket-lang.org/

The CS version compiles to native code rather than portable bytecode,
so pretty much everything in the distribution is somewhat larger. It
adds up quickly.

George

[Bertrand Augereau]

I'm just adding noise, but I just wanted to thank you for sharing your
plans, it's much appreciated, even when we don't answer in general.

Yep, told you so. Noise :)

> --
> You received this message because you are subscribed to the Google Groups "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to racket-users...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/racket-users/20200730064636.29e%40sirmail.smtp.cs.utah.edu.

[Matthew Flatt]

That's still the best explanation I have, but I also think there must
be something more to it.

For example, the Chez Scheme boot files in uncompressed form add up to
about 8 times the size of compiled Racket BC executable, but Chez
Scheme doesn't have 8 times the functionality of the Racket BC
executable (so it should have 8 times as much machine code). The
machine code generated by Chez Scheme for its boot files is less
compact than machine code generated by GCC or LLVM for Racket BC's
implementation --- but, again, I don't think it's a factor of 8. So,
I'm optimistic that I've so far overlooked something that can make a
big difference.

I've concentrated more on understanding the difference in the run-time
memory footprints, and the difference there is not nearly so large.
Racket CS now sometimes has a smaller memory footprint than Racket BC
(e.g., peak memory use for a distribution build).

Matthew

[George Neuner]

Hi Matthew,

I don't know details of the Racket bytecode, but I'm assuming that it is
a mix of simpler operations that map directly to one or a few native
instructions, and more complex operations that compile to (the
equivalent of) a small function.  Probably some of these functions can
be inlined, but I expect there still would be some that can't.

To understand the difference in code size, you would need to look at the
lengths of bytecode instructions vs their native code equivalents, and
account for instances of those operations that can be inlined and for
calls to the functions for operations that can't.


As I said above, I don't know details of Racket bytecode, but a lot of
virtual machines use 8 or 16 bit opcodes and (allowing for immediate
operands) have average instruction lengths of 3..4 bytes. Contrast this
with, e.g., x86-64 code in which the average instruction is 5..6 bytes 
[larger if many instructions require length prefixes (e.g., 32-bit ops
in 64-bit code)].  The difference can add up very quickly.

George

[Gustavo Massaccesi]

The human friendly version of the bytecode is explained in https://docs.racket-lang.org/raco/decompile.html?q=decompile#%28mod-path._compiler%2Fdecompile%29 . The human friendly version looks similar to the expanded version of a program that you get with the Macro Debugger (when the macros are not hidden), but without the syntax scopes and with a few additional quirks.

(Note that the bytecode production has a few additional steps after expansion, for example inlining, constant propagation and folding.)

The disk version is more compact, but IIRC it is not documented.

Gustavo

--
You received this message because you are subscribed to the Google Groups "Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to racket-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/racket-users/c3324f5f-19cd-1e0f-3d83-08787474bede%40comcast.net.

[Sam Tobin-Hochstadt]

Note that Matthew's point was not about bytecode, but about the machine code in the Racket BC executable vs the machine code in the Chez kernel plus boot files. Especially if you look pre-7.0, there is very little bytecode in the Racket BC executable. 

Sam

To view this discussion on the web visit https://groups.google.com/d/msgid/racket-users/CAPaha9My_paizR3P_XWLkRshPmj4hy0m%2B%3DZ0mrjt-p9uXTdtYQ%40mail.gmail.com.

[George Neuner]

On 8/1/2020 3:48 PM, Sam Tobin-Hochstadt wrote:
> Note that Matthew's point was not about bytecode, but about the
> machine code in the Racket BC executable vs the machine code in the
> Chez kernel plus boot files. Especially if you look pre-7.0, there is
> very little bytecode in the Racket BC executable.
>
> Sam

Got it.  I missed the context when I read Matthew's message.

However, my comment about average code size remains:  If the Chez code
uses explicit length prefixes on all instructions that allow them (so as
to be unambiguous about intent) ... and the GCC code for the BC version
does not ... then the Chez code EASILY could be much larger.

George