Happy Module Day!

[Jay McCarthy]

Please join me in recognizing a historic day for Racket.

It has been 15 years since the creation of the Racket module system,
which was introduced to the world with the modesty typical of Matthew,
in a commit containing the single character, ".":

https://github.com/samth/old-plt/commit/79104357a355132eaf13fb3d8dd1d1a514414b53

From humble beginnings, module.c has gone from 340 lines to 12,729
lines today (about 2 lines per day). We have seen amazing features
like submodules and functions like
syntax-local-lift-module-end-declaration.

So much of what we now think of as essential to the Racket way would
not be possible without our unique and powerful module system and
phase-based compilation of it.

May I suggest a reading of "Composable and Compilable Macros: You Want
it When?" as an appropriate way to celebrate?

https://www.cs.utah.edu/plt/publications/macromod.pdf

Enjoy!

Jay

--
Jay McCarthy
Associate Professor
PLT @ CS @ UMass Lowell
http://jeapostrophe.github.io

"Wherefore, be not weary in well-doing,
for ye are laying the foundation of a great work.
And out of small things proceedeth that which is great."
- D&C 64:33

[Robby Findler]

Happy Module Day!

Robby

> --
> You received this message because you are subscribed to the Google Groups "Racket Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to racket-dev+...@googlegroups.com.
> To post to this group, send email to racke...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/racket-dev/CAJYbDakijD7u8o5fPvPVmysPrSByV7venBB3otqPVuOGW-a07A%40mail.gmail.com.
> For more options, visit https://groups.google.com/d/optout.

[Brian Adkins]

On Friday, December 11, 2015 at 7:31:25 AM UTC-5, Jay McCarthy wrote:
> From humble beginnings, module.c has gone from 340 lines to 12,729
> lines today (about 2 lines per day). We have seen amazing features
> like submodules and functions like
> syntax-local-lift-module-end-declaration.
>
> So much of what we now think of as essential to the Racket way would
> not be possible without our unique and powerful module system and
> phase-based compilation of it.

Excuse my ignorance of Racket internals & implementation, but I'm curious about why the module system is implemented in C vs. Racket. Can someone who is familiar with it provide some insight?

Thanks,
Brian

[Jay McCarthy]

On Fri, Dec 11, 2015 at 7:55 AM, Brian Adkins <lojic...@gmail.com> wrote:
> Excuse my ignorance of Racket internals & implementation, but I'm curious about why the module system is implemented in C vs. Racket. Can someone who is familiar with it provide some insight?

I'll do my best. A large number of features are implemented in C.
Generally, you could divide those features in C into the categories
"runtime", "virtual machine", and "compiler". (Don't read too much in
to this division because the compiler is available at runtime and they
are all primitive functions of the virtual machine.)

By virtual machine, I mean the execution of byte-code forms, like
making closures, doing garbage collection, making lists, and other
data structures, connecting to C, etc. By runtime, I mean useful
functions like filtering lists, converting numbers to do arithmetic,
printing to ports, etc. By compiler, I mean expanding syntax then
compiling it into byte-code.

Historically, I'd say that these are all together in C because our
macro system means the compiler has to run code inside of the virtual
machine and have access to all those useful runtime functions. (Other
things are in C because the FFI came "late" to Racket and we haven't
gone back and implemented things like IO with the FFI, although
Matthew did convert all of the GUI from C to Racket a few years ago.)

Presently it would be pretty reasonable to convert C runtime code to
Racket (to take a random example: a few years ago, `map` was moved[*]
from C to Racket and performance improved due to the JIT. [* = it is
actually still there but your Racket program's don't use it, just
other internal C code calls that map.]. It's just unclear whether
that's a good use of resources at this time. Matthew plans on porting
the syntax expander part of the compiler into Racket and another
graduate student is working on converting the compiler itself to
Racket.

It is pretty implausible at this point to convert the VM itself and
get rid of everything in the runtime. You might imagine that we could
make a compiler turn the bytecode directly into assembly, but we'd
still need to have a runtime written in something to do stuff like GC.
But few languages like Racket (i.e. that have a runtime) do anything
like that.

I hope this is helpful and I hope it is accurate,

[Robby Findler]

Well, the current "racket" repo is only a small part of what we
distribute. You might try that same thing on the standard
distribution.

Robby


On Fri, Dec 11, 2015 at 7:15 AM, Brian Adkins <lojic...@gmail.com> wrote:

> I ran the following commands on the latest repo:
>
> find . -name \*.c | xargs wc # 310,178 lines
> find . -name \*.h | xargs wc # 54,558 lines
> find . -name \*.rkt | xargs wc # 211,357 lines
>
> If those can be trusted, Racket consists of 364,736 (63%) C and 211,357 (37%) Racket. I was under the impression that the C code was in the minority, but I guess that's not the case.
>
> I suppose it makes little difference to me as an application programmer. It just makes me happy to have more of a language implemented in itself :)
>
> --
> You received this message because you are subscribed to the Google Groups "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to racket-users...@googlegroups.com.

[Brian Adkins]

On Friday, December 11, 2015 at 8:17:39 AM UTC-5, Robby Findler wrote:
> Well, the current "racket" repo is only a small part of what we
> distribute. You might try that same thing on the standard
> distribution.
>
> Robby

Interesting. When I download the "Unix Source" from racket-lang.org, I get:

.c = 308,651
.h = 54,424
.rkt = 43,544
.scm = 9,340

So 363,075 (87%) C and 52,884 (13%) Racket.

[Robby Findler]

I think something is wrong with your command. When I run "find . -name
\*.rkt | xargs wc" in my git checkout (that has the standard
distribution installed), I get 121200 lines. When I run just the "find
. -name \*.rkt" part and then sum up the numbers with a little racket
script, I get 2289056.

Robby

> --

> You received this message because you are subscribed to the Google Groups "Racket Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to racket-dev+...@googlegroups.com.
> To post to this group, send email to racke...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/racket-dev/3d15377e-cd9f-4ad9-9881-68f117417611%40googlegroups.com.

[Sam Tobin-Hochstadt]

I think you're counting all the package checkouts under extra-pkgs, and possibly other package installs as well. 45k sounds like what I remember for just the main collects directory.

Sam

You received this message because you are subscribed to the Google Groups "Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to racket-users...@googlegroups.com.

[Robby Findler]

Yes, that was the point (well, that and the point that something goes
wrong with xargs).

Brian: Sam and I don't agree on the facts. What I'm saying is
"Racket's main distribution consistss of a lot of Racket-implemented
goodies that, in many languages, would count as part of the compiler
(and much more beyond)." And the rest is about lies, damn lies, and
statistics. :)

Robby

[Matthew Flatt]

Jay's explanation sounds right to me.

Naturally, we've wanted to move more code out of C for a long time. I
think the pieces are finally moving into place so that it will really
start to happen over the next year or so (but it's always difficult to
predict).

FWIW, I'm puzzled by the behavior of `find . -name \*.rkt | xargs wc`,
but using

(for/fold ([n 0]) ([f (in-directory)]
#:when (regexp-match? #rx"[.]rkt$" f))
(+ n (length (file->lines f))))

on the v6.3 source distribution, I get

.c: 308,651 (where roughly 10% is a GC that we don't use anymore)
.h: 54,424
.rkt: 823,030

[Brian Adkins]

On Friday, December 11, 2015 at 9:06:16 AM UTC-5, Matthew Flatt wrote:
> Jay's explanation sounds right to me.
>
> Naturally, we've wanted to move more code out of C for a long time. I
> think the pieces are finally moving into place so that it will really
> start to happen over the next year or so (but it's always difficult to
> predict).
>
> FWIW, I'm puzzled by the behavior of `find . -name \*.rkt | xargs wc`,
> but using
>
> (for/fold ([n 0]) ([f (in-directory)]
> #:when (regexp-match? #rx"[.]rkt$" f))
> (+ n (length (file->lines f))))
>
> on the v6.3 source distribution, I get
>
> .c: 308,651 (where roughly 10% is a GC that we don't use anymore)
> .h: 54,424
> .rkt: 823,030

I think maybe an xargs limit is being reached or something - when I scroll up, I see another total line is displayed for .rkt, so it's:

779,431
43,544

For a total of 822,975 Racket lines. Sorry for the confusion.

[Benjamin Greenman]

FWIW, http://www.dwheeler.com/sloccount/ might be a better way to accomplish this once it supports Racket detection. I just opened a feature request for that at https://sourceforge.net/p/sloccount/feature-requests/20/ and am hoping there is interest.

You could also use sloc [1], which already supports Racket. Or just trust github's count (47.6% Racket, 34% C at [2])

[1] https://github.com/flosse/sloc

[2] https://github.com/racket/racket