Clarify project policy on racket2 syntax

464 views
Skip to first unread message

Atlas Atlas

unread,
Aug 11, 2019, 10:51:53 AM8/11/19
to Racket Users

I don't known how racket is managed. Can someone clarify for me the future of Racket.


Is abandoning s-expressions is sealed decision?


What chances that this will happen? 10% 50% 80% 100%?

Manfred Lotz

unread,
Aug 11, 2019, 10:58:29 AM8/11/19
to racket...@googlegroups.com
As far as I understood the discussion we had here the chance of
abandoning s-expressions is 0%.

The discussion was (and is presumably ongoing) about enhancing
Racket ...

--
Manfred

Atlas Atlas

unread,
Aug 11, 2019, 2:32:32 PM8/11/19
to Racket Users
I just see pretty broad discussion of new non s-expression syntax on GitHub with lots of code examples and even some draft specification.

And I just cannot understand whats going on.

воскресенье, 11 августа 2019 г., 17:58:29 UTC+3 пользователь Manfred Lotz написал:

Robby Findler

unread,
Aug 11, 2019, 2:47:20 PM8/11/19
to Atlas Atlas, Racket Users
Matthew posted an (IMO) clear explanation of the state of the thinking here earlier. tl;dr: sexpressions will never be abandoned and backwards compatibility with existing languages will be maintained for the foreseeable future.

... but read his message if you are worried.  I believe it is included in the github discussion you mention. 

Robby

--
You received this message because you are subscribed to the Google Groups "Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to racket-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/racket-users/6e41152a-970a-4c67-81b4-f57e303b26ce%40googlegroups.com.

Neil Van Dyke

unread,
Aug 11, 2019, 4:19:06 PM8/11/19
to Racket Users
Atlas, I get the impression, from the impassioned discussion thus far,
that a lot of the community really wants an s-expression syntax (and
print form for data) to be not merely backwards-compatibile supported,
*but to remain fully a "first-class citizen"*.

I suspect that the top-level requirements and the nature of community
involvement will become more clear in a few weeks, after everyone
returns from summer vacations, and has a chance to catch to catch their
breath and discuss.

Racket has approx. a two-decade history of good stability and
engineering, so I think we can have a little patience, while the various
confusing new things -- the Conservancy, community relationship and
process, Racket2, whatever happens now that the Chez backend is firming
up -- all get figured out and get solid starts.


(Relevant aside: I want to mention that I'm also hoping to see Honu, in
particular, promoted better in Racket.  Once I looked at the paper, the
other day, I was surprised that it hadn't really been promoted already
for practical use.  It's additional complexity beyond s-expressions, but
the approach looks clever.  I'll be interested to see how people use it
in practice.  That might start with a better Racket manual for Honu. 
And maybe one of the the textbooks ends up using Honu for a teaching
language.  And maybe someone shows a DSL construction tutorial using
Honu to do the parsing work.  That doesn't have to mean Honu bumps
s-expression to "second-class citizen" status, such as in Racket manuals
or ongoing development.  Nor does that have to mean that Honu becomes
the form that might be displayed when data and syntax objects, such as
when viewing the intermediate representation of Racket as a
language-oriented programming platform or construction kit.)

Atlas Atlas

unread,
Aug 12, 2019, 2:37:54 AM8/12/19
to Racket Users
My question was not about backwards compatibility, but about adopting new default syntax.
For me it is as good as dropping s-expressions because only default\main syntax is what really mater for me.
Sorry for not expressing myself clearly enough.

воскресенье, 11 августа 2019 г., 21:47:20 UTC+3 пользователь Robby Findler написал:

Atlas Atlas

unread,
Aug 12, 2019, 2:45:18 AM8/12/19
to Racket Users
Thank you for your answer. I wait then to see.

>Honu
Yes, documentation is really lacking.


воскресенье, 11 августа 2019 г., 23:19:06 UTC+3 пользователь Neil Van Dyke написал:

Robby Findler

unread,
Aug 12, 2019, 8:49:45 AM8/12/19
to Atlas Atlas, Racket Users
Sounds like you're going to take a wait-and-see attitude, which sounds
wise to me, but you are also welcome to participate in the discussion!

As for adopting-new-syntax vs backwards-compatibility, does it help if
I were to tell you that anything new will always be "opt in", in the
sense that existing programs will continue to work completely
unchanged with no special annotations or anything else like that
required? There is the issue of documentation that others have raised
and how various writings about Racket might change to show example
programs -- that may well change depending on how successful the
new-syntax venture is, but Racket has been committed to backwards
compatibility in a way that few programming languages are. There are
piles of lecture notes (in the form of slide presentations written in
Racket) from the late 90s (so not in any continuous integration system
anywhere, as far as I know) that still run fine in today's Racket for
example.

Robby
> --
> You received this message because you are subscribed to the Google Groups "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to racket-users...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/racket-users/e72b6c20-56a1-4d29-90e1-228877ea8f4c%40googlegroups.com.

Vincent St-Amour

unread,
Aug 12, 2019, 10:21:38 AM8/12/19
to Robby Findler, Atlas Atlas, Racket Users
On Mon, 12 Aug 2019 07:49:30 -0500,
Robby Findler wrote:
>
> There are piles of lecture notes (in the form of slide presentations
> written in Racket) from the late 90s (so not in any continuous
> integration system anywhere, as far as I know) that still run fine in
> today's Racket for example.

Not only do they run, some of them are still being actively developed. ;)

Vincent


> On Mon, Aug 12, 2019 at 1:37 AM Atlas Atlas <peacekee...@gmail.com> wrote:
> >
> > My question was not about backwards compatibility, but about adopting new default syntax.
> > For me it is as good as dropping s-expressions because only default\main syntax is what really mater for me.
> > Sorry for not expressing myself clearly enough.
> >
> > воскресенье, 11 августа 2019 г., 21:47:20 UTC+3 пользователь Robby Findler написал:
> >>
> >> Matthew posted an (IMO) clear explanation of the state of the thinking here earlier. tl;dr: sexpressions will never be abandoned and backwards compatibility with existing languages will be maintained for the foreseeable future.
> >>
> >> ... but read his message if you are worried. I believe it is included in the github discussion you mention.
> >>
> >> Robby
> >>
> > --
> > You received this message because you are subscribed to the Google Groups "Racket Users" group.
> > To unsubscribe from this group and stop receiving emails from it, send an email to racket-users...@googlegroups.com.
> > To view this discussion on the web visit https://groups.google.com/d/msgid/racket-users/e72b6c20-56a1-4d29-90e1-228877ea8f4c%40googlegroups.com.
>
> --
> You received this message because you are subscribed to the Google Groups "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to racket-users...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/racket-users/CAL3TdOMFG610kXQwr8RPAOb5Cx%3DoF_-_v1bUUxwXth8X16Li3g%40mail.gmail.com.

Neil Van Dyke

unread,
Aug 12, 2019, 10:50:03 AM8/12/19
to Racket Users
Robby, I'm still not certain we all have a shared understanding of some
of the concerns and where we all stand, so please let me try to get at
that some of that:

> As for adopting-new-syntax vs backwards-compatibility, does it help if I were to tell you that anything new will always be "opt in", in the sense that existing programs will continue to work completely unchanged with no special annotations or anything else like that required?

I suspect, to many industry software engineers, this phrasing sounds
like "deprecated", which is something we understand.  It's not something
anyone likes to hear, unless we've already moved away from it, and we
think that deprecating that is a positive sign.

> There are piles of lecture notes (in the form of slide presentations written in Racket) from the late 90s (so not in any continuous integration system anywhere, as far as I know) that still run fine in today's Racket for example.

I think this argument doesn't address the concerns of software engineers
who know the history of Racket (and of countless possible parallels
elsewhere).

For one example, from Racket specifically, how the change to pair
mutability was done meant that some of those modules in "compatibility"
dialects no longer interoperate well with modern modules.  That's a big one.

For a lesser example, which nevertheless was a problem in real-world
practice: one of the changes to C extensions meant being locked to the
old, non-default GC (missing out on enhancements, and concern it was
more likely to break in the future, since most people had been pushed
along to using the new thing, until that scary C extension code could be
disturbed again).

Another example was people who were using PLaneT's
multiple-installed-package versions and SemVer-like updating, when that
was deprecated and the functionality lost.

From an engineering perspective, over the last couple decades, Racket
has done a good job, overall.  And an outstanding job, as academic-led
projects go.  But, at this point, I think software engineers should want
a clear understanding of top-level requirements for Racket, so they have
an idea of what to expect.

Some degree of trust factors into assessments of whether adopting or
staying with such-and-such tech makes sense, and I'd think that arguing
"some old CS101 lecture slides probably still work" is going to increase
concern rather than reduce it. :)

Some people (here, and in other venues) are skittish or turned off by
various recent commotion.  At this point I could allay some of their
concerns by mentioning multiple mitigation/transition options to them,
but I'd prefer to keep all the value of the community we've built around
Racket.

Moving forward... Software engineers have learned to expect modern
platform pitchers to often be disingenuous, or at least
over-enthusiastic.  If core states, utterly unambiguously, and speaking
as one, the top-level requirements that will guide Racket, and
everything else clearly follows from those requirements, then I think
that will increase confidence.  (Even if some of the articulated
priorities are not ideal for our needs, we know what to plan with, with
some confidence.)

Robby Findler

unread,
Aug 12, 2019, 10:57:15 AM8/12/19
to Neil Van Dyke, Racket Users
Points well taken, Neil. My messages were probably better unsent.

Robby
> --
> You received this message because you are subscribed to the Google Groups "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to racket-users...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/racket-users/35d7b52a-abd7-e593-bcec-7f25ecde8f8d%40neilvandyke.org.

Brian Adkins

unread,
Aug 12, 2019, 2:15:02 PM8/12/19
to Racket Users
Thank you Neil for articulating concerns that I feel are common to a larger number of people than it may seem. As the old saying goes, "the difference between practice and theory is much greater in practice than in theory".

I'm hopeful that we will receive clarification soon. If not, despite all the wonderful things that Racket provides over other Scheme implementations currently, it will become increasingly difficult for me to continue investing in a Racket codebase if I need to wait N years for a concrete implementation of Racket2 before really knowing the subtle consequences.

The fact that I'm currently a full-time Racket developer should indicate that I don't need the same stability/predictability that's provided by mainstream languages, but I do need _enough_ stability/predictability :)

It might be an attractive option for the core team to have the community wait for N years while Racket2 is built before coming to any conclusions, but I really don't think that's a viable option.

Kira

unread,
Aug 13, 2019, 4:08:37 AM8/13/19
to Racket Users
I do feel different concerns and not even sure about some of them, so I will try to explain myself if this is in some interest to someone.
And perhaps better understand myself in process.

1. I am value my time, I am not so young for learning whole new thing each week.
2. I came to Racket somewhere 2 years ago, and it was advertised to me as LISP with reasonable infrastructure(coming from C# this is what I am used to), otherwise I would never have spent time on Racket. (I find however that libs not in great shape).
3. I have few libs already that I was planed to publish, few prototypes and few starter projects. I invested 2 years of my time to Racket.
4. News that Racket can became non lisp is devastating for me. It just crosses out the 2 years of my life. I am not happy even of jokes of such kind. And no efforts for backward compatibility can change this, they just don't matter.

5. I am however see a lots of places where Racket can improve. And I am not afraid of changes that break things. I am not fun of maintaining backward compatibility in the cost of progress, modern technologies is fast evolving and this is reality that we all must faced.
(Some programming languages manage to absorb change, but withstand progress - Alan J. Perlis)
If this is real improvements, then they a paying for themselves. Some degree of compatibility good of course, but this is not something that lots of effort should be putted, especially when there is a huge problems to solve.
Lets break things to improve. Real improvements will be worth to rewrite some lib's from scratch.
And the benefits of improvements should be clearly communicated to community.

What I see as fields to improve is infrastructure in first place.
- 1 Better and more consistent standard library. (date time processing, fold map for collections of any type, new number types perhaps? as uint16 uint32 uint64 int16 int32 int64 and homogeneous vectors with them)
- 2 Improvements on type system
- 3 Official auto documentation mechanism
- 4 Dr.Racket overall improvements and multi threading support
- 5 Internationalization support for documentation and tooling
- 6 Syntax changes.

Of course, it can be worth to make one major syntax change in the first place, and then improve other fields on top of this.
I already pointed out on what considerations syntax changes can be provided in different places.
I will repeat it here one more time:
https://www.youtube.com/watch?v=uEFrE6cgVNY
https://www.youtube.com/watch?v=R3zEOsh8AnQ
https://www.youtube.com/watch?v=Ps3mBPcjySE
https://www.youtube.com/watch?v=Sg4U4r_AgJU
In general I want to see perceptual science and statistical research as basis for this changes not abandoning common sense and practicality.


Although I decided to wait for a couple of weeks, I do not feel in a position to invent a new big programming language right now.
And I really don't have time to wait.
I have already begun to study an alternatives for my practical tasks. I need a tool on what I can rely on.
Programming language must solve problems, not creating them.

Perhaps it is actually good for Racket to move from s-expressions, personally I can suggest shocking for lispers/schemers changes staying within s-expressions.

I just feel overall cheated and false advertised. I was advertised stable tool, and now I am somehow involved in new programming language design.
I need some time to digest this further, but overall this is shocking experience.

Long answer I wrote...

понедельник, 12 августа 2019 г., 15:49:45 UTC+3 пользователь Robby Findler написал:
Sounds like you're going to take a wait-and-see attitude, which sounds
wise to me, but you are also welcome to participate in the discussion!

As for adopting-new-syntax vs backwards-compatibility, does it help if
I were to tell you that anything new will always be "opt in", in the
sense that existing programs will continue to work completely
unchanged with no special annotations or anything else like that
required?

Robby

Kira

unread,
Aug 13, 2019, 4:40:50 AM8/13/19
to Racket Users
понедельник, 12 августа 2019 г., 15:49:45 UTC+3 пользователь Robby Findler написал:

As for adopting-new-syntax vs backwards-compatibility, does it help if
I were to tell you that anything new will always be "opt in", in the
sense that existing programs will continue to work completely
unchanged with no special annotations or anything else like that
required?


I see this common idea that technical means(opt in changes for example) can resolve such king of issues, but they cant.

Because this is conceptual plane, political if you wish. The is no tool that can return my time, right?


So only solution can be conceptual as well. In the plane of ideas, of project goals.


For example, main site states : "Mature, stable".

This doesn't seems as true if major syntax changes is planned?


Another statement from main site:

"Racket is a general-purpose programming language as well as the world’s first ecosystem for language-oriented programming."


This sounds good, but what about research platform? I hear sometimes that Racket is a academic research platform for language-oriented programming.

Being "mature and stable general-purpose programming language" is something not really good in mixing with academic research effort.

And I see a lots of positioning as educational platform, and this is whole another field.


So I can identify three poorly compatible areas:


1. General-purpose programming language with language-oriented programming futures.

This involves industrial professional approach to handle all sorts of things from documentation and tooling to syntax changes.
More responsible changes, more restrained and incremental, more support for infrastructure.


2. Language-oriented programming research effort.

This involves rapid changes that break all sorts of things, limited support of any kind besides actual research efforts.

This also implies all sorts of weird and wonderful syntax changes.


3. Teaching platform.

This implies simplicity of all kinds.

But also consistency and a strong theoretical foundation.

Documentation tools, performance measuring tools, OpenCL libraries, is what unnecessary here.

Performance measuring tools for example in no need here as well.



There must be conceptual solution that resolves conflict. Idea that describes how this contradictions can coexists.

Prioritization of goals.
Conflict resolution descriptions.

Etc etc.


And then technical solutions can be developed.

For example:
1. Decide that Racket must be industrial tool. When research and teaching activities must be provided on this platform as product of language-oriented programming possibilities of it.
2. Decide that Racket is mostly research project not ready for production, but perhaps in future, when research effort shows good results it will be stabilized and turned to industrial tool.

3. Some form of symbiosis of two approaches where it stays production grade instrument but also provide research facilities it the way that benefits business and science.
4. To reduce such a volume of work research and infrastructure to an educational utility, in my opinion, is cruel.

5. Make 3 separate projects with different goals.

6. Etc etc etc.


When solution is found it must be clearly stated in front project page, and in all levels of documentation and guidance.

Matthew Flatt

unread,
Aug 14, 2019, 10:39:52 AM8/14/19
to Racket Users
The Racket project leadership [see signature at end] hasn't had a
chance to meet and discuss since RacketCon. When it does meet, we
should be able to offer a plan for both the future development of
Racket and the process of involving everyone in that development.

Let's start by reminding everyone that Racket has always played several
roles simultaneously. It has been a solid vehicle for

- commercial developments,
- research experiments,
- education at several levels, and
- hobbyists' pleasure.

A vehicle for commercial software and educational delivery, in
particular, needs stability, so we have provided that to a high degree.

A research tool needs flexibility. We have accommodated flexibility,
and Racket has gained

- a new garbage collector,
- a syntax system seamlessly integrated with documentation,
- a syntax-parse system that everybody migrates to,
- a contract system that seeks its equal, and
- a type system that accommodates almost all idioms.

None of this initially affected Racket's usability as a commercial or
educational vehicle, and all of this has benefited people in these
communities, eventually. When research didn't work out or was not
completed to a production level, such as

- the initial experimentation with Honu,
- our experiments with parallel futures, and
- class initialization factories.

then we made sure it would not affect Racket's stability. You may not
even have heard about this efforts. On some occasions, we have broken
backwards compatibility (including in version 7.4), but always with a
transition path and generally with positive community feedback.

Moving from Racket to Racket2 involves two technical projects. We
consider the "consistency" and "genericity" vectors uncontroversial.
The syntax-design experiment is a different story. We are

- likely to proceed with the experiment
- because people seem interested
- and willing to spend significant resources on this project.

It's important for the community to understand this last point. The
resource issue is why we can't just quietly perform some experiments
and then integrate it, like we have done before. Doing so would make
some of our work, our discussions, and our predictions easier, but it
would not be fair.

The syntax-design experiment will still be structured as independent of
the main Racket distribution. So, for quite some time, core Racket will
experience only the "consistency" and "genericity" changes. You
can count on Racket's stability and usability as you have for 20 years.

Because the syntax part in particular is an experiment, we do not know
exactly how Racket will look at the end. Matthew's messages have
offered some guesses. Contrary to those guesses, the syntax part of the
experiment might fail completely, and it might fail later rather than
sooner. And because of this, too, we want to engage the community for
the entire migration path.

We understand the discomfort that comes with uncertainly, but Racket
needs room to evolve. From our perspective, this is far from the first
point of uncertainly or contention during Racket's history. Our
experience is that Racket users stick around through experiments and
changes that are made with earnest effort and in good faith. We will
continue to act on these principles.

- Jay, Matthew, Matthias, Robby, and Sam

Brian Adkins

unread,
Aug 14, 2019, 12:51:25 PM8/14/19
to Racket Users
On Wednesday, August 14, 2019 at 10:39:52 AM UTC-4, Matthew Flatt wrote:
The Racket project leadership [see signature at end] hasn't had a
chance to meet and discuss since RacketCon. When it does meet, we
should be able to offer a plan for both the future development of
Racket and the process of involving everyone in that development.

[...]


- Jay, Matthew, Matthias, Robby, and Sam

Thank you for that preliminary statement; I look forward to hearing the plan after the project leadership has had a chance to meet.

I want to offer just one suggestion for consideration when you meet. I've talked to a number of people from various language backgrounds and practices to (hopefully) balance my own biases on this matter, and I think it's possible that the syntax experiment may be unique with respect to the types of changes one might expect in a language community. My suggestion is that if the syntax experiment is successful, and the new syntax is chosen as the default language for the Racket community, the existing s-expression syntax of #lang racket may need to be treated in such a way as to avoid all appearances of deprecation. This might include, for example:

* Not using adjectives such as "deprecated", "legacy", "unstable", etc.

* Not putting the link to #lang racket documentation at the very bottom of this page: https://docs.racket-lang.org/ where R5RS, Scheme, MZScheme are located.

* Not using phrases such as: "Do not use #lang racket to start new projects; #lang racket2 is the preferred language" which can be seen here: https://docs.racket-lang.org/scheme/index.html

I understand that, if the new syntax is successful, you will want to promote it as the default/main language, and that you'll want to spend most of your resources on the new language; however, I don't think it's a foregone conclusion that #lang racket would then need to go the way of the other legacy languages which, while they may not have "gone away", and there may be programs that "still run", are certainly not viewed as attractive languages with which to use for an ambitious new project.


However, if after meeting together, the project leadership does not feel this way, then please be direct in your communication so those of us who have invested much in a #lang racket codebase can make an informed decision about how to proceed.

Thanks,
Brian Adkins

 
Reply all
Reply to author
Forward
0 new messages