Allow custom read-cond features

[Herwig Hochleitner]

Hi,

so now that read-cond is well-understood, I'd like to introduce custom features to my clojurescript build. A ticket for adding the necessary configuration to the compiler has been declined for lack of previous discussion: https://dev.clojure.org/jira/browse/CLJS-2396

So, I'd like to see this added to the language. My use-case is compiling stuff for react on web, react-native-ios, react-native-android and react on nashorn.

Is anybody else interested in adding custom features to their builds?

Open issues:

- Option interface:

  - On cljs, this would probably be a compiler option

  - On clj, this could be a java `-D` option

- Only allow qualified keywords as custom features?

kind regards

[Alex Miller]

Both the Clojure reader and tools.reader are able to handle custom conditionals now and have the ability to specify the active "features" as options when invoking the reader. So if you are in control of reading, this is partially available now.

What you're asking for is the ability to activate features when the platform (Clojure or ClojureScript) is in control of the reading. This implies many additional questions:

1) How are active features specified on each platform? For ClojureScript, how do you separate the features active for Clojure when compiling macros from .clj/.cljc vs compiling ClojureScript from .cljs/.cljc?

2) When are the expanded features used? Currently conditional reading is only enabled in .cljc files in both Clojure and ClojureScript. Does that need to be expanded to .clj and .cljs files? How does it affect .clj files used in ClojureScript for macros?

3) Do you need to support feature combinations like (and :cljs :react/ios)?

4) Do you need to support dynamic runtime extension? How does that affect ClojureScript?

Prior design page which hints at a few of these near the end:

https://dev.clojure.org/display/design/Reader+Conditionals

That's a lot of things to figure out. Creating a design page and starting to figure out the alternatives etc is the path towards talking about this. I think you have the benefit of a pretty good concrete use case here - it would be good to elaborate a bit more on that and to consider conditional reading vs the alternatives.

--
You received this message because you are subscribed to the Google Groups "Clojure Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure-dev+unsubscribe@googlegroups.com.
To post to this group, send email to cloju...@googlegroups.com.
Visit this group at https://groups.google.com/group/clojure-dev.
For more options, visit https://groups.google.com/d/optout.

[Herwig Hochleitner]

2018-02-14 17:23 GMT+01:00 Alex Miller <al...@puredanger.com>:

Both the Clojure reader and tools.reader are able to handle custom conditionals now and have the ability to specify the active "features" as options when invoking the reader. So if you are in control of reading, this is partially available now.

Yes, I just don't want to replace the compilation pipeline.

What you're asking for is the ability to activate features when the platform (Clojure or ClojureScript) is in control of the reading. This implies many additional questions:

Well, I can answer them from my perspective:

1) How are active features specified on each platform? For ClojureScript, how do you separate the features active for Clojure when compiling macros from .clj/.cljc vs compiling ClojureScript from .cljs/.cljc?

The compiler should be in charge of this to guard the possibility of passing source read with :read-cond :preserve

As suggested in my previous post, straight up compiler options could be used. Another possibility would be `data_readers` - like descriptor, but I'm in favor of compiler options, because read-cond is for alternate builds, not composable, project-wide config.

 

2) When are the expanded features used? Currently conditional reading is only enabled in .cljc files in both Clojure and ClojureScript. Does that need to be expanded to .clj and .cljs files? How does it affect .clj files used in ClojureScript for macros?

Leave it for just `cljc`. We can still grow to support `clj` and `cljs` later.

Macro separation is a hairy concern. I'd go with the flow of what's already there:

- for clj macros, clojure's read-cond is used

- for cljs (self-hosted) macros, clojurescript's read-cond is used

FWIW, macro separation in cljc is hairy as it is and I would not attempt to solve this as part of read-cond

 

3) Do you need to support feature combinations like (and :cljs :react/ios)?

Another concern, separate to this request. True, overlapping feature spaces make boolean combinators more desirable, however, they are not a requirement. People can just define :cljs.react/ios, for now.

 

4) Do you need to support dynamic runtime extension? How does that affect ClojureScript?

Not at all. read-cond is just about the `R` in repl and it should stay that way.

[Thomas Heller]

That's a lot of things to figure out. Creating a design page and starting to figure out the alternatives etc is the path towards talking about this. I think you have the benefit of a pretty good concrete use case here - it would be good to elaborate a bit more on that and to consider conditional reading vs the alternatives

I opened the ticket since I wanted to to be able to have conditional requires in ClojureScript.

In Clojure you can just call (when condition (require 'some.ns) ...) at runtime. This is actually used in quite a lot of places and very useful. This however is not possible in ClojureScript due to the limitations imposed by the Closure Compiler. All requires are completely static and cannot be modified. Unlike Clojure on the JVM, ClojureScript does run on a variety of different platforms. It may run in the browser, node, react native etc. In my case I needed to exclude a require since including it broke the build for nodejs. It will be fairly common that you'll want to re-use your React components from the Client for a server-side render (or elsewhere).

One such package would be CodeMirror. It does not run in node due to heavy DOM interop. It is not required for server-side render. It should just not be included.

Other "solutions" for this problem I have seen include either using a global variable and assigning that somewhere outside the build itself (eg. via :preamble) or using different source paths to include empty dummy namespaces. Both options are clunky to use and not always feasible to do. Sometimes you'll just might just need to omit one function call. Another option that I relied on heavily previously was using a :closure-defines variable and hope that the Closure Compiler would be able to eliminate the unusued code. This works to a certain extent but is also not 100% reliable and you'll still end up compiling everything.

(ns my.cool.component

  (:require

   [reagent.core :as r]

   #?@(:node []

       :cljs [["codemirror" :as cm]])))

I implemented this feature in shadow-cljs and just set :compiler-options {:reader-features #{:node}} for the node build. Simple. For Clojure or maybe even the newer cljs.main a command line option/system property could be used.

The features only apply when reading ClojureScript since :clj only has one runtime: the JVM. Clojure can already do dynamic require so my use-case doesn't apply at all. Clojure happily ignores the unknown features when reading so they don't interfere with anything. Personally I didn't need any boolean feature combinations yet. Although my use is rather limited to 3 uses at the moment.

Self-hosted macros are hairy at the moment but could actually become simpler if they had their own :read-cond feature to begin with.

The question whether the :reader-features should be enabled in .cljs files came up in a discussion with Kevin Lynaghk and my argument was that it might be useful but would probably break tools and the default CLJS compiler so I opted to staying with .cljc files.

In the case of shadow-cljs I didn't need to change the compiler at all since I already handle the reader. The change was trivial though and should be equally simple for ClojureScript.

   

[Herwig Hochleitner]

​Actually, the lack of this feature wouldn't hurt nearly as bad, if the infrastructure around Clojurescript was prepared to deal with custom readers. As it stands, I'm counting no less than 7 direct calls  (in 6 distinct files) to read, with hardcoded features in Clojurescript's code base.

How do people feel about a :reader compiler flag, that we could use to swap out the read - phase in full?