Introducing Boot v2 with a streamlined CLJS workflow
Micha Niskin
Laurent PETIT
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
Laurent PETIT
--
Micha Niskin
Hi, sorry for the late reply! Boot pretty much takes care of that part for you, as long as you follow a few basic rules (I will be adding a “how to be a good citizen of the boot-o-sphere” section to the wiki on github soon):
Tasks don’t fish around in the filesystem directly to find things to compile or otherwise operate on. Instead, tasks use functions in
boot.corethat return immutable sets ofjava.io.Fileobjects from boot-managed temp directories. These functions present a sort of overlay filesystem with files in anonymous temporary directories. This allows tasks to be completely decoupled from the filesystem layout. Additionally, this makes it possible for boot to shuffle files around and use hardlinks and such to craft the classpath and the build fileset in different ways during the build cycle. In this way boot can emulate immutability and lexical and dynamic scope for things on the filesystem.Tasks don’t create files in the filesystem directly. Instead, tasks use functions in
boot.corethat create various flavors of anonymous, boot-managed temp directories (the ones mentioned in item 1, in fact). An important concept in boot is that the output of any task is part of the input for the next task in the pipeline. This is the property that supports the amazing composition of tasks that is possible with boot, without needing to generate miles of boilerplate configuration.The boot-managed temp directory lifespan is one build session only. This means one JVM, basically. The temp directories are stored in the
.bootdirectory in the project root. The next time you run boot it cleans out any old temp dirs in there (they are not cleaned up on exit because you may want to look in them if something goes wrong with the build; stack traces could be referencing source files in these temp dirs).The only directories that boot knows about that are not temp dirs it created are the ones you specify in the
build.bootfile viaset-env!(i.e. the:src-paths,:rsc-paths, and:tgt-pathkeys). The source and resource paths are not molested by boot in any way (no files in there are ever deleted, moved, modified etc.). The target directory, on the other hand, is completely owned by boot—boot will overwrite files or delete them in there as it sees fit. Boot ensures that the target directory always contains only those files the build process emits for that specific run, and doesn’t allow any stale files to hang out in there.
What this all means is that there is something of a tradeoff: boot never persists files that could become stale, so there is no need for a clean task, but on the other hand some things then need to be rebuilt instead of just hanging out in the target dir. We think this is an okay tradeoff because boot’s composition capabilities make it really easy to incrementally run any build process at all using the built-in watch task. In return you get 100% deterministic builds.
We’ll be talking about the details of the whole temporary filesystem machinery soon. Have fun playing with boot!
You received this message because you are subscribed to a topic in the Google Groups "Clojure" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/clojure/YAckwMSfZkY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to clojure+u...@googlegroups.com.
Laurent PETIT
Laurent PETIT
Alan Dipert
--Micha Niskin
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--Laurent Petit
--Laurent Petit
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
You received this message because you are subscribed to a topic in the Google Groups "Clojure" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/clojure/YAckwMSfZkY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to clojure+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscribe@googlegroups.com.
--Laurent Petit--Laurent Petit
Laurent PETIT
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
Henrik Eneroth
Micha Niskin
Henrik Eneroth
Micha Niskin
I don’t understand why Aether isn’t finding -pre20 via the coordinates [boot "RELEASE"]. I can see the jar file and pom in the repo: https://clojars.org/repo/boot/boot/2.0.0-pre20/. Do you have any idea why Pomegranate wouldn’t be fetching that? Is there some subtlety of the sorting order that I’m missing, do you think?
Micha Niskin
Does adding BOOT_CHANNEL=DEV boot -u do the trick? (We aren’t pushing snapshots currently so it should be the same…in theory.)
Micha Niskin
It looks like Clojars is doing what I thought it would do: https://clojars.org/repo/boot/boot/maven-metadata.xml
<release>2.0.0-pre20</release>
Henrik Eneroth
Laurent PETIT
:-)
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
Henrik Eneroth
Micha Niskin
Henrik, the -d option simply adds dependencies to the project. The main use case for this option is to pull in dependencies when you have no formal “project” and no build.boot file. Mainly it’s used in concert with the repl task so you can quickly and easily try something out:
$ boot -d aysylu/loom repl
boot.user=> (use 'loom.graph)
nil
boot.user=> (graph [1 2] [2 3] {3 [4] 5 [6 7]} 7 8 9)
...
However, if a dependency pulled in via -d (not transitive deps) contains namespaces in which tasks are defined, these namespaces may be tagged with ^:boot/export-tasks metadata. Boot will find these namespaces and refer all public vars into your build.boot environment.
So if the reagent jar contained task namespaces you should see them when you do
$ boot -d reagent
Note that you don’t need to supply the group-id for reagent, and if you want the latest release you can also omit the version.
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to a topic in the Google Groups "Clojure" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/clojure/YAckwMSfZkY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to clojure+u...@googlegroups.com.
Micha Niskin
Oh, I forgot to say, the ^:boot/export-tasks convention makes boot into a sort of application container. You can use boot to pull code from the internet and run it. This is really nice when combined with Docker, for example, but I’m sure we haven’t even found the coolest uses for this yet. We’re pretty excited about this aspect of boot in particular and would love to see how people find ways to use it in their workflows.
Henrik Eneroth
Micha Niskin
Henrik, can you make a github repo with what you were trying to do in your second question? It would help to be able to see your whole setup.
The add-src task is perhaps confusingly named (Names Are Hard). Its purpose is to shuffle files around in the file set, moving files that would normally be considered “source” files into the stream of artifacts that end up being packaged. It’s roughly equivalent to setting Leiningen’s :omit-source key to false (but it’s a composable task, not a global setting).
I’ll try to explain how this works. To model all the filesystem side effects that building software entails we must establish some types of files, aligned along some basis of user intent. We ended up with two categories: input and output files (again, Names Are Hard, so please bear with me).
input – files that MUST be processed further before being packaged or emitted to the project target directory (eg. Java source files, CLJS source files, etc.)
output – files that MAY be processed further, but can be included in the packaged artifact or emitted to the project target directory (eg. compiled JavaScript, Java bytecode, etc.). Note that this also includes files that might pass through the build system completely unmolested to end up in the final result, like your
index.htmlfile for instance. These are also called “resources” in Java parlance, I believe.
A major concept in boot has that the output of a task is always part of the input of subsequent tasks. Tasks shouldn’t know or care what is to be done to the artifacts they emit; they shouldn’t know or care who or what might be further processing them. Thus, both input and output files are on the classpath. This means that both types serve as input for tasks, which is, I admit, somewhat confusing.
When you set up your environment in boot you use the :src-paths and :rsc-paths arguments to set-env!. The :src-paths denote input dirs, :rsc-paths output dirs.
This brings us to tempdirs. Tasks that emit files do so in anonymous temp dirs they obtain from boot core. These temp dirs come in a few flavors, but there are two main classes of temp dir a task may create: input or output, naturally. The choice is left to the task writer, who decides based on his intent. For example, the cljs-repl task emits a CLJS namespace containing the websocket connection code the client needs to run to connect to the REPL server. This namespace is emitted into an input type temp dir. Thus the CLJS namespace is compiled into the application JS file, but the CLJS source does not appear in the app webroot.
Finally, we arrive at add-src. This task creates a temp dir of the output type. It then obtains a list of input type files from boot core and moves them into this dir. Then when the jar task for example runs later in the pipeline, these source files will be included in the jar.
I’m not sure how clear this is, I’m trying to figure out the best way to explain this stuff and this is one of the first iterations :)
You received this message because you are subscribed to a topic in the Google Groups "Clojure" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/clojure/YAckwMSfZkY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to clojure+u...@googlegroups.com.
Laurent PETIT
--
Laurent
Micha Niskin
- flagging part of its output as "input" type is a way to declare to boot that the files there MUST be processed further, at some point, or else there's an error somewhere?Just to check that I correctly understood the concepts :- it's upfront, in the task itself, that the decision of whether what the task produces is of type "input" or "output".
- the granularity of input / output files is the directory (it's at the directory level that content is flagged as "input" or "output"). Is it recursive (I guess so), for subdirectories of the tagged directory?
- add-src somehow pipelines some files tagged as "input" in the input of the task as "output" for the rest of the pipeline, thus ensuring those files will reach the target/ directory (and they may or may not be subsequently modified on their way to target/, this is somewhat an orthogonal concern). Right?
Some remarks (I know that Names Are Hard):- :src-paths and :rsc-paths are already difficult to distinguish when one does not have diseases related to ordering of letters. Maybe falling back to longer names could fix this: :source-paths and :resource-paths
Micha Niskin
...
Catonano
Oh, I forgot to say, the
^:boot/export-tasksconvention makes boot into a sort of application container. You can use boot to pull code from the internet and run it. This is really nice when combined with Docker, for example, but I’m sure we haven’t even found the coolest uses for this yet. We’re pretty excited about this aspect of boot in particular and would love to see how people find ways to use it in their workflows.--Micha Niskin
I'm not sure I understand what you mean and I'm especially curious about the relationship between this and Docker