If Clojure is to blame for the majority of the startup time, why doesn't ClojureScript proportionally slow down the JavaScript startup time also?
992 views
Skip to first unread message
Didier
Jul 14, 2017, 6:20:28 PM7/14/17
to Clojure
This link: https://dev.clojure.org/display/design/Improving+Clojure+Start+Time says that the Java startup time is ~94 ms, while Clojure boot time is ~640 ms. That's a ~680% increase.
On my machine the java start time is: ~1042 ms, and the Clojure start time is around ~3108 ms. A ~298% increase.
When I time the startup time of lumo against node, I get ~1190 ms for node, and ~1523 ms for lumo. A ~128% increase only.
Does Clojure perform more initialization then ClojureScript? And would that explain the much higher overhead Clojure has on top of the JVM, versus ClojureScript?
Gary Trakhman
Jul 14, 2017, 7:24:33 PM7/14/17
to Clojure
My mental model explains most of this away due to the different load patterns of java vs v8. In JVM clojure, functions are essentially 1:1 with classes, so in a functional language and standard lib, that's a lot of classes. Java will eagerly classload your entire app and deps before doing any work. V8 takes an upfront hit on parsing into javascript, but the JS target fits the structure of clojure itself much more closely. More time (not sure how much compared to JVM) will be spent JITing hot code paths, but that won't show up in the startup cost.
You can get JVM clojure REPL startup a little faster by AOT compiling all your deps, but it's generally not worth it in a development flow.
I think lumo does some extra tricks to keep this upfront cost down that are v8-specific: https://anmonteiro.com/2016/11/the-fastest-clojure-repl-in-the-world/
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Didier
Jan 24, 2018, 4:09:54 AM1/24/18
to Clojure
So if that is true, then the slow startup times are to blame on Java. I understand that Clojure does a lot of extra stuff at startup that a plain Java main method would not, but those extra things Clojure does are slow, because of the way the JVM handles them, where as they are quite fast on V8 or other JavaScript engines.
Gary Verhaegen
Jan 24, 2018, 8:39:48 AM1/24/18
to clo...@googlegroups.com
I've read a mot of discussions around that but have never myself looked into the details, so the following may be completely wrong.
My understanding is slightly different: the JVM does not actually load a class before it is first accessed by running code. What takes a long time for the JVM and not for JS engines, I think, is not loading the code, but creating objects.
For Clojure to run, all of your functions need to exist as objects in memory. Java bytecode does not have any mechanism to describe objects, so when Clojure starts up, it has to first load the code that defines the classes, then load the code that creates instances of those classes, and then execute that code to actually get the function objects. Only then can it start even thinking about looking at your own code. AOT thus just saves on parsing string files into Clojure lists, but it cannot really help with the problem that all of these objects still have to be created again from scratch at each start up.
The javascript engines have a way of representing objects directly (aka json), so this is much less of a problem: a lot of that object creation can be done at compile time and the js engine can just load objects in memory directly from the js file. You can go further (like Lumo does) by preloading these objects in memory directly as part of the executable bundle, skipping the part where you need to parse a string.
So in my understanding it mostly boils down to the ability to define objects directly, so they can just be loaded in memory, versus having to load and execute code to create the objects.
Gary Trakhman
Jan 24, 2018, 10:10:34 AM1/24/18
to clo...@googlegroups.com
@garyv, I haven't tested/profiled in a really long time, but I recall the majority of startup time being spent in Class.forName. I'm not sure how to verify startup exactly, but it should be easy to profile a top-level repl 'require' call from the user ns, which is doing essentially the same work, except it wouldn't show the difference between direct-linked code and nondirect, and you'd have to be careful to AOT your project so you're not just profiling the compiler.
Alex Miller
Jan 24, 2018, 1:48:10 PM1/24/18
to Clojure
On Wednesday, January 24, 2018 at 9:10:34 AM UTC-6, Gary Trakhman wrote:
@garyv, I haven't tested/profiled in a really long time, but I recall the majority of startup time being spent in Class.forName.
There was a bug (CLJ-1529) related to excessive use of Class.forName() in the compiler (particularly visible in macro-heavy code) but that was fixed in Clojure 1.7. Very approximately in the same ballpark CLJ-2210 was fixed in Clojure 1.9. I doubt that you would see this a hotspot on this today (beyond the fact that loading a lot of code loads a lot of classes).
I'm not sure how to verify startup exactly, but it should be easy to profile a top-level repl 'require' call from the user ns, which is doing essentially the same work, except it wouldn't show the difference between direct-linked code and nondirect, and you'd have to be careful to AOT your project so you're not just profiling the compiler.On Wed, Jan 24, 2018 at 8:39 AM Gary Verhaegen <gary.ve...@gmail.com> wrote:I've read a mot of discussions around that but have never myself looked into the details, so the following may be completely wrong.My understanding is slightly different: the JVM does not actually load a class before it is first accessed by running code. What takes a long time for the JVM and not for JS engines, I think, is not loading the code, but creating objects.For Clojure to run, all of your functions need to exist as objects in memory. Java bytecode does not have any mechanism to describe objects, so when Clojure starts up, it has to first load the code that defines the classes, then load the code that creates instances of those classes, and then execute that code to actually get the function objects. Only then can it start even thinking about looking at your own code. AOT thus just saves on parsing string files into Clojure lists, but it cannot really help with the problem that all of these objects still have to be created again from scratch at each start up.
Loading AOT'ed Clojure code will require loading the classes and initializing each var in the loaded namespaces. Loading non-AOT'ed Clojure code requires first reading and compiling the code, and also loading the classes and initializing vars. Of all of these steps, loading classes is generally the fastest - Java has spent a lot of time optimizing the load of classes from jars (and there's even more we could potentially take advantage of). When I have profiled it, reading and compiling usually end up being about the same order of magnitude. Var initialization varies a lot (you could be doing anything in there), but certainly having work to do for every var has a cost that adds up, particularly if you load 100s of namespaces on startup (which is not uncommon in web apps).
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscribe@googlegroups.com.
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscribe@googlegroups.com.
Didier
Jan 25, 2018, 12:24:14 PM1/25/18
to Clojure
Based on the profiling performed here http://clojure-goes-fast.com/blog/clojures-slow-start/ on 1.9,
It looks like clojure/core__init.load is the slowest bit with load and requiring spending the most time on, as well as loadClass, followed by clojure/lang/Var.invoke.
Wouldn't ClojureScript also need to initialize all Vars and execute top level forms? You'd think that part would be slower in JavaScript no?
Mikhail Gusarov
Jan 25, 2018, 1:22:06 PM1/25/18
to clo...@googlegroups.com
cljs vars are not reified.
Stephen Nelson
Jan 25, 2018, 3:29:07 PM1/25/18
to clo...@googlegroups.com
The JVM is concurrent whereas Javascript VMs are singled-threaded. Var initialisation in Clojure uses concurrency primitives to ensure that namespaces are always consistent for all threads, whereas Clojurescript simply assigns to variables. Our profiling of Clojure on the JVM indicates that a lot of the time spent in namespace initialisation is spent in concurrency structures, and this seems much more significant for non-clojure-core namespaces.
Timothy Baldridge
Jan 25, 2018, 3:53:03 PM1/25/18
to clo...@googlegroups.com
Really, any comparisons between JS and JVM startup times are not useful at all. For a long list of reasons, not limited to:
* CLJS doesn't have Vars, CLJ does
* JS VMs are highly tuned for excellent startup times
* JVM Bytecode is a binary format expressing classes, JS files are text files expressing functions and data transformations
* CLJS doesn't have Vars, CLJ does
* JS VMs are highly tuned for excellent startup times
* JVM Bytecode is a binary format expressing classes, JS files are text files expressing functions and data transformations
* JVM is class based, JS is prototype based
* JVM is multithreaded and designed to handle several GB of data, JS VMs are single threaded and designed to often deal with less than a GB of data.
* HotSpot (and most) JVMs are method JITS, JS engines often use many profiling methods that are closer to that of a tracing JIT
* The JVM performs extensive security analysis on loaded bytecode, JS engines disallow most security flaws via syntax restrictions
* CLJS supports tree shaking and minifcation.
All this adds up to an apples-to-oranges comparison. They are completely different platforms, the numbers between the two won't be close, and drawing any sort of comparison between the two isn't very useful at all.
--
* No multithreaded safety in vars is not what takes up most of the startup time in Clojure. We're talking about single threaded access (var initing isn't multithreaded) to a primitive that is updated via a CAS. The highest cost you'll see accessing a var in this situation is somewhere in the range of a cache miss or two.
* JVM is multithreaded and designed to handle several GB of data, JS VMs are single threaded and designed to often deal with less than a GB of data.
* HotSpot (and most) JVMs are method JITS, JS engines often use many profiling methods that are closer to that of a tracing JIT
* The JVM performs extensive security analysis on loaded bytecode, JS engines disallow most security flaws via syntax restrictions
* CLJS supports tree shaking and minifcation.
All this adds up to an apples-to-oranges comparison. They are completely different platforms, the numbers between the two won't be close, and drawing any sort of comparison between the two isn't very useful at all.
--
* No multithreaded safety in vars is not what takes up most of the startup time in Clojure. We're talking about single threaded access (var initing isn't multithreaded) to a primitive that is updated via a CAS. The highest cost you'll see accessing a var in this situation is somewhere in the range of a cache miss or two.
On Thu, Jan 25, 2018 at 1:26 PM, Stephen Nelson <ste...@montoux.com> wrote:
The JVM is concurrent whereas Javascript VMs are singled-threaded. Var initialisation in Clojure uses concurrency primitives to ensure that namespaces are always consistent for all threads, whereas Clojurescript simply assigns to variables. Our profiling of Clojure on the JVM indicates that a lot of the time spent in namespace initialisation is spent in concurrency structures, and this seems much more significant for non-clojure-core namespaces.
On Fri, Jan 26, 2018 at 7:22 AM Mikhail Gusarov <dott...@dottedmag.net> wrote:
cljs vars are not reified.
On Thu, 25 Jan 2018, at 18:24, Didier wrote:
> Based on the profiling performed here
> http://clojure-goes-fast.com/blog/clojures-slow-start/ on 1.9,
>
> It looks like clojure/core__init.load is the slowest bit with load and
> requiring spending the most time on, as well as loadClass, followed by
> clojure/lang/Var.invoke.
>
> Wouldn't ClojureScript also need to initialize all Vars and execute top
> level forms? You'd think that part would be slower in JavaScript no?
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to clojure+unsubscribe@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscribe@googlegroups.com.
“One of the main causes of the fall of the Roman Empire was that–lacking zero–they had no way to indicate successful termination of their C programs.”
(Robert Firth)
(Robert Firth)
Didier
Jan 25, 2018, 11:34:13 PM1/25/18
to Clojure
Really, any comparisons between JS and JVM startup times are not useful at all.
I'd say its useful to me, if anything because I've learned quite a few things already from this comparison, and I hope to learn more.
If Var reification is part of the slowness of Clojure startup over Java's, and ClojureScript managed without it, yet it delivered 90% of the practical benefits of Var reification, I could see a potential here maybe. Something similar to a dynamic or AOT. Flagging some Vars which need not be reified, or compiling to non reified Vars if fast startup time is what the user would rather have. I could see this being desirable for use cases like AWS Lambda, or small scripts, if it had a drastic startup improvement.
The concurrency issue is interesting, and would definitely be unique to Clojure over ClojureScript. First time I hear about this being a possible bottleneck for startup time. Are the concurrency checks java's? Or are these higher level Clojure concurrency checks? Is there possible improvement from the Clojure side of them to make them faster?
@tbc++ seems to imply the apples to oranges are due to the platforms, and I can accept this to be true, except that I've seen suggestions that the slow startups are due to either Clojure's implementation causing startup bottlenecks, or Clojure's core design preventing any kind of fast startup implementation. Meaning the culprit is Clojure and not the JVM. But as I find myself thinking about ClojureScript, I wonder if this is true. If Clojure's core design is the bottleneck, then why doesn't ClojureScript suffer too? And so, maybe its because it has altered some core design choices, like concurrency and Var reification. Is it just that? Or is there also bottleneck in the current Clojure implementation on the JVM, which were implemented more efficiently on ClojureScript? I know Alex Miller is working on some of those, and every version of Clojure optimizes, but I've also heard nothing here is expected to create a game changer improvement (appart from possibly grouping functions under one class). And if its neither the implementation, or the core design choices, then I have to conclude that it is in fact the JVM's fault, in that its core design is making what Clojure does on startup more expensive then say the V8 design. Which one of those three things (or possible combination is it) is what I'm trying to seek answer for here.
Thanks everyone,
Didier
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to clojure+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
Timothy Baldridge
Jan 26, 2018, 12:21:54 AM1/26/18
to clo...@googlegroups.com
>> If Var reification is part of the slowness of Clojure startup over Java's
It's really not. If you completely removed vars from Clojure on the JVM, Clojure wouldn't be faster by any measurable amount. If someone has benchmarks that say that Vars themselves are the cause of Clojure JVM's slower startup, then I'd love to see those benchmarks, because there's most likely a serious flaw in the measurements.
It's really not. If you completely removed vars from Clojure on the JVM, Clojure wouldn't be faster by any measurable amount. If someone has benchmarks that say that Vars themselves are the cause of Clojure JVM's slower startup, then I'd love to see those benchmarks, because there's most likely a serious flaw in the measurements.
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscribe@googlegroups.com.
Timothy Baldridge
Jan 26, 2018, 12:22:00 AM1/26/18
to clo...@googlegroups.com
I would suggest everyone here read this document: https://dev.clojure.org/display/design/Improving+Clojure+Start+Time It goes into a lot of detail on prior work in this area, and I'd say the benchmarks on it should be quite informative.
But let me also go into a bit more detail into how CLJS differs from CLJ in how it accesses global function definitions:
In CLJs, clojure core contains a lot of definitions that look something like this:
clojure.core.conj = function () {... clojure.core.some_fn(...) . ....}
The cost of creating that global function is then the cost of 2 lookups (clojure in the global namespace, core in clojure), the creating of a JS function object, and the assigning of that function to the core object. The internal call to some_fn doesn't matter at load-time because of Javascript's late binding.
In the JVM this happens:
1) A class is loaded that implements the IFn or AFn interfaces
2) The methods on this class must be loaded and the bytcode parsed, and validated
3) The static constructor on this method is run
4) Inside the static constructor there are calls to find Vars. In the case of the above function this would be a lookup to clojure.core.some_fn. This is done for every var referenced by this function
5) Any non-native constants must be new'd up. This includes hashmaps, vectors, lists, keywords (which must be interned), symbols, etc.
But let me also go into a bit more detail into how CLJS differs from CLJ in how it accesses global function definitions:
In CLJs, clojure core contains a lot of definitions that look something like this:
clojure.core.conj = function () {... clojure.core.some_fn(...) . ....}
The cost of creating that global function is then the cost of 2 lookups (clojure in the global namespace, core in clojure), the creating of a JS function object, and the assigning of that function to the core object. The internal call to some_fn doesn't matter at load-time because of Javascript's late binding.
In the JVM this happens:
1) A class is loaded that implements the IFn or AFn interfaces
2) The methods on this class must be loaded and the bytcode parsed, and validated
3) The static constructor on this method is run
4) Inside the static constructor there are calls to find Vars. In the case of the above function this would be a lookup to clojure.core.some_fn. This is done for every var referenced by this function
5) Any non-native constants must be new'd up. This includes hashmaps, vectors, lists, keywords (which must be interned), symbols, etc.
5) The clojure.core.conj var is created
6) The contents of the var are set to the function that was loaded (and new'd). Synchronization doesn't matter here as there are no other threads that can see this Var.
7) The metadata for the function is created. Most of the time this includes creating a hashmap with at least 2-3 values.
6) The contents of the var are set to the function that was loaded (and new'd). Synchronization doesn't matter here as there are no other threads that can see this Var.
7) The metadata for the function is created. Most of the time this includes creating a hashmap with at least 2-3 values.
8) The metadata for the var is set.
The systems in play here are completely different. So I re-state what I said before: it's apples-to-oranges.
But I'd love to see more benchmarks in this thread, so far we only have opinions.
The systems in play here are completely different. So I re-state what I said before: it's apples-to-oranges.
But I'd love to see more benchmarks in this thread, so far we only have opinions.
Nathan Fisher
Jan 26, 2018, 12:47:32 AM1/26/18
to clo...@googlegroups.com
It's not a huge contributor but Class loading isn't free and Clojure could optimise it in a couple of places. I think with a couple of months effort in optimisation you might be able to get it down to 500ms for a simple "hello world". I'm doubtful you can get it much lower than that. TBH I'm not sure if the engineering effort would necessarily be worth it for many users.
I wanted to test dynamic loading and docker before I shared the article more broadly but here's the results from implementing prefetching in Clojure;
It's a part of a series that I'm slowly plodding through as I have time. I hope to cover off the 4 items from the conclusion of this article;
Cheerio!
Nathan
Nathan Fisher
Alex Miller
Jan 26, 2018, 1:32:01 PM1/26/18
to Clojure
With a few custom patches (which are pending in jira) + AOT + direct linking + lazy vars + var meta elision + jvm flags, I have gotten raw startup as low as ~450 ms. Past that, radical changes are probably required which are unlikely to be worth doing.
On your conclusions:
1. minimize CAS'es - I'm doubtful this would make much difference. Hotspot is usually pretty good at optimizing that stuff in single-threaded uncontended scenarios. I will wave in the general direction of immutable namespaces to a place that I think is an opportunity to improve a range of issues like this.
2. Lazy vars effectively do this (in tandem with AOT and direct linking) and it can help 20-30% (presuming you have compiled stuff to use). However, most startup cases don't have AOT compiled code to work with and thus this won't help at all.
3. Prefetching - as you found, I would not expect this to be useful enough to be worth the pain and fragility involved.
4. jaotc - I think this *is* likely to be useful for cases where you can pre-stage stuff like this, in particular core itself is a perfect opportunity. People have already tested this (https://mjg123.github.io/2017/10/04/AppCDS-and-Clojure.html) and found it useful. But again, requires pre-AOT'ed code.
I don't think most of these changes help at all though in the most common case where you are spinning up a project with a lot of non-AOT'ed code - the 10s of seconds of read/compile/init times dwarf this stuff. Rich and I working on an idea to help with this.
Justin Smith
Jan 26, 2018, 1:39:06 PM1/26/18
to clo...@googlegroups.com
a nitpick on point 1 - I would assume you can't expect hotspot to improve anything in the timescale of a program startup
am I missing something here?
Nathan Fisher
--
Alex Miller
Jan 26, 2018, 2:48:00 PM1/26/18
to Clojure
On Friday, January 26, 2018 at 12:39:06 PM UTC-6, Justin Smith wrote:
a nitpick on point 1 - I would assume you can't expect hotspot to improve anything in the timescale of a program startup
hard to say.... there are a lot of lock optimizations. I'm not well-versed enough in that stuff to be able to tell, but certainly uncontended locking is exactly the kind of thing you want to optimize.
Nathan Fisher
Jan 26, 2018, 2:58:13 PM1/26/18
to clo...@googlegroups.com
Hi Alex,
That’s cool!
I was looking to explore alternatives optimisations to what you had already outlined and which I could potentially do with little effort. I started writing something to do parallel loads back in Sept/Oct. and was replicating some of what Buck/Blaze/Basel do in terms of a DAG. I wanted “pristine” Clojure VMs for each node in the dependency graph but startup time was an issue and I couldn’t see an easy way to bring it down.
For point one I was noticing some amount of non-trivial latency in Var interning and RT map. Agree Java CAS performance won’t be the issue but (I think) there’s something in Clojure that I’m working to improve my understanding. For the map I noticed once I eliminated class loading via prefetching the map initialisation in RT was a non-trivial amount of time (double digit ms). Not something that I think needs prioritisation by any means but just a “huh wonder why that is?”.
Justin in terms of Java it’s not a matter of JIT or anything like that. It’ll be relatively efficient out of the box, it’s a handful of cpu instructions to execute and is around 20 CPU cycles per try (eg nanoseconds). It should succeed on or about the first time within the context of Clojure startup. An atomic load side depends on how far up the memory chain it has to go but shouldn’t be “too expensive” either, more expensive than a local variable for sure but not enough to worry about in most cases.
Cheers,
Nathan
That’s cool!
I was looking to explore alternatives optimisations to what you had already outlined and which I could potentially do with little effort. I started writing something to do parallel loads back in Sept/Oct. and was replicating some of what Buck/Blaze/Basel do in terms of a DAG. I wanted “pristine” Clojure VMs for each node in the dependency graph but startup time was an issue and I couldn’t see an easy way to bring it down.
For point one I was noticing some amount of non-trivial latency in Var interning and RT map. Agree Java CAS performance won’t be the issue but (I think) there’s something in Clojure that I’m working to improve my understanding. For the map I noticed once I eliminated class loading via prefetching the map initialisation in RT was a non-trivial amount of time (double digit ms). Not something that I think needs prioritisation by any means but just a “huh wonder why that is?”.
Justin in terms of Java it’s not a matter of JIT or anything like that. It’ll be relatively efficient out of the box, it’s a handful of cpu instructions to execute and is around 20 CPU cycles per try (eg nanoseconds). It should succeed on or about the first time within the context of Clojure startup. An atomic load side depends on how far up the memory chain it has to go but shouldn’t be “too expensive” either, more expensive than a local variable for sure but not enough to worry about in most cases.
Cheers,
Nathan
- sent from my mobile
Nathan Fisher
Jan 26, 2018, 3:45:46 PM1/26/18
to clo...@googlegroups.com
Also what I was referencing in terms of the CAS was deferring submission until the namespace is fully initialised. Something akin to a SQL transaction where it could use mutable state and then transform it to the immutable map at commit (eg end of the file). Until a ns fully parsed and processed (by a single thread) there wouldn’t be any need for the intermediate data structure to be thread-safe. Kind of like Guava data structures but for Namespaces.
Aleš Roubíček
Jan 28, 2018, 3:32:51 AM1/28/18
to Clojure
In Lumo case there is heavy optimisation of startup time in not loading CLJS from JS sources at all. They use V8 Startup Snapshots. See https://anmonteiro.com/2016/11/the-fastest-clojure-repl-in-the-world/ for more details.
Dne sobota 15. července 2017 0:20:28 UTC+2 Didier napsal(a):
Dne sobota 15. července 2017 0:20:28 UTC+2 Didier napsal(a):
Herwig Hochleitner
Jan 28, 2018, 5:51:59 PM1/28/18
to clo...@googlegroups.com
With the help of criu [1], I can get clojure's startup time on jdk 1.8 down to around 300 - 350 ms [2]
This could be considered cheating, because it only works on linux with `CONFIG_CHECKPOINT_RESTORE=y`, but it also demonstrates, what can be achieved with kernel support.
scott stackelhouse
Jan 31, 2018, 9:24:05 PM1/31/18
to Clojure
I was just reading about Docker+CRIU.
For me, the slow startup isn't much of a bother for normal operations of my apps. They are generally long running services. But development is where I am bothered by it the most. It can be painful to restart a repl and it's very disruptive to my ability to stay focussed.
CRIU seems like it could actually help in this case. In particular, paired with Docker, I can just restart from a known good checkpoint with editor loaded, repl up and connected, etc. That would be very nice.
Herwig Hochleitner
Feb 1, 2018, 5:47:41 AM2/1/18
to clo...@googlegroups.com
2018-02-01 3:24 GMT+01:00 scott stackelhouse <scott.sta...@gmail.com>:
CRIU seems like it could actually help in this case. In particular, paired with Docker, I can just restart from a known good checkpoint with editor loaded, repl up and connected, etc. That would be very nice.
Yep, it nice. Although, with CRIU, you have to be very careful how far you initialize your program (open files, network sockets, ....) and you have to disable java's hsperfdata files (because they get deleted after the first run and are then missing for CRIU's restore), but I suppose the larger a Program is, the more CRIU can save time for require / load.
Reply all
Reply to author
Forward
0 new messages