Running Clojure apps using less memory.

[Jason Basanese]

Hello all! I'm new to this forum. My name Is Jason Basanese and I am a relatively new clojurist.

I recently began hosting my first small full stack Clojure web app. While doing this I ran into memory problems with my admittedly small load server. The maximum it can take in temp memory is 741MB. Too much of that memory is consumed giving an error when I try to run two REPLs on the server. One for testing and editing code and the other for leaving the app running. My other dynamic content websites which use php directly with Apache use minimal memory. Why is it that running a Clojure app like such "lein run -m clojureweb.core/serve" takes up so much memory? Here is the code of the function I am running with that command. 

(defn serve [& args]

  (org.apache.log4j.BasicConfigurator/configure)

  (run-server

   (logger/wrap-with-logger

    (reload/wrap-reload app))

   {:port 8080}))

Are the logger or the wrap-reload making it take up so much memory? Is there a better way to run Clojure apps that is less robust? Or is using a relatively large amount of memory just a bullet to bite when using Clojure? Yes I know an obvious solution would just be to upgrade the server, but where's the fun in that?

Requests for more detail, comments, answers and opinions all welcome!

[Michael Willis]

You can tune memory usage by passing parameters to the JVM:

http://docs.oracle.com/cd/E19900-01/819-4742/abeik/index.html

The sample leiningen project file has an example of how to pass parameters to the JVM:

https://github.com/technomancy/leiningen/blob/master/sample.project.clj#L264

Hope that helps,
Michael Willis

[Gary Trakhman]

I always work in a single process, REPL and server, there's no need to separate them out generally.

You might have better luck with the G1 collector, in addition to JVM tuning.  The G1 has the useful property of giving memory back to the OS when a GC is performed.

Use the command-line flag: -XX:+UseG1GC

If this is confusing, in short, PHP is 'interpreted', thus it's leaner and slower than a 'compiled' language like clojure.  There's a lot of stuff that needs to be loaded for clojure to work (the compiler lives in memory), and you'll take a hit in startup-time and memory footprint.  Some of this is java-specific, probably CLJS on V8 can do better in terms of memory footprint.

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Nando Breiter]

If you want to get more deeply into JVM memory tuning, perhaps jClarity's Censum tool can help: https://www.jclarity.com/censum/. It will run an analysis of your specific applications memory usage patterns and make suggestions how to configure the JVM for it.

Aria Media Sagl

+41 (0)76 303 4477 cell
skype: ariamedia

[Josh Tilles]

Hi Jason,

I have a hunch that you have more Java processes running than you need and/or are aware of. For example, by running your Clojure app via lein run -m clojureweb.core/serve, you have two Java processes: one for Leiningen, and a separate one (started by Leiningen) for your app. It’s unlikely that you need Leiningen’s JVM to stick around, so you should probably try starting your app with the command lein trampoline run -m clojureweb.core/serve. (For details, see “Server-side Projects” in the Leiningen tutorial.)

Also, you didn’t mention how you were starting your REPL(s), but I’m going to assume that you were using lein repl; if so, that’s the source of yet another Java process or two. In order to refrain from needing to keep other JVMs running, you might want to use Clojure 1.8’s built-in socket server REPL instead.

Cheers,

Josh

P.S. You mentioned that you’re relatively new to Clojure—welcome! =)

[Jason Basanese]

Thanks guys! Going through your various tips helped me get the memory usage down below the limit. Now things are running nicely.