Ensime memory usage

[Brad Noriega]

Hi,

I'm using emacs + ensime for remote coding through ssh. I realized
that as soon as I start M-x ensime , the memory usage peaks very high
on my system almost 90%. Code completion with tab takes 1-2 minutes to
appear.

I've tried to use exclude-memory-index on my .ensime file, but I'm not
sure if it's the right way since I'm n00b in regex.

1. Can somebody explain me why there is double backslash that is used
in the user manual:
:exclude-from-index ("com\\\\.sun\\\\..\*" "com\\\\.apple\\\\..
\*")

If i want to exclude classes from org.eclipse and org.slf4j, is
this the correct way :
:exclude-from-index ("org\\\\.eclipse\\\\..\*" "org\\\\.slf4j\\\\..
\*")

Is it possible to white-list indexing instead ?

2. My server is small-ish 512MB RAM, but I dont run anything other
than emacs plus ssh. Is this why ensime cant run smoothly ? What's the
minimum memory req. ?

project infos : emacs 23.2.1, ensime 2.9.2-SNAPSHOT-0.8.0, scala
2.9.1, akka 1.2 with spray 0.8.0-RC3.

Thanks,
Brad

[Jason Fager]

> I've tried to use exclude-memory-index on my .ensime file, but I'm not
> sure if it's the right way since I'm n00b in regex.
>
> 1. Can somebody explain me why there is double backslash that is used
> in the user manual:
>    :exclude-from-index ("com\\\\.sun\\\\..\*" "com\\\\.apple\\\\..
> \*")
>
>    If i want to exclude classes from org.eclipse and org.slf4j, is
> this the correct way :
>    :exclude-from-index ("org\\\\.eclipse\\\\..\*" "org\\\\.slf4j\\\\..
> \*")

Hmm. I'm not in a place to doublecheck it right now, but I'm 99% sure
it should just be "\\". I remember having to manually replace the
'\\' sequence with a single '\' to get the regex engine to recognize
it as an escape character; I think down in Scala, that would look like
.replace("\\\\", "\\"); maybe that's what got copied into the docs?

>   Is it possible to white-list indexing instead ?

Nope. Would probably be pretty easy to add, though, I'll look at it
this weekend.

> 2. My server is small-ish 512MB RAM, but I dont run anything other
> than emacs plus ssh. Is this why ensime cant run smoothly ? What's the
> minimum memory req. ?

My ensime sessions generally run around 450MB, though I've spent some
time getting my index blacklist tweaked to get there. When I'm next
in front of my real computer, I can post my config.

- Jason

[Aemon Cannon]

512 sounds like cutting it close. I'm guessing you're getting a lot of swapping on compilation.

[Brad Noriega]

I tried running the same project in 2G machine with the newest spray
release, now it runs smooth with code completion, etc.
However, memory usage is still huge, somewhere around 1.2G, after
initial loading and analyzer ..
I'm guessing this is caused by loading unnecessary dependencies into
RAM ?
As a comparison, IntelliJ Idea takes only about 550MB total for the
same project.

This makes me opt for using IntelliJ for the time being in my local
machine.
Would be glad if there is an option for white listing indexing, hence
reducing memory usage.

Thanks guys.

[Aemon Cannon]

Whitelisting packages for indexing seems a very reasonable feature addition.

https://github.com/aemoncannon/ensime/issues/209

Have you tried your project with the Eclipse IDE? I'm curious how big that will

be in memory. Eclipse and ENSIME both use the Scala Presentation Compiler, 

which is the chief consumer of RAM (at least in ENSIME).

[Adam Alix]

I have had crazy memory consumption using ENSIME on OS X Lion with Emacs 23.1.90.

With a 821 file project, my ENSIME JVM is at about 1.2GB after analyzing.  It continues to grow up to about 2GB after a few saves; should its memory footprint continue to grow this quickly?  After 3 saves of a small file, I'm at 2GB of usage.

Adam

[Aemon]

Hi, Adam. Sorry for the delay in responding.

That's a big project.

It's hard to diagnose without a picture of how memory consumption changes over time.

For example, I just had a look at ensime-in-ensime. The size resident in ram starts out at about 700MB and grows

 to about 850MB as I touch files (dozens) and save each file several times. At that point growth is much slower. 

This is about what I would expect given that ensime is slowly populating a cache of the types in the project, and 

at some point this cache contains the majority of types.

Recommendation:

You can save quite a bit of memory by whitelisting or blacklisting packages 

in the indexer. The manual needs updating on this (i'll fix tonight).

If the growth does not stop at some point than please do follow up