Extremely poor performance

[Sarah Gerweck]

I am having issues with recent nightlies wherein I cannot edit nontrivial Scala classes because it hangs the IDE for a long time (20 seconds?) after every single edit. If I pause typing for more than perhaps 250 ms, my IDE hangs.

I've tried turning off both semantic highlighting and "more accurate" semantic highlighting, but it hasn't seemed to help me. It was never nearly this bad until recently unless I turned on "more accurate" highlighting.

The Scala Presentation Compiler thread is about a hundred stack calls deep inside scala.tools.nsc.interactive.Global.backgroundCompile, which seems to be the culprit here. This does not seem to happen in the background.

This is an IDE breaker for me, and I'm literally using vi to do all my Scala development right now.

I am on Eclipse 4.2 Indigo and Java 7.

[virtualeyes]

vi, ouch

By nontrivial, how many LOCs? Does it affect all such scala source files, or only a couple? How much memory have you allocated to Eclipse?

I hit a similar issue while editing a Play framework scala file which was hitting the Scala 22 tuple limit; IDE ground to a halt anytime I tried to edit the offending code block.

[Sarah Gerweck]

Sorry for the slow reply, have been very busy lately....

It seems to get progressively much worse over time as I use my IDE. Most of my Scala files have maybe 200 lines, but I am in a pretty large JavaEE project and the Scala areas make use of things like implicits and lots of little classes.

It's not a memory issue: I allocate 1.5 GB to Eclipse. (I only have 4 GB RAM on this box, so I do sometimes get some penalty from swapping, but this seems to be a CPU-bound issue. I'm on a Core i5, so it's not a terribly slow box.)

It seems pretty consistently that the Scala Presentation compiler spends massive amounts of time doing type checking. I've turned off all the semantic syntax highlighting options because they make it even worse. I wouldn't mind so much if it spent some time in the background checking types, but it seems to block the UI while it does its thing, which makes the Scala editor unusable.

Most of my coworkers have either switched to IntelliJ or stopped using the Scala editors in one way or another.

Is there a way to just turn off this type checking? That would be a tremendous help for me.

[Sarah Gerweck]

One last note: this doesn't happen only after making edits. I also get long pauses while Eclipse is "Updating occurrence annotations." E.g., the Scala file I was just trying to edit was about 400 lines and it proved impossible to use Eclipse's Scala editor.

I'd be happy to provide stack dumps or whatever else might be useful. Unfortunately though, I can't really provide the sources or the project I'm using.

[Mat]

I'm currently making a similar experience: there is a little (and sometimes not so little) pause after virtually every keystroke when editing large files. 

I deactivated semantic highlighting and "mark occurrences" and assigned 2GB heap in eclipse.ini. 

Working is now possible on a i7 machine, but nevertheless, those pauses become pretty annoying after a while.  

I have no idea what in the presentation compiler is causing them, but a preliminary workaround would

be to perform whatever time consuming operation not after each and every keystroke but only

if the user has remained inactive for a certain amount of time. Ypresentation-delay looks like

an implementation of this idea, but doesn't seem to have any visible effect here.

Juno SR1,  Scala IDE for Eclipse 2.1.0.nightly-2_09-201210150313-3c0ede2

[anli]

It is interesting both Sarah and Mat use Juno. Have you tried Eclipse 3.8.1 with the same Scala IDE plugin (Juno plugin does match Eclipse 3.8.x)?

[iulian dragos]

Thanks for getting back. Performance when editing is very important, and we need to get to the bottom of this.

1. To rule out memory issues, did you enable 'Show Heap Status' in Preferences? You'd see then if the used memory is really close to the edge. Please tell us how that looks.

2. Turn off 'Mark occurrences' (CMD-ALT-O on mac, probably Ctrl-Alt-O on PC). This is one of the actions that happen on every keystroke

3. Tunr off semantic highlighting (if you didn't already)

3. Try out Eclipse Indigo. I recently used Juno for a few hours, and it definitely feels much slower.

4. Does it happen in every file, or only some files? You may encounter one of the corner cases of the type checker (see https://issues.scala-lang.org/browse/SI-5862)

There is no way to turn off type checking, and all type-checking is done in the background thread (but sometimes the UI thread needs some result of type-checking, and needs to wait). We need to identify what is running in the UI thread when you type.

iulian

--
« Je déteste la montagne, ça cache le paysage »
Alphonse Allais

[iulian dragos]

On Sat, Oct 20, 2012 at 12:18 AM, Sarah Gerweck <sarah...@gmail.com> wrote:

One last note: this doesn't happen only after making edits. I also get long pauses while Eclipse is "Updating occurrence annotations." E.g., the Scala file I was just trying to edit was about 400 lines and it proved impossible to use Eclipse's Scala editor.

I'd be happy to provide stack dumps or whatever else might be useful. Unfortunately though, I can't really provide the sources or the project I'm using.

Thread dumps gold! Please do that.

iulian

 

[virtualeyes]

Juno is definitely part of the problem here.

If you are switching between scala source file and an xml/html/js/css/
coffee etc. file in another tab, good luck, can be a delay of 3-4
seconds before cursor gains focus in target non-scala file.

As far as the Scala presentation compiler itself, standalone (without
sbt), it's actually pretty snappy in my experience. The only times I
notice a marked slowdown in performance is via sbt, particularly
incremental builds; at that point the IDE gets noticeably laggy.

There is, I believe, an ongoing issue with semantic highlighting and a
hanging thread, so I have grudgingly given up that very nice (on the
eyes) feature, which does give a slight performance boost (though not
much).

Memory is not the issue, IMO. Makes no difference whether I allocate
1GB or 2GB memory to Eclipse with 8GB available in the machine. Heap
generally stays below threshold; GCs can (but not always) occur during
sbt clean/compile and eclipse project clean, but otherwise around
300MB. FWIW, for smaller projects you can probably get away with 1GB
memory and not experience any performance impact due insufficient
memory resources.

Have a 1st generation core i7 laptop with SSD. Running Java 7, Fedora
14, Juno latest, and Scala IDE from @2 weeks ago.

On Oct 20, 10:14 am, iulian dragos <jagua...@gmail.com> wrote:
> Thanks for getting back. Performance when editing is very important, and we
> need to get to the bottom of this.
>
> 1. To rule out memory issues, did you enable 'Show Heap Status' in
> Preferences? You'd see then if the used memory is really close to the edge.
> Please tell us how that looks.
> 2. Turn off 'Mark occurrences' (CMD-ALT-O on mac, probably Ctrl-Alt-O on
> PC). This is one of the actions that happen on every keystroke
> 3. Tunr off semantic highlighting (if you didn't already)
> 3. Try out Eclipse Indigo. I recently used Juno for a few hours, and it
> definitely feels much slower.
> 4. Does it happen in every file, or only some files? You may encounter one

> of the corner cases of the type checker (seehttps://issues.scala-lang.org/browse/SI-5862)

>
> There is no way to turn off type checking, and all type-checking is done in
> the background thread (but sometimes the UI thread needs some result of
> type-checking, and needs to wait). We need to identify what is running in
> the UI thread when you type.
>
> iulian
>

> >>> deep inside scala.tools.nsc.**interactive.Global.**backgroundCompile,
> >>> which seems to be the culprit here. This does *not* seem to happen in

[virtualeyes]

I should say, max heap is approached during sbt clean/compile and
eclipse project clean

[iulian dragos]

On Sun, Oct 21, 2012 at 6:54 AM, virtualeyes <sit...@gmail.com> wrote:

Juno is definitely part of the problem here.

If you are switching between scala source file and an xml/html/js/css/
coffee etc. file in another tab, good luck, can be a delay of 3-4
seconds before cursor gains focus in target non-scala file.

As far as the Scala presentation compiler itself, standalone (without
sbt), it's actually pretty snappy in my experience. The only times I
notice a marked slowdown in performance is via sbt, particularly
incremental builds; at that point the IDE gets noticeably laggy.

There is, I believe, an ongoing issue with semantic highlighting and a
hanging thread, so I have grudgingly given up that very nice (on the
eyes) feature, which does give a slight performance boost (though not
much).

Please give it a try again, the issue should be fixed in recent nightlies (https://github.com/scala-ide/scala-ide/pull/208 is one part, the other part is in the presentation compiler).

iulian

[virtualeyes]

updating now, will be thrilled if the latest update solves the
semantic highlighting issue ;-)

Thanks

On Oct 21, 12:47 pm, iulian dragos <jagua...@gmail.com> wrote:

> On Sun, Oct 21, 2012 at 6:54 AM, virtualeyes <sit1...@gmail.com> wrote:
> > Juno is definitely part of the problem here.
>
> > If you are switching between scala source file and an xml/html/js/css/
> > coffee etc. file in another tab, good luck, can be a delay of 3-4
> > seconds before cursor gains focus in target non-scala file.
>
> > As far as the Scala presentation compiler itself, standalone (without
> > sbt), it's actually pretty snappy in my experience. The only times I
> > notice a marked slowdown in performance is via sbt, particularly
> > incremental builds; at that point the IDE gets noticeably laggy.
>
> > There is, I believe, an ongoing issue with semantic highlighting and a
> > hanging thread, so I have grudgingly given up that very nice (on the
> > eyes) feature, which does give a slight performance boost (though not
> > much).
>

> Please give it a try again, the issue should be fixed in recent nightlies (https://github.com/scala-ide/scala-ide/pull/208is one part, the other part

[virtualeyes]

I almost want to cry seeing the deep blue of my vals back in place --
code is relaxing to look at again ;-)

Not sure if the hung thread issue is resolved, will see today...

Thanks for working on the problem regardless, semantic highlighting is
such a help when working with dense scala code blocks.

On Oct 21, 12:47 pm, iulian dragos <jagua...@gmail.com> wrote:

> On Sun, Oct 21, 2012 at 6:54 AM, virtualeyes <sit1...@gmail.com> wrote:
> > Juno is definitely part of the problem here.
>
> > If you are switching between scala source file and an xml/html/js/css/
> > coffee etc. file in another tab, good luck, can be a delay of 3-4
> > seconds before cursor gains focus in target non-scala file.
>
> > As far as the Scala presentation compiler itself, standalone (without
> > sbt), it's actually pretty snappy in my experience. The only times I
> > notice a marked slowdown in performance is via sbt, particularly
> > incremental builds; at that point the IDE gets noticeably laggy.
>
> > There is, I believe, an ongoing issue with semantic highlighting and a
> > hanging thread, so I have grudgingly given up that very nice (on the
> > eyes) feature, which does give a slight performance boost (though not
> > much).
>

> Please give it a try again, the issue should be fixed in recent nightlies (https://github.com/scala-ide/scala-ide/pull/208is one part, the other part

[Mat]

I've tried Eclipse Indigo SR2 now instead of Juno, and with this editing Scala files feels much faster. Also, using Java 6 instead of 7 seems to help a bit.

[Sarah Gerweck]

(OP here.) This does not seem to have fixed the issue, though it is definitely better than it was. 

Turning off the occurrence annotations has helped quite a bit, although I'm not so sure I can do my job effectively if I turn off that feature across the Java code. I spend more time reviewing code than writing it these days, and the project has more Java than Scala. Can we get an option to turn off occurrence annotations only in Scala if it doesn't perform well enough to be used?

As far as the basic editor nonresponse, it still seems like I'm having the exact same symptoms with the latest nightlies. When I'm editing Scala files in the Scala editor, if I pause typing for more than a maybe 500 ms or move the cursor, the editor decides it needs to do type checking. This is a slow operation, and it blocks the editor until it's done, so that I can't even type. It ends up looking like a long pause after every few keys, which is still pretty much unusable.

I'll try to get you some stack traces in the next few days.

Thanks,

  Sarah

On Sunday, October 21, 2012 3:47:37 AM UTC-7, Iulian Dragos wrote:

Please give it a try again, the issue should be fixed in recent nightlies (https://github.com/scala-ide/scala-ide/pull/208 is one part, the other part is in the presentation compiler).

iulian

[Mirco Dotta]

As far as the basic editor nonresponse, it still seems like I'm having the exact same symptoms with the latest nightlies. When I'm editing Scala files in the Scala editor, if I pause typing for more than a maybe 500 ms or move the cursor, the editor decides it needs to do type checking.

Do you know you can tweak that number? Scala > Run Setup Diagnostic

This is a slow operation, and it blocks the editor until it's done, so that I can't even type.

That is unexpected, because the typechecker stops as soon as you start typing in the editor. If that doesn't happen, we need at least a thread dump to know what's going on.

It ends up looking like a long pause after every few keys, which is still pretty much unusable.

I'll try to get you some stack traces in the next few days.

Yes, please, do so. Furtheremore, use the latest nightly and re-enable Mark Occurrences, as there has been some small improvement and also added some more logging (make sure to set the log level to DEBUG - http://scala-ide.org/docs/helium/features/logging.html). When the IDE is unresponsive, have a look at the log, it may turn out to contain quite some useful information.

-- Mirco

[Mat]

On Wednesday, October 24, 2012 9:11:58 AM UTC+2, Mirco Dotta wrote:

As far as the basic editor nonresponse, it still seems like I'm having the exact same symptoms with the latest nightlies. When I'm editing Scala files in the Scala editor, if I pause typing for more than a maybe 500 ms or move the cursor, the editor decides it needs to do type checking.

Do you know you can tweak that number? Scala > Run Setup Diagnostic

When I experienced typing delays, I increased this value and also tried various values for Ypresentation-delay, but that didn't have any visible effect.  
(I cannot reproduce this using my current environment (with Indigo), since there are no delays anymore.)

[Sebastian Nozzi]

Can confirm for Eclipse Juno, Scala Plugins installed to an Eclipse J2EE edition from:

http://download.scala-ide.org/sdk/e38/scala210/stable/site

Running on Windows XP, JDK 1.7. On a Play2 project.

It completely froze the IDE. Also happened to me with the pre-packaged Scala-IDE.

:-(

[iulian dragos]

Could you please report a stack trace?

--
You received this message because you are subscribed to the Google Groups "Scala IDE User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-ide-use...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[iulian dragos]

You could use Svelto to automate this process:

https://github.com/dragos/svelto

It will automatically take a thread dump if the UI thread is blocked for more than a (configured) amount of time.

cheers,

iulian

[Sebastian Nozzi]

With the latest release-candidate I didn't have the problem any more.

So, I'll "downgrade" (?) to stable again and try to reproduce it, with the tool you suggested. If I have anything I'll let you know. 

Thanks,

Sebastian

[Sebastian Nozzi]

So far I can not reproduce the problem. It's not as bad as yesterday. But maybe this is something to get started.

In this thread dump there is a mention of a process waiting for 5 seconds. If I manage again to freeze Eclipse, I'll be back :-D

[Sarah Gerweck]

If you're not on Juno SR2, you definitely want to upgrade. Juno came with a lot of performance bugs, and they interact with the slow parts of Scala IDE (i.e., type checking) especially badly. 

I also think both Juno and Scala IDE have a tendency to slow down over time, so you may not notice real problems until you keep using the IDE for a while.

One thing I've found is absolutely required with Scala in Eclipse is turning off auto-build for projects as they get past a couple dozen source files. I just manually do a build from time to time or after completing a set of changes. Between this and Juno SR2, Scala IDE is usable for me even on pretty large projects.

—Sarah

On Jun 24, 2013, at 1:46 PM, Sebastian Nozzi wrote:

So far I can not reproduce the problem. It's not as bad as yesterday. But maybe this is something to get started.

In this thread dump there is a mention of a process waiting for 5 seconds. If I manage again to freeze Eclipse, I'll be back :-D

--
You received this message because you are subscribed to a topic in the Google Groups "Scala IDE User" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/scala-ide-user/fQ276wcs_tU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to scala-ide-use...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

<threadDumps.zip>

[iulian dragos]

Thanks Sebastian, please follow up if you manage to reproduce the slow-down.

In the attached thread dumps I see two instances:

- a block for the weaving agent loading a class (I assume this happens only when new bundles are initialized, so fairly rare after startup)

- a block while asking for completions, more precisely in the InSynth plugin. You could try disabling InSynth completions, in the `Content Assist/Advanced` tab in your Eclipse Preferneces

Don't hesitate to let us know if you find something else!

Regarding auto-building, I'm also building on-demand. It fits better with my style of work, but I know people building on save and being happy with it. It's a matter of preference and number of cores, I guess.

thanks,

iulian

On Mon, Jun 24, 2013 at 10:46 PM, Sebastian Nozzi <sebn...@gmail.com> wrote:

So far I can not reproduce the problem. It's not as bad as yesterday. But maybe this is something to get started.

In this thread dump there is a mention of a process waiting for 5 seconds. If I manage again to freeze Eclipse, I'll be back :-D

--
You received this message because you are subscribed to the Google Groups "Scala IDE User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-ide-use...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Nils Kilden-Pedersen]

On Sat, Oct 20, 2012 at 3:14 AM, iulian dragos <jagu...@gmail.com> wrote:

On Sat, Oct 20, 2012 at 12:18 AM, Sarah Gerweck <sarah...@gmail.com> wrote:

One last note: this doesn't happen only after making edits. I also get long pauses while Eclipse is "Updating occurrence annotations." E.g., the Scala file I was just trying to edit was about 400 lines and it proved impossible to use Eclipse's Scala editor.

I'd be happy to provide stack dumps or whatever else might be useful. Unfortunately though, I can't really provide the sources or the project I'm using.

Thread dumps gold! Please do that.

Iulian, I have submitted 4 tickets with thread dumps, yet they are still "New" and doesn't appear anyone has looked at them.

• https://scala-ide-portfolio.assembla.com/spaces/scala-ide/tickets/1001626
• https://scala-ide-portfolio.assembla.com/spaces/scala-ide/tickets/1001634
• https://scala-ide-portfolio.assembla.com/spaces/scala-ide/tickets/1001643
• https://scala-ide-portfolio.assembla.com/spaces/scala-ide/tickets/1001660

Am I doing something wrong when submitting them?

[Simon Schäfer]

> Am I doing something wrong when submitting them?

Don't be disappointed if no one reacts to your tickets. There are just
too much to handle them all, especially the ones like yours that have a
difficult behavior to reproduce.

[Sebastian Nozzi]

Hi!

Attached goes my last threadDump file... Today I had a large freeze with the InSynth plugin.

I will now switch back to the latest RC (where these "hiccups" don't seem to happen ;-)

Thanks for taking time in looking into this!

Cheers,

Sebastian

[iulian dragos]

On Tue, Jun 25, 2013 at 4:33 PM, Nils Kilden-Pedersen <ni...@kilden-pedersen.net> wrote:

On Sat, Oct 20, 2012 at 3:14 AM, iulian dragos <jagu...@gmail.com> wrote:

On Sat, Oct 20, 2012 at 12:18 AM, Sarah Gerweck <sarah...@gmail.com> wrote:

One last note: this doesn't happen only after making edits. I also get long pauses while Eclipse is "Updating occurrence annotations." E.g., the Scala file I was just trying to edit was about 400 lines and it proved impossible to use Eclipse's Scala editor.

I'd be happy to provide stack dumps or whatever else might be useful. Unfortunately though, I can't really provide the sources or the project I'm using.

Thread dumps gold! Please do that.

Iulian, I have submitted 4 tickets with thread dumps, yet they are still "New" and doesn't appear anyone has looked at them.

• https://scala-ide-portfolio.assembla.com/spaces/scala-ide/tickets/1001626
• https://scala-ide-portfolio.assembla.com/spaces/scala-ide/tickets/1001634
• https://scala-ide-portfolio.assembla.com/spaces/scala-ide/tickets/1001643
• https://scala-ide-portfolio.assembla.com/spaces/scala-ide/tickets/1001660

Hi Nils,

They all look to be frozen in a similar fashion. Unfortunately, there's a race condition in the presentation compiler that we didn't manage to reproduce (you seem to be hitting it fairly often). I'll try to look into it a bit more (I wonder if you see the same thing with 2.10, all of these traces are based on the 2.9 version).

iulian

 

Am I doing something wrong when submitting them?

 

iulian

 

--
« Je déteste la montagne, ça cache le paysage »
Alphonse Allais

--
You received this message because you are subscribed to the Google Groups "Scala IDE User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-ide-use...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[iulian dragos]

Hi!

I suggest you uninstall InSynth, or at least disable it from the Contant Assist Advanced preferences. It seems to be causes all these lags. Ivan Kuraj, the author of this plugin, might be able to fix it, but in the meantime just live without it ;-)

cheers,

iulian

--
You received this message because you are subscribed to the Google Groups "Scala IDE User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-ide-use...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Ivan Kuraj]

Hi,

Yes, after inspecting the stack trace I can concur that indeed InSynth seems to be the problem.

Sebastian, is it possible to recreate this situation so we can have a closer look?

Cheers,
Ivan