Analyzer memory consumption after updating to Dart 2.19

[Dmitry Zhifarsky]

Hi, yesterday after updating to Flutter 3.7 and Dart 2.19 I've started to notice higher analyzer memory consumption and high latency to save a file after the "save file" shortcut is pressed.

So, before the Dart 2.19 the repo I have (https://github.com/dart-code-checker/dart-code-metrics) was consuming around ~1 GB (with the DCM analyzer plugin), but now it consumes up to 4 times more (see the attachments). Note that this is a pure Dart repo with no Flutter code.

As for saving a file, I use VS Code and have "dart fix" enabled via 

`"editor.codeActionsOnSave": {

   "source.fixAll": true,

   "source.organizeImports": true,

},`
and after the update I've started to regularly (but not in every file and haven't noticed any pattern yet) see a VS Code notification with "Saving file..." which can take like tens of seconds to complete on a small files (like 100 LoC). 

Should I create an issue or something might be wrong on my end?

[Slava Egorov]

 Hi Dmitry!

@Jens Johansen could you take a look at this? 

We will probably need to take some heap snapshots to get an idea of what is going on. 

--
You received this message because you are subscribed to the Google Groups "analyzer-scalability" group.
To unsubscribe from this group and stop receiving emails from it, send an email to analyzer-scalabi...@dartlang.org.
To view this discussion on the web visit https://groups.google.com/a/dartlang.org/d/msgid/analyzer-scalability/d9962e27-5c69-41ff-82ad-705c36111af0n%40dartlang.org.
For more options, visit https://groups.google.com/a/dartlang.org/d/optout.

[Samuel Rawlins]

Thanks for the report, Dmitry!

One question, to clarify: do you have any analyzer plugins _running_ in VS Code? Or just editing one? If you do have one running, does the situation change if you disable it? Thanks!

[Jacob Richman]

Do you have precise repro steps? When I open VSCode on the root folder in g...@github.com:dart-code-checker/dart-code-metrics.git, I only see 1GB of memory usage from the analyzer.

To view this discussion on the web visit https://groups.google.com/a/dartlang.org/d/msgid/analyzer-scalability/4e258102-0f17-4aa8-915c-db0c0790a937n%40dartlang.org.

[Dmitry Zhifarsky]

Yes, I have the plugin enabled, disabling it shows normal memory consumption, but the plugin code hasn't changed after I updated to 2.19. Can anything got broken in the integration part?

пятница, 27 января 2023 г. в 21:56:37 UTC+4, sraw...@google.com:

[Dmitry Zhifarsky]

Just writing code / tests, switching between branches, nothing special. No exact steps, but the problem is constantly present.

пятница, 27 января 2023 г. в 22:09:29 UTC+4, jac...@google.com:

[Brian Wilkerson]

... but the plugin code hasn't changed after I updated to 2.19. Can anything got broken in the integration part?

Did you update the version of the `analyzer` package that the plugin is using? (It isn't necessary to do so, I'm just trying to narrow down the cause of the issue.)

To view this discussion on the web visit https://groups.google.com/a/dartlang.org/d/msgid/analyzer-scalability/235f5c4b-336d-48a3-a762-8b043584a942n%40dartlang.org.

[Jacob Richman]

Repro steps:

git checkout 2984e3b8a023aaee1466dd1e16ba33239ca194e7
git check origin/master

repeat 20X without doing anything else and memory usage grows fairly quickly. Not as much as described in the bug but I got it to 1.70 GB before getting bored.

[Dmitry Zhifarsky]

Previous analyzer update (to 5.4.0) was 3 weeks ago, but we published the new DCM version only 4 days ago. So there might be a chance it's actually the update of the analyzer, not SDK, but I'm not sure how to measure it properly.

To Jacob:
I have the VS Code opened in the background doing nothing and it's already 2GB. And it's just growing slowly.

пятница, 27 января 2023 г. в 22:16:34 UTC+4, brianwi...@google.com:

[Dmitry Zhifarsky]

2.7 GB 🙂 interesting part that the real mem consumption is also high

пятница, 27 января 2023 г. в 22:23:48 UTC+4, Dmitry Zhifarsky:

[Dmitry Zhifarsky]

It went up to 3 GB and stopped growing (even the coolers have stopped making noise). 

пятница, 27 января 2023 г. в 22:25:29 UTC+4, Dmitry Zhifarsky:

[Konstantin Shcheglov]

  Meanwhile, in the Observatory.

  9 or 10 plugins running, after switching to a branch and back to `master`.

  Not all of them fit on the screen.

You received this message because you are subscribed to a topic in the Google Groups "analyzer-scalability" group.
To unsubscribe from this topic, visit https://groups.google.com/a/dartlang.org/d/topic/analyzer-scalability/J6PydDEthKo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to analyzer-scalabi...@dartlang.org.
To view this discussion on the web visit https://groups.google.com/a/dartlang.org/d/msgid/analyzer-scalability/5ad81558-11f8-4f60-832d-0e9e364a99e0n%40dartlang.org.

For more options, visit https://groups.google.com/a/dartlang.org/d/optout.

--

Konstantin Scheglov

Software Engineer

Google, Inc.

[Dmitry Zhifarsky]

Checked the DCM plugin version, it's actually an old version running, so I'm not sure that the problems is with the analyzer update.

To Konstantine:
Can I collect anything from my device that would be helpful?

пятница, 27 января 2023 г. в 22:34:26 UTC+4, sche...@google.com:

[Dmitry Zhifarsky]

Dart info log:

пятница, 27 января 2023 г. в 22:43:32 UTC+4, Dmitry Zhifarsky:

[Dominik Roszkowski]

Unfortunately I was not able to capture logs, but this is what happened to me today. I believe this was a rogue process left running after I closed VS Code an hour before. What should be the steps if it ever happens to me that would help you diagnose the issue?

Cheers,

Dominik Roszkowski

To view this discussion on the web visit https://groups.google.com/a/dartlang.org/d/msgid/analyzer-scalability/777e85bd-df8c-4644-aa71-605d854b6ca3n%40dartlang.org.

[Konstantin Shcheglov]

On Fri, Jan 27, 2023 at 10:43 AM Dmitry Zhifarsky <dimann...@gmail.com> wrote:

Checked the DCM plugin version, it's actually an old version running, so I'm not sure that the problems is with the analyzer update.

To Konstantine:
Can I collect anything from my device that would be helpful?

  I don't think so.

  This looks like a bug in the Dart analysis server - we should either stop, or kill plugins.

  Apparently we fail to do this, and this is what causes the memory leak - a lot of isolates running plugins.

[Dmitry Zhifarsky]

>  I don't think so.

👍

As for long saving - I have a stable repro on the version.dart file we have https://github.com/dart-code-checker/dart-code-metrics/blob/master/lib/src/version.dart. It happens literally every version update (see the attachment). Can I provide any additional info to help fix this?

суббота, 28 января 2023 г. в 01:25:00 UTC+4, sche...@google.com:

[Samuel Rawlins]

Hi Dmitry, I unfortunately cannot reproduce. I did the following

1. git clone g...@github.com:dart-code-checker/dart-code-metrics.git dart-dart-code-metrics

2. cd dart-dart-code-metrics
3. Install Dart 2.19

4. In VS Code, open a window to dart-dart-code-metrics/ dir.

5. In problems panel I see 5 diagnostics.

6. In Dart Code settings, add `"source.fixAll": true, "source.organizeImports": true,` to `editor.codeActionsOnSave`.
7. Open `lib/src/version.dart`.

8. Repeatedly change the string and save the file.

It always saves instantaneously. Can you give us some more data points?

* VS Code version (I'm on 1.74.2)

* Dart Code version (I'm on 3.58.0)

* OS (I'm on Mac Ventura 13.1)

* In the Analysis Server Diagnostics page:

  * the Analysis server version: (I'm on 1.33.4), Dart SDK (2.19.0 (stable)), 

  * Communications tab, like  average latency and maximum latency

  * number of contexts in the Contexts tab (I have 2)

  * Exceptions tab?

  * Timing tab?

[Dmitry Zhifarsky]

Interesting, tried to update the file right after the IDE has loaded and also got an instant response.

But, I usually update this file after writing code / switching between branches and according to this guide https://github.com/dart-code-checker/dart-code-metrics/blob/master/RELEASE_GUIDE.md. And now I see that the pubspec version update (which restarts the analysis server and with all the extra plugins might be the reason for such slow response) is listed there before the version update.

Could you please try to switch between branches and follow this guide? I was not able to repro a long delay (only like 5-6 seconds). If I get the long delay again, I'll report it here.

Note: "format on save" setting is also enabled.

VS Code: 1.74.3
Dart Code: 3.58
OS: (see the attachments)
Analysis server version: 1.33.4
Dart SDK: 2.19.0 (stable) (Mon Jan 23 11:29:09 2023 -0800) on "macos_x64"
Communications tab: (see the attachments)
Contexts: 2
Exceptions: empty
Timing: (see the attachments)

вторник, 31 января 2023 г. в 04:34:27 UTC+4, sraw...@google.com:

[Samuel Rawlins]

Thanks for all of the stats. Nothing seems amiss. I cannot reproduce slow saves even after many rapid `git checkout`s (I used `git checkout 2984e3b8a023aaee1466dd1e16ba33239ca194e7` and `git checkout origin/master` as Jacob used). Let us know if you reproduce again.

[Dmitry Zhifarsky]

Here is the video, reproduced just now. Maybe the device you have is too good? 😅

вторник, 31 января 2023 г. в 22:28:31 UTC+4, sraw...@google.com:

[Samuel Rawlins]

OK there is one difference between your recording and my experience, I believe. When I save a pubspec.yaml file, VS Code pops up with a toast saying that `pub get` is running, but I don't see that in your video. Do you not have `pub get` automatically running? I'll have to look up which setting that is, probably a Dart Code setting.

[Dmitry Zhifarsky]

Yes, it's disabled. I disabled it long ago, even don't remember the exact reason (something with performance / general convenience).

вторник, 31 января 2023 г. в 23:44:52 UTC+4, sraw...@google.com: