Really slow analyzer

[Samuel Peirson]

Hi! We are currently struggling with what feels like extremly slow analysis in out Flutter project, and are wondering if there is a concrete way to determine what might be causing this.

Any help or feedback in this regard will be appreciated - happy to provide any additional detail.

Thanks

[Jeff Ward]

I was just talking about this in the Dart Community Discord, but figured my problems were unique to my project.

Some more information on my side, as what I'm working on is open source (link: https://github.com/fuzzybinary/godot_dart)

Godot Dart generates a large amount of code to interact with Godot (around 1000 files), which is held in three main libraries, and every other file needs to usually needs to import least one of them. On top of that it has a large FFI file that's imported into most files.

Even if I'm only working in the main `godot_dart` package, initial analysis takes about 47 seconds in Linux (Zorin) with a Intel 13700K. Subsequent analysis is near instant, but modifying any code file in the repo seems to cause almost a full re-analyze.

More frustring, (though this may be specific to Visual Studio Code) is that formatting files, even small files, also seems to take a really long time, and VSCode won't save the file until it has finished the format. I'm not sure why project size affects formatting individual files, unless the hang in the analyzer is causing the format command to get locked up.  This slows development time significantly as it seems like contention between the formatter and the analyzer affects all the IDE features, from saving files, to completion, to error detection.

I'm curious if there's something in the package that I can fix that would speed up analysis speed.

[Brett Sutton]

I'm seeing similar issues in a project with 40kloc - vs code 

https://github.com/bsutton/hmb/tree/wip

Slow rename which sometimes just doesn't complete. 

Slow on first pass to analyse.

Slow analysis after adding a package. 

Slow displaying a call hierarchy.

I've also seen the format/save issue but not recently. 

--
You received this message because you are subscribed to the Google Groups "Dart Analyzer Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to analyzer-discu...@dartlang.org.
To view this discussion visit https://groups.google.com/a/dartlang.org/d/msgid/analyzer-discuss/4b8369fc-e70c-4749-9ea7-301f4a19479bn%40dartlang.org.

[Brian Wilkerson]

Thanks for bringing this to our attention.

We have been investigating the performance issues, and are working on ways to significantly improve it. One of the things that we know causes problems is having large library cycles. The analyzer works better on code that has a more layered architecture. The tools shouldn't enforce a particular coding practice, so we're working on solving that problem, but reducing circular dependencies in the library graph can improve performance (and some people believe that doing so is good for other reasons).

There have been other reports recently of issues related to the speed of the formatter. We're looking into it, but I don't have any information at this point.

To view this discussion visit https://groups.google.com/a/dartlang.org/d/msgid/analyzer-discuss/CAGYsEwgte-mTx3u6X0ss6r3SBFVjzoivO-51Uh61Exxp1h%3DyBQ%40mail.gmail.com.

[brianwilkerson]

Out of curiosity, have you noticed any performance problems when running `dart format` over a single file?

[Brett Sutton]

I don't run dart fix over a single file - I always run it across the entire project. 

To view this discussion visit https://groups.google.com/a/dartlang.org/d/msgid/analyzer-discuss/cfa9c7c3-99d5-4390-ae09-cda1aff7f032n%40dartlang.org.

[Jeff Ward]

When running from the command line, dart fix, dart format and dart analyze are fairly fast when constrained to a single file or single directory (2s max for dart fix, 0.11s for dart format)  But even for minor changes in my package, dart analyze can take significant time if I jump up to the root, even for small changes (42 seconds on last run).

I'm guessing, but I think the VSCode slowness comes from it trying to dart fix / dart format across the whole project on save, and / or it's waiting on the analyzer, and that causes a command backup.

I'm going to try to refactor a bit in godot_dart and maybe see if I can avoid importing one of the libraries (the one with 1000 generated parts), and instead only export it.

By the way, using this package doesn't seem to be too much of an issue. It's only when editing the package itself that I get slowness. For others facing a similar issue, if it is possible to refactor a subset of your code into another package, that might speed your analyzer.

To view this discussion visit https://groups.google.com/a/dartlang.org/d/msgid/analyzer-discuss/CAGYsEwgeeS4-k95GXs_DYYM3gWQSvpkR9kmE9wYpFj0jGeX2mA%40mail.gmail.com.

[Danny Tuppeny]

> But even for minor changes in my package, dart analyze can take significant time if I jump up to the root, even for small changes (42 seconds on last run).

Do you have steps that reliably reproduce this kind of slow analysis? (for example starting from a specific commit and making a specific change in a specific file)?

> I'm guessing, but I think the VSCode slowness comes from it trying to dart fix / dart format across the whole project on save, and / or it's waiting on the analyzer, and that causes a command backup.

One way to check this could be to disable format-on-save and fix-on-save and see how long you see "Analyzing..." in the status bar after making the same change. If it's approximately the same, the fix/format is probably just blocked behind the reanalysis. If it seems faster, then they might be contributing to it. In either case, a good repro for this could be quite useful for profiling and testing.

[Jeff Ward]

Do you have steps that reliably reproduce this kind of slow analysis? (for example starting from a specific commit and making a specific change in a specific file)?

Yes - Godot Dart is actually a great stress test for the analyzer at this point. (https://github.com/fuzzybinary/godot_dart).  You first have to generate all of the Godot interop (run dart bin/binding_generator.dart in tools/binding_generator).  The file that I was working on that has the most trouble is the signals,dart file (https://github.com/fuzzybinary/godot_dart/blob/main/src/dart/godot_dart/lib/src/core/signals.dart). Adding a private method to SignalCallable was enough to get a slow analyze pass.

From further testing, it does appear that the number of imports does not seem to matter, but the number of places the target file is imported, or possibly the number of places the target class is used. Since SignalCallable and its subclasses are used in most of the generated code, that might be part of the problem.

 

> I'm guessing, but I think the VSCode slowness comes from it trying to dart fix / dart format across the whole project on save, and / or it's waiting on the analyzer, and that causes a command backup.

One way to check this could be to disable format-on-save and fix-on-save and see how long you see "Analyzing..." in the status bar after making the same change. If it's approximately the same, the fix/format is probably just blocked behind the reanalysis. If it seems faster, then they might be contributing to it. In either case, a good repro for this could be quite useful for profiling and testing.

I'll give this a try, but my guess is that format / fix is blocked behind the analyzer. If VSCode's wait order is analyze -> fix -> format -> save, then that's definitely the case.

 

To view this discussion visit https://groups.google.com/a/dartlang.org/d/msgid/analyzer-discuss/d8aae7c9-ebb6-4340-b4c5-24c26b7db8ddn%40dartlang.org.

[Danny Tuppeny]

Thanks, I can repro your issue. Modifying that file triggers work that takes around 60 seconds on my machine. Practically all of that time is spent analyzing engine_classes.dart:

+++ Compute analysis result for D:\Dev\Temp Projects\godot_dart\src\dart\godot_dart\lib\src\gen\engine_classes.dart.
Work in D:\Dev\Temp Projects\godot_dart\src\dart\godot_dart
+++ Prepare linked bundles.
[librariesTotal: 0][librariesLoaded: 0][librariesLinked: 0][librariesLinkedTimer: 0 ms][bytesGet: 0][bytesPut: 0]
--- Prepare linked bundles in 0 ms.
Computed new analysis result.
--- Compute analysis result for D:\Dev\Temp Projects\godot_dart\src\dart\godot_dart\lib\src\gen\engine_classes.dart in 59794 ms.

That library is made up of almost a thousand parts that are analyzed as one. I'll defer to Brian or Konstantin about whether there might be a better way to structure this that might analyze faster as I'm less familiar with the analyzer internals.

> I'll give this a try, but my guess is that format / fix is blocked behind the analyzer. If VSCode's wait order is analyze -> fix -> format -> save, then that's definitely the case.

VS Code doesn't directly wait for any analysis (the analysis server analyzes as files change and pushes diagnostics to VS Code), however an operation like computing fixes will require a resolved AST for the latest version of the document, so this will have to wait on analysis to complete for that library. From the timings I see, I don't think the format/fixes are themselves a contributor here, I think they're just blocked behind the analysis. I believe formatting should only require a parsed (not resolved) result which shouldn't need to wait for a resolved result, though I think the analysis of a single library may be synchronous and therefore won't allow the format request to start while the thread busy.

[Jeff Ward]

I did have similar issues when it was split, for what it's worth. I was actually hoping that merging them together would make things better, though there was a secondary reason as well because of a Godot Dart use case (I need to be able to find classes by name, which is easier if they're in a single library / package).

I'm wondering if I can easily turn off analysis for that file, as it's generated. I only need it to be analyzed right after re-generation, but I'm not sure if that's an option.

I'm interested to hear if Brian or Konstantin have any recommendations.

To view this discussion visit https://groups.google.com/a/dartlang.org/d/msgid/analyzer-discuss/d6245eb0-4185-4a41-9314-cedb35f0b0e4n%40dartlang.org.