Reason behind design decisions to couple Linter and Analyzer

[Patt O'Brien]

Hi Analyzer Team,

Thanks for all of your effort on the analyzer ecosystem, over the past year especially. The APIs are definitely becoming way more intuitive.

I'm looking to get some insight into past design decisions involving the linter and analyzer, in hopes that some insight would save me a couple days/weeks of experimenting with my own project. I've just completed a prototype Analyzer Plugin that I'm creating to help me define custom lints. With a single lint/code assist complete, I'm now exploring the feasibility and performance of scaling my plugin to N lints. FWIW, I enjoy learning about/building tools like this, so I'd say I'm 50/50 with doing this for learning as well as the possibility of contributing an experimental prototype for others to use. :)

A specific question: Why are all lint Rules defined in a separate Linter package and connected to the Analyzer via the NodeLintRegistry, while the code Corrections themselves are defined within the AnalysisServer package? Is there a reason the same level of abstraction could not be applied to the Corrections? (I think I read somewhere that performance was the reason for tighter coupling within lints + analyzer)

Thanks in advance for any insight you have here. I'm looking forward to sharing not only my plugin experience with the community, but also the lint use-cases that I'd like for my projects (once they're more concretely defined).

Best,

Patt

[Brian Wilkerson]

Why are all lint Rules defined in a separate Linter package and connected to the Analyzer via the NodeLintRegistry, while the code Corrections themselves are defined within the AnalysisServer package? Is there a reason the same level of abstraction could not be applied to the Corrections?

The lints are defined in a separate package primarily for historic reasons. We initially thought we'd ship a separate linter tool, but later decided that it would be better for users if they didn't need to run separate tools, install separate IDE plugins, etc. in order to get all of the analysis support we were building.

We've thought a few times about merging the `linter` package into the `analyzer` package, but we have a significant number of  external contributors to the `linter` package and believe that part of the reason for that is the familiarity of the GitHub workflow (as opposed to the workflow using Gerrit that's required to contribute to the `sdk` repo). Not wanting to discourage external contributions, we have so far decided not to merge them or even to move the `linter` package into the `sdk` repo (even though doing so would have some benefits in terms of our process).

The reason the "Corrections" (or "quick fixes", in server implementation terms) aren't in the `linter` package is because they depend on too much functionality that's already in the `analysis_server` package. We don't want to publish the `analysis_server` package, which makes it impossible for the `linter` package to depend on it.

(I think I read somewhere that performance was the reason for tighter coupling within lints + analyzer)

Correct. We have, as you know, a proof-of-concept implementation of plugins, but they're not performant enough for us to officially ship the support or recommend that anyone use it. If we are able to solve the performance issues, then we might well consider making the linter be a plugin to the analyzer. But until the performance issues have been addressed, making the linter a plugin would not be in the best interests of our users.

I'm happy to answer any other specific questions you might have.

--
You received this message because you are subscribed to the Google Groups "Dart Analyzer Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to analyzer-discu...@dartlang.org.
To view this discussion on the web visit https://groups.google.com/a/dartlang.org/d/msgid/analyzer-discuss/a230778a-2a73-448b-88cf-db262eff5a10n%40dartlang.org.

[Patt O'Brien]

Seriously, thanks so much for all of this info Brian. I've been collecting my thoughts while taking your notes into consideration, as I've been revisiting the analyzer code. It's great to know that the organization of linter/analyzer code is mostly because of historical reasons, and that there's already an interest in decoupling the systems.

It's interesting that you brought up lint contributors being a primary consideration for past decisions, as collaboration is part of what I'm getting after with my own plugin project. 

Basically, I've been wondering what the implications would be if there was a better ecosystem for creating / sharing lint rules. The analyzer_plugin package is a huge step in the right direction, but can we do even better by not requiring an author of a single lint (which may not make sense to be added to the greater "linter" package) to release and maintain their own plugin? Could more analyzer use cases be unlocked by facilitating a more mix-and-match, collaborative lint ecosystem, based upon a single analyzer plugin? Use cases such as:

- Defining single-use project-specific rules, e.g. a UI Design System that only allows use of globally-defined brand colors

- Using analysis errors to teach developers how to properly use various architectures, design patterns, etc.

- Package developers creating package-specific refactorings or code assists to more effectively distribute package-specific boilerplate and guidelines

As someone who's building up an app dev team around Flutter and (eventually) Dart for backend, being able to pick and choose from a large variety of lints would also help me create very strict opinionated guidelines for newly-hired devs to adhere to, which would significantly speed up the onboarding process and allow me to utilize more junior devs while being able to more efficiently "scale" senior-level expertise in the form of rules.

Performance implications aside, all of the above is technically possible using analyzer_plugin, but I think the barrier to entry would still be too high to facilitate the open sharing of diverse and oftentimes conflicting individual rules. A potential solution that I hope to make a POC of this week would be a pub-like system of downloading and caching a single dart file containing a class that inherits from LintRule, which would be dynamically bootstrapped to the analyzer plugin. In the end, a configuration file would look very similar to the list/map of rules added to "linter" section of analysis_options.yaml, despite being a melting pot of a dozen different architectures, packages, and opinions.

I know this proposal poses significant challenges and implications, and I feel like it would probably make most sense as a community-driven project, but I'm curious if these challenges / solutions is something that you've thought of and/or if you see the future of analyzer_plugin facilitating some of these use cases?

[Osama Alraddadi]

Hi Patt, 

> Basically, I've been wondering what the implications would be if there was a better ecosystem for creating / sharing lint rules. The analyzer_plugin package is a huge step in the right direction, but can we do even better by not requiring an author of a single lint (which may not make sense to be added to the greater "linter" package) to release and maintain their own plugin? 

You may find the following package/project interesting by the way:

https://pub.dev/packages/custom_lint

I believe its maintainers share a similar vision to the one you outlined but I'm not sure to what extent it can achieve all the points you mentioned.   

I hope you find it useful if you don't already know about it and thanks for starting such an interesting discussion.

Thanks,

Osama 

[Patt O'Brien]

> I believe its maintainers share a similar vision to the one you outlined but I'm not sure to what extent it can achieve all the points you mentioned. 

Thanks for the notes, and yes I have seen the Invertase package :) It's a huge step in the right direction and I think collaboration would make more sense than duplicating their work. I wanted to first get a working PoC for fetching and dynamically initializing lints (which I was able to finish up today), and so a draft proposal detailing a hypothetical architecture is the next logical step I think. Not sure exactly how perfect of a fit the custom_lint APIs will be, but I foresee our ideas being compatible.

Not sure if you're following the open issue against the SDK, but I plan on posting this idea there to get some feedback; would love to get your thoughts at that time.

[Brian Wilkerson]

... I'm curious if these challenges / solutions is something that you've thought of and/or if you see the future of analyzer_plugin facilitating some of these use cases?

We've definitely thought about it. If the analyzer_plugin package has any future it will be to facilitate something like this.

Not sure if you're following the open issue against the SDK, ...

The link for "open issue" is to search results for "dart custom lint", so I'm not sure which issue you were referring to.

... but I plan on posting this idea there to get some feedback; would love to get your thoughts at that time.

I am eager to see your thoughts and ideas.