MonoDevelop Addin Ideas (again)
2 views
Skip to first unread message
Michael Hutchinson
Jun 12, 2008, 7:14:36 PM6/12/08
to Gendarme
Hey all,
I've just committed the MonoDevelop.CodeAnalysis addin from Google
Code to MonoDevelop extras, after fixing up the autotools build and
making sure it works against trunk. However, when using it I ran into
some usability problems, so I thought I'd start another thread of
ideas (since it's a bit late to resurrect the old one).
Requirements
==========
* MonoDevelop shouldn't link against Gendarme, at least until the APIs
are stable. It certainly shouldn't ever load the rules assemblies
(because of memory use and uinstability), but might need to scan them
for options in order to generate a GUI.
* Users should be able to specify ignoring specific violations from a
context menu on the list of violations.
* Users should be able to select which rules they want to use.
* Rules should be grouped by severity.
* The analysis should be an MSBuild target, so that it can be executed
from the command line with xbuild.
* Users must be able to jump to the offending code from the violations
list (this is the entire point of fully integrating it in MD).
Possible features
=================
* Multiple rules profiles -- allow the user to specify different
profiles of rules, so that it's possible to run just the critical
ones, or a full scan.
* Allow running certain analysis profiles as a postbuild step.
* Rules violations could display an explanation and example.
* Analysis history -- display a graph of pass/fail history.
* Allow suppressing violations from autogenerated code (could be as
simple as ignoring anything from a *.designer.* file, or could use the
MSBuild attributes that VS uses to mark autogenerated code files)
* GUI for rule options.
* Seach/filter violations and rules, since the lists could get pretty
big.
* Ability to download new/updated rules from Mono.Addins repositories.
* Rules rated both on severity and probability of being correct.
* Allow requiring code analysis without before checking in to VCS.
* Use the MD Code DOM to find things not listed in MDB files.
* Automatic/suggested fixes using MD refactoring tools, e.g. "Click
here to rename the methods to..."
Questions
=========
* Should the gendarme and smokey addins be merged, or simply share
some base functionality (widgets etc)? I'd suggest the latter, because
I don't think there's any way to unify the options and output cleanly,
especially since many of the rules are duplicated.
* Should the options be stored in the project files, or special
gendarme files? I think that cluttering the project files would be a
bad idea.
* Should the analysis options be per-project or per-solution? Or
should there be a way of overriding solution preferences in a project?
Is it worth scanning multiple assemblies at once? More radically (but
more flexibly), could there be such a thing as an "Analysis project"
that can reference a set of projects and constian different analysis
profile files?
* What will be the relationship in MD with other analysis tools?
Sharing an "Analyse" menu and setting section?
Other analysis tools
====================
* C# source style analyser: http://code.msdn.microsoft.com/sourceanalysis
* Smokey: http://code.google.com/p/smokey/
* FXCop, when MD runs on windows
* A number of commercial tools
Interesting links
=================
Analyse menu: http://blogs.msdn.com/fxcop/archive/2007/09/22/new-for-visual-studio-2008-the-analyze-menu.aspx
Loaction lookup:
http://blogs.msdn.com/fxcop/archive/2007/05/12/faq-why-is-file-and-line-information-available-for-some-warnings-in-fxcop-but-not-for-others.aspx
Code metrics: http://blogs.msdn.com/fxcop/archive/2007/10/03/new-for-visual-studio-2008-code-metrics.aspx
Check-in policies:
http://blogs.msdn.com/fxcop/archive/2007/09/20/new-for-visual-studio-2008-code-analysis-policy-improvements.aspx
Per-project custom spelling dictionary:
http://blogs.msdn.com/fxcop/archive/2007/08/20/new-for-visual-studio-2008-custom-dictionaries.aspx
Suppressing messages: http://blogs.msdn.com/fxcop/archive/2006/12/21/two-new-features-for-orcas.aspx
Sharing settings between project using MSBuild:
http://blogs.msdn.com/fxcop/archive/2006/11/16/faq-how-do-i-share-managed-code-analysis-rule-settings-over-multiple-projects-david-kean.aspx
--
Michael
I've just committed the MonoDevelop.CodeAnalysis addin from Google
Code to MonoDevelop extras, after fixing up the autotools build and
making sure it works against trunk. However, when using it I ran into
some usability problems, so I thought I'd start another thread of
ideas (since it's a bit late to resurrect the old one).
Requirements
==========
* MonoDevelop shouldn't link against Gendarme, at least until the APIs
are stable. It certainly shouldn't ever load the rules assemblies
(because of memory use and uinstability), but might need to scan them
for options in order to generate a GUI.
* Users should be able to specify ignoring specific violations from a
context menu on the list of violations.
* Users should be able to select which rules they want to use.
* Rules should be grouped by severity.
* The analysis should be an MSBuild target, so that it can be executed
from the command line with xbuild.
* Users must be able to jump to the offending code from the violations
list (this is the entire point of fully integrating it in MD).
Possible features
=================
* Multiple rules profiles -- allow the user to specify different
profiles of rules, so that it's possible to run just the critical
ones, or a full scan.
* Allow running certain analysis profiles as a postbuild step.
* Rules violations could display an explanation and example.
* Analysis history -- display a graph of pass/fail history.
* Allow suppressing violations from autogenerated code (could be as
simple as ignoring anything from a *.designer.* file, or could use the
MSBuild attributes that VS uses to mark autogenerated code files)
* GUI for rule options.
* Seach/filter violations and rules, since the lists could get pretty
big.
* Ability to download new/updated rules from Mono.Addins repositories.
* Rules rated both on severity and probability of being correct.
* Allow requiring code analysis without before checking in to VCS.
* Use the MD Code DOM to find things not listed in MDB files.
* Automatic/suggested fixes using MD refactoring tools, e.g. "Click
here to rename the methods to..."
Questions
=========
* Should the gendarme and smokey addins be merged, or simply share
some base functionality (widgets etc)? I'd suggest the latter, because
I don't think there's any way to unify the options and output cleanly,
especially since many of the rules are duplicated.
* Should the options be stored in the project files, or special
gendarme files? I think that cluttering the project files would be a
bad idea.
* Should the analysis options be per-project or per-solution? Or
should there be a way of overriding solution preferences in a project?
Is it worth scanning multiple assemblies at once? More radically (but
more flexibly), could there be such a thing as an "Analysis project"
that can reference a set of projects and constian different analysis
profile files?
* What will be the relationship in MD with other analysis tools?
Sharing an "Analyse" menu and setting section?
Other analysis tools
====================
* C# source style analyser: http://code.msdn.microsoft.com/sourceanalysis
* Smokey: http://code.google.com/p/smokey/
* FXCop, when MD runs on windows
* A number of commercial tools
Interesting links
=================
Analyse menu: http://blogs.msdn.com/fxcop/archive/2007/09/22/new-for-visual-studio-2008-the-analyze-menu.aspx
Loaction lookup:
http://blogs.msdn.com/fxcop/archive/2007/05/12/faq-why-is-file-and-line-information-available-for-some-warnings-in-fxcop-but-not-for-others.aspx
Code metrics: http://blogs.msdn.com/fxcop/archive/2007/10/03/new-for-visual-studio-2008-code-metrics.aspx
Check-in policies:
http://blogs.msdn.com/fxcop/archive/2007/09/20/new-for-visual-studio-2008-code-analysis-policy-improvements.aspx
Per-project custom spelling dictionary:
http://blogs.msdn.com/fxcop/archive/2007/08/20/new-for-visual-studio-2008-custom-dictionaries.aspx
Suppressing messages: http://blogs.msdn.com/fxcop/archive/2006/12/21/two-new-features-for-orcas.aspx
Sharing settings between project using MSBuild:
http://blogs.msdn.com/fxcop/archive/2006/11/16/faq-how-do-i-share-managed-code-analysis-rule-settings-over-multiple-projects-david-kean.aspx
--
Michael
Sebastien Pouliot
Jun 12, 2008, 9:03:23 PM6/12/08
to Gendarme
Hello Michael,
On Jun 12, 7:14 pm, Michael Hutchinson <m.j.hutchin...@gmail.com>
wrote:
them.
However I don't mind them since I believe people should not be waiting
on the results, any more than they should be waiting on the compiler,
to continue their work. A bit more about this later...
There are no breaking change planned atm (unlike the previous
milestone) but enhancements/addition are planned. Anyway I'm not ready
to promise API stability yet (but the number of existing rules makes
me very hesitant to break backward compatibility ;-).
> * Users should be able to specify ignoring specific violations from a
> context menu on the list of violations.
The base runner (in the framework) already support this and the
"ignore" source is not hard-coded. The consoler runner use a file,
with a basic syntax, while the wizard runner does not (by design)
support it.
> * Users should be able to select which rules they want to use.
This is also possible with the base runner. The console runner support
"rule sets" defined in the rules.xml file (by default), the wizard let
the user select them individually or by namespace/category.
> * Rules should be grouped by severity.
Rules are not based on severity, defects are. While many rules have a
"static" severity (and confidence level) quite a few of them will
select a different severity, or confidence, based on the information
it has available. Filtering/ordering on both properties (severity and
confidence) should be possible inside MD.
There are two changes planned for the next (after Mono 2.0 branch)
Gendarme milestone:
1. A new severity level "Audit" for things that cannot be fixed, e.g.
ReviewSecurityAssertsRule would be an example. Runners implementing
"ignore list" are primordial for the rules to be useful (since the
defect will always be reported);
2. By default the console runner will report only defects > Low. This
means Audit and Low will be skipped (by default). The number of
existing rules (and the # defects found) make it hard to concentrate
on the "top" defects for new users.
> * The analysis should be an MSBuild target, so that it can be executed
> from the command line with xbuild.
New runner are easy to build, default behaviors are mostly provided by
the base runner (part of the framework).
Nestor already has one (not yet committed) for a NAnt task.
> * Users must be able to jump to the offending code from the violations
> list (this is the entire point of fully integrating it in MD).
Yes - but that's the *basic*, not the whole, point IMO. Doing so get
us to the VS.NET level (i.e. much better than today ;-) but it does
not provide the right information at the right time to the developers.
E.g. even if runner hides Audit/Low defects the developer should be
warned (somehow) visually* if he's working on a type/methods that
needs fixes. If you're gonna spend sometime to understand a method
then it may be nice to know, from the start, that it's flagged as
complex (Maintainability.AvoidComplexMethodsRules) and that the code
might perform poorly because it cause too much unboxing
(Performance.AvoidUnneededUnboxingRule). This will likely influence
how the developer will work on the code.
* no blinking required ;-)
> Possible features
> =================
>
> * Multiple rules profiles -- allow the user to specify different
> profiles of rules, so that it's possible to run just the critical
> ones, or a full scan.
This already exists, multiple "sets" inside a single rules.xml file
(e.g. each project could have it's own rules.xml equivalent).
> * Allow running certain analysis profiles as a postbuild step.
Definitively - this is how the "early warning system" explained above
would work best (for it's defect source).
It should also be possible to have different sets for "Debug" and
"Release" (or any other defined). E.g. We already have one rule* (to
be moved) and other planned for a "Release" category - i.e. thing that
would make sense only at the late stage of a project.
(*) DisableDebuggingCodeRule
> * Rules violations could display an explanation and example.
Each rule already provides it's own (attribute-based) Problem,
Solution and Url. The extended description and examples are, right
now, on the wiki (accessible using the Url) - but I keep thinking to
move them into SVN (or even the source code) and generate the wiki
documentation from them.
> * Analysis history -- display a graph of pass/fail history.
yes! :-)
> * Allow suppressing violations from autogenerated code (could be as
> simple as ignoring anything from a *.designer.* file, or could use the
> MSBuild attributes that VS uses to mark autogenerated code files)
This is already done, when applicable to the rule, when the code use
one of the two attributes defined for autogenerated code.
* System.CodeDom.Compiler.GeneratedCodeAttribute
* System.Runtime.CompilerServices.CompilerGeneratedAttribute
There are also a few special exceptions made by some rules.
> * GUI for rule options.
Rules properties can be changes using rules.xml (reflection based). We
have been talking to flag the properties with attributes since this
would ease the GUI job to build (and display) the rules options.
> * Seach/filter violations and rules, since the lists could get pretty
> big.
/me nods
> * Ability to download new/updated rules from Mono.Addins repositories.
That would be nice since, at some point, I want some domain-specific
rules (e.g. Moonlight, Unit tests, Rules about rules ;-) that should
not be deployed with the default Gendarme installation.
> * Rules rated both on severity and probability of being correct.
Severity and Confidence already exists - but it's not about the rule
but the defects found by the rule.
> * Allow requiring code analysis without before checking in to VCS.
Good one!
> * Use the MD Code DOM to find things not listed in MDB files.
Could be tricky, or not, depending on how MD interfaces with Gendarme.
> * Automatic/suggested fixes using MD refactoring tools, e.g. "Click
> here to rename the methods to..."
Yes! I know nothing about how refactoring works in MD but the changes
needed to fix some defects are quite simple.
E.g. UseStringEmptyRule
> Questions
> =========
>
> * Should the gendarme and smokey addins be merged, or simply share
> some base functionality (widgets etc)? I'd suggest the latter, because
> I don't think there's any way to unify the options and output cleanly,
> especially since many of the rules are duplicated.
I also prefer the later.
> * Should the options be stored in the project files, or special
> gendarme files? I think that cluttering the project files would be a
> bad idea.
There's no need to clutter the MD config files - just adding a pointer
to the gendarme configuration file(s) should be enough.
> * Should the analysis options be per-project or per-solution? Or
> should there be a way of overriding solution preferences in a project?
> Is it worth scanning multiple assemblies at once? More radically (but
> more flexibly), could there be such a thing as an "Analysis project"
> that can reference a set of projects and constian different analysis
> profile files?
To be to most useful the configuration needs to be solution (not
project) specific. Some rules (few right now, it's a recent feature)
can work on the "assembly set", i.e. find defects by analyzing several
assemblies.
E.g. Smells.AvoidSpeculativeGeneralityRule
> * What will be the relationship in MD with other analysis tools?
> Sharing an "Analyse" menu and setting section?
If MD allows different "analyze actions" (e.g. for gendarme a rule
set) to be executed then each one should be bound to a specific tool.
> Interesting links
> =================
I probably seen a good number of them but I'll look again :-)
Thanks
Sebastien
On Jun 12, 7:14 pm, Michael Hutchinson <m.j.hutchin...@gmail.com>
wrote:
> Hey all,
>
> I've just committed the MonoDevelop.CodeAnalysis addin from Google
> Code to MonoDevelop extras, after fixing up the autotools build and
> making sure it works against trunk. However, when using it I ran into
> some usability problems, so I thought I'd start another thread of
> ideas (since it's a bit late to resurrect the old one).
>
> Requirements
> ==========
>
> * MonoDevelop shouldn't link against Gendarme, at least until the APIs
> are stable. It certainly shouldn't ever load the rules assemblies
> (because of memory use and uinstability), but might need to scan them
> for options in order to generate a GUI.
This might cause performance issues - Dan can tell you more about
>
> I've just committed the MonoDevelop.CodeAnalysis addin from Google
> Code to MonoDevelop extras, after fixing up the autotools build and
> making sure it works against trunk. However, when using it I ran into
> some usability problems, so I thought I'd start another thread of
> ideas (since it's a bit late to resurrect the old one).
>
> Requirements
> ==========
>
> * MonoDevelop shouldn't link against Gendarme, at least until the APIs
> are stable. It certainly shouldn't ever load the rules assemblies
> (because of memory use and uinstability), but might need to scan them
> for options in order to generate a GUI.
them.
However I don't mind them since I believe people should not be waiting
on the results, any more than they should be waiting on the compiler,
to continue their work. A bit more about this later...
There are no breaking change planned atm (unlike the previous
milestone) but enhancements/addition are planned. Anyway I'm not ready
to promise API stability yet (but the number of existing rules makes
me very hesitant to break backward compatibility ;-).
> * Users should be able to specify ignoring specific violations from a
> context menu on the list of violations.
"ignore" source is not hard-coded. The consoler runner use a file,
with a basic syntax, while the wizard runner does not (by design)
support it.
> * Users should be able to select which rules they want to use.
"rule sets" defined in the rules.xml file (by default), the wizard let
the user select them individually or by namespace/category.
> * Rules should be grouped by severity.
"static" severity (and confidence level) quite a few of them will
select a different severity, or confidence, based on the information
it has available. Filtering/ordering on both properties (severity and
confidence) should be possible inside MD.
There are two changes planned for the next (after Mono 2.0 branch)
Gendarme milestone:
1. A new severity level "Audit" for things that cannot be fixed, e.g.
ReviewSecurityAssertsRule would be an example. Runners implementing
"ignore list" are primordial for the rules to be useful (since the
defect will always be reported);
2. By default the console runner will report only defects > Low. This
means Audit and Low will be skipped (by default). The number of
existing rules (and the # defects found) make it hard to concentrate
on the "top" defects for new users.
> * The analysis should be an MSBuild target, so that it can be executed
> from the command line with xbuild.
the base runner (part of the framework).
Nestor already has one (not yet committed) for a NAnt task.
> * Users must be able to jump to the offending code from the violations
> list (this is the entire point of fully integrating it in MD).
us to the VS.NET level (i.e. much better than today ;-) but it does
not provide the right information at the right time to the developers.
E.g. even if runner hides Audit/Low defects the developer should be
warned (somehow) visually* if he's working on a type/methods that
needs fixes. If you're gonna spend sometime to understand a method
then it may be nice to know, from the start, that it's flagged as
complex (Maintainability.AvoidComplexMethodsRules) and that the code
might perform poorly because it cause too much unboxing
(Performance.AvoidUnneededUnboxingRule). This will likely influence
how the developer will work on the code.
* no blinking required ;-)
> Possible features
> =================
>
> * Multiple rules profiles -- allow the user to specify different
> profiles of rules, so that it's possible to run just the critical
> ones, or a full scan.
(e.g. each project could have it's own rules.xml equivalent).
> * Allow running certain analysis profiles as a postbuild step.
would work best (for it's defect source).
It should also be possible to have different sets for "Debug" and
"Release" (or any other defined). E.g. We already have one rule* (to
be moved) and other planned for a "Release" category - i.e. thing that
would make sense only at the late stage of a project.
(*) DisableDebuggingCodeRule
> * Rules violations could display an explanation and example.
Solution and Url. The extended description and examples are, right
now, on the wiki (accessible using the Url) - but I keep thinking to
move them into SVN (or even the source code) and generate the wiki
documentation from them.
> * Analysis history -- display a graph of pass/fail history.
> * Allow suppressing violations from autogenerated code (could be as
> simple as ignoring anything from a *.designer.* file, or could use the
> MSBuild attributes that VS uses to mark autogenerated code files)
one of the two attributes defined for autogenerated code.
* System.CodeDom.Compiler.GeneratedCodeAttribute
* System.Runtime.CompilerServices.CompilerGeneratedAttribute
There are also a few special exceptions made by some rules.
> * GUI for rule options.
have been talking to flag the properties with attributes since this
would ease the GUI job to build (and display) the rules options.
> * Seach/filter violations and rules, since the lists could get pretty
> big.
> * Ability to download new/updated rules from Mono.Addins repositories.
rules (e.g. Moonlight, Unit tests, Rules about rules ;-) that should
not be deployed with the default Gendarme installation.
> * Rules rated both on severity and probability of being correct.
but the defects found by the rule.
> * Allow requiring code analysis without before checking in to VCS.
> * Use the MD Code DOM to find things not listed in MDB files.
> * Automatic/suggested fixes using MD refactoring tools, e.g. "Click
> here to rename the methods to..."
needed to fix some defects are quite simple.
E.g. UseStringEmptyRule
> Questions
> =========
>
> * Should the gendarme and smokey addins be merged, or simply share
> some base functionality (widgets etc)? I'd suggest the latter, because
> I don't think there's any way to unify the options and output cleanly,
> especially since many of the rules are duplicated.
> * Should the options be stored in the project files, or special
> gendarme files? I think that cluttering the project files would be a
> bad idea.
to the gendarme configuration file(s) should be enough.
> * Should the analysis options be per-project or per-solution? Or
> should there be a way of overriding solution preferences in a project?
> Is it worth scanning multiple assemblies at once? More radically (but
> more flexibly), could there be such a thing as an "Analysis project"
> that can reference a set of projects and constian different analysis
> profile files?
project) specific. Some rules (few right now, it's a recent feature)
can work on the "assembly set", i.e. find defects by analyzing several
assemblies.
E.g. Smells.AvoidSpeculativeGeneralityRule
> * What will be the relationship in MD with other analysis tools?
> Sharing an "Analyse" menu and setting section?
set) to be executed then each one should be bound to a specific tool.
> Interesting links
> =================
I probably seen a good number of them but I'll look again :-)
Thanks
Sebastien
Michael Hutchinson
Jun 12, 2008, 9:09:44 PM6/12/08
to Gendarme
> Other analysis tools
> ====================
>
> * C# source style analyser:http://code.msdn.microsoft.com/sourceanalysis
Hmm, I *thought* that was MS-PL when I looked at it before, but it
> ====================
>
> * C# source style analyser:http://code.msdn.microsoft.com/sourceanalysis
doesn't seem to be now...
Michael Hutchinson
Jun 12, 2008, 9:21:56 PM6/12/08
to Gendarme
On Jun 12, 9:09 pm, Michael Hutchinson <m.j.hutchin...@gmail.com>
wrote:
REALLY restrictive. I suspect it was a mistake on their part :-/
At least I know I'm not going crazy.
--
Michael
wrote:
> > * C# source style analyser:http://code.msdn.microsoft.com/sourceanalysis
>
> Hmm, I *thought* that was MS-PL when I looked at it before, but it
> doesn't seem to be now...
According to the page history, the licence changed today. It's now
>
> Hmm, I *thought* that was MS-PL when I looked at it before, but it
> doesn't seem to be now...
REALLY restrictive. I suspect it was a mistake on their part :-/
At least I know I'm not going crazy.
--
Michael
Sebastien Pouliot
Jun 12, 2008, 9:22:43 PM6/12/08
to gend...@googlegroups.com
and you were probably right! That's their stupid web site.
FxCop itself is (or was) listed as MS-PL before you download it, but the
license when installing (and once installed) is/was *pretty different*!
Sebastien
p.s. I say "is/was" because I didn't check if they fixed their mistake
or not.
Reply all
Reply to author
Forward
0 new messages