Custom template rule isn't triggered

[Lonzak]

I  have created a custom rule based on the S3688 template.
As written in its description it can also be triggered by a regex.
I want to exclude certain classes by package.

I used the following regex (examplary);
org\.apache\.commons\.lang3\.builder\..+

There are two usages in a class (not a test class!) but those are not found. I also varied the regex but it isn't working.
(e.g. import org.apache.commons.lang3.builder.ToStringBuilder and the actual "ToStringBuilder.someMethod()" in the code...)

Any hints how to get it to work?

[Lonzak]

Has anybody tried to use a custom rule at all?
Since there is no answer this seems like a bug to me. If it is not then some examples for rule S3688  template would really help.

[G. Ann Campbell]

Hi,

Any time I'm debugging something regex-related, my first step is to fall back to something like *.*. Have you tried that sort of thing?

Ann

P.S. The standard courtesies (Hi, Thanks, ...) are appreciated in this group.

[Lonzak]

Hi,

yeah I am a bit stackoverflow influenced where those formalities are skipped. (There are even people editing posts and removing those).
But to the question:

I tried lots of variations:
org.apache.commons.lang3.builder*.*
org.apache.commons.lang3.builder.*
org.apache.commons.lang3.builder.**
...

So the template may be functional for classes but not for packages. Can you add a bug for that?
Thanks :-)

[Lonzak]

Hi Ann,

can you give me the bugticket for this bug?
https://jira.sonarsource.com/browse/SONARJAVA

Thank you,
L.

[G. Ann Campbell]

Hi,

It's working for me:

Ann

---

G. Ann Campbell | SonarSource

Product Manager

@GAnnCampbell

https://sonarsource.com

--
You received this message because you are subscribed to a topic in the Google Groups "SonarQube" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sonarqube/8zsTkjlH7E0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sonarqube+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/4a596f43-6fa7-4a4a-b5f9-b23886535a8c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Lonzak]

Ok, that is strange because whatever I tried it is not working for me:

To unsubscribe from this group and all its topics, send an email to sonarqube+...@googlegroups.com.

[G. Ann Campbell]

Hi,

Are you providing libraries to your analysis? 

Also, did we establish your version of SonarJava?

Ann

---

G. Ann Campbell | SonarSource

Product Manager

@GAnnCampbell

https://sonarsource.com

To unsubscribe from this group and all its topics, send an email to sonarqube+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/d4e18747-dd76-4e79-a836-9ac2c2f8b099%40googlegroups.com.

[G. Ann Campbell]

Oops! One more thing. Your 2nd screenshot shows an import of a forbidden class, but not a use. You're not going to get issues on import statements.

---

G. Ann Campbell | SonarSource

Product Manager

@GAnnCampbell

https://sonarsource.com

[Lonzak]

We are using SonarServer 6.7 with SonarJava 4.15.

I added a screenshot showing the usage in that class. There are never imports without an actual usage in our classes :-)

What do you mean with "Are you providing libraries to your analysis?"

I analyze the full (multimodule) maven project which contains all external libraries...

Regards,

L.

[Lonzak]

Somehow the image wasn't transfered, so here it comes again:

[Lonzak]

Hi Ann,

Are you providing libraries to your analysis?

I am still curious how I am supposed to provide libraries to my analysis?

As mentioned it is a multi module maven project so everything should be there but maybe I miss something?

Thanks
L.

[Michael Gumowski]

Hello Lonzak,

Maybe there is indeed a problem with the scan of multi-module project... Before pushing further the investigation, I would need a reproducer.

Could you prepare a small project reproducing the issue, and share your configuration of the rule? 

Something simple like an almost empty maven project with 2 tiny modules having one single class using apache commons.

Thanks,

Michael

--
You received this message because you are subscribed to the Google Groups "SonarQube" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sonarqube+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/c94b8ae5-84b4-4e44-a0f7-71da4f088500%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Michael Gumowski | SonarSource

Software Developer, Language Team
https://www.sonarsource.com

[Michael Gumowski]

Hello Lonzak,

I finally had the time to investigate the issues you where having, thanks a lot for the reproducer you sent (the Futurama example made me laugh).

I identified a two distinct issues:

• Regarding template rule "Track uses of disallowed classes":
  The implementation of the rule is currently not targeting method invocations nor package declarations, meaning that any method call such as MyForbiddenType.foo() is never going to raise any issue... This will be handled by the following ticket: SONARJAVA-2743
• Regarding template rule "Track comments matching a regular expression":
  On this one I think it's only a configuration problem. We use java.util.regex.Pattern.compile(regularExpression, Pattern.DOTALL) to compile the regex provided to the templated rule. This pattern is then tested directly against the FULL content (as string) of each "COMMENT" (in term of Abstract Syntax Tree). It term of implementation, it means that we use the "matches()" method rather than the "find()", consequently, we expect that the whole comment should match the regex. I gave a try to the following and the regex you provided:
  
  public static void main(String[] args) {
    Pattern compile = Pattern.compile("\\bTODO\\s(?!\\d{6}\\b)", Pattern.DOTALL);
    System.out.println("find()");
    System.out.println(compile.matcher("// TODO").find());               // <<<---- false
    System.out.println(compile.matcher("// TODO continue").find());      // <<<---- true
    System.out.println("matches()");
    System.out.println(compile.matcher("// TODO").matches());            // <<<---- false
    System.out.println(compile.matcher("// TODO continue").matches());   // <<<---- false
  }
  
  Now, by changing your regext to a DOTALL format, I have the following: .*\bTODO\s(?!\d{6}\b).*
  I successfully raise an issue:
  
  

I consequently think that there is no reason to change the rule itself. I'm however going to modify the description of RSPEC-124 in order to make it very explicit that we expect the DOTALL pattern to be used in the regex, and that the pattern is applied against the whole comment content.

Now, in conclusion, I'm glad to see that these two issues are not related at all with the project being multi-module, but simply flaws/bad documentation in rule.

Thanks again for your feedback (and for insisting)!

Cheers,

Michael

[TvT]

Hi Michael,

thanks for your insight and your effort!

Your explaination of the regex really helps, so if you add that to the rule template everybody benefits.

Out of curiousity:
You mentioned that the template rule "Track uses of disallowed classes" "does not target method invocations nor package declarations ..."

But what does it currently trigger then?

Regards,

Lonzak

2018-04-30 17:19 GMT+02:00 Michael Gumowski <michael.gumowski@sonarsource.com>:

Hello Lonzak,

I finally had the time to investigate the issues you where having, thanks a lot for the reproducer you sent (the Futurama example made me laugh).

I identified a two distinct issues:

• Regarding template rule "Track uses of disallowed classes":
  The implementation of the rule is currently not targeting method invocations nor package declarations, meaning that any method call such as MyForbiddenType.foo() is never going to raise any issue... This will be handled by the following ticket: SONARJAVA-2743
• Regarding template rule "Track comments matching a regular expression":
  On this one I think it's only a configuration problem. We use java.util.regex.Pattern.compile(regularExpression, Pattern.DOTALL) to compile the regex provided to the templated rule. This pattern is then tested directly against the FULL content (as string) of each "COMMENT" (in term of Abstract Syntax Tree). It term of implementation, it means that we use the "matches()" method rather than the "find()", consequently, we expect that the whole comment should match the regex. I gave a try to the following and the regex you provided:
  
  public static void main(String[] args) {
    Pattern compile = Pattern.compile("\\bTODO\\s(?!\\d{6}\\b)", Pattern.DOTALL);
    System.out.println("find()");
    System.out.println(compile.matcher("// TODO").find());               // <<<---- false
    System.out.println(compile.matcher("// TODO continue").find());      // <<<---- true
    System.out.println("matches()");
    System.out.println(compile.matcher("// TODO").matches());            // <<<---- false
    System.out.println(compile.matcher("// TODO continue").matches());   // <<<---- false
  }
  
  Now, by changing your regext to a DOTALL format, I have the following: .*\bTODO\s(?!\d{6}\b).*
  I successfully raise an issue:
  
  

I consequently think that there is no reason to change the rule itself. I'm however going to modify the description of RSPEC-124 in order to make it very explicit that we expect the DOTALL pattern to be used in the regex, and that the pattern is applied against the whole comment content.

Now, in conclusion, I'm glad to see that these two issues are not related at all with the project being multi-module, but simply flaws/bad documentation in rule.

Thanks again for your feedback (and for insisting)!

Cheers,

Michael

[Michael Gumowski]

Hey,

The rule is targeting any usage of the class as Type, so it will detect issues when a class is used to declare variable, parameters, as return type, or when extending a class. For instance, if excluding "org.bar.B", we will get the following issues in this code (not that the static method invocation is not detected [line 8], as you reported):

Hope this clarifies how the rules work at the moment.

Cheers,

Michael