Include / exclude files from scan

[Nicolas MERVAILLIE]

Hi 

I've tried the new 1.1.4 release with the possibility to exclude some files from analysis.

This is really great as scanning is much much faster now.

However, in my spring boot applications use case, I have a spring boot fat jar with the following structure :

/

|- /lib/

   |- spring.jar       <- I'd like to exclude this

   |- aspectj.jar      <- I'd like to exclude this

   |- my-app-dependency.jar      <- I'd like to scan this 

   |- ...

|- /classes

   |- abc.class

   |- ...

|- other stuff....

Considering the following include / exclude configuration : 

file.exclude=/lib/*

file.include=/lib/my-app-*

All jars in the lib folder are ignored. I do not get my custom dependencies to be scanned.

Is there any specific reason to give higher precedence to exclude patterns than to include patterns ?

Cheers 

Nicolas

[Dirk Mahler]

Hi Nicolas,

the implementation follows the semantics of ant filesets where include has precedence over exclude. Changing the precedence would disallow use cases like "include everything execpt on single element"...

Just thinking about supporting an "!" operator, e.g.

file.exclude=/lib/*,!/lib/my-app-*

Could this approach solve your problem.

Opinions?

Cheers,

Dirk

------ Originalnachricht ------

Von: "Nicolas MERVAILLIE" <nmerv...@gmail.com>

An: "jQAssistant" <jqass...@googlegroups.com>

Gesendet: 22.11.2016 16:10:20

Betreff: [jQAssistant] Include / exclude files from scan

--
You received this message because you are subscribed to the Google Groups "jQAssistant" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jqassistant...@googlegroups.com.
To post to this group, send email to jqass...@googlegroups.com.
Visit this group at https://groups.google.com/group/jqassistant.
To view this discussion on the web visit https://groups.google.com/d/msgid/jqassistant/6f565802-7727-4354-980f-58edfb880193%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Nicolas Mervaillie]

Hi Dirk

I guess precedence of inclusion / exclusion is use case specific.

I came to the same conclusion as you about the need of a "!" operator.

It could make sense to have just 'file.include' with negation operator instead of both 'file.include' and 'file.exclude'. Precedence would be determined by the order in the list.

What could also be useful is a java PathMatcher implem. It would allow glob and regex filtering.
https://docs.oracle.com/javase/7/docs/api/java/nio/file/FileSystem.html#getPathMatcher%28java.lang.String%29

WDYT ?

Cheers

Nicolas

To unsubscribe from this group and stop receiving emails from it, send an email to jqassistant+unsubscribe@googlegroups.com.

To post to this group, send email to jqass...@googlegroups.com.
Visit this group at https://groups.google.com/group/jqassistant.
To view this discussion on the web visit https://groups.google.com/d/msgid/jqassistant/6f565802-7727-4354-980f-58edfb880193%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "jQAssistant" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jqassistant/5Bnl3eqPbh4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jqassistant+unsubscribe@googlegroups.com.

To post to this group, send email to jqass...@googlegroups.com.
Visit this group at https://groups.google.com/group/jqassistant.

To view this discussion on the web visit https://groups.google.com/d/msgid/jqassistant/em77e54756-f252-46d4-b82d-22f0807acd89%40mythenmetz.

[Dirk Mahler]

Hi Nicolas,

I guess precedence of inclusion / exclusion is use case specific.

I think so ;-)

I came to the same conclusion as you about the need of a "!" operator.

It could make sense to have just 'file.include' with negation operator instead of both 'file.include' and 'file.exclude'. Precedence would be determined by the order in the list.

It could make sense but it needs good explanation to the users. Let me give an example: a WAR file contains the following files:

/app/myapp.js

/WEB-INF/classes/MyApp.class
/WEB-INF/lib/myapp-lib1.jar
/WEB-INF/lib/myapp-lib2.jar
/WEB-INF/lib/otherlib.jar

I want all files but not the other libs. Then the filter would look like this:

file.filter=*,!/WEB-INF/lib/*.jar,/WEB-INF/lib/myapp*.jar

- the filter expressions are evaluated from left to right

- each positive filter adds its matching elements to the result

- each negative filter removes its matching elements from the result

The example in a mathematical style:

((*) ^ (!/WEB-INF/lib/*.jar)) v (/WEB-INF/lib/myapp*.jar)

What could also be useful is a java PathMatcher implem. It would allow glob and regex filtering.
https://docs.oracle.com/javase/7/docs/api/java/nio/file/FileSystem.html#getPathMatcher%28java.lang.String%29

Thanks for the hint, I'll have a look at it. The current implementation is based on FilenameUtils from commons-io.

Cheers,

Dirk

To unsubscribe from this group and stop receiving emails from it, send an email to jqassistant...@googlegroups.com.

To post to this group, send email to jqass...@googlegroups.com.
Visit this group at https://groups.google.com/group/jqassistant.

To view this discussion on the web visit https://groups.google.com/d/msgid/jqassistant/CANn_wiufhzLDLv5fTq-ezvG%2BuGCv6DqLYLiJ5UMXzMMQwn0%3DoQ%40mail.gmail.com.

[Nicolas MERVAILLIE]

Hi Dirk,

Was rather thinking of smtg like this :

/app/myapp.js

/WEB-INF/classes/MyApp.class

/WEB-INF/lib/myapp-lib1.jar

/WEB-INF/lib/myapp-lib2.jar

/WEB-INF/lib/otherlib.jar

/WEB-INF/lib/anotherlib.jar

First match in the filter list "wins", decides if file is included or not.

* All files but not external libs

file.filter=/WEB-INF/lib/myapp*.jar,!/WEB-INF/lib/*.jar,**/*

* Only a the specific file myapp-lib1.jar

file.filter=/WEB-INF/lib/myapp-lib1.jar,!**/*

* Only files containing myapp

file.filter=**/*myapp*,**/*myApp*,!**/*

* All files but those containing myapp

file.filter=!**/*myapp*,!**/*myApp*,**/*

Maybe, to make it more user friendly, and do not have to rely on the order in the list, precedence could be defined with a notion of "more specific pattern". ie : 

The most specific path could have precedence over others. /WEB-INF/lib/myapp-lib1.jar would take precedence over **/*.jar

If a pattern is more specific could be determined by 

- length of the path : the longest paths have precedence (naive implementation, as done in some frameworks like spring)

- number of path segments ? 

Let me know what you think.

Could a PR be helpful ?

Cheers

Nicolas