Walkmod resolution of .* imports

[mkea...@e-c-group.com]

Hello,

I am excited about integrating walkmod into our build process but have ran across a few issues.

1) .* imports are not cleaned. Part of our library is dynamically generated from a customer API. These classes all have the same block of .* imports with the majority not needed for any one class. However, it appears that cleanup only occurs for the classes directly imported. Am I missing something?

2) I can't find any guide for walkmod.xml configuration. Are there default variables? What is the correct way to set properties? etc. Is there a guide for this?

3) The reader and writer directory don't seem to match. Example - if reader is set for 'src/com/company/generated' and writer is set for the same... the files are actually written at 'src/com/company/generated/com/company/generated'. It seems like you always need to write to the 'src' directory or the files will not be placed correctly. Again -- am I just missing something?

I figured I would write a single post rather than split them up since this is a forum :)

Thanks!

[walkmod]

Hello,

I answer your questions above:

El lunes, 23 de noviembre de 2015, 2:09:17 (UTC+1), mkea...@e-c-group.com escribió:

Hello,

I am excited about integrating walkmod into our build process but have ran across a few issues.

1) .* imports are not cleaned. Part of our library is dynamically generated from a customer API. These classes all have the same block of .* imports with the majority not needed for any one class. However, it appears that cleanup only occurs for the classes directly imported. Am I missing something?

The imports-cleaner-plugin was the first plugin we did to easily test if the walkmod workflow works. In order to clean wildcard imports, walkmod needs to apply a semantic analysis and thus, open the jars of your classpath and discover the classes that appear inside a package. To perform it, you need to use the walkmod-dead-code-cleaner-plugin (url: https://github.com/rpau/walkmod-dead-code-cleaner-plugin). Follow the instructions of the Github webpages, because it tells you how to set your classpath if your project is maven or gradle.

If you experiment any problem during the execution, do not hesitate to open an issue here or in the plugin, because the semantic analysis, even I have tested it with the guava source code, it is still very experimental.

2) I can't find any guide for walkmod.xml configuration. Are there default variables? What is the correct way to set properties? etc. Is there a guide for this?

The XML configuration file follows a DTD and thus, your XML editor, should help you to write configurations. However, since walkmod 2.0, you can create or upgrade configurations from your command line. Moreover, there is a command called walkmod inspect, that allows to know the properties of any plugin. IF you type just walkmod in your command line and press enter, you will be able to see the list of available commands.

For example, the necessary instructions in order to add the new plugin (if you have a maven project) and assuming that we are in the root folder of the project and starting without any walkmod.xml. 

 walkmod add -R dead-code-cleaner 

 walkmod add-provider -R maven

 walkmod apply

My intention is to upgrade the documentation, to rewrite everything from commands, but currently I am involved in integrating walkmod in another projects. Would you be interested to help me :) ? If so, I can talk in private to collaborate to do so ;) Obviously, this collaboration should be explicitly promoted through the webpage. 

3) The reader and writer directory don't seem to match. Example - if reader is set for 'src/com/company/generated' and writer is set for the same... the files are actually written at 'src/com/company/generated/com/company/generated'. It seems like you always need to write to the 'src' directory or the files will not be placed correctly. Again -- am I just missing something?

Yes, the path never should be part of the package. The path is just the source directory.  If you would like to filter a subset, you need to put it as an includes as follows:

 <reader path="src">

            <include wildcard="com/company/generated/"></include>

  </reader>

<writer path="src/generated" />

 

I figured I would write a single post rather than split them up since this is a forum :)

Yes, no problem. Any kind of feedback no matter the way, is always welcome :)

Thanks!

[mkea...@e-c-group.com]

Thank you for the quick reply!

> To perform it, you need to use the walkmod-dead-code-cleaner-plugin (url: https://github.com/rpau/walkmod-dead-code-cleaner-plugin). Follow the instructions of the Github webpages, because it tells you how to set your classpath if your project is maven or gradle.

That makes sense. I was wondering if the behavior was intentional.

I did attempt to install the dead-code-cleaner-plugin but have been unable to get it to work. I am trying to run in from within a Maven build.

When I run it according to the instructions it says to add the walkmod-maven-plugin. However, I get issues when running maven from within a maven build. --

>[ERROR] Error executing Maven.
>[ERROR] Unable to create injector, see the following errors:
>1) No scope is bound to org.apache.maven.SessionScoped.
> at org.apache.maven.ReactorReader.class(Unknown Source)
> at ClassRealm[plexus.core, parent: null] (via modules: org.eclipse.sisu.wire.WireModule -> org.eclipse.sisu.plexus.PlexusBindingModule)
>1 error

I decided to take the maven specific portions out of the walkmod.xml file which resulted in a different error that appears to the plugins ability to detect the classpath.

>ERROR [main] - TRANSFORMATION CHAIN (default) FAILS
>An exeception has been produced during the null transformation - >org.walkmod.util.location.LocationImpl@316cda31
>...
>Caused by: There is no available project classpath to apply a semantic analysis - >org.walkmod.util.location.LocationImpl@316cda31
> at org.walkmod.javalang.walkers.DefaultJavaWalker.visit(DefaultJavaWalker.java:120)
> at org.walkmod.javalang.walkers.DefaultJavaWalker.accept(DefaultJavaWalker.java:74)

I will try looking back through your code later but any pointers would be helpful.

> My intention is to upgrade the documentation, to rewrite everything from commands, but currently I am involved in integrating walkmod in another projects. Would you be interested to help me :) ? If so, I can talk in private to collaborate to do so ;) Obviously, this collaboration should be explicitly promoted through the webpage. 

I am definitely interested and am willing to contribute once I become more familiar with the project.

Thanks again!

[walkmod]

Hi,

Could you send me your walkmod.xml and your pom file?

I don't like so much the message of "null" transformation :-)

Have a nice day!

[mkea...@e-c-group.com]

Just a note -- I was trying to track down the error.

In DefaultJavaWalker.java --

ClassLoader cl = getClassLoader();

if (requiresSemanticAnalysis) {
if (cl != null) {
SymbolVisitorAdapter<HashMap<String, Object>> visitor = new SymbolVisitorAdapter<HashMap<String, Object>>();
visitor.setClassLoader(cl);
visitor.visit(cu, new HashMap<String, Object>());
} else {
throw new WalkModException(

"There is no available project classpath to apply "

+ "a semantic analysis");
}
}

It appears that the ClassLoader is set following the null check. Admittedly I am new to this code base, but should the visitor set the classloader before 'ClassLoader cl = getClassLoader();' is called?

[walkmod]

The configuration provider, in your case the walkmod-maven-plugin puts into a map the value of the "classLoader" property. 

Check: https://github.com/rpau/walkmod-maven-plugin/blob/master/src/main/java/org/walkmod/maven/providers/ClassLoaderConfigurationProvider.java

By IoC (Spring), walkmod-core sets the value into the walker and all the components (e.g transformations) that need it.

So, if it is null something was wrong in the configuration or in your classpath resolution. This is the reason why I ask for the configuration file and the pom.xml file.

[mkea...@e-c-group.com]

Looking through the code -- are their currently just two configuration providers? I am not using Spring and the walkmod-maven-plugin seems to not work with the maven-walkmod-plugin.

walkmod.xml (I explicitely removed the walkmod-maven-plugin after testing it) --

<!DOCTYPE walkmod PUBLIC "-//WALKMOD//DTD" "http://www.walkmod.com/dtd/walkmod-1.1.dtd">
<walkmod>
<plugins>
<plugin artifactId="walkmod-dead-code-cleaner-plugin" groupId="org.walkmod" version="1.0.0" />
</plugins>
<chain name="default">
<reader path="src">
<include wildcard="co/ecg/toolkit/generated" />
</reader>
<transformation type="walkmod:commons:unused-declarations-cleaner" />
<writer path="src" type="org.walkmod:walkmod-java-formatter-plugin:writer">
<param name="configFile">dist/conf/schema/ECG-Java-Code-Formatter.xml</param>
</writer>
</chain>
</walkmod>

pom.xml (It is very long with multiple steps so I am just listing the walkmod portion) -

<!-- Clean Imports and Format -->
<plugin>
<groupId>org.walkmod</groupId>
<artifactId>maven-walkmod-plugin</artifactId>
<version>1.2</version>
<executions>
<execution>
<phase>process-resources</phase>
<goals>
<goal>apply</goal>
</goals>
<configuration>
<printError>true</printError>
<configFile>walkmod.xml</configFile>
</configuration>
</execution>
</executions>
</plugin>

Our currently lifecycle follows --

validate - Pre-compile code that will be used to generate API code
generate-sources - Generate code from APIs and other classes created dynamically
process-sources - Add generated sources to classpath
process-resources - Run walkmod to clean generated code
compile, etc.

[walkmod]

Hi,

Yes, I was thinking about. walkmod-maven-plugin builds the project (mvn install) and if the maven-walkmod-plugin is inside, it turns into a recursive execution. Anyway, you need to set the walkmod-maven-plugin into your configuration, because otherwise there is no classpath to compute the semantic analysis.

Let me fix it and I notify you ASAP (today or during this week)

Regarding Spring, it is not about your project, I mean that walkmod-core uses internally Spring to instantiate components during the execution workflow. 

[walkmod]

Hi,

Fixed :)

Wait for an hour to be certain that maven repository is sync.

And then, upgrade your library versions:

maven-walkmod-plugin = 1.3

walkmod-dead-code-cleaner-plugin = [1.0.0,)

maven-walkmod-plugin = [2.0.3,)

I have added tests to verify if it works. Let me know if it has worked to you, too :)

Sorry for the bugs, I usually use walkmod from the command line.

[walkmod]

Wait, I am trying something else.

[walkmod]

Sorry, I need to leave for a while. It is not fixed at all yet. Try to run the configuration from the command line until I fixed it.

[walkmod]

Solved, in the maven-walkmod-plugin (pom.xml), you need to use it exactly as follows (notice the skipWalkmod property):

    <groupId>org.walkmod</groupId>

                <artifactId>maven-walkmod-plugin</artifactId>

                <version>1.3</version>

                <executions>

                    <execution>

                        <phase>generate-sources</phase>

                        <goals>

                            <goal>apply</goal>

                        </goals>

                        <configuration>

                            <skipWalkmod>${skipWalkmod}</skipWalkmod>

                        </configuration>

                    </execution>

                </executions>

            </plugin>

[mkea...@e-c-group.com]

Thanks for continuing to work on this. It is possible that my configuration is too complicated currently. If this seems like a lost cause let me know.

I made the changes you suggested. If 'skipWalkmod' is set to true then the walkmod module does not run during the maven build which defeats the point. If it is set to false I get the same error when walkmod attempts to run another maven build from within maven a maven build. See Below -

----

[ERROR] Error executing Maven.
[ERROR] Unable to create injector, see the following errors:

1) No scope is bound to org.apache.maven.SessionScoped.
at org.apache.maven.ReactorReader.class(Unknown Source)
at ClassRealm[plexus.core, parent: null] (via modules: org.eclipse.sisu.wire.WireModule -> org.eclipse.sisu.plexus.PlexusBindingModule)

1 error
ERROR [main] - Invalid configuration
org.walkmod.conf.ConfigurationException: Error executing: clean mvn install -DskipTests in /Users/mkeathley/Workspace/eclipse/AlpacaLibrary
at org.walkmod.maven.providers.MavenProject.build(MavenProject.java:181)
at org.walkmod.maven.providers.MavenProject.resolveClassLoader(MavenProject.java:248)
at org.walkmod.maven.providers.ClassLoaderConfigurationProvider.load(ClassLoaderConfigurationProvider.java:82)
at org.walkmod.impl.DefaultConfigurationAdapter.prepare(DefaultConfigurationAdapter.java:55)
at org.walkmod.conf.ConfigurationManager.<init>(ConfigurationManager.java:76)
at org.walkmod.conf.ConfigurationManager.<init>(ConfigurationManager.java:81)
at org.walkmod.WalkModFacade.apply(WalkModFacade.java:329)
at org.walkmod.mojos.ApplyMojo.execute(ApplyMojo.java:32)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:862)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:286)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:197)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: java.lang.Exception: Error executing: clean mvn install -DskipTests in /Users/mkeathley/Workspace/eclipse/AlpacaLibrary
... 30 more

----

It seems like there needs to be a way to get the classpath and classloader from the currently running maven build when it runs the maven-walkmod-module. I can set the modules lifecycle to AFTER compile so it should have everything needed to perform the ananlysis.

Thoughts?

[walkmod]

Hi,

You don't need to specify a value for skipWalkmod. The value should be exactly "${skipWalkmod}". Take a look to this example, which is my test case :)

https://github.com/rpau/walkmod-maven-plugin/tree/master/src/test/recursive

If it still doesn't work, I suggest to continue the discussion using Skype. Is it ok for you? I am going to send you my skype account with a private message. 

<skipWalkmod>${skipWalkmod}</skipWalkmod>