Applying one transformation config to multiple files found by pattern

[Hauke]

the current implementation of the maven-config-processor expects
exactly one "input" file as parameter per transformation. I would like
to apply the same transformation config file to a series of files
(e.g. matched by some ANT-stlye pattern like "**/*.xml").
Does anyone have an idea if this can be accomplished with the current
maven-config-processor or a similar plugin?

My current idea would be to extend the current maven-config-processor
implementation by adding 2 new parameters, one for the base directory
and one for the pattern of files to search for. Then each file
matching the pattern would be prcoessed with the same transformation
config.
In case useOutputDirectory is set, the files will be created in
directories relatively to the outputDirectory, otherwise the original
files will be overwritten.

Any other thoughts?

[Hauke]

I created the functionality to define "input" with a wildcard pattern
to find files below a (new parameter!) inputDir.
Also added some basic tests but struggled a bit with the more complex
tests because I am not very familar with EasyMock and did not want to
create any temp files during JUnit tests.

Below you can find the Patch, you can can now for example specify the
following transformation:

<configuration>
<useOutputDirectory>true</useOutputDirectory>
<outputDirectory>${project.build.directory}/merged</outputDirectory>
<transformations>
<transformation>
<input>**/test*.xml</input>
<inputDir>${basedir}/src/main/resources/</inputDir>
<config>${basedir}/src/main/resources/config.xml</config>
</transformation>
</transformations>
</configuration>


Here comes the patch (@Leandro, feel free to integrate the patch into
Trunk)

---- snip ---

Index: src/main/java/com/google/code/configprocessor/
Transformation.java
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- src/main/java/com/google/code/configprocessor/Transformation.java
(revision 635)
+++ src/main/java/com/google/code/configprocessor/Transformation.java
(revision )
@@ -26,14 +26,37 @@
public static final String XML_TYPE = "xml";

/**
- * File to process.
+ * File(s) to process.
+ * If a pattern-based String with wildcards (e.g. <code>**\/
*.xml</code>) is supplied,
+ * all files below the {@link #getInputDir()} directory matching
the pattern will be processed
- *
+ *
- * @parameter
+ * The implementation is utilizing {@link
org.apache.tools.ant.DirectoryScanner} for the pattern matching,
+ * e.g. it allows to use single ("*") and double wildcards ("**")
for matching arbitrary characters or directories.
+ *
+ * Examples:
+ * <table>
+ * <tr>
+ * <td><pre>*.xml</pre></td><td>matches all XML files in the base
directory</td>
+ * </tr>
+ * <tr>
+ * <td><pre>**\/*.xml</pre></td><td>matches all XML files in any
subfolder</td>
+ * </tr>
+ * </table>
+ *
+ * @parameter input a file name or pattern denoting files to be
processed
* @required
*/
private String input;

- /**
+ /**
+ * A base directory under which files with a specific pattern
shall be searched.
+ * The pattern must be specified using the <code>input</
code>parameter
+ *
+ * @parameter
+ */
+ private String inputDir;
+
+ /**
* Output file to generate the result of processing.
*
* @parameter
@@ -72,7 +95,11 @@
return input;
}

+ public String getInputDir() {
+ return inputDir;
+ }
+
- public String getOutput() {
+ public String getOutput() {
return output;
}

@@ -90,9 +117,13 @@

public void setInput(String input) {
this.input = input;
+ }
+
+ public void setInputDir(String inputDir) {
+ this.inputDir = inputDir;
- }
+ }

- public void setOutput(String output) {
+ public void setOutput(String output) {
this.output = output;
}

Index: src/main/java/com/google/code/configprocessor/
ConfigProcessor.java
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- src/main/java/com/google/code/configprocessor/ConfigProcessor.java
(revision 635)
+++ src/main/java/com/google/code/configprocessor/ConfigProcessor.java
(revision )
@@ -15,19 +15,24 @@
*/
package com.google.code.configprocessor;

-import static com.google.code.configprocessor.util.IOUtils.*;
+import com.google.code.configprocessor.expression.ExpressionResolver;
+import com.google.code.configprocessor.io.FileResolver;
+import com.google.code.configprocessor.log.LogAdapter;
+import
com.google.code.configprocessor.parsing.ProcessingConfigurationParser;
+import com.google.code.configprocessor.processing.Action;
+import com.google.code.configprocessor.processing.ActionProcessor;
+import
com.google.code.configprocessor.processing.properties.PropertiesActionProcessor;
+import
com.google.code.configprocessor.processing.xml.XmlActionProcessor;
+import org.apache.commons.lang.StringUtils;
+import org.apache.tools.ant.DirectoryScanner;

import java.io.*;
-import java.util.*;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;

-import com.google.code.configprocessor.expression.*;
-import com.google.code.configprocessor.io.*;
-import com.google.code.configprocessor.log.*;
-import com.google.code.configprocessor.parsing.*;
-import com.google.code.configprocessor.processing.*;
-import com.google.code.configprocessor.processing.properties.*;
-import com.google.code.configprocessor.processing.xml.*;
-import org.apache.commons.lang.StringUtils;
+import static com.google.code.configprocessor.util.IOUtils.close;
+import static
com.google.code.configprocessor.util.IOUtils.forceMkdirs;

public class ConfigProcessor {

@@ -82,46 +87,121 @@
}

public void execute(ExpressionResolver resolver, Transformation
transformation) throws ConfigProcessorException, IOException {
- File input = fileResolver.resolve(transformation.getInput());
+ String input = transformation.getInput();
+
- File config = fileResolver.resolve(transformation.getConfig());
+ File config =
fileResolver.resolve(transformation.getConfig());
- File output;
+ if (!config.exists()) {
+ throw new ConfigProcessorException("Configuration file ["
+ config + "] does not exist");
+ }
+
+ if (input != null && input.contains("*")) {
+ // input parameter specifies a wildcard pattern, we need
a base input directory
+ File inputDir =
fileResolver.resolve(transformation.getInputDir());
+ if (!StringUtils.isBlank(transformation.getOutput())) {
+ throw new ConfigProcessorException("Cannot specify
output file if wildcard pattern based input is given");
+ }
+ getLog().info("Using wildcard pattern based input [" +
input + "] with base directory [" + inputDir + "]");
+ List<File> inputFiles= getMatchingFiles(inputDir, input);
+ for (File inputFile : inputFiles) {
+ String type = getInputType(transformation,
inputFile);
+ File outputFile;
+ if (actualOutputDirectory != null) {
+ // calculate a relative path below the output
directory based on the input file
+ outputFile = new File(actualOutputDirectory,
inputDir.toURI().relativize(inputFile.toURI()).getPath());
+ createOutputFile(outputFile);
+ }
+ else {
+ outputFile = inputFile;
+ }
+ process(resolver, inputFile.getPath(), inputFile,
outputFile, transformation.getConfig(), config, type);
+ }
+ }
+ else {
+ File inputFile =
fileResolver.resolve(transformation.getInput());
+ if (!inputFile.exists()) {
+ throw new ConfigProcessorException("Input file [" +
inputFile + "] does not exist");
+ }
- //use input file as output file if output is not set
+ //use input file as output file if output is not set
+ File output;
- if (StringUtils.isBlank(transformation.getOutput()))
{
+ if (StringUtils.isBlank(transformation.getOutput())) {
- output = input;
+ output = inputFile;
- } else {
- output = new File(actualOutputDirectory,
transformation.getOutput());
+ } else {
+ output = new File(actualOutputDirectory,
transformation.getOutput());
+ createOutputFile(output);
- }
+ }
- String type = getInputType(transformation);
+ String type = getInputType(transformation, inputFile);
+ process(resolver, transformation.getInput(), inputFile,
output, transformation.getConfig(), config, type);
+ }

- if (!input.exists()) {
- throw new ConfigProcessorException("Input file [" + input + "]
does not exist");
- }
+ }
- if (!config.exists()) {
- throw new ConfigProcessorException("Configuration file [" + config
+ "] does not exist");
- }
- createOutputFile(output);

- process(resolver, transformation.getInput(), input, output,
transformation.getConfig(), config, type);
+ /**
+ * scans all files below the given {@param baseDirectory} using
the supplied {@param pattern}.
+ * All files matching the {@param pattern} are returned.
+ * The implementation is utilizing {@link DirectoryScanner} for
the pattern matching, e.g. it allows
+ * to use single ("*") and double wildcards ("**") for matching
arbitrary characters or directories.
+ *
+ * Examples:
+ * <table>
+ * <tr>
+ * <td><pre>*.xml</pre></td><td>matches all XML files in the base
directory</td>
+ * </tr>
+ * <tr>
+ * <td><pre>**\/*.xml</pre></td><td>matches all XML files in any
subfolder</td>
+ * </tr>
+ * </table>
+ * @param baseDirectory the base directory under which files
shall be searched
+ * @param pattern the directory and file name pattern that files
shall match
+ * @return the {@link List} of {@link File}s that match the given
pattern
+ *
+ * @throws ConfigProcessorException
+ */
+ protected List<File> getMatchingFiles(File baseDirectory, String
pattern) throws ConfigProcessorException {
+ if (!baseDirectory.exists()) {
+ throw new ConfigProcessorException("Base directory [" +
baseDirectory + "] does not exist");
- }
+ }
+ if (!baseDirectory.isDirectory()) {
+ throw new ConfigProcessorException("File [" +
baseDirectory + "] is not a directory");
+ }
+ if (pattern == null || pattern.length() == 0) {
+ throw new ConfigProcessorException("Invalid pattern
["+pattern+"]");
+ }
+ DirectoryScanner scanner = new DirectoryScanner();
+ scanner.setBasedir(baseDirectory);
+ scanner.setIncludes(new String[]{pattern});
+ scanner.setCaseSensitive(false);
+ scanner.scan();
+ String[] fileNames = scanner.getIncludedFiles();
+ List<File> files = new ArrayList<File>();
+ for (String fileName : fileNames) {
+ files.add(new File(baseDirectory, fileName));
+ }
+ return files;
+ }

- /**
+ /**
* Detects input file type.
+ * If {@link
com.google.code.configprocessor.Transformation#getType()} is not null,
this type ist used,
+ * otherwise it is tried to guess the type from the given {@param
input} File based on the file extension
+ * (.properties or .xml).
+ * If no type could be found or guessed, {@link
Transformation#XML_TYPE} is used.
*
- * @param transformation
+ * @param transformation which can have an explicit type set
+ * @param input file from which the type can be guessed if
transformation parameter does not contain a type
- * @return Input file type.
+ * @return Input file type.
*/
- protected String getInputType(Transformation transformation) {
+ protected String getInputType(Transformation transformation, File
input) {
String type;

if (transformation.getType() == null) {
- if (transformation.getInput().endsWith(".properties")) {
+ if (input.getName().endsWith(".properties")) {
type = Transformation.PROPERTIES_TYPE;
- } else if (transformation.getInput().endsWith(".xml")) {
+ } else if (input.getName().endsWith(".xml")) {
type = Transformation.XML_TYPE;
} else {
if (getLog() != null) {
getLog().warn(
- "Could not auto-detect type of input [" +
transformation.getInput() +
+ "Could not auto-detect type of input [" + input +
"], assuming it is XML. It is recommended that you configure
it in your pom.xml (tag: transformations/transformation/type) to avoid
errors");
}
type = Transformation.XML_TYPE;
Index: pom.xml
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- pom.xml (revision 635)
+++ pom.xml (revision )
@@ -120,6 +120,12 @@
<version>2.0</version>
<scope>test</scope>
</dependency>
+
+ <dependency>
+ <groupId>org.easymock</groupId>
+ <artifactId>easymockclassextension</artifactId>
+ <version>2.5.2</version>
+ </dependency>
</dependencies>

<build>
Index: src/test/java/com/google/code/configprocessor/
ConfigProcessorTest.java
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- src/test/java/com/google/code/configprocessor/
ConfigProcessorTest.java (revision )
+++ src/test/java/com/google/code/configprocessor/
ConfigProcessorTest.java (revision )
@@ -0,0 +1,82 @@
+/*
+ * Copyright (C) 2009 Leandro de Oliveira Aparecido
<lehp...@gmail.com>
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied.
+ * See the License for the specific language governing permissions
and
+ * limitations under the License.
+ */
+package com.google.code.configprocessor;
+
+import org.junit.Before;
+import org.junit.Test;
+
+import java.io.File;
+
+import static org.easymock.classextension.EasyMock.createStrictMock;
+import static org.easymock.classextension.EasyMock.expect;
+import static org.junit.Assert.assertEquals;
+
+public class ConfigProcessorTest {
+
+ private ConfigProcessor configProcessor = null;
+
+ @Before
+ public void setup() throws Exception {
+ configProcessor = new ConfigProcessor("UTF-8", 80, 4, null,
null, false, null, null, null);
+ }
+
+ @Test(expected = ConfigProcessorException.class)
+ public void testNotADirectory() throws Exception {
+ File baseDir = createStrictMock(File.class);
+ expect(baseDir.isDirectory()).andReturn(false);
+ configProcessor.getMatchingFiles(baseDir, "*.xml");
+ }
+
+ @Test(expected = ConfigProcessorException.class)
+ public void testNullPattern() throws Exception {
+ File baseDir = createStrictMock(File.class);
+ expect(baseDir.isDirectory()).andReturn(true);
+ expect(baseDir.exists()).andReturn(true);
+ configProcessor.getMatchingFiles(baseDir, null);
+ }
+
+ @Test(expected = ConfigProcessorException.class)
+ public void testEmptyPattern() throws Exception {
+ File baseDir = createStrictMock(File.class);
+ expect(baseDir.isDirectory()).andReturn(true);
+ expect(baseDir.exists()).andReturn(true);
+ configProcessor.getMatchingFiles(baseDir, "");
+ }
+
+ @Test(expected = ConfigProcessorException.class)
+ public void testInvalidDirectory() throws Exception {
+ File baseDir = createStrictMock(File.class);
+ expect(baseDir.isDirectory()).andReturn(true);
+ expect(baseDir.exists()).andReturn(false);
+ configProcessor.getMatchingFiles(baseDir, "*.xml");
+ }
+
+ @Test
+ public void testGetTypeFromTransformation() throws Exception {
+ Transformation transformation = new Transformation();
+ transformation.setType(Transformation.PROPERTIES_TYPE);
+ assertEquals(Transformation.PROPERTIES_TYPE,
configProcessor.getInputType(transformation, null));
+ }
+
+ @Test
+ public void testGuessTypeFromInputFile() throws Exception {
+ Transformation transformation = new Transformation();
+ assertEquals(Transformation.PROPERTIES_TYPE,
configProcessor.getInputType(transformation, new
File("test.properties")));
+ assertEquals(Transformation.XML_TYPE,
configProcessor.getInputType(transformation, new File("test.xml")));
+ assertEquals(Transformation.XML_TYPE,
configProcessor.getInputType(transformation, new
File("test.something")));
+ }
+
+}

[Leandro de Oliveira]

Hi Hauke, I'll take a look at the code and integrate it, then I'll
make a new release. Thank you! May I add you as a contributor to the
project?

2012/2/9 Hauke <hauke...@googlemail.com>:

> --
> You received this message because you are subscribed to the Google Groups "maven-config-processor-users" group.
> To post to this group, send email to maven-config-p...@googlegroups.com.
> To unsubscribe from this group, send email to maven-config-process...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/maven-config-processor-users?hl=en.
>

[lehphyro]

I've created the issue 28 [1] for this feature and committed your
changes. Could you please verify that it's working correctly?

[1] http://code.google.com/p/maven-config-processor-plugin/issues/detail?id=28

On Feb 9, 6:48 pm, Leandro de Oliveira <lehph...@gmail.com> wrote:
> Hi Hauke, I'll take a look at the code and integrate it, then I'll
> make a new release. Thank you! May I add you as a contributor to the
> project?
>

> 2012/2/9 Hauke <hauke.k...@googlemail.com>:

> ...
>
> read more »