Sequential creation of related columns using apply-operation command

[Gofran Shukair]

Hi everyone,

I am working on Google refine extension that requires applying a number of grefine json operations after creating the project .

I implemented a custom apply-operations command  which will be called after lunching the /project page.

Basically the json operations are created manually on Google refine then I extracted the history into a file that and I am reading in the command doPost function and apply them to the project as the following :

public void doPost(HttpServletRequest request, HttpServletResponse response)

throws ServletException, IOException {

Project project = getProject(request);

String jsonString = getOperationFile();

try {

JSONArray a = ParsingUtilities

.evaluateJsonStringToArray(jsonString);

int count = a.length();

for (int i = 0; i < count; i++) {

JSONObject obj = a.getJSONObject(i);

reconstructOperation(project, obj);

}

if (project.processManager.hasPending()) {

respond(response, "{ \"code\" : \"pending\" }");

} else {

respond(response, "{ \"code\" : \"ok\" }");

}

} catch (JSONException e) {

respondException(response, e);

}

}

The file includes "core/column-addition-by-fetching-urls" operations that receive json data , then other operations create columns from the splitting that json data in the previously created columns 

The problem that : The function "createProcess" of the  "ColumnAdditionByFetchingURLsOperation" checks if baseColumnName of the operation exists in the project.columnModel before creating it, and throw an exception.

That is happening because the creation process of the _baseColumnName is still pending or not fully executed. 

public Process createProcess(Project project, Properties options) throws Exception {

        Column column = project.columnModel.getColumnByName(_baseColumnName);

        if (column == null) {

            throw new Exception("No column named " + _baseColumnName);

        }

        if (project.columnModel.getColumnByName(_newColumnName) != null) {

            throw new Exception("Another column already named " + _newColumnName);

        }

        

        Engine engine = createEngine(project);

        engine.initializeFromJSON(_engineConfig);

        

        Evaluable eval = MetaParser.parse(_urlExpression);

        

        return new ColumnAdditionByFetchingURLsProcess(

            project, 

            engine,

            eval,

            getBriefDescription(null)

        );

    }

I understand that related columns operations should be executed sequentially as their order in the json file.

should i consider this as bug ? should i report about it in the Google code issues ?

--
Thanks
Gofran Shukair

[David Huynh]

Hi Gofran,

That does sound like a bug! Please file it!

David

[David Huynh]

Gofran,

I believe I've fixed it. Could you please test?

David

[Gofran Shukair]

The following operation order dose not work:

1-core/add-column-by-fetching-urls named X

2-core/text-transform for X or  core/column-addition depending on the column X 

because the same check is executed for the second operation in "createHistoryEntry" of the   "TextTransformOperation" or "ColumnAdditionOperation"

Column column = project.columnModel.getColumnByName(_columnName);

        if (column == null) {

            throw new Exception("No column named " + _columnName);

--
Thanks
Gofran Shukair

[David Huynh]

Thanks for testing, Gofran. I've marked the issue as Accepted rather than Fixed.

Thinking more about this issue, I believe it needs a non-trivial fix, involving going over all operations. There is also some complication with the interactive mode. We will release 2.5 first and then fix this issue for 2.6 or 2.5.1.

David