new function released to allow "transform all"

[qi cui]

I added "transform all" support to allow the operation to be applied to multiple columns as we discussed before. It is under "All" menu where you can apply the operation to more than 1 column you choose. 

It is basically a enhanced function for the "transform" which is only applicable to one column. 

Please help to use it and test it. 

Cheers.

Qi 

[Sumit Kulkarni]

Great !

[Sumit Kulkarni]

What about clustering OpenRefine for High availability, fault tolerance, redundancy, and mostly for the performance?

Can we have cluster of OpenRefine? if yes then how?

How much big and massy data can be handled by single OpenRefine application (If my data is around 10-20 Million rows or larger than that)?

On Thursday, June 1, 2017 at 8:12:37 AM UTC+5:30, qi cui wrote:

[Andrea Zanni]

wow, this is great!

but in which version is this new feature available?

Aubrey

--
You received this message because you are subscribed to the Google Groups "OpenRefine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openrefine+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Owen Stephens]

Doesn't seem to work for me.

I get the Transform option, but then when I write some GREL and click 'OK' nothing happens. 

In the JS Console I see "ReferenceError: Can't find variable: ExpressionColumnDialog"

To unsubscribe from this group and stop receiving emails from it, send an email to openrefine+...@googlegroups.com.

[qi cui]

Fixed. Forgot one file. Please try again

[qi cui]

@Aubrey  Only in the repository. No official release yet.

To unsubscribe from this group and stop receiving emails from it, send an email to openrefine+...@googlegroups.com.

[qi cui]

OpenRefine's current architecture only support standalone which means it has its limitation such as your server capacity, JVM, file size, data complexity and browser limitation etc. It is very hard to give you a straightforward answer for you questions due to those combination.

This is one reply regarding this:

https://www.quora.com/What-is-the-largest-amount-of-data-Google-Refine-can-ingest

There are some projects trying to scale the openrefine by clustering it:

• https://github.com/fusepoolP3/p3-batchrefine
  
• https://groups.google.com/d/msg/openrefine/TOgkfvI4ZV4/JIHtnZGKMzkJ

There was some discussion to integrate with HDFS/Spark. But the effort is non-trivial to implement this.  If everybody want to explore more on this option, let me know. 

[peter.m...@hmri.org.au]

I assume to use this I need to download the latest version of OR

Peter

[Thad Guidry]

You can easily build the Development version at any time with our step-by-step instructions.  Only Apache Ant and Java JDK are needed to be installed on your computer.  https://github.com/OpenRefine/OpenRefine/wiki/Installation-Instructions#development-version

Ask for help on our mailing list if you get stuck or confused with instructions.

-Thad

+ThadGuidry  

[Ettore Rizza]

I finally had the opportunity to build the development version and to test the function "transform all". So far, it seems to work like a charm. The only remark I could make is about the "undo", which only applies to the very last transformation. It might be more logical that "undo" cancels all transformations in this case. 

But that's a detail. It's really a good job and a useful improvement. Thank you very much Qi !

[qi cui]

@Rizza Thank you for your test. could you please submit an issue for this? 

[magdmartin]

Ok I finally got time to test it. Great work Qi this will pleased many users! 

[Enrique Mota]

Good Job!!!

Could anybody provide an example about how to use "Transform All" I don't understand how to do a transformation  in several columns at the same time.

Thank you!!!

[Ettore Rizza]

It's very simple: you click on "All" -> "Transform", you type a GREL formula, and then you specify if you want to apply it to all or some columns only. 

Here is a visual example with the formula value.toNumber().

[Thad Guidry]

Ettore and Owen,

Do you mind putting something up on our Wiki about our new feature under https://github.com/OpenRefine/OpenRefine/wiki/Column-Editing ?

Make a new section on that page like we have for https://github.com/OpenRefine/OpenRefine/wiki/Column-Editing#splitting-columns called "Transform All Cells in Columns" or whatever.  You can add your animated GIF as well.

Actually, it probably makes more sense to users for Transform All to be located under Cell Editing https://github.com/OpenRefine/OpenRefine/wiki/Cell-Editing rather than Column Editing. 

I'll let you guys decide where it should fit  :)

Thanks !

-Thad

+ThadGuidry  

[Enrique Mota]

Thak you very much!!

Next time you come to spain I will you invite to a beer ;-)

[Ettore Rizza]

Be careful, I record this kind of thing in a small notebook. ;)

[Gorav Seth]

I just discovered the 'All' option by searching google for a way to apply a function to all cells (remove whitespace in my case)

This is great! FYI - I took a look and didnt see anything on the cell editing page on the wiki as suggested in this thread.  It would be great to indicate that this option exists!

Thanks

Gorav

[Owen Stephens]

Hi Gorav

I've added some information on this to https://github.com/OpenRefine/OpenRefine/wiki/Cell-Editing#editing-by-transforming

If you ever come across something missing from the documentation please do feel free to add it - as far as I know you just need to setup a Github account to edit the wiki, no special permissions are needed.

Best wishes

Owen