OpenRefine 3.5-beta2 is out

106 views
Skip to first unread message

Antonin Delpeuch (lists)

unread,
Oct 25, 2021, 3:45:47 PM10/25/21
to OpenRefine
Hello!

The second beta release of OpenRefine 3.5 is out! You can download it
from here:

https://openrefine.org/download.html

We are looking forward to your bug reports and hope to publish a stable
version very soon!

Happy refining,

Antonin

Parthasarathi Mukhopadhyay

unread,
Oct 26, 2021, 11:33:26 AM10/26/21
to openr...@googlegroups.com
Dear Antonin

I tried beta 2 of OpenRefine 3.5 with the command ./refine -i 0.0.0.0 -m 4000m -d /home/myrefine/.local/share/openrefine/3.5/ and it is opening as usual but the console produces following messages with a few warnings -

Using refine.ini for configuration
./refine: line 852: [: 4000m: integer expression expected
You have 7862M of free memory.
Your current configuration is set to use 4000m of memory.
OpenRefine can run better when given more memory. Read our FAQ on how to allocate more memory here:
https://github.com/OpenRefine/OpenRefine/wiki/FAQ-Allocate-More-Memory
No host specified while binding to interface 0.0.0.0, guessing localhost.
/usr/lib/jvm/default-java/bin/java -cp server/classes:server/target/lib/* -Xms256M -Xmx4000m -Drefine.memory=4000m -Drefine.max_form_content_size=1048576 -Drefine.verbosity=info -Dpython.path=main/webapp/WEB-INF/lib/jython -Dpython.cachedir=/home/psm/.local/share/google/refine/cachedir -Drefine.data_dir=/home/psm/.local/share/openrefine/3.5/ -Drefine.webapp=main/webapp -Drefine.port=3333 -Drefine.interface=0.0.0.0 -Drefine.host=* com.google.refine.Refine
Starting OpenRefine at 'http://localhost:3333/'

log4j:WARN No appenders could be found for logger (org.eclipse.jetty.util.log).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

Is there any way to avoid these warnings (as I'm not getting any of these warnings in OR 3.4.1 by applying the same call)? And another question is that is there a more sophisticated way to stop an OR server apart from CTRL+C (this applies to all versions of OR)?

Regards

--
You received this message because you are subscribed to the Google Groups "OpenRefine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openrefine+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine/3ca4eec0-c4d9-091c-1ac7-47614c61a258%40antonin.delpeuch.eu.

Antonin Delpeuch (lists)

unread,
Oct 27, 2021, 2:38:09 AM10/27/21
to openr...@googlegroups.com
Hi Parthasarathi,

Thanks for trying it out!

> No host specified while binding to interface 0.0.0.0, guessing localhost.

This message simply means that the startup script will launch your
browser at http://localhost:3333/ by default. It is not meant to be an
error message. Perhaps we should rephrase it?

> ./refine: line 852: [: 4000m:*integer expression expected*

I think the expected syntax is "4000M", not "4000m". If I try to launch
OpenRefine 3.4.1 with `./refine -m 4000m", I also get the same error, so
I do not think we have changed anything there.

Best,
Antonin

On 26/10/2021 17:33, Parthasarathi Mukhopadhyay wrote:
> Dear Antonin
>
> I tried beta 2 of OpenRefine 3.5 with the command ./refine -i 0.0.0.0 -m
> 4000m -d /home/myrefine/.local/share/openrefine/3.5/ and it is opening
> as usual but the console produces following messages with a few warnings -
>
> Using refine.ini for configuration
> ./refine: line 852: [: 4000m:*integer expression expected*
> You have 7862M of free memory.
> Your current configuration is set to use 4000m of memory.
> OpenRefine can run better when given more memory. Read our FAQ on how to
> allocate more memory here:
> https://github.com/OpenRefine/OpenRefine/wiki/FAQ-Allocate-More-Memory
> <https://github.com/OpenRefine/OpenRefine/wiki/FAQ-Allocate-More-Memory>
> *No host specified while binding to interface 0.0.0.0, guessing localhost.*
> /usr/lib/jvm/default-java/bin/java -cp
> server/classes:server/target/lib/* -Xms256M -Xmx4000m
> -Drefine.memory=4000m -Drefine.max_form_content_size=1048576
> -Drefine.verbosity=info -Dpython.path=main/webapp/WEB-INF/lib/jython
> -Dpython.cachedir=/home/psm/.local/share/google/refine/cachedir
> -Drefine.data_dir=/home/psm/.local/share/openrefine/3.5/
> -Drefine.webapp=main/webapp -Drefine.port=3333
> -Drefine.interface=0.0.0.0 -Drefine.host=* com.google.refine.Refine
> Starting OpenRefine at 'http://localhost:3333/ <http://localhost:3333/>'
>
> *log4j:WARN No appenders could be found for logger
> (org.eclipse.jetty.util.log).
> log4j:WARN Please initialize the log4j system properly.
> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig
> <http://logging.apache.org/log4j/1.2/faq.html#noconfig> for more info.*
> *
> *
> Is there any way to avoid these warnings (as I'm not getting any of
> these warnings in OR 3.4.1 by applying the same call)?**And another
> question is that is there a more sophisticated way to stop an OR server
> apart from CTRL+C (this applies to all versions of OR)?
> *
> *
> Regards*
> *
>
> On Tue, Oct 26, 2021 at 1:15 AM Antonin Delpeuch (lists)
> <li...@antonin.delpeuch.eu <mailto:li...@antonin.delpeuch.eu>> wrote:
>
> Hello!
>
> The second beta release of OpenRefine 3.5 is out! You can download it
> from here:
>
> https://openrefine.org/download.html
> <https://openrefine.org/download.html>
>
> We are looking forward to your bug reports and hope to publish a stable
> version very soon!
>
> Happy refining,
>
> Antonin
>
> --
> You received this message because you are subscribed to the Google
> Groups "OpenRefine" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to openrefine+...@googlegroups.com
> <mailto:openrefine%2Bunsu...@googlegroups.com>.
> <https://groups.google.com/d/msgid/openrefine/3ca4eec0-c4d9-091c-1ac7-47614c61a258%40antonin.delpeuch.eu>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "OpenRefine" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to openrefine+...@googlegroups.com
> <mailto:openrefine+...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/openrefine/CAGM_5uYE4XU38cA8XXWVN3imsjC%2B6%3D_6NkDhpyqOo%3DJvqSX-bA%40mail.gmail.com
> <https://groups.google.com/d/msgid/openrefine/CAGM_5uYE4XU38cA8XXWVN3imsjC%2B6%3D_6NkDhpyqOo%3DJvqSX-bA%40mail.gmail.com?utm_medium=email&utm_source=footer>.

Thad Guidry

unread,
Oct 27, 2021, 10:44:04 AM10/27/21
to openr...@googlegroups.com
Hi Antonin,

Actually either capital M or lowercase m can be used.  This goes back to the earliest days of Java Options arguments -

https://docs.oracle.com/en/java/javase/11/tools/java.html#GUID-3B1CE181-CD30-4178-9602-230B800D4FAE
For options that require an argument, the argument may be separated from the option name by a space, a colon (:), or an equal sign (=), or the argument may directly follow the option (the exact syntax differs for each option). If you’re expected to specify the size in bytes, then you can use no suffix, or use the suffix k or K for kilobytes (KB), m or M for megabytes (MB), or g or G for gigabytes (GB). For example, to set the size to 8 GB, you can specify either 8g, 8192m, 8388608k, or 8589934592 as the argument. If you are expected to specify the percentage, then use a number from 0 to 1. For example, specify 0.25 for 25%.



To unsubscribe from this group and stop receiving emails from it, send an email to openrefine+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine/313af2d0-3f5b-a563-3859-b270a68af5bf%40antonin.delpeuch.eu.

Thad Guidry

unread,
Oct 27, 2021, 10:57:54 AM10/27/21
to openr...@googlegroups.com
Looks like the Linux startup got changed 2 years ago where the refine Linux start script now introduces a wrongful parsing of the token which is a String and asks for less than or equivalent to an Integer of 256.

Updated refine to fix #388 by aakankasha028 · Pull Request #2490 · OpenRefine/OpenRefine · GitHub


The refine.bat Windows start script does not have that, nor is it really needed.
I think someone can just fix the refine Linux start script to check the value itself instead of the whole token.  4000  versus  "4000m"

Parthasarathi Mukhopadhyay

unread,
Oct 27, 2021, 12:00:21 PM10/27/21
to openr...@googlegroups.com
Yes, some more observations in this issue:

If I run it like - ./refine -i 0.0.0.0 -m 4000 -d /home/psm/.local/share/openrefine/3.5/

that error is not there any more

Using refine.ini for configuration

You have 7862M of free memory.
Your current configuration is set to use 4000 of memory.

but it always says this (and not starting OR):

Error occurred during initialization of VM
Initial heap size set to a larger value than the maximum heap size

My refine.ini settings is this:

REFINE_MEMORY=6500M
REFINE_MIN_MEMORY=3500M

With the same settings in refine.ini, when I call like ./refine -i 0.0.0.0 -m 4000m -d /home/psm/.local/share/openrefine/3.5/
It returns that error message but starts OR without any issue.

Using refine.ini for configuration
./refine: line 852: [: 4000m: integer expression expected

You have 7862M of free memory.

Regards

Antonin Delpeuch (lists)

unread,
Oct 28, 2021, 4:15:02 AM10/28/21
to openr...@googlegroups.com
Thank you! Then that is indeed something that looks worth fixing before
a 3.5 stable release.

Antonin
> <https://github.com/OpenRefine/OpenRefine/pull/2490>
>
> The refine.bat Windows start script does not have that, nor is it
> really needed.
> I think someone can just fix the refine Linux start script to check
> the value itself instead of the whole token.  4000  versus  "4000m"
>
> Thad
> https://www.linkedin.com/in/thadguidry/
> <https://www.linkedin.com/in/thadguidry/>
> https://calendly.com/thadguidry/ <https://calendly.com/thadguidry/>
>
>
> On Wed, Oct 27, 2021 at 9:43 AM Thad Guidry <thadg...@gmail.com
> <mailto:thadg...@gmail.com>> wrote:
>
> Hi Antonin,
>
> Actually either capital M or lowercase m can be used.  This goes
> back to the earliest days of Java Options arguments -
>
> https://docs.oracle.com/en/java/javase/11/tools/java.html#GUID-3B1CE181-CD30-4178-9602-230B800D4FAE
> <https://docs.oracle.com/en/java/javase/11/tools/java.html#GUID-3B1CE181-CD30-4178-9602-230B800D4FAE>
>
> For options that require an argument, the argument may be
> separated from the option name by a space, a colon (:), or
> an equal sign (=), or the argument may directly follow the
> option (the exact syntax differs for each option). If you’re
> expected to specify the size in bytes, then you can use no
> suffix, or use the suffix |k| or |K| for kilobytes (KB), |m|
> or |M| for megabytes (MB), or |g| or |G| for gigabytes (GB).
> For example, to set the size to 8 GB, you can specify either
> |8g|, |8192m|, |8388608k|, or |8589934592| as the argument.
> If you are expected to specify the percentage, then use a
> number from 0 to 1. For example, specify |0.25| for 25%.
>
>
> Thad
> https://www.linkedin.com/in/thadguidry/
> <https://www.linkedin.com/in/thadguidry/>
> https://calendly.com/thadguidry/ <https://calendly.com/thadguidry/>
>
>
> On Wed, Oct 27, 2021 at 1:38 AM Antonin Delpeuch (lists)
> <li...@antonin.delpeuch.eu <mailto:li...@antonin.delpeuch.eu>>
> wrote:
>
> Hi Parthasarathi,
>
> Thanks for trying it out!
>
>  > No host specified while binding to interface 0.0.0.0,
> guessing localhost.
>
> This message simply means that the startup script will
> launch your
> browser at http://localhost:3333/ <http://localhost:3333/>
> <http://localhost:3333/> <http://localhost:3333/
> <mailto:li...@antonin.delpeuch.eu
> <mailto:li...@antonin.delpeuch.eu>>> wrote:
> >
> >     Hello!
> >
> >     The second beta release of OpenRefine 3.5 is out! You
> can download it
> >     from here:
> >
> > https://openrefine.org/download.html
> <https://openrefine.org/download.html>
> >     <https://openrefine.org/download.html
> <https://openrefine.org/download.html>>
> >
> >     We are looking forward to your bug reports and hope
> to publish a stable
> >     version very soon!
> >
> >     Happy refining,
> >
> >     Antonin
> >
> >     --
> >     You received this message because you are subscribed
> to the Google
> >     Groups "OpenRefine" group.
> >     To unsubscribe from this group and stop receiving
> emails from it,
> >     send an email to
> openrefine+...@googlegroups.com
> <mailto:openrefine%2Bunsu...@googlegroups.com>
> >     <mailto:openrefine%2Bunsu...@googlegroups.com
> <mailto:openrefine%252Buns...@googlegroups.com>>.
> <mailto:openrefine%2Bunsu...@googlegroups.com>
> > <mailto:openrefine+...@googlegroups.com
> <mailto:openrefine%2Bunsu...@googlegroups.com>>.
> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/openrefine/CAGM_5uYE4XU38cA8XXWVN3imsjC%2B6%3D_6NkDhpyqOo%3DJvqSX-bA%40mail.gmail.com
> <https://groups.google.com/d/msgid/openrefine/CAGM_5uYE4XU38cA8XXWVN3imsjC%2B6%3D_6NkDhpyqOo%3DJvqSX-bA%40mail.gmail.com>
>
> >
> <https://groups.google.com/d/msgid/openrefine/CAGM_5uYE4XU38cA8XXWVN3imsjC%2B6%3D_6NkDhpyqOo%3DJvqSX-bA%40mail.gmail.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/openrefine/CAGM_5uYE4XU38cA8XXWVN3imsjC%2B6%3D_6NkDhpyqOo%3DJvqSX-bA%40mail.gmail.com?utm_medium=email&utm_source=footer>>.
>
> --
> You received this message because you are subscribed to the
> Google Groups "OpenRefine" group.
> To unsubscribe from this group and stop receiving emails
> from it, send an email to
> openrefine+...@googlegroups.com
> <mailto:openrefine%2Bunsu...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/openrefine/313af2d0-3f5b-a563-3859-b270a68af5bf%40antonin.delpeuch.eu
> <https://groups.google.com/d/msgid/openrefine/313af2d0-3f5b-a563-3859-b270a68af5bf%40antonin.delpeuch.eu>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "OpenRefine" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to openrefine+...@googlegroups.com
> <mailto:openrefine+...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/openrefine/CAChbWaOMXWVdfNdU1cJgy2Y80nPTUUz-fRPZcbFh5KM0Sx8t3Q%40mail.gmail.com
> <https://groups.google.com/d/msgid/openrefine/CAChbWaOMXWVdfNdU1cJgy2Y80nPTUUz-fRPZcbFh5KM0Sx8t3Q%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "OpenRefine" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to openrefine+...@googlegroups.com
> <mailto:openrefine+...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/openrefine/CAGM_5ubP79ZoFMGDAwwOqXiMDnUWpz%2BJNfreBh%2B%2BdQZAWTOezQ%40mail.gmail.com
> <https://groups.google.com/d/msgid/openrefine/CAGM_5ubP79ZoFMGDAwwOqXiMDnUWpz%2BJNfreBh%2B%2BdQZAWTOezQ%40mail.gmail.com?utm_medium=email&utm_source=footer>.

Reply all
Reply to author
Forward
0 new messages