OpenRefine 3.6-rc1 create project

[AdaNeedLearn]

Hi, 

I just installed the newest version of OpenRefine, and tried to create a project.

However, it has been 10 hours. The progress shows almost done.

And in the command window, the only information I am getting is 

12:33:38.975 [                   refine] POST /command/core/get-importing-job-status (999ms)

Did I do something wrong? Or is it normal?

Or the dataset is too big?

Looking forward to your help.

Thanks.

[AdaNeedLearn]

The command window information looks like this. 

12:35:36.967 [                   refine] POST /command/core/get-importing-job-status (998ms)
12:35:37.967 [                   refine] POST /command/core/get-importing-job-status (1000ms)
12:35:38.970 [                   refine] POST /command/core/get-importing-job-status (1003ms)
12:35:39.970 [                   refine] POST /command/core/get-importing-job-status (1000ms)
12:35:40.977 [                   refine] POST /command/core/get-importing-job-status (1007ms)
12:35:41.978 [                   refine] POST /command/core/get-importing-job-status (1001ms)
12:35:42.974 [                   refine] POST /command/core/get-importing-job-status (996ms)
12:35:43.976 [                   refine] POST /command/core/get-importing-job-status (1002ms)
12:36:25.977 [                   refine] POST /command/core/get-importing-job-status (42001ms)
12:37:25.977 [                   refine] POST /command/core/get-importing-job-status (60000ms)
12:38:25.971 [                   refine] POST /command/core/get-importing-job-status (59994ms)
12:39:25.979 [                   refine] POST /command/core/get-importing-job-status (60008ms)
12:40:25.978 [                   refine] POST /command/core/get-importing-job-status (59999ms)
12:41:25.968 [                   refine] POST /command/core/get-importing-job-status (59990ms)
12:42:25.977 [                   refine] POST /command/core/get-importing-job-status (60009ms)
12:43:25.980 [                   refine] POST /command/core/get-importing-job-status (60003ms)
12:44:25.976 [                   refine] POST /command/core/get-importing-job-status (59996ms)
12:45:25.983 [                   refine] POST /command/core/get-importing-job-status (60007ms)
12:46:25.982 [                   refine] POST /command/core/get-importing-job-status (59999ms)
12:47:25.972 [                   refine] POST /command/core/get-importing-job-status (59990ms)
12:48:25.977 [                   refine] POST /command/core/get-importing-job-status (60005ms)
12:49:25.976 [                   refine] POST /command/core/get-importing-job-status (59999ms)
12:50:25.978 [                   refine] POST /command/core/get-importing-job-status (60002ms)
12:51:25.971 [                   refine] POST /command/core/get-importing-job-status (59993ms)
12:52:25.979 [                   refine] POST /command/core/get-importing-job-status (60008ms)
12:53:25.973 [                   refine] POST /command/core/get-importing-job-status (59994ms)
12:54:25.970 [                   refine] POST /command/core/get-importing-job-status (59997ms)
12:55:25.971 [                   refine] POST /command/core/get-importing-job-status (60001ms)
12:56:25.967 [                   refine] POST /command/core/get-importing-job-status (59996ms)
12:57:25.982 [                   refine] POST /command/core/get-importing-job-status (60015ms)
12:58:25.968 [                   refine] POST /command/core/get-importing-job-status (59986ms)
12:59:25.969 [                   refine] POST /command/core/get-importing-job-status (60001ms)

[Antonin Delpeuch (lists)]

Hello,

Does the interface indicate that you are running out of memory? This could happen if you are importing a large file.

In that case, you could try increasing the memory allocated to OpenRefine: https://docs.openrefine.org/manual/installing#increasing-memory-allocation

I wonder if you have the same problem with OpenRefine 3.5.

Cheers,

Antonin

--
You received this message because you are subscribed to the Google Groups "OpenRefine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openrefine+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine/a3bf3158-5060-4ee8-976e-2ec30e6b5582n%40googlegroups.com.

[AdaNeedLearn]

I used to create project with smaller dataset. 

The open refine page shows: almos done... Memory usage: 32% (339/1073MB). 

So, I am not sure if it's too big and running out of memory. 

This is the first time I am creating the project with larger file. 

It is a .xlsx file that is 9 MB size. 

I will install the extension and try again. 

Thanks.

[AdaNeedLearn]

I changed my OpenRefine back to 3.5. I can create in a second now. 

[Antoine Beaubien]

Little question to you: can you share your XLSX file with us?

Regards,

   Antoine

To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine/0400379b-6bbd-47de-8329-7a9d0671bf6en%40googlegroups.com.

[AdaNeedLearn]

No problem. 
But how can I share it with you? 

I tried to attach it to this message but it shows the message I am sending is too long.

[Owen Stephens]

Hi

My experience is that it's usually easiest to put the file onto something like Google Drive or Dropbox and share a link

Best wishes

Owen

 

[Xue Bai]

Thank you so much.

The file is under this link.

https://docs.google.com/spreadsheets/d/18rVuUEO2aH5xPZ-PbHgFJ3RKLZ9R-DM-/edit?usp=sharing&ouid=115878709752210841176&rtpof=true&sd=true

--
You received this message because you are subscribed to a topic in the Google Groups "OpenRefine" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/openrefine/jD_9eI4UdIA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to openrefine+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine/d4912bdb-3230-4ef7-8fc1-93d79ca1c63en%40googlegroups.com.

[Owen Stephens]

Thank you for this

I've tried importing that file in both OpenRefine 3.5.2 and OpenRefine 3.6rc1, in both cases I have 4Gb (4096Mb) memory allocated to OpenRefine

In both cases it imports for me OK. However, with less memory assigned (e.g. the default 1Gb / 1024Mb) then it does not import successfully - I get a message similar to the one you report.

My guess would be it's just a memory issue

Have you tried increasing the memory allocated to OpenRefine 3.6rc1?

When you start OpenRefine in the console/terminal window you should see messages like:

Using refine.ini for configuration
-------------------------------------------------------------------------------------------------
You have 1561M of free memory.
Your current configuration is set to use 4096M of memory.
OpenRefine can run better when given more memory. Read our FAQ on how to allocate more memory here:
https://docs.openrefine.org/manual/installing#increasing-memory-allocation

Based on the error message that you've reported: almost done... Memory usage: 32% (339/1073MB). 

Then you only have a lower amount of memory allocated currently

Best wishes

Owen

[Xue Bai]

I did not try to increase the memory for 3.6.1. I tried another way to solve this problem. I convert the excel file to .CVS file instead of .xsls. I can successfully upload the file into the OpenRefine. 

To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine/c7782895-d92f-424c-93d1-2657fb1886d6n%40googlegroups.com.

[Xue Bai]

Sorry about my typo, 

It was .CSV and .xlsx.

To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine/c7782895-d92f-424c-93d1-2657fb1886d6n%40googlegroups.com.

[Xue Bai]

I changed the Memory size to 4096 mb just now, and it's also working now! 

Thank you so much.

To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine/c7782895-d92f-424c-93d1-2657fb1886d6n%40googlegroups.com.

[Antonin Delpeuch (lists)]

Brilliant! Thank you very much all for the collective debugging!

Antonin

You received this message because you are subscribed to the Google Groups "OpenRefine" group.

To unsubscribe from this group and stop receiving emails from it, send an email to openrefine+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine/CACgKyf0Y%3DMJtOJ-8cjPkXN4E74WasLBx-V%3DH51yGcF-rLj9SPw%40mail.gmail.com.