How to do intermittent save of a project?

[Pauline]

When working in EXCEL, I can hit Save as often as I want to save the file.  In OpenRefine, I know how to export the project.  But is there a way to intermittently save the work I have done in the project without exporting it to a project?  Once I accidentally closed the OpenRefine tab and lost all my work.

Pauline

[Tom Morris]

OpenRefine saves your projects automatically every few minutes if there have been any changes to them since they were last saved. Closing a browser tab has no effect on the OpenRefine server (it has no way of knowing it went away). If you interrupt the server using ctrl/C at the shell or in a command window, it should attempt to save all "dirty" projects as well. The only way it should be possible to lose any data is if you shutdown the computer or kill the server while it still has unsaved projects.

Tom

On Tue, May 3, 2016 at 4:10 PM, Pauline <psmi...@ewu.edu> wrote:

When working in EXCEL, I can hit Save as often as I want to save the file.  In OpenRefine, I know how to export the project.  But is there a way to intermittently save the work I have done in the project without exporting it to a project?  Once I accidentally closed the OpenRefine tab and lost all my work.

Pauline

--
You received this message because you are subscribed to the Google Groups "OpenRefine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openrefine+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Pauline]

Hi Tom,

   I think I opened the project and did work on it and then closed the tab accidentally.  After that, I'm like, it's gone!  I have not saved the project as a project at all. 

   So I did a test just now.  I imported a csv file and exported it to a project.  Then I did an edit.  Then I closed the tab.  Then I re-opened OpenRefine and imported the tar.gz file.  The edit was not there.

Pauline

[Thad Guidry]

Pauline,

Edits are NOT saved immediately.

If you want to save a project, simply go to the OpenRefine console or terminal window and do a CTRL-C and wait for the prompt to terminate (Y/N) and select Y.

Outside of a manual save as above...OpenRefine does save your project and edits to it every 15 minutes. 

https://github.com/OpenRefine/OpenRefine/blob/c103cdcbff49cde90409855d45f4e50b5a8349d1/main/src/com/google/refine/ProjectManager.java#L66

Try out the CTRL-C quick saving and see if that helps you with your idea of doing a Save Project.

(The Export Project is of course another way and should have all edits stored in the Undo/Redo History on the left side.  Facets and their state are not saved however, but there are some OpenRefine Extensions that can help with that if you need: http://openrefine.org/download.html )

Thad

+ThadGuidry

[Owen Stephens]

Hi Pauline,

The workflow you describe isn't how OpenRefine works - that's why you didn't see the edit in the file. When you import a csv file, or import a project file, once you've done this successfully the project is created in OpenRefine, and is accessed in OpenRefine - this is completely separate copy of the data to the file you imported from. So in the scenario you describe you are importing a project, making changes in OpenRefine (which will be saved in the new copy of the data OpenRefine has made when you did the import), and then re-importing the original project - which as you note, has not changed at all.

You should still be able to see the edited copy in OpenRefine as the original project you imported - that's where all your changes are stored. (and this is stored in what OpenRefine calls the 'Workspace Directory' - but most of the time you don't need to know anything about this as OpenRefine will happily store data there, and keep track of it for you).

The 'Export Project' option is only really needed either when you want to share an OpenRefine project with another OpenRefine user or if you want to take an extra backup of the project you are working on.

Hope that makes sense

Owen

[Pauline]

Hello,

  First of all, thank you very much for the responses.  Unfortunately, I thought OpenRefine is like EXCEL in terms of saving edits in the project.  I went to GitHub and read the brief documentation about importing and exporting, in addition to trying to understand the explanations posted in this thread.

  What I want to find out here is how I use OpenRefine in my workflow.  For example, I import a csv file and do edits and delete data.  I am half way through but not quite finish the project.  It's time to go home. I would like to be able to open the project the next morning and continue with my work.  

  What should I do to make that happen?

Thanks for your time!

Pauline

On Tuesday, May 3, 2016 at 1:10:31 PM UTC-7, Pauline wrote:

[Owen Stephens]

Hi Pauline,

When you've finished your work for the day, you can shutdown OpenRefine - see the 'shutdown' instructions for the relevant operating system on this page https://github.com/OpenRefine/OpenRefine/wiki/Installation-Instructions

Next time you want to work on the project, start up OpenRefine in the usual way. In the Browser interface go to the "Open Project" tab (this should show by default) and you should see a list of your projects. Click on the one you want to work with. This should open your project in just the state you left it when you last shutdown OpenRefine

Hope this makes sense.

Owen

[Pauline]

Hello,

  I imported a csv file, did some edits.  Then I did Ctrl+C in the command line and the console automatically shut down.  I closed the browser tab.  Then I clicked open OpenRefine. Then I had to click on Open Project.  It showed a list of files I had worked with up to last week.  It doesn't show any I worked this week, and not the one I did testing a minute ago.  Then I clicked on the workspace directory, and saw the folder of my test file.  There are several folders inside.  I think I shouldn't have to go to this directory.

Pauline

On Tuesday, May 3, 2016 at 1:10:31 PM UTC-7, Pauline wrote:

[Owen Stephens]

Hi Pauline,

No you definitely shouldn't have to go to the workspace directory - any projects you work on should show in the Open Project list.

After you've imported the project, but before you close down the console, can you see the project in the 'Open Project' list?

Owen

[Pauline]

Hi Owen,

  I have the feeling that I must be doing something wrong.  So I describe exactly the step I took.  If I didn't mention a step, I didn't do it.  Here it goes:

  I activate OpenRefine and have the browser opened.  With Create project highlighted, I clicked to find my csv file.  I clicked create project.  Then I did some edits.  Then I clicked the Open ... button on the top right  A second browser tab opened up and showed the beginning page of OpenRefine.  I then clicked on "Open project" on the left.  It showed a list of files, e.g. txt, csv files, but the last files on the top are a week ago.  I did not see my testing file, which is called British library file.csv in the list. (I did not hit Ctrl +C either - not in your instruction).

 Please see picture below.

Thank you for your time.
Pauline

On Tuesday, May 3, 2016 at 1:10:31 PM UTC-7, Pauline wrote:

[Owen Stephens]

Hi Pauline,

I'd definitely expect to see the project you've just created in the Open Projects screen at that point.

Could you select all the text from the command window and paste it in here to see if that gives any clues as to why you aren't seeing the project at this point?

Thanks

Owen

[Pauline]

Here is the screenshot of the console.  Thank you for your time. --- Pauline

On Tuesday, May 3, 2016 at 1:10:31 PM UTC-7, Pauline wrote:

[Owen Stephens]

Would it be possible for you to select all the text in the console and past the whole thing in here? I think you can do this by right clicking on the console window (sorry don't usually use Windows so I'm a bit rusty) and there are some options for copying the text?

Thanks

Owen

[Pauline]

Oh, I am sorry.  I have only got a partial screenshot.  Here when I did Ctrl+A and Ctrl + C to select an copy the text, the console shut down on me.

I tried right click, nothing happened.

Here are the whole screen pictures:

On Tuesday, May 3, 2016 at 1:10:31 PM UTC-7, Pauline wrote:

[Pauline]

Got to get off work.  Talk to you tomorrow.  Thank you for your help!  --- Pauline

On Tuesday, May 3, 2016 at 1:10:31 PM UTC-7, Pauline wrote:

[Owen Stephens]

I'm afraid I'm stumped at the moment.

Could you try the URL (while OpenRefine is running)

http://127.0.0.1:3333/project?project=1501781915090

This Project ID shows in your first screenshot of the console, and so I'm guessing it should be the URL for your project - lets see if that works

Owen

[Pauline]

Hi Owen,

   Yes.  I clicked on the project link, and it opened up the updated project with my edits.  Then I clicked "Open..." button, and clicked open project, it was still showing projects up to last week.  I clicked Browse workspace directory, I saw the project I tested yesterday, together with the projects from last week.

     I think the problem maybe that this screen is not refreshed.  I found a mention of a bug back in 2015 on the Internet that renaming a project in the list did not refresh to show the new name.  I tested the renaming and it didn't refresh. 

     Maybe the problems are related.

Pauline

 

On Tuesday, May 3, 2016 at 1:10:31 PM UTC-7, Pauline wrote:

[Owen Stephens]

Thanks Pauline - that means that we know it is saving the project as we'd expect - which is good!

You might want to try:

Clearing the cache on your browser - see if that leads to the list of projects refreshing

Trying opening http://127.0.0.1:3333 in a different browser and see if in that display you see a longer list of projects 

I'm suggesting these to try to check that it isn't your browser caching an older view of the project list

Owen

[Pauline]

Hi Owen,

    After I cleared the cache, now it is showing  a longer list, including files I opened yesterday.

   Now my question is:  do I have to clear the cache every time I open a project? Well, if I can't find the project in the list.....
Is this a normal behavior?  Is there a setting in OpenRefine I can set to always refresh the list?

  Appreciate your help!

Pauline

On Tuesday, May 3, 2016 at 1:10:31 PM UTC-7, Pauline wrote:

[Tom Morris]

On Fri, May 6, 2016 at 11:18 AM, Pauline <psmi...@ewu.edu> wrote:

    After I cleared the cache, now it is showing  a longer list, including files I opened yesterday.

   Now my question is:  do I have to clear the cache every time I open a project? Well, if I can't find the project in the list.....
Is this a normal behavior?  Is there a setting in OpenRefine I can set to always refresh the list?

You shouldn't have to clear your browser cache or do anything special to have the list of projects displayed correctly. This is the first I've heard of any problem of this nature.

What web browser are you using? 

Tom

[Owen Stephens]

Yay!

I've never seen this behaviour before, so I'm a bit puzzled - it maybe something related to your specific computer/browser/network setup I guess - do you have anything special/different setup in terms of caching in your browser?

Otherwise I think I'd just recommend clearing the cache regularly - I think there are some browser extensions that make this a one-click job

Owen

[Pauline]

I have firefox 45.0.2.

I don't have any special set up about the cache that I know of.  Now I know that I can open a project that would intermittently save my edits.  If I can't find the project, clear the cache.

  Thanks for helping me understand OpenRefine.  This software has helped me tremendously with my work.

Thank you!

Pauline

On Tuesday, May 3, 2016 at 1:10:31 PM UTC-7, Pauline wrote: