Same .leo file open from different leo instances

[Ville M. Vainio]

I may have reported this before, but I often have a problem of having
the same .leo file open in many leo instances. This leads to data loss
as I'm not sure which has the most recent mods.

On opening a .leo file, I'd encourage to check whether it's open in
another leo process, and complain loudly if it is.

You can use c.db, g.app.db or old fashioned lock files
(~/.leo/locks/workbook.leo.lock) to do this.

[Matt Wilkie]

> On opening a .leo file, I'd encourage to check whether it's open in
> another leo process, and complain loudly if it is.

...and perhaps show unified diff on attempted save.

--
-matt

[Edward K. Ream]

On Fri, Apr 27, 2012 at 5:32 AM, Ville M. Vainio <viva...@gmail.com> wrote:
> I may have reported this before

https://bugs.launchpad.net/leo-editor/+bug/711158

> On opening a .leo file, I'd encourage to check whether it's open in
> another leo process, and complain loudly if it is.
>
> You can use c.db, g.app.db or old fashioned lock files
> (~/.leo/locks/workbook.leo.lock) to do this.

Good timing. I looked at this a bit yesterday. I'll see what I can do today.

Edward

[Edward K. Ream]

On Fri, Apr 27, 2012 at 11:11 AM, Edward K. Ream

>> You can use c.db, g.app.db or old fashioned lock files
>> (~/.leo/locks/workbook.leo.lock) to do this.
>
> Good timing.  I looked at this a bit yesterday.  I'll see what I can do today.

Rev 5285 fixes this using g.app.db. This is fundamentally clean,
although there were quite a few complications.

In particular, Leo now always allocates the PickleshareDB cache, even
when caching isn't otherwise used.

The code passes all hand tests and unit tests, but problems could
easily remain. Please report problems immediately.

Edward

[Ville Vainio]

Edward, I'd appreciate if you didn't use the word 'cache' ,:). C.db is
a general database that happens to be used as a cache, but many use
cases are totally unrelated to caching.

Sent from my Windows Phone
From: Edward K. Ream
Sent: 4/27/2012 7:15 PM
To: leo-e...@googlegroups.com
Subject: Re: Same .leo file open from different leo instances

--
You received this message because you are subscribed to the Google
Groups "leo-editor" group.
To post to this group, send email to leo-e...@googlegroups.com.
To unsubscribe from this group, send email to
leo-editor+...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/leo-editor?hl=en.

[Edward K. Ream]

On Fri, Apr 27, 2012 at 11:40 AM, Ville Vainio <viva...@gmail.com> wrote:
> Edward, I'd appreciate if you didn't use the word 'cache' ,:)

Fair enough. And now g.db is available even when --no-cache is in
effect. I'll ensure that c.db always exists immediately.

EKR

[Ville M. Vainio]

The logic seems to have a problem, it complained me about open leo
file (workbook.leo) when I was launching the first leo instance.

Typically, you should store process id of the leo process keeping the
file open in g.app.db. That way, if you see that the file is reported
to be open by a process that no longer exists, you know it's stale
data.

(e.g.

{
"workbook.leo" : [1222, 2323]
}

... would indicate workbook.leo is open by leo instances running at
pid 1222 and 2323.

You can get pid by os.getpid().

Finding out if process is running is a bit harder, e.g.

http://stackoverflow.com/questions/568271/check-if-pid-is-not-in-use-in-python

On Fri, Apr 27, 2012 at 8:42 PM, Edward K. Ream <edre...@gmail.com> wrote:

> On Fri, Apr 27, 2012 at 12:28 PM, Edward K. Ream <edre...@gmail.com> wrote:
>
>> I'll ensure that c.db always exists immediately.
>

> Done at rev 5286.  All unit tests pass.
>
> Edward

[Terry Brown]

On Sat, 28 Apr 2012 22:46:02 +0300

"Ville M. Vainio" <viva...@gmail.com> wrote:

> The logic seems to have a problem, it complained me about open leo
> file (workbook.leo) when I was launching the first leo instance.
>
> Typically, you should store process id of the leo process keeping the
> file open in g.app.db. That way, if you see that the file is reported
> to be open by a process that no longer exists, you know it's stale
> data.

Did this ever get resolved? I'm still seeing this - certainly relying
on pid entries in g.db to be removed during regular Leo exit and not
checking that pid entries are real during load won't work, in cases
where exit is not regular.

Cheers -Terry

[Edward K. Ream]

On Mon, May 7, 2012 at 10:44 AM, Terry Brown <terry_...@yahoo.com> wrote:

>> The logic seems to have a problem, it complained me about open leo
>> file (workbook.leo) when I was launching the first leo instance.
>>
>> Typically, you should store process id of the leo process keeping the
>> file open in g.app.db. That way, if you see that the file is reported
>> to be open by a process that no longer exists, you know it's stale
>> data.
>
> Did this ever get resolved?  I'm still seeing this - certainly relying
> on pid entries in g.db to be removed during regular Leo exit and not
> checking that pid entries are real during load won't work, in cases
> where exit is not regular.

There are still problems when exit is irregular.

Edward

[Terry Brown]

On Mon, 7 May 2012 11:07:48 -0500

"Edward K. Ream" <edre...@gmail.com> wrote:

> There are still problems when exit is irregular.

Which are apparently permanent? My default opening of Leo opens four
outlines in tabs, so getting the message four times is problematic.

I was starting to investigate when I tripped over the g.app.db being
None from leoBridge issue.

Cheers -Terry

[Terry Brown]

On Mon, 7 May 2012 11:22:50 -0500
Terry Brown <terry_...@yahoo.com> wrote:

> I was starting to investigate when I tripped over the g.app.db being
> None from leoBridge issue.

Additional note - answering No to the first of these questions will
suppress any further questions, loading all remaining files, even when
they're already open in another Leo instance.

Cheers -Terry

[Terry Brown]

On Mon, 7 May 2012 11:26:47 -0500

Terry Brown <terry_...@yahoo.com> wrote:

> > I was starting to investigate when I tripped over the g.app.db being
> > None from leoBridge issue.

Whoops, wrong quote, I meant to quote:

> Which are apparently permanent? My default opening of Leo opens four
> outlines in tabs, so getting the message four times is
> problematic.

[Terry Brown]

On Sat, 28 Apr 2012 22:46:02 +0300
"Ville M. Vainio" <viva...@gmail.com> wrote:

> Typically, you should store process id of the leo process keeping the
> file open in g.app.db. That way, if you see that the file is reported
> to be open by a process that no longer exists, you know it's stale
> data.
>
> (e.g.
>
> {
> "workbook.leo" : [1222, 2323]
> }

But how can you tell the process no longer exists? What if the
pid is now in use by a different process? Seems likely to happen
if you're opening Leo about the say point in your system startup
each time. Might not be the exact same pid, but it could be
getting a pid in a range which is likely to be in use. Seems to me you
want to check the command line for the given pid, and I can't see a pure
python way to do that.

There's a module out there called psutil, but apart from being another
dependency we'd have to ship, it uses .c, so isn't pure python.

Cheers -Terry

[SegundoBob]

On Monday, May 7, 2012 9:07:48 AM UTC-7, Edward K. Ream wrote:

There are still problems when exit is irregular.

I don't have any suggestions for directly eliminating the irregular exit problems, but I do suggest that using a "file descriptor" based scheme might be better than your pid based scheme.

If exclusive file locking has not already been considered and rejected, it might be worthwhile considering it.  I believe file locking can be implemented using Python Standard Library modules fnctl for Unix and OSX; and msvcrt for Windows.

http://docs.python.org/library/fcntl.html#fcntl.lockf

    fcntl.lockf(fd, operation[, length[, start[, whence]]])

    I think fnctl works under OSX because I found a patch to fcntl that fixed fcntl behavior under OSX.
   
http://docs.python.org/library/msvcrt.html

    msvcrt.locking()

I assume that if a process terminates without explicitly releasing a lock, then the lock remains in effect a least until the system is rebooted.  I hope, but do not know, that a lock does not persist over a reboot.  These and many other details which I don't know will determine if file descriptor basedfile locking is practical and desirable.  I'm sorry I can't be of more help.
 

[Terry Brown]

On Wed, 9 May 2012 17:20:47 -0700 (PDT)
SegundoBob <bhos...@ieee.org> wrote:

> I assume that if a process terminates without explicitly releasing a lock,
> then the lock remains in effect a least until the system is rebooted. I
> hope, but do not know, that a lock does not persist over a reboot. These
> and many other details which I don't know will determine if file descriptor
> basedfile locking is practical and desirable. I'm sorry I can't be of more
> help.

Good thought, but there are often cases where you *do* want to open the
file twice, and I'm not sure if that could be done with exclusive locks.

The pid scheme isn't being used at the moment either, because I can't
find a cross-platform (or even a platform specific) way to check that
the pid is still a running instance of Leo. Well, you could parse the
output of subprocess.Popen('ps'), but that's klunky.

So currently g.app.db just holds a list of all open files, each Leo
instance adds and removes files from it, and the user's prompted for
confirmation if a file's already on the list - and then prompted to
remove all entries for the file to clean up after an unclean exit.
Probably works well enough, certainly fulfills the need to catch
multiple openings and warn the user about the data loss risk they pose.

Cheers -Terry