Bookmarks pane disappears

48 views
Skip to first unread message

lewis

unread,
Dec 25, 2016, 4:55:57 PM12/25/16
to leo-editor
Hi Terry,

I recently switched from python 3.5.2/PyQt5 v5.7.0 to python 2.7.13/PyQt4 and noticed that my bookmarks pane no longer displays.
Do I have to somehow reactivate the bookmark pane?

Regards
Lewis

Leo Log Window
Leo 5.4, build 20161225051646, Sun Dec 25 05:16:46 EST 2016
Git repo info: branch = master, commit = a14ca3603bc9
Python 2.7.13, PyQt version 4.8.7
Windows 10 AMD64 (build 10.0.14393)

Terry Brown

unread,
Dec 25, 2016, 9:29:26 PM12/25/16
to leo-e...@googlegroups.com
On Sun, 25 Dec 2016 13:55:56 -0800 (PST)
lewis <lewi...@operamail.com> wrote:

> Hi Terry,
>
> I recently switched from python 3.5.2/PyQt5 v5.7.0 to python
> 2.7.13/PyQt4 and noticed that my bookmarks pane no longer displays.
> Do I have to somehow reactivate the bookmark pane?

Yes, if you just select the top node of bookmarks in your outline and
do Alt-X: bookmarks-show that should fix it, the layout might have an
extra pane the first time, but that should go away on reload.

I think what's happening is that the identity of the node holding the
bookmarks is held in the per outline persistent DB,
c.db['_leo_bookmarks_show'] and that's tied to the outline by the path
to the .leo file or a hash of the path, not sure. If somehow the path
or its hash differs at all between Python versions (even though it
resolves to the same path), you don't get the same c.db, and have to
identify the node again.

Not 100% sure about that, but that's my best guess.

Cheers -Terry

> Regards
> Lewis
>
> *Leo Log WindowLeo 5.4, build 20161225051646, Sun Dec 25 05:16:46 EST
> 2016Git repo info: branch = master, commit = a14ca3603bc9Python
> 2.7.13, PyQt version 4.8.7Windows 10 AMD64 (build 10.0.14393) *

lewis

unread,
Dec 26, 2016, 8:40:47 AM12/26/16
to leo-editor
It isn't enough to Alt-X: bookmarks-show. The bookmarks reload in new pane but if you close the leo file, you are prompted to save, and the bookmarks settings are not saved.
The file reloads as if no changes had been made to the bookmarks.

I found the best way is right-click 'Insert' (hover 'Insert an empty pane here') then select the bookmarks node you want to load, then select 'Action' > bookmarks. This reloads the bookmarks setup.
Then you need to right-click over the pane splitter 'Save Layout' enter a name and OK to save it. If you don't save the layout all the previous bookmarks actions are lost.

This reminded me that I had saved layout names in my leo file, but Leo could not find the layout names.
So these layout names (identity?) are saved to the per outline persistent DB.

The fact that I can lose this setup data feels like a bug to me.

Regards
Lewis

Terry Brown

unread,
Dec 26, 2016, 11:51:44 AM12/26/16
to leo-e...@googlegroups.com
On Mon, 26 Dec 2016 05:40:47 -0800 (PST)
lewis <lewi...@operamail.com> wrote:

> It isn't enough to Alt-X: bookmarks-show. The bookmarks reload in new
> pane but if you close the leo file, you are prompted to save, and the
> bookmarks settings are not saved.
> The file reloads as if no changes had been made to the bookmarks.
>
> I found the best way is right-click 'Insert' (hover 'Insert an empty
> pane here') then select the bookmarks node you want to load, then
> select 'Action' > bookmarks. This reloads the bookmarks setup.
> Then you need to right-click over the pane splitter 'Save Layout'
> enter a name and OK to save it. If you don't save the layout all the
> previous bookmarks actions are lost.
>
> This reminded me that I *had* saved layout names in my leo file, but
> Leo could not find the layout names.
> So these layout names (identity?) are saved to the per outline
> persistent DB.
>
> The fact that I can lose this setup data feels like a bug to me.

Ok, so I think it might still be the c.db issues, but perhaps not only
the tracking of the node holding bookmarks, but also the tracking of
the active layout for a particular outline, I think they're both stored
in c.db.

Hmm, so layouts themselves are saved in g.app.db. They shouldn't
disappear with a change in Python version. So if you'd done
right-click Load Layout, your previously saved layout should have been
on the list, and selecting it would have been sufficient to set it for
Python 2's version of c.db.

I think I need to poke around a bit more to confirm that it is changes
in the association of c.db with the same outline between versions of
Python that's driving this problem, and if so see if anything can be
done about it.

Cheers -Terry

> Regards
> Lewis
>

Terry Brown

unread,
Dec 26, 2016, 8:30:13 PM12/26/16
to leo-e...@googlegroups.com
On Mon, 26 Dec 2016 10:51:41 -0600
"'Terry Brown' via leo-editor" <leo-e...@googlegroups.com> wrote:

[snip]
> I think I need to poke around a bit more to confirm that it is changes
> in the association of c.db with the same outline between versions of
> Python that's driving this problem, and if so see if anything can be
> done about it.

I think we were both basically on track, although the exact problem is
a bit simpler than Py 2/3 having different file paths or hashes thereof
for a particular file. It's this line:

https://github.com/leo-editor/leo-editor/blob/4d28c69/leo/core/leoCache.py#L405

c.db values are always written with the highest available pickle
protocol, which is 2 in Python 2 and 4 in Python 3. Python 3's default
is 3. So in Python 2 Leo fails to decode the *value* and reports it as
a *key* error. Or silently fails in bookmarks.py's case.

So the solution would be a hard coded constant of 2 in place of
pickle.HIGHEST_PROTOCOL there.

Differences in protocols are explained here:
https://docs.python.org/3/library/pickle.html#data-stream-format

I've made this change to prevent the problem going forward:

https://github.com/leo-editor/leo-editor/commit/0cf4999

But there's nothing that can be done to fix it retrospectively, Python
2 can't depickle protocols higher than 2, which is what Python 3
wrote. So the problem only affects people using 3, then going back to
2, but that's probably not uncommon for various reasons.

Cheers -Terry

Terry Brown

unread,
Dec 26, 2016, 8:57:15 PM12/26/16
to leo-e...@googlegroups.com
On Mon, 26 Dec 2016 19:30:11 -0600
"'Terry Brown' via leo-editor" <leo-e...@googlegroups.com> wrote:

> But there's nothing that can be done to fix it retrospectively, Python
> 2 can't depickle protocols higher than 2,

But... we could write Python 3 code to recursively convert all the
pickles in ~/.leo/db/ to protocol 2 - would that help you Lewis? It
would undo the effects of this issue, but maybe you've covered all the
impacted cases, or can't run Python 3 on the machine in question? It
would just vanilla Python 3, not Qt required.

Cheers -Terry

lewis

unread,
Dec 26, 2016, 9:24:40 PM12/26/16
to leo-editor
I can run Python 3 on the machine in question. It was only a temporary switch back to python 2, and that occurs infrequently.
I leave it to you for an elegant simple solution :)

Regards
Lewis

Edward K. Ream

unread,
Dec 27, 2016, 3:32:27 AM12/27/16
to leo-editor
On Mon, Dec 26, 2016 at 8:30 PM, 'Terry Brown' via leo-editor <leo-e...@googlegroups.com> wrote:

I think we were both basically on track, although the exact problem is
a bit simpler than Py 2/3 having different file paths or hashes thereof
for a particular file.  It's this line:

https://github.com/leo-editor/leo-editor/blob/4d28c69/leo/core/leoCache.py#L405

​Excellent sleuthing, Terry. 
 
So the solution would be a hard coded constant of 2 in place of
pickle.HIGHEST_PROTOCOL there.

​This seems like the simplest and best solution. I appreciate your attention to detail in rev 0cf4999.

As you say later, we could write Python 3 code to recursively convert all the pickles in ~/.leo/db/ to protocol 2, but for now there seems to be no great need to do that.

Lewis, does the clear-all-caches command make the problem better or worse?

Edward

lewis

unread,
Dec 27, 2016, 7:10:47 AM12/27/16
to leo-editor
Ran alt-x clear-all-caches
I thought this command would clear all caches. It only cleared the cache of the file currently open, on a per file basis.
Output to log is
clearing cache at directory...
C:\Users\lewis\.leo\db\workbook.leo_16f42a527a16b5a0203701b0b93b1da6
clearing cache at directory...
C:\Users\lewis\.leo\db\workbook.leo_4a6694c973447ee8ff01c0da5d57cf4c
done


When I reloaded my leo file:
1. all the bookmarks were cleared
2. the saved layout names were not deleted, and still available to reload

I'm not quite sure what you mean by better or worse. If 'clear-all-cahes' simulates the effect of switching from python 3 to 2, and considering the saved layout names were still available to reload, then things are probably better :)

Leo Log Window
Leo 5.4, build 20161226191821, Mon Dec 26 19:18:21 CST 2016
Git repo info: branch = master, commit = 0cf49992c997

Python 2.7.13, PyQt version 4.8.7
Windows 10 AMD64 (build 10.0.14393)


Regards
Lewis

lewis

unread,
Dec 27, 2016, 7:23:19 AM12/27/16
to leo-editor
As requested here is the full log from loading another of my leo files, with messages about Unpickling errors:


Leo Log Window
Leo 5.4, build 20161226191821, Mon Dec 26 19:18:21 CST 2016
Git repo info: branch = master, commit = 0cf49992c997
Python 2.7.13, PyQt version 4.8.7
Windows 10 AMD64 (build 10.0.14393)
loadOnePlugin: can not load enabled plugin: leo.plugins.stickynotes_plus
Unpickling error - Python 3 data accessed from Python 2?
Unpickling error - Python 3 data accessed from Python 2?
Unpickling error - Python 3 data accessed from Python 2?
reading: N:\leo\Scripting_tutorial.leo
Unpickling error - Python 3 data accessed from Python 2?
reading: @file hello.py
Unpickling error - Python 3 data accessed from Python 2?
Unpickling error - Python 3 data accessed from Python 2?
reading: @auto-md Python for Leo code academy
Unpickling error - Python 3 data accessed from Python 2?

Internal Leo error in bodyString
not unicode: '@others\n\nWarning: this node is ignored when writing this file.\n\n@language md\n@tabwidth -4\n'
Called from writeFile,makeCacheList,__get_b,bodyString
Please report this error to Leo's developers

Internal Leo error in bodyString
not unicode: ''
Called from makeCacheList,makeCacheList,__get_b,bodyString
Please report this error to Leo's developers

Internal Leo error in bodyString
not unicode: ''
Called from makeCacheList,makeCacheList,__get_b,bodyString
Please report this error to Leo's developers

Internal Leo error in bodyString
not unicode: ''
Called from makeCacheList,makeCacheList,__get_b,bodyString
Please report this error to Leo's developers

Internal Leo error in bodyString
not unicode: ''
Called from makeCacheList,makeCacheList,__get_b,bodyString
Please report this error to Leo's developers

Internal Leo error in bodyString
not unicode: ''
Called from makeCacheList,makeCacheList,__get_b,bodyString
Please report this error to Leo's developers

Internal Leo error in bodyString
not unicode: ''
Called from makeCacheList,makeCacheList,__get_b,bodyString
Please report this error to Leo's developers

Internal Leo error in bodyString
not unicode: ''
Called from makeCacheList,makeCacheList,__get_b,bodyString
Please report this error to Leo's developers

Internal Leo error in bodyString
not unicode: ''
Called from makeCacheList,makeCacheList,__get_b,bodyString
Please report this error to Leo's developers

Internal Leo error in bodyString
not unicode: ''
Called from makeCacheList,makeCacheList,__get_b,bodyString
Please report this error to Leo's developers

Internal Leo error in bodyString
not unicode: ''
Called from makeCacheList,makeCacheList,__get_b,bodyString
Please report this error to Leo's developers

Internal Leo error in bodyString
not unicode: ''
Called from makeCacheList,makeCacheList,__get_b,bodyString
Please report this error to Leo's developers

Internal Leo error in bodyString
not unicode: ''
Called from makeCacheList,makeCacheList,__get_b,bodyString
Please report this error to Leo's developers

Internal Leo error in bodyString
not unicode: ''
Called from makeCacheList,makeCacheList,__get_b,bodyString
Please report this error to Leo's developers

Internal Leo error in bodyString
not unicode: ''
Called from makeCacheList,makeCacheList,__get_b,bodyString
Please report this error to Leo's developers

Internal Leo error in bodyString
not unicode: ''
Called from makeCacheList,makeCacheList,__get_b,bodyString
Please report this error to Leo's developers

Internal Leo error in bodyString
not unicode: ''
Called from makeCacheList,makeCacheList,__get_b,bodyString
Please report this error to Leo's developers

Internal Leo error in bodyString
not unicode: ''
Called from makeCacheList,makeCacheList,__get_b,bodyString
Please report this error to Leo's developers

Internal Leo error in bodyString
not unicode: ''
Called from makeCacheList,makeCacheList,__get_b,bodyString
Please report this error to Leo's developers

Internal Leo error in bodyString
not unicode: ''
Called from makeCacheList,makeCacheList,__get_b,bodyString
Please report this error to Leo's developers

Internal Leo error in bodyString
not unicode: ''
Called from makeCacheList,makeCacheList,__get_b,bodyString
Please report this error to Leo's developers

Internal Leo error in bodyString
not unicode: ''
Called from makeCacheList,makeCacheList,__get_b,bodyString
Please report this error to Leo's developers

Internal Leo error in bodyString
not unicode: ''
Called from makeCacheList,makeCacheList,__get_b,bodyString
Please report this error to Leo's developers

Internal Leo error in bodyString
not unicode: ''
Called from makeCacheList,makeCacheList,__get_b,bodyString
Please report this error to Leo's developers

Internal Leo error in bodyString
not unicode: ''
Called from makeCacheList,makeCacheList,__get_b,bodyString
Please report this error to Leo's developers

Internal Leo error in bodyString
not unicode: ''
Called from makeCacheList,makeCacheList,__get_b,bodyString
Please report this error to Leo's developers

Internal Leo error in bodyString
not unicode: ''
Called from makeCacheList,makeCacheList,__get_b,bodyString
Please report this error to Leo's developers

Internal Leo error in bodyString
not unicode: ''
Called from makeCacheList,makeCacheList,__get_b,bodyString
Please report this error to Leo's developers

Internal Leo error in bodyString
not unicode: ''
Called from makeCacheList,makeCacheList,__get_b,bodyString
Please report this error to Leo's developers

Internal Leo error in bodyString
not unicode: ''
Called from makeCacheList,makeCacheList,__get_b,bodyString
Please report this error to Leo's developers

Internal Leo error in bodyString
not unicode: ''
Called from makeCacheList,makeCacheList,__get_b,bodyString
Please report this error to Leo's developers
read 2 files in 0.17 seconds
Unpickling error - Python 3 data accessed from Python 2?
read outline in 0.20 seconds

lewis

unread,
Dec 29, 2016, 6:20:56 PM12/29/16
to leo-editor
Hi Terry,

An update after switching from running Python 2.7.13/PyQt version 4.8.7 to the very latest Python 3.6.0/PyQt version 5.7.1

If I do a Right-click over splitter, no Load layout menu appears for previously saved layouts.

Leo Log Window
Leo 5.4, build 20161229051601, Thu Dec 29 05:16:01 EST 2016
Git repo info: branch = master, commit = 1f5e6a261b90
Python 3.6.0, PyQt version 5.7.1
Windows 10 AMD64 (build 10.0.14393) SP0


Regards
Lewis
Reply all
Reply to author
Forward
0 new messages