NEW: cloud storage plugin for Leo

[Terry Brown]

The new leo_cloud plugin allows subtrees within a .leo file to be
stored in the cloud. It should be possible to support various cloud
platforms, currently git is supported (i.e. you can use GitLab or
GitHub or your own remote git server).

A leo_cloud subtree has a top node with a headline that starts with
'@leo_cloud'. The rest of the headline is ignored. The body of this
top node is used to describe the cloud service, e.g.:

type: Git
remote: g...@gitlab.com:tnbrown/leo_cloud_storage.git
local: ~/.leo/leo_cloud/gitlab_leo_cloud_storage
ID: shortcuts
read_on_load: ask
write_on_save: ask

The first three lines can be repeated with different IDs to store
different subtrees at the same remote cloud location.

read_on_load: / write_on_save: can be yes, no, or ask. If it's not one
of those three, there's a warning dialog.

There's also a file system backend, which would look like this:

type: FileSystem
root: ~/DropBox/leo_cloud
ID: my_notes
read_on_load: ask
write_on_save: ask

The FileSystem backend was meant to be for development, but of course
if you map it into a folder that is sync'ed externally, as shown above,
it can serve as a cloud adapter too.

In addition to the Git and FileSystem cloud types it should be possible
to add many others - Google Drive, OneDrive, DropBox, AWS, WebDAV,
sFTP, whatever.

Note: https://gitlab.com/ gives you free private repos., in case you
didn't know.

The data stored is basically headline, body, and uA (unknown
attributes). The caveat is that it must be JSON serializable, this is
to avoid pickle flavor issues. I don't think this will cause problems
except for legacy datetime objects from the todo.py plugin and set()s
in the tags plugin. I think both can be fixed easily - a custom JSON
writer can write datetime as iso string time and sets as lists, and the
tags plugin can coerce lists to sets. I think the todo.py plugin
already reads iso string time values.

My intended use was a common synchronized todo list across machines,
which this achieves. (note to self, make sure todo icons are refreshed
properly).

An unintended bonus is that you can use it to sync. your settings
across machines easily too. Because Leo is brilliant ;-), this:

@settings
@keys
@leo_cloud
@shortcuts

"just works", so now your shortcuts etc. can be stored on a central
server.

Lightly tested, but seems to work - testing and other feedback
appreciated.

Cheers -Terry

[Terry Brown]

p.s. for the Git cloud adapter, `git` needs to be on the path.

[Edward K. Ream]

On Sun, Sep 24, 2017 at 3:40 PM, Terry Brown <terry...@gmail.com> wrote:

The new leo_cloud plugin allows subtrees within a .leo file to be
stored in the cloud.  It should be possible to support various cloud
platforms, currently git is supported (i.e. you can use GitLab or
GitHub or your own remote git server).

​Thanks for this.  I've updated LeoPluginsRef.leo with the appropriate @auto nodes. They will be part of Leo 5.6 final, coming later today.

Edward

[john lunzer]

Just read through this properly and it sounds like a step into the future (or present). 

The part about cloudifying/centralizing @settings would be friggin fantastic. sFTP might be nice to have for people behind restrictive firewalls.

Great job, thank you for your contributions.

[Terry Brown]

On Thu, 28 Sep 2017 04:48:46 -0700 (PDT)
john lunzer <lun...@gmail.com> wrote:

> Just read through this properly and it sounds like a step into the
> future (or present).

:-) thanks, I think it's quite significant. It was quite simple in its
current form, reading / writing whole sub-trees. Maybe it will get
more granular in the future, but it's already a big feature addition.

Now my web links, todo items, and @shortcuts are sync'ed between my
work and home machines, within workbook.leos that are otherwise quite
different.

> The part about cloudifying/centralizing @settings would be friggin
> fantastic. sFTP might be nice to have for people behind restrictive
> firewalls.

I can look at an sFTP adapter. Note that GitHub/GitLab can use HTTPS,
which will work with most restrictive firewalls.

Also I should stress (for general audience) that using GitHub/GitLab in
this context doesn't mean you're using git / need to understand git, it
just means your using those services to provide cloud storage.

Cheers -Terry

[Terry Brown]

On Thu, 28 Sep 2017 04:48:46 -0700 (PDT)
john lunzer <lun...@gmail.com> wrote:

> sFTP might be nice to have for people behind restrictive firewalls

Would a dependency on paramiko be an issue for you? That seems to be
the python lib. for sFTP.

Jake's @sftp plugin uses it.

Cheers -Terry

[Kent Tenney]

I've been looking at a couple syncing issues:

- between work and home (I currently schlep a laptop back and forth)

- between my comfortable, tweaked, gui workstation using heavily

  customized Leo, and files of interest on bare bone servers.

leo_cloud looks like it will address the first one very nicely.

I's a big deal.

For the 2nd one I'm using the Gvim scp capability:

# gvim scp://user@server//path/to/files/

I've been looking at kludges which would mimic vim's scp://

capability while working from Leo, they mostly involve rsync and inotify,

If I was a better programmer I'd try to write

@scp-auto and @scp-clean file types.

Just saying.

Thanks,

Kent

On Sun, Sep 24, 2017 at 3:40 PM, Terry Brown <terry...@gmail.com> wrote:

--
You received this message because you are subscribed to the Google Groups "leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email to leo-editor+unsubscribe@googlegroups.com.
To post to this group, send email to leo-e...@googlegroups.com.
Visit this group at https://groups.google.com/group/leo-editor.
For more options, visit https://groups.google.com/d/optout.

[Terry Brown]

On Thu, 28 Sep 2017 09:53:10 -0500
Kent Tenney <kte...@gmail.com> wrote:

> For the 2nd one I'm using the Gvim scp capability:
> # gvim scp://user@server//path/to/files/
>
> I've been looking at kludges which would mimic vim's scp://
> capability while working from Leo, they mostly involve rsync and
> inotify,

Take a look at Jake's sftp plugin, I think that's basically what that
does. I don't think there's much real difference between sftp and scp.

It adds @sftp nodes.

Cheers -Terry

[Kent Tenney]

I've settled on lsyncd https://github.com/axkibe/lsyncd

It will transparently maintain one or more mirrors of a directory tree.

Just what I want, with minimal effort.

Thanks,

Kent

Cheers -Terry

[Edward K. Ream]

On Sun, Sep 24, 2017 at 3:40 PM, Terry Brown <terry...@gmail.com> wrote:

The new leo_cloud plugin allows subtrees within a .leo file to be
stored in the cloud. 

​Rev cd45289 adds a mention of leo_cloud.py to LeoDocs.leo and adds a lightly edited version of the rest of this post to the module-level docstring in leo_cloud.py.

Terry, please review the docstring and make any necessary edits.

Edward

[Terry Brown]

Thanks for doing that - feel like I missed this msg. originally.

Cheers -Terry

[Matt Wilkie]

In addition to the Git and FileSystem cloud types it should be possible
to add many others - Google Drive, OneDrive, DropBox, AWS, WebDAV,
sFTP, whatever.

Note: https://gitlab.com/ gives you free private repos., in case you
didn't know.

 

How to use Dropbox (safely) for a private git repo:

http://www.anishathalye.com/2015/08/19/git-remote-dropbox/