Serving a streaming local foder with JupyterLab or Jupyter server

[marco caggioni]

Hi,

First of all thanks for all the work and the great product you are developing!

I have been using notebook from the Ipython notebook version and is great to see the evolution especially the new Jupyterlab.

I wanted to share a use case to get your feedback and possibly ideas to build on it:

Recently box.com released a box drive application https://www.box.com/drive that once installed on the PC creates a virtual folder in the local file system that provides access to all the files in your cloud account.

The beauty of it is that files are available and the search is very fast without the need to download them on the local disk.

Serving that folder with Jupyter or Jupyterlab server works perfectly, I can search notebook, open, edit save files and load data files as if I had the files on my computer but at the same time the file are immediately updated on the box account and box provides also a version control (i don't use it very much but is there and in principle I could go back and check the last 100 version of the notebook)

Also if I share the folder with someone else, the files produced by a notebook analysis, for example, are available to all the collaborators without having to move anything. 

I find this workflow very useful since I can also switch computer, for example PC to Mac and, provided I have box drive installed, I can  work on the same files but with the local python server. 

My question to you is:

Is this something commonly used? if anybody experimented similar or alternative approach would be nice to hear different experiences.

I know box.com allows to add label to the files so I was thinking if I add a json label to some of my file I could use the BOX API to query the files similarly to what I would do to querying a Mongodb object or gridfs file. I'm sure if it is a good idea, the speed would not probably be comparable but I don't necessarly have the problem of big data, more the problem to keep them organized and available to multiple user that already have a box account.

It is easy for me to ask collaborators to upload files to box to make them available for analysis but would be much harder to create a workflow for them to upload to a Mongodb for example

So in a sense I see Box or goggle drive which will have a streaming option (i think now is on the business version) as a simple object oriented database that can store the notebooks, the data analyzed by the notebook and the output generated by the notebook in the same place and provides on top of the  ability to serve the folder directly by Jupyter server to do computation on the data, the possibility for users that do not use Jupyter to explore datafiles and output with a nice user interface.

I also think that when eventually there will be viewer for the ipynb file directly on box, dropbox, google drive this would be a very simple workflow to share notebook analysis in organization that do not have the need to use more advanced workflow management like git.

Just wanted to share this use case since it really simplified my life and would like to learn more if the Jupyterlab environment may enable to improve the use of file streaming services.

thanks

    Marco