How to share a Jupyter installation for team notebook development

[Heath Raftery]

I originally posted this on Stack Overflow seeing there were some relevant discussions there, but am getting no action. I see there was a related thread in this group last year, but it trailed off into minutia. Thought there might be some further insight available from this group.

Copy and paste from the original SO post follows:

I'm developing a Jupyter Notebook for my team to use to catalogue and analyse some proprietary data. I'm ready to share it with the team for on-going execution and development. The team generally have Windows 10 workstations and are skilled engineers, though not data scientists. No one currently uses Jupyter.

I now realise I might have thoroughly misjudged Jupyter's ability to support this sort of working environment.

Option 1: Individual installations

This is the worst case scenario. Anyone that wants to run or modify the notebook needs to install Jupyter. Anaconda is probably the best way to go, but its a big, ugly, scary install. Worse, every user will have to install and manage additional libraries. Any notebook change that requires a kernel change will have to be manually applied to each installation.

Surely, being client-server, this is not the intention of Jupyter.

Option 2: One server, many clients

The obvious alternative is to host the Jupyter server on a network accessible computer and have all users connect to it with a browser. That way there's only one shared installation to manage and each user just needs a URL to access it.

But there's a gotcha - the server expects the notebook to be on its own file system! So every user will access the same notebook file. This makes version control very problematic - no one can check out their own copy of the notebook for independent edit and commit sessions. Instead, changes will overwrite the only copy, and commits/reverts/diffs will have to be done on the server (or by mounting the server's file system).

Option 3: Server in Docker image, each user runs a container

Docker to the rescue? That way we can build/maintain one server image (and even version control it) and each user only needs to have a Docker engine installed to instantiate the image (which is a friendly 8GB download!!). They connect to their own container which, with a bit of scripting trickery, will be pointing at their own copy of the notebook.

This option only took 20 hours to investigate before discovering that it fundamentally sucks. Working with the kernel is tricky with lots of new skills necessary. But more showstopping - nothing that shows a Qt window will work. The qtconsole we can do without, but part of our notebook shows a File Open dialog and the best way to do that is with a Qt Widget. With the server in a Docker Container expecting an X Windows environment, and the client in a Windows browser, the Widget cannot be shown.

The Qt issue was the last of many, many issues trying to get the Docker option running. Everything from matplotlib to path mapping, from os library calls to ipywidgets needed to be investigated, tweaked, Googled, chopped and changed to work. I'm fairly convinced that these dramas would be on-going.

Conclusion

There are lots of discussions around Jupyter version control. There's lots of options for read-only sharing. And there's even a project for runtime-building a Docker container to provide executable access to a notebook. But there is scant advice on using Jupyter in a team environment.

Given the endless complications when the server is not natively running on the same machine as the client, I'm starting to believe Option 1 is the only sane way to go. Before I go to my colleagues with the crappy news, are there any other suggestions?

[Matthias Bussonnier]

Hi Heath,

You seem to have missed https://gitter.im/jupyterhub/jupyterhub, which
is what you should investigate if you want to have many users, and if
you want an all in browser environment; look at
https://github.com/jupyterlab/jupyterlab which is still in development
but should be way better in manipulating non notebook files.

Though it seem like you have competing requirement, you both want
things to run on each user machine and have a single server for all
users. You'll likely have to release one of these requirement to find
a solution that fits you.

You might want to read
http://zero-to-jupyterhub.readthedocs.io/en/latest/ that is still
highly in flux. It show how to deploy using Docker but you don't need
to .
--
M

> --
> You received this message because you are subscribed to the Google Groups
> "Project Jupyter" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jupyter+u...@googlegroups.com.
> To post to this group, send email to jup...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jupyter/47d8ccd3-f522-4011-8ba3-0ab48f785ad2%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Roland Weber]

Hello Heath,

if you want to centralize the library management for the kernel runtime, you could run the Jupyter kernel gateway on a shared server. The users could then run their own copy of nb2kg as a browser-based frontend to that gateway. However, I have no idea how you would manage file upload, or a console-based frontend in that scenario.

Depending on how much effort you want to invest, you can do all sorts of customizations in Jupyter. The notebook server doesn't require the notebooks to be in the file system. That's only the out of the box behavior. You can write your own ContentManager and put the notebooks wherever you want them. For example, in the Data Science Experience (commercial offering), we're storing notebooks in a Cloudant database. With some tricks, the ContentManager can also access the user identity and manage notebooks per user. Maybe it would be sufficient for your scenario to implement a file-based ContentManager with user-specific directories? But I think you'd also have to implement a multi-user authentication module. Out of the box, notebook servers are running for a single user.

cheers,

  Roland

[1] https://github.com/jupyter/kernel_gateway/

[2] https://github.com/jupyter/kernel_gateway_demos/tree/master/nb2kg

[Heath Raftery]

Matthias,

Appreciate the response. I didn't mention JupyterHub because it struck me as solving a fairly different problem, ie. that of providing a user login page in front of an otherwise fairly typical Jupyter server. Is that the right way to think of it? If so, I can't see that it gets me any closer. User management is not an issue here, it's notebook file management. One Jupyter server is enough, provided users can use their own notebook file. Is that something JupyterHub supports? It seems like a beast to investigate if it turns out not to solve the problem.

JupyterLab looks interesting, but again, seems to be solving a different problem.

My requirements are not that things run on each user machine, but that the store is on each user machine and all the running happens on the server. Maybe that is a pipe dream, since so many client-server architectures expect the store to be "in the cloud". The best example of what I want that I can think of off the top of my head is https://www.draw.io - all the user interface, file editing, rendering, exporting, etc. happens on the server, but you can use your own files. To be fair, the result ain't pretty, as straddling the local/remote divide is cumbersome at best.

Heath

[Heath Raftery]

Roland,

Yes, that's the concept! One kernel instance doing the heavy lifting, with every user just installing a relatively lightweight and static notebook server. If that project ever reaches maturity, it may be of use. At the moment it looks like it has all sorts of dramas bridging the divide between notebook and kernel.

Specifying a different ContentManager is an interesting approach too. Sounds like a bit too much work at this stage, but I can imagine a situation where the server prompts the user for access to their preferred store and forms a connection to it. Perhaps Matthias' suggestion around JupyterHub could help manage the user identity part. Again, too much effort for us to justify at this stage unfortunately.

Regards,

Heath

[Matthias Bussonnier]

Well, if you are running code in a kernel then you **need** both your
data store and computation to be on the same machine.
Otherwise you have no way of having open('filename') to work.

So if hosing on a server is ok with you, your best chance is a shared
filesystem where at least user data is mounted on the server.
That's (IMHO) the least worse situation.

You don't need Jupyter and Python libraries to be on users machines
these can be system wide on your server(s), but for each user you
mount their home directory. I would suggest [1] as a read, it's
starting to show its age, but shows you what can be done.

Any way as soon as you need multiple users you want to look at
JupyterHub because the single-user jupyter server have this assumption
: only one user is using it. And if you try to avoid this you will get
into painful situations.

The things you have to figure out are how you want to architecture
your shared filesystem, mounting the server on machines or have
machine being nfs-servers mounting on the main jupyter-hub server.
--
M
1: https://developer.rackspace.com/blog/deploying-jupyterhub-for-education/

> https://groups.google.com/d/msgid/jupyter/415f0a2b-695e-4d22-8495-53893400cbd9%40googlegroups.com.

[Heath Raftery]

Ah yes, that's all making a lot of sense, thank you very much.

If I ever make any progress down that path, I'll report back!

[Jean Bigboute]

Thank you for posting this and I hope you are successful. As an occasional Jupyter user, I have struggled to understand its use for collaborative work when so much depends on future developments and additional infrastructure. In my work environment we can't deploy IT services such as servers, containers etc. on our own. We don't always have access to help sites either. It is hard to know when to push and when to punt. I don't feel so alone any more.

JBB

[Matthias Bussonnier]

On Tue, Jun 6, 2017 at 4:12 PM, Jean Bigboute <jeanbi...@gmail.com> wrote:
> Thank you for posting this and I hope you are successful. As an occasional Jupyter user, I have struggled to understand its use for collaborative work when so much depends on future developments and additional infrastructure. In my work environment we can't deploy IT services such as servers, containers etc. on our own. We don't always have access to help sites either. It is hard to know when to push and when to punt. I don't feel so alone any more.

The usage for collaborative work should become easier soon. We had our
dev meeting last week, and a version that allow some[1] syncing via
google drive is almost ready (Ian did and does a fantastic job). You
will probably see that in JupyterLab in one of the next releases (at
least as an extension).
Finding the right balance between consumers of Jupyter that want to
deploy that on large number of nodes with hundreds of users – and
single individual with limited permissions is hard. We have to make
some concessions, and that's also why various frontends/backend
exists. But IFAICT, google drive integration should need jupyterLab,
one extension and an internet connection to be functional no more.
That that should help a lot.

The documentation can definitively get improvement – and as you might
have seen in an earlier mail Jessica is starting to work with us on
that and we are thrilled – but we can't cover all the ground. We also
tend to assume more and more that people have internet to get help if
you have an interest if having our docs offline and we don't provide a
link please open an issue and we'll try to find time to make that
happen. If you have experience in deploying in a difficult environment
and have manage to go around some pain point, please let us know – or
even better make a PR against our documentation. We'd love to have a
clearer way of expressing the advantages drawback of each deployment
but so far didn't had the time to sit down and write these as things
are moving really fast even for us.

Cheers,
--
Matthias

[1] kernel state is not shared.

>
> JBB

>
> --
> You received this message because you are subscribed to the Google Groups "Project Jupyter" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to jupyter+u...@googlegroups.com.
> To post to this group, send email to jup...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/0fac0fc7-8ab1-4c40-89f0-18abfbb5a5f2%40googlegroups.com.

[Jean Bigboute]

Matthias and Team:

On Tuesday, June 6, 2017 at 5:04:29 PM UTC-7, Matthias Bussonnier wrote:

On Tue, Jun 6, 2017 at 4:12 PM, Jean Bigboute <jeanbi...@gmail.com> wrote:
> Thank you for posting this and I hope you are successful.  As an occasional Jupyter user, I have struggled to understand its use for collaborative work when so much depends on future developments and additional infrastructure.  In my work environment we can't deploy IT services such as servers, containers etc. on our own.  We don't always have access to help sites either.  It is hard to know when to push and when to punt.  I don't feel so alone any more.

The usage for collaborative work should become easier soon. We had our
dev meeting last week, and a version that allow some[1] syncing  via
google drive is almost ready (Ian did and does a fantastic job). You
will probably see that in JupyterLab in one of the next releases (at

least as an extension). ...

 

Thanks for your prompt response.  With due respect, I would like to point out that Google Drive doesn't solve the problem.  In corporate environments where proprietary data and algorithms are at stake, it is highly unlikely that users will be allowed access to Google Drive, Github, NBViewer, StackOverflow, etc.  That is certainly true with my employer so I can only post completely generic questions and then again from my own account.  In the present IT security environment (which is only going to get more restrictive) any type of added service requires multiple levels of scrutiny and review all tied into business cases.  Bypassing the system or attempting to do so will be detected quickly and the people responsible will get fired.  Unless the product or service is incredibly compelling (see below) no one will go through the effort to get a product approved because the odds of success are practically zero.

Additionally, I looked up JupyterLab just now and read that
"JupyterLab computational environment. This is a very early preview, and is not suitable for general usage yet. "
This has been the case for quite some time.
 

Finding the right balance between consumers of Jupyter that want to

deploy that on large number of nodes with hundreds of users – ...

[deletia]

The documentation can definitively get improvement – and as you might
have seen in an earlier mail Jessica is starting to work with us on
that and we are thrilled – but we can't cover all the ground. We also
tend to assume more and more that people have internet to get help if
you have an interest if having our docs offline and we don't provide a
link please open an issue and we'll try to find time to make that
happen. If you have experience in deploying in a difficult environment
and have manage to go around some pain point, please let us know – or
even better make a PR against our documentation. We'd love to have a
clearer way of expressing the advantages drawback of each deployment
but so far didn't had the time to sit down and write these as things
are moving really fast even for us.

 

And this is the key point.  I have been following IPython then Jupyter now Jupyter +Lab and +Hub since about 2014.  The pace of developments is incredibly fast and I admire the team for doing so much.  But, I think the mindset is very much from academia.  I love academia, its ethos, and the exuberance but Jupyter is now fully promoting into other areas which have very different cultures and needs.  Documentation, user testing, stability, self-containment,... these are all important and I don't think are being given sufficient consideration.  There's a fascinating blogpost on making Jupyter Notebooks more reproducible:

https://jakevdp.github.io/blog/2017/03/03/reproducible-data-analysis-in-jupyter/

The author uses Github as a repository and a standalone text editor among other tools to accomplish this.  In one of the comments, he remarks "I agree that collaborating on notebooks is tough. I mostly avoid it, and instead collaborate on plain-text documents like markdown and code files."  Then, he recommends the nbdime project as another workaround.  I bring him into this discussion because I think he is incredibly smart, an advocate for the project, highly experienced with the tool,  and yet still has to find ways to do things which, on the surface, are asserted as built in.

Please excuse the frustration that is coming though.  I think I am a well-wisher to the project but after nearly three years of observation, I suggest, again respectfully, that you won't always be reaching the elite.  If Jupyter is to succeed, it will have to be used by rank-and-file slobs like myself in environments where we have pressures to produce.  We aren't dumb but neither can we 't be diving into the gizzards of servers, source code, and containers.  We need stuff to work so we can get on with our jobs.  'Real Soon Now' only goes so far.





 

[Yuvi Panda]

For collaboration, I set up my git to use
https://github.com/jupyter/nbdime and then use git / github to
collaborate like I do with all of my other software projects.

I think part of the confusion is that 'collaboration' can mean so many
different things, and almost everyone is talking about something else
when they say 'collaboration'. There's multi user environments
(JupyterHub, with containers and all of that), real time multi user
document editing (JupyterLab + google drive), 'make .ipynb files
importable like .py files' (github.com/ipython/ipynb, still a WIP) and
'make .ipynb files work with git!'
(https://github.com/jupyter/nbdime). They are all targeted at
completely different people - JupyterHub info is completely
incomprehensible to someone wanting nbdime, and can be super
frustrating too.

I'm not entirely sure how to make the situation better tho.e


--
Yuvi Panda T
http://yuvi.in/blog

[Brian Granger]

Jean,

Thanks for the feedback! The pain you are feeling around collaboration
isn't unique. We are hearing the same things from many users and that
feedback is strongly driving our roadmap...but it takes time...years
in many cases to build a robust architecture that supports these
things. I realize that telling you "we are building things...some day
soon..." doesn't help you ability to get things today. But we do
appreciate your sharing about your usage case and experiences.

Cheers,

Brian

> --
> You received this message because you are subscribed to the Google Groups
> "Project Jupyter" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jupyter+u...@googlegroups.com.
> To post to this group, send email to jup...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/jupyter/ebc062a3-4ae1-4635-960a-4d9ce62ea605%40googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.

--
Brian E. Granger
Associate Professor of Physics and Data Science
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgra...@calpoly.edu and elli...@gmail.com

[Matthias Bussonnier]

But, we have to start somewhere. We definitively do not have the
bandwidth, and likely miss experience in designing realtime
collaboration protocols and implementing a server that does it. At
least by having GoogleDrive as a backend we can show that the frontend
does support that, and hopefully someone will implement a backend that
can be hosted on site by those that desire it. We are aware that not
everybody can use GoogleDrive, or any other services because of
network restriction or simply because they are on Air-Gapped network.
Though we are funded by non-profit and the majority of our users can,
so that's the first step we took. We did leave room for changing the
implementation though. It may not be one of our priority.

Services like CoCalc (ex SageMathCloud) do provide on premise
deployment of Jupyter with RT collaboration that are completely
custom. CoCalc have some restriction on what you can do in notebooks –
for security reasons – and is one of the answer we give to users that
have custom needs. WE understand that you may not have your word to
say in your institution, but if a lot of users really want something
like this, you can likely contract with William (CoCalc creator), or
even Continuum (Anaconda), I'm pretty sure they will be happy for you
to consult with one of their employee under NDA. In the end the
success of these companies and that they can sell support is in part
what allow the project to be successful, survive financially and be
open source. Being on the Open Source and Academic Side, even if I
care about you as a user, and I will do my best to help you, this is
(I think) the right place to tell "This one place where commercial
support is available".

>
> Additionally, I looked up JupyterLab just now and read that
> "JupyterLab computational environment. This is a very early preview, and is
> not suitable for general usage yet. "
> This has been the case for quite some time.

Well we should probably update this, we don't want to say it's ready
for prime time yet (but soon, and I'll let the pleasure to announce
that to the team behind it). Though it is quite stable , already
deploy in a couple of commercial offer (Last one we learned of was
r-brain). It's just that the standard of "Ready" for JupyterLab will
be way higher than the current notebook, and the team is spending a
large chunk of time to make sure the transition is as seamless a
possible. We want people that try jupyterLab one to not go back to the
classical notebook because we break their workflow as otherwise they
will go back and not bother to re-try another time. And if there is
one thing we don't want it is the Python 2 vs Python 3 debacle once
more.

>>
>> Finding the right balance between consumers of Jupyter that want to
>> deploy that on large number of nodes with hundreds of users – ...
>
> [deletia]
>>
>>
>> The documentation can definitively get improvement – and as you might
>> have seen in an earlier mail Jessica is starting to work with us on
>> that and we are thrilled – but we can't cover all the ground. We also
>> tend to assume more and more that people have internet to get help if
>> you have an interest if having our docs offline and we don't provide a
>> link please open an issue and we'll try to find time to make that
>> happen. If you have experience in deploying in a difficult environment
>> and have manage to go around some pain point, please let us know – or
>> even better make a PR against our documentation. We'd love to have a
>> clearer way of expressing the advantages drawback of each deployment
>> but so far didn't had the time to sit down and write these as things
>> are moving really fast even for us.
>
>
> And this is the key point. I have been following IPython then Jupyter now
> Jupyter +Lab and +Hub since about 2014. The pace of developments is
> incredibly fast and I admire the team for doing so much.

These days most of the new things happening in Jupyter are actually
not from the core team.

> But, I think the
> mindset is very much from academia. I love academia, its ethos, and the
> exuberance but Jupyter is now fully promoting into other areas which have
> very different cultures and needs. Documentation, user testing, stability,
> self-containment,... these are all important and I don't think are being
> given sufficient consideration.

That's weird to hear as most of the development in JupyterLab these
days is done by corporate sponsors and get a lot of feedback from
internal needs. You can quickly grep through the main commiters of Lab
and you will see that mostly they work at Bloomberg and Continuum.io.
I'm pretty sure they have internal deployment with customization and
that some of their network policies are likely as constrained as your.
So yes their IT does deploy this – which is not the case for you. But
they do run it, which should be a large selling point. Microsoft,
IBM,... are also deploying notebooks often with commercial offers, and
while maybe not as much involve in day-to day commits to the core
components on the GitHub side they do share feedback and contribute a
lot of extensions.

> There's a fascinating blogpost on making
> Jupyter Notebooks more reproducible:
>
> https://jakevdp.github.io/blog/2017/03/03/reproducible-data-analysis-in-jupyter/
>
> The author uses Github as a repository and a standalone text editor among
> other tools to accomplish this. In one of the comments, he remarks "I agree
> that collaborating on notebooks is tough. I mostly avoid it, and instead
> collaborate on plain-text documents like markdown and code files." Then, he
> recommends the nbdime project as another workaround. I bring him into this
> discussion because I think he is incredibly smart, an advocate for the
> project, highly experienced with the tool, and yet still has to find ways
> to do things which, on the surface, are asserted as built in.

Yes, and jake did a lot of efforts, complained and ask a lot of
questions as well, even wrote books using notebooks, which was not an
easy thing.
We can't perfectly accommodate everyone workflow (we do our best),
things are getting way easier than they were, and the Jupyter is
flexible enough to accommodate custom format if you wish – [1][2][3]
store notebooks as "flat" formats editable with any text editor ; with
the constraint of partially losing some guaranties like metadata. And
so far a lot of people are using Git for full notebooks and it works.
There is some frustration – yes – but so far every-time someone was
frustrated enough they came and fixed it ; in the core or as an
extension. That's how most of us started this project to scratch our
own itch and if someone take the time to scratch their itch; write a
flat format that have a good enough experience and is widespread
enough we'll gradely move that into the core.

> Please excuse the frustration that is coming though.

No worries, we are happy that user like you come and express these.
It's the kind of feedback that we can cite when going through
agencies/lab/companies and show them that their users are in needs of
change and functionalities but that we don't have the
bandwidth/funding. And hopefully convince them to either fund us or
contribute back.

> I think I am a
> well-wisher to the project but after nearly three years of observation, I
> suggest, again respectfully, that you won't always be reaching the elite.
> If Jupyter is to succeed, it will have to be used by rank-and-file slobs
> like myself in environments where we have pressures to produce. We aren't
> dumb but neither can we 't be diving into the gizzards of servers, source
> code, and containers. We need stuff to work so we can get on with our jobs.
> 'Real Soon Now' only goes so far.

Like you we have limited time, and can only do a best-effort to
provide things that work out of the box. The simplicity with how you
can get things today is so much better than a couple of years ago. We
are not yet at the install simplicity of something like the AppStore
or similar but it is not _that_ far, and hopefully with your help we
can get to it faster.

Thanks again for reaching out, and keep in mind that if you can't ask
things publicly you can alway try to reach our via private channels.

Cheers,

--
Matthias

P.S: Apologies for the too long response but it's already written...

[1] https://github.com/rossant/ipymd
[2] https://github.com/aaren/notedown
[3] https://github.com/takluyver/nbexplode

[Jean Bigboute]

Matthias and Brian:

I appreciate your taking the time to consider my concerns and to respond.  A few short thoughts:

1) Prioritization and starting somewhere:  I understand that this is a big problem and that free (as in speech) has its limitations

2) Contributing: I sincerely wish I could contribute to open source projects. I just don't have the computing chops.  I'll see about making a donation (free as in beer)

3) Contracting for a private deployment and service:  Agreed and when the tools are farther along with respect to my employer's business, I'm open to trying to get the funding to work with suppliers when there's a real chance of purchasing their services

4) JupyterLab:  I have installed it and already I like it.  It uses the screen better and I appreciate the simple things like "Save As" and the dockable doc browser is a nice touch.   I hope there's a variable inspector and breakpointing feature in the works.  This is one of those areas where Matlab earns its fees - their UI is wonderful

5) Containers:  There must be something to them.  Nature has chimed in: http://www.nature.com/news/software-simplified-1.22059
"Software simplified: Containerization technology takes the hassle out of setting up software and can boost the reproducibility of data-driven research."

A final thought before I go back to lurking.  I read the successful funding proposal on the Jupyter site and was taken aback by  'computational narrative.'  Story over substance is bad news (cf. Edward Tufte and Powerpoint) and if that corporate management mindset leaks into academia, it's all over.  There is nothing more beautiful than proof. 

With thanks for the positive discussion [and apologies to Heath Raftery for hijacking his thread],

--- JBB

[Matthias Bussonnier]

Hi Jean,

On Wed, Jun 7, 2017 at 9:17 PM, Jean Bigboute <jeanbi...@gmail.com> wrote:
> Matthias and Brian:
>
> I appreciate your taking the time to consider my concerns and to respond. A
> few short thoughts:
>
> 1) Prioritization and starting somewhere: I understand that this is a big
> problem and that free (as in speech) has its limitations
>
> 2) Contributing: I sincerely wish I could contribute to open source
> projects.

> I just don't have the computing chops.

You are likely underestimating yourself. Feel free to have a look at
my first PR[1] to IPython a couple of years ago. If you are on this
mailing list then you have the skills to contribute, that's one on the
main reason we do not have a -dev vs -user mailing list. We are
trying to keep a list of easy issues [2], and for example IPython even
just tags PRs that need manual backporting [3] because of git
conflict. We're also always happy to guide you through any of these
steps !

> I'll see about making a
> donation (free as in beer)

That will also be really appreciated as well. And IFAIU it's tax
deductible in the US, we're looking into how to have something similar
in Europe (help welcome if you have any skill in this domain as well).

>
> 3) Contracting for a private deployment and service: Agreed and when the
> tools are farther along with respect to my employer's business, I'm open to
> trying to get the funding to work with suppliers when there's a real chance
> of purchasing their services
>
> 4) JupyterLab: I have installed it and already I like it. It uses the
> screen better and I appreciate the simple things like "Save As" and the
> dockable doc browser is a nice touch.

> I hope there's a variable inspector
> and breakpointing feature in the works. This is one of those areas where
> Matlab earns its fees - their UI is wonderful

Yes, there are plans, but making it cross language is tough. We've
heard of some Companies interested in exploring that.

>
> 5) Containers: There must be something to them. Nature has chimed in:
> http://www.nature.com/news/software-simplified-1.22059
> "Software simplified: Containerization technology takes the hassle out of
> setting up software and can boost the reproducibility of data-driven
> research."

Containers are not a silver bullet. In particular in replicability vs
reproducibility, and compostability.
But it helps.

Thanks !
--
Matthias