Jupyterlab extension takes up too much space after built

[Chen Wang]

Hi mailing list,

We're working on a React based Jupyterlab extension, and want to install inside a docker image. Since this docker image already has a lot of other stuff, the size is already pretty big (~ 7G). Since it's still not ready to be open source yet so it's not on NPM yet. I installed by doing jupyter labextension link appFolder and then jupyter lab. After we install the Jupyterlab extension locally, it takes up 1.4G more space... I'm wondering if there's other clean way to install it. 

we have tried a few things:

• After installing we noticed there're new folders and contents generated in the folder ../share/jupyter/lab, such as /extensions, /schemas, /settings, staging ...
  
• We tried to remove the node_modules under staging folder, which seems to take the most spaces, and our extension still performs unaffectedly. However, manually removing stuff is very awkward to do since we're trying use a docker makefile.
  
• We also tried to instead of installing, just replace and move the whole lab folder and place it in the same place in our Docker container. It worked, but the whole Jupyterlab seems to be affected, namely it complains about versions for a few plugins.

I'm wondering if you guys have any methods to either install the extension and then clean out the intermediate files during the build process? Or alternatively there's a way pre-build the extension and just deploy it?

Thank you so much!

[Nicholas Bollweg]

JupyterLab might just part of the problem! But anyhow, try:

- install jupyterlab and nodejs
- add your source
- build your extensions and all dependencies as npm-pkg.tgz
- copy the extensions right into PREFIX/share/jupyter/extensions
- jupyter lab build
- remove nodejs
- rm -rf PREFIX/share/jupyter/lab/staging ~/.npm ~/.cache/yarn
- probably some more stuff, like extra copies of yarn.js (4mb), but staging is the big deal.
- clean all your package managers
- squash your image at the end

All this assumes you are...

- not planning on permitting your end users to _(un)install_ (vs (en|dis)able) extensions
- not using nodejs for anything else
- want to do it all in one docker file with just your Babel/typescript/whatever too chain

Some notes on this:
- The biggest I've ever made a production lab/static/ is about 45mb, which included A number of in-browser language runtimes
- don't do that if you can avoid it
- use webpack visualizer and duplicate checking tools to figure out if you or an extension you're using is messing stuff up
- Once you get to some serious size (main.xxx.js > 5mb), it's important to make sure you are lazy loading... I'm sure most upstream extension would accept PRs

Another way to approach this is the builder pattern... Using pip or conda or rpm or docker or tar, do a build of the extensions you want, and pack up the lab directory (minus staging) and use that as an input to your docker build.

The best place to do this is probably in the same package managers you are using for jupyterlab (e.g pip/conda), and hard pin to the version of jupyterlab. Heck you can even make /static/ read-only.

If you do want to let users install random extensions, you're pretty much out of luck, supporting many users doing "modern frontend development" is an escheresque bouncy castle covered in barbed wire.

Sorry this isn't nice to work with yet in a deployed seeing. We're trying. As long as webpack is in the loop, this is going to be painful and slow. I've even started looking at systemjs and http/2 on tornado.