Can not see the custom.js or log files when using jupyter notebook cmd to launch a IPython notebook

[nju08e...@gmail.com]

Hi, all
           Currently, I am using Anaconda3-4.1.1-Linux-x86_64.sh(currently the newest one) to help me install a Jupyter environment for me to use Jupyter Notebook, Using Anaconda, I successfully installed jupyter, and when I am using jupyter notebook to start a notebook, I found almost no extra files produced, expecially the custom.js file that I want to use later. I know some files are produced in the home directory of the current user, that's the default, and then I referenced the website http://jupyter.readthedocs.io/en/latest/migrating.html  told that, I set value to the parameter as JUPYTER_DATA_DIR and JUPYTER_CONFIG_DIR, like following:

firstly,
unset XDG_RUNTIME_DIR
then
export JUPYTER_CONFIG_DIR=/MY_CONFIG_DIR
export JUPYTER_DATA_DIR=/MY_DATA_DIR

then run
jupyter notebook --generate-config
jupyter notebook

the config file really produced in MY_CONFIG_DIR, and some files like nbserver-20374.json produced in runtime dir of MY_DATA_DIR, but I definitely can not find the dirs as custom and files as custom.js, Could anyone help me or tell me where are the files' location?

Thanks in advance.


[Thomas Kluyver]

On 27 July 2016 at 11:36, <nju08e...@gmail.com> wrote:

the config file really produced in MY_CONFIG_DIR, and some files like nbserver-20374.json produced in runtime dir of MY_DATA_DIR, but I definitely can not find the dirs as custom and files as custom.js, Could anyone help me or tell me where are the files' location?


The custom files are not created automatically, but you can put them at e.g. $JUPYTER_CONFIG_DIR/custom/custom.js

[薛亚兰]

Thanks Thomas, then I knew how to put the custom.js file, and I tried, it worked, Thanks again.

however,  I have another question about the notebook local directory, what parameter we should set to change it, since in my example, I really have no idea why the local directory of mine should be this value,
Serving notebooks from local directory: /tmp/nonroot/jupyter/work
I run the jupyter notebook at this dir /tmp/nonroot/jupyter/, and I already set the  c.NotebookApp.notebook_dir to be another value, but the local directory keeps the same, everytime I start a notebook, it is this value /tmp/nonroot/jupyter/work/. 

Do you have any idea about these above dir associated issues?

Thanks in advance! 

--
You received this message because you are subscribed to a topic in the Google Groups "Project Jupyter" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jupyter/VF5RHpncN2k/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jupyter+u...@googlegroups.com.
To post to this group, send email to jup...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/CAOvn4qh%2BkSQ_cdGj6M28ZanoXJTNpEgiTrT7AWYc%3D-5XX_anww%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Yalan  Xue ， SIT  Lab of   Nanjing  University 

 Address: Nanjing ,China。

[MinRK]

On Thu, Jul 28, 2016 at 9:32 AM, 薛亚兰 <nju08e...@gmail.com> wrote:

Thanks Thomas, then I knew how to put the custom.js file, and I tried, it worked, Thanks again.

however,  I have another question about the notebook local directory, what parameter we should set to change it, since in my example, I really have no idea why the local directory of mine should be this value,
Serving notebooks from local directory: /tmp/nonroot/jupyter/work
I run the jupyter notebook at this dir /tmp/nonroot/jupyter/, and I already set the  c.NotebookApp.notebook_dir to be another value, but the local directory keeps the same, everytime I start a notebook, it is this value /tmp/nonroot/jupyter/work/. 

Do you have any idea about these above dir associated issues?

Setting c.NotebookApp.notebook_dir should indeed set the default notebook directory. This will be overridden by command-line arguments, e.g.

jupyter notebook /path/to/notebooks

or

jupyter notebook --notebook-dir /path/to/notebooks

You might verify that this config is picked up, by starting the notebook server with --debug.

-MinRK

Thanks in advance! 

2016-07-28 4:40 GMT+08:00 Thomas Kluyver <tak...@gmail.com>:

On 27 July 2016 at 11:36, <nju08e...@gmail.com> wrote:

the config file really produced in MY_CONFIG_DIR, and some files like nbserver-20374.json produced in runtime dir of MY_DATA_DIR, but I definitely can not find the dirs as custom and files as custom.js, Could anyone help me or tell me where are the files' location?


The custom files are not created automatically, but you can put them at e.g. $JUPYTER_CONFIG_DIR/custom/custom.js

--
You received this message because you are subscribed to a topic in the Google Groups "Project Jupyter" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jupyter/VF5RHpncN2k/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jupyter+u...@googlegroups.com.
To post to this group, send email to jup...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/CAOvn4qh%2BkSQ_cdGj6M28ZanoXJTNpEgiTrT7AWYc%3D-5XX_anww%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Yalan  Xue ， SIT  Lab of   Nanjing  University 

 Address: Nanjing ,China。

--
You received this message because you are subscribed to the Google Groups "Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jupyter+u...@googlegroups.com.

To post to this group, send email to jup...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/CACNymyti%3DfBzxrU56qTk3fMaeFPYit%3DcwQUoYMve2_%2B9VqqCyg%40mail.gmail.com.

​

[薛亚兰]

Thanks MinRK, found the problem of this notebook.dir issue.
And another problem came when I used the old way as 00-pyspark-setup.py to start pyspark with Jupyter notebook. We want to combine Spark 1.6.1 and Jupyter notebook for usage, and originally, we use ipython 3.2.1, and use python setup.py to install it, then use ipython profile create to create all the configure files and also it will produce the startup dir in the profile_dir, then we put the 00-pyspark-setup.py  in startup dir, then the sparkcontext will be produced successfully.

However, currently, we want to move to jupyter 4.1, and use Anaconda to help us to install it, but seems the 00-pyspark-setup.py file does not executed anymore, the following are my configurations:

export PYSPARK_SUBMIT_ARGS="--deploy-mode client --master $masterurl pyspark-shell"
PYSPARK_PYTHON=Anaconda/installdir/bin/python
PYSPARK_DRIVER_PYTHON=Anaconda/installdir/bin/jupyter

and then create dir $IPYTHONDIR/profile_default/startup/, and then put the 00-pyspark-setup.py in it, using cmds like:
cp 00-pyspark-setup.py $IPYTHONDIR/profile_default/startup/

and then start the jupyter notebook:
 jupyter notebook --notebook-dir=$JUPYTER_NOTEBOOK_DIR

I just see all the files are produced  as structures like this in IPYTHONDIR:

drwx------. 3 root root   67 Aug  1 15:38 config
drwx------. 2 root root    6 Aug  1 15:40 extensions
-rw-------. 1 root root 2415 Aug  1 15:42 ipythonlog
drwx------. 2 root root    6 Aug  1 15:40 nbextensions
drwx------. 3 root root   52 Aug  1 15:42 notebooks
drwx------. 7 root root   85 Aug  1 15:58 profile_default

in profile_default dir:
drwx------. 2 root root     6 Aug  1 15:40 db
-rw-------. 1 root root 28672 Aug  1 15:40 history.sqlite
drwx------. 2 root root     6 Aug  1 15:40 log
drwx------. 2 root root     6 Aug  1 15:40 pid
drwx------. 2 root root     6 Aug  1 15:40 security
drwx------. 2 root root    45 Aug  1 15:40 startup

but however, notebook APP can be started well, and we can access its urls, but unfortunately, the sparkcontext is not initialized successfully, I can guarantee the master url and spark_home is set right and both of them works well. But did something changed in Jupyter compared to original ipython, or did I miss some steps?

Really need support from you, I am blocked by this.

the following are the content of my 00-pyspark-setup.py file:

import os
import sys

spark_home = os.environ.get('SPARK_HOME', None)
if not spark_home:
      raise ValueError('SPARK_HOME environment variable is not set')
sys.path.insert(0, os.path.join(spark_home, 'python'))
tmpDir = os.path.join(spark_home,'python/lib/')
tmpStr = "py4j"
joinDir = tmpDir
for root, dirs, files in os.walk(tmpDir):
        for f in files:
                if tmpStr in f:
                        joinDir = os.path.join(tmpDir,f)
                        break
if joinDir == tmpDir:
        raise ValueError('py4j library is not set in spark_home dir')

sys.path.insert(0, joinDir)
execfile(os.path.join(spark_home, 'python/pyspark/shell.py'))

Thanks in advance!

To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/CAHNn8BUJ%2BUz-UYqwb4se0Z_Vux08f-d8xLLKRoVw_q9d8DQpZg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[MinRK]

IPython profiles have not changed with the transition to Jupyter. The only difference is that Jupyter itself (the notebook server) is no longer configured by the IPython config, only the IPython kernel is.

 

Really need support from you, I am blocked by this.

the following are the content of my 00-pyspark-setup.py file:

import os
import sys

spark_home = os.environ.get('SPARK_HOME', None)
if not spark_home:
      raise ValueError('SPARK_HOME environment variable is not set')
sys.path.insert(0, os.path.join(spark_home, 'python'))
tmpDir = os.path.join(spark_home,'python/lib/')
tmpStr = "py4j"
joinDir = tmpDir
for root, dirs, files in os.walk(tmpDir):
        for f in files:
                if tmpStr in f:
                        joinDir = os.path.join(tmpDir,f)
                        break
if joinDir == tmpDir:
        raise ValueError('py4j library is not set in spark_home dir')

sys.path.insert(0, joinDir)
execfile(os.path.join(spark_home, 'python/pyspark/shell.py'))

Thanks in advance!

Can you verify that any of this file is executed, e.g. is `spark_home` defined? Is `joinDir` defined? Can you start a regular IPython terminal int he same context, so that it will load this same profile? One disadvantage of the notebook is that errors raised in your startup files will not be easily visible.

-MinRK

 

To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/CACNymysQj1%3DaDTbeoedmYaqsX_sKMdxAKYb1Pbq9MzeUx-Mdig%40mail.gmail.com.