well, the PYTHONPATH is an environment variable, which resets if you close and re-open your shell. It is just in the nature of the thing. To (artificially) add it more permanently, you could add something along the lines of
export PYTHONPATH=/your/caffe/dir/python
to your .bashrc. Or, you just do something like
import sys
sys.path.append('your/caffe/dir/python')
# and then import caffe
import caffe
in your python scripts that use caffe. The third option would be to really install (copy) the pycaffe package to your standard python directories. But I would not recommend that, it is inconvenient to have something installed there not maintained by your package manager, and you'd have to reinstall it there every time to pull and recompile caffe.
Jan