import caffe successfully, but can't find "layers"

[BK Zero]

i did

make
make test
make runtest

successfully. and i can run the mnist example.

but when i did make pycaffe,i got only 

zero@ubuntu:~/Documents/caffe-master$ make pycaffe

CXX/LD -o python/caffe/_caffe.so
python/caffe/_caffe.cpp

touch python/caffe/proto/__init__.py

PROTOC (python) src/caffe/proto/caffe.proto

zero@ubuntu:~/Documents/caffe-master$

is this normal? it seems strange. and when i enter python, and import caffe, it cannot find "layers"

so i use command dir(caffe) to check it, and i see

>>> import caffe

>>> dir(caffe)

['__doc__', '__loader__', '__name__', '__package__', '__path__', '__spec__']

>>> 

btw, i set the PYTHONPATH like this

export PYTHONPATH=/home/zero/Documents/caffe-master/build/python:$PYTHONPATH

can you give me son hints about this?

[Jan]

The output you get from "make pycaffe" is ok. That is all it does. Seems there is no error in the compilation. On the other hand it is very strange that when you import caffe it does not really import everything, and even without an error message... Maybe it's a python version thing? Do you work with 2.7 or 3.x?

Jan

[BK Zero]

thanks for your reply. yes, i use anaconda3.5. 

this problem may be caused by the mismatching between the boost version and the python version. 

when i set the PYTHONPATH equals /home/zero/caffe-master/python, as the caffe website did, i got this error message, 

ImportError: /home/zero/Documents/caffe-master/python/caffe/_caffe.so: undefined symbol: _ZN5boost6python6detail11init_moduleER11PyModuleDefPFvvE

i google it, and someone says it is because the code" is compiled against Python 3, but being loaded by Python 2. "

and i remember days ago, when i didnot install the boost1.55, the make error message says, "can not fing -lpython2.7"or something like this. but the anaconda version is always 3.5.

so i check the makfile.config, and guess maybe i should uncomment the "PYTHON_LIBRARIES := boost_python3 python3.5m", should i ?

when i did this, and make caffe again, it says can not find -lboost_python3.

what should i do now?

在 2016年4月27日星期三 UTC+8下午6:42:51，Jan写道：

[Jan]

Well, first of all you should decide on a python version you want to work with ;-). I am assuming for now this is 2.7. Check that following items are fulfilled:

• In Makefile.config, PYTHON_INCLUDE is set to the correct directories for python 2.7, in Ubuntu 14.04 these are as specified in the current Makefile.config.example. If you want to use anaconda (which is really unnecessary on Ubuntu), uncomment the respective lines (which are the corresponding paths in the anaconda distribution.Also make sure that you have a python 2.7 of Anaconda installed instead of the 3.5!
• The lines for python 3.5 stay commented.
• Comment/Uncomment the PYTHON_LIB lines as needed (depending on whether you use anaconda or not).

Also check that you have the boost-python and python-dev packages installed (for python 2.7!).

I do not recommend anaconda on a Linux system, since the libraries it provides often clash with the libraries already present on the system, which usually results in hard-to-find and ugly-to-handle errors. In every modern linux distribution there is already a working python distribution present anyway, so there is no need for anaconda.

Jan

[BK Zero]

o, i am sorry, what i mean is i want to work with python3, so i choose anaconda3.5. 

i dont quite understand, if i use anaconda2.7, you suggest uncomment the anaconda lines, but should the python2.7 lines stay uncomment?

the PYTHON_LIBRARIES in python3.5 confuse me, because there is no corresbonding item in python2.7 lines and anaconda lines. they have PYTHON_LIB, but not PYTHON_LIBRARIES. so my question is, if i want to work in pyhon3.5 with anaconda, should i uncomment it?

how i configure the makefile.config is like this, i think it is no problem. but i just cannot import caffe correctly.

## Refer to http://caffe.berkeleyvision.org/installation.html

# CPU-only switch (uncomment to build without GPU support).

CPU_ONLY := 1

# open for OpenBlas

BLAS := atlas

# Verify anaconda location, sometimes it's in root.

ANACONDA_HOME := $(HOME)/anaconda3

PYTHON_INCLUDE := $(ANACONDA_HOME)/include \

          $(ANACONDA_HOME)/include/python3.5m \

          $(ANACONDA_HOME)/lib/python3.5/site-packages/numpy/core/include \

# Uncomment to use Python 3 (default is Python 2)

# PYTHON_LIBRARIES := boost_python3 python3.5m

# We need to be able to find libpythonX.X.so or .dylib.

# PYTHON_LIB := /usr/lib

PYTHON_LIB := $(ANACONDA_HOME)/lib

# Whatever else you find you need goes here.

INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include

LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib

BUILD_DIR := build

DISTRIBUTE_DIR := distribute

# The ID of the GPU that 'make runtest' will use to run unit tests.

TEST_GPUID := 0

# enable pretty build (comment to see full commands)

Q ?= @

在 2016年4月27日星期三 UTC+8下午8:04:29，Jan写道：

[Jan]

I have used anaconda with caffe, so I am not 100%, but you should probably use ONLY the anaconda-related settings for PYTHON_LIB and PYTHON_INCLUDE. You are right, there is no PYTHON_LIBRARIES setting for python2.7, I am not sure why that is... Maybe it is set to default values for python2.7 by the Makefile and only replaced with the PYTHON_LIBRARIES setting in the Makefile.config if uncommented.

As an alternative you could try to use cmake to build caffe. The configuration is a bit easier imho, and you don't need to set many paths since cmake finds most stuff on its own. Be aware that the cmake build mechanism is completely unrelated to the Makefile.config settings, it is a second, alternative method of building the (basically same) caffe binaries from the same source files.

Jan

[BK Zero]

ok，i will try cmake later. actually i am trying parallel with python2.7 and python3.5 in two different virtual machine. and i got failed in python3.5.

i install all the package from a pure system, and this time i carefully install boost by "apt-get install libboost1.55-all-dev",but i got the same err msg "undefined symbol: _ZN5boost6python6detail11init_moduleER11PyModuleDefPFvvE"

and with 2.7, i also got some problems.

maybe you are right, i should not use the anacondas, i will try again tomorrow.

在 2016年4月27日星期三 UTC+8下午11:10:53，Jan写道：

[BK Zero]

btw, when i set the PYTHONPATH to /caffehome/build/python, i can also import caffe but got no layers etc.

在 2016年4月27日星期三 UTC+8下午11:10:53，Jan写道：

I have used anaconda with caffe, so I am not 100%, but you should probably use ONLY the anaconda-related settings for PYTHON_LIB and PYTHON_INCLUDE. You are right, there is no PYTHON_LIBRARIES setting for python2.7, I am not sure why that is... Maybe it is set to default values for python2.7 by the Makefile and only replaced with the PYTHON_LIBRARIES setting in the Makefile.config if uncommented.