Issues installing bisque from source

42 views
Skip to first unread message

mce...@ucsb.edu

unread,
Oct 2, 2018, 6:41:06 PM10/2/18
to bisque bioimage
Hi all, I'm working to install bisque from source, and I'm running into a couple errors walking through the tutorial here


I'm going to try to log all the different errors I'm getting here, but I think the significant one regards imgcnv, and it is the last part highlighted below.

My errors begin running bisque-bootstrap.py:

  cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
  In file included from mahotas/numpypp/array.hpp:17:0,
                   from mahotas/_histogram.cpp:10:
  mahotas/numpypp/numpy.hpp:11:33: fatal error: numpy/ndarrayobject.h: No such file or directory
  compilation terminated.
  error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
  
  ----------------------------------------
  Failed building wheel for mahotas
  Running setup.py clean for mahotas
Failed to build mahotas

So, I activate bqenv and run pip install -r requirements.txt, but everything, including mahotas, appears to be satisfied, so I continue.

paver setup server appears to run without issues.

Running bq-admin setup, I use all the default variables unless otherwise noted. Choosing to deploy all static resources to public directory leads to errors. I'm not entirely sure what this option means, but I'm doing this as an exercise and not planning on deploying any public servers any time soon, so I continue on. 

I choose N to Change a site variable? 
N to enable mail delivery.

After choosing to initialize the new database,

15:11:51,741 ERROR [bq.websetup] Issue loading ingest_service = bq.ingest.controllers.ingest_server
Traceback (most recent call last):
  File "/home/austin/bisque/bqcore/bq/websetup/schema.py", line 23, in setup_schema
    service = x.load()
  File "/home/austin/bisque/bqenv/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 
2332, in load
    return self.resolve()
  File "/home/austin/bisque/bqenv/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 
2338, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/home/austin/bisque/bqserver/bq/ingest/controllers/ingest_server.py", line 19, in <module>
    from bq import image_service
  File "/home/austin/bisque/bqserver/bq/image_service/__init__.py", line 2, in <module>
    from api import *
  File "/home/austin/bisque/bqserver/bq/image_service/api.py", line 62, in <module>
    from controllers.service import ImageServiceController as LocalImageServer
  File "/home/austin/bisque/bqserver/bq/image_service/controllers/service.py", line 32, in <module>
    from .imgsrv import ImageServer, getOperations
  File "/home/austin/bisque/bqserver/bq/image_service/controllers/imgsrv.py", line 51, in <module>
    from .converters.converter_imgcnv import ConverterImgcnv
  File "/home/austin/bisque/bqserver/bq/image_service/controllers/converters/converter_imgcnv.py", l$
ne 53, in <module>
    imgcnvlib = ctypes.cdll.LoadLibrary(imgcnv_lib_name)
  File "/usr/lib/python2.7/ctypes/__init__.py", line 440, in LoadLibrary
    return self._dlltype(name)
  File "/usr/lib/python2.7/ctypes/__init__.py", line 362, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: libimgcnv.so: cannot open shared object file: No such file or directory

then four similar errors with the same root issue. I run bq-admin server start anyways, and unsurprisingly I see a nearly page with title "Server Error" and the uninformative line "An Unexpected exception:"


I stop the server.

I take the following action:
(bqenv) austin@karlsplatz:~/bisque$ apt list | grep libimgcnv

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

libimgcnv/unstable 2.2.2-1 amd64


As I read on in the tutorial, I see mention of imgcnv, but I am running on Ubuntu 16.04, so I would assume that following these instructions (https://biodev.ece.ucsb.edu/projects/bisquik/wiki/CbiDebian) should work. However, I ran those lines previously and am still having this issue, so I try again, and here's some potentially relevant output from apt-get update

AppStream cache update completed, but some metadata was ignored due to errors.
Reading package lists... Done
W: http://biodev.ece.ucsb.edu/debian/archive/unstable/InRelease: Signature by key 68D85999C316C55215620BC6632DA85E7BAAED9E uses weak digest algorithm (SHA1)
W: http://www.openprinting.org/download/printdriver/debian/dists/lsb3.2/Release.gpg: Signature by key F8897B6F00075648E248B7EC24CBF5474CFD1E2F uses weak digest algorithm (SHA1)


Output from install attempts:
(bqenv) austin@karlsplatz:~/bisque$ sudo apt-get install imgcnv libimgcnv
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 imgcnv : Depends: libavcodec56 (>= 10:2.6.9) but it is not installable
          Depends: libavformat56 (>= 10:2.6.9) but it is not installable
          Depends: libavutil54 (>= 10:2.6.9) but it is not installable
          Depends: libgdcm2.4 but it is not installable
          Depends: libswresample1 (>= 10:2.6.9) but it is not installable
          Depends: libswscale3 (>= 10:2.6.9) but it is not installable
 libimgcnv : Depends: libavcodec56 (>= 10:2.6.9) but it is not installable
             Depends: libavformat56 (>= 10:2.6.9) but it is not installable
             Depends: libavutil54 (>= 10:2.6.9) but it is not installable
             Depends: libgdcm2.4 but it is not installable
             Depends: libswresample1 (>= 10:2.6.9) but it is not installable
             Depends: libswscale3 (>= 10:2.6.9) but it is not installable
E: Unable to correct problems, you have held broken packages.

 
The other two packages (libopenjpeg5 and libopenslide-dev)  appear to install without issue.

Ultimately, it seems like there is an issue with imgcnv, and maybe there are other issues at work here too. I include all this detail in case there's something (like an option in setup) that I am choosing incorrectly.

mce...@ucsb.edu

unread,
Oct 4, 2018, 4:04:31 PM10/4/18
to bisque bioimage
Hi all, I'm trying to install imgcnv and libimgcnv but this time using aptitude, which is supposed to sort out dependency problems.

I tried:

(bqenv) austin@karlsplatz:~/bisque$ sudo aptitude install imgcnv libimgcnv
The following NEW packages will be installed:
  imgcnv{b} libimgcnv{b}
The following packages will be REMOVED:
  libappindicator1{u} libindicator7{u} libjpeg62{u} linux-signed-image-4.13.0-38-generic{u}
  linux-signed-image-4.13.0-39-generic{u} linux-signed-image-4.13.0-41-generic{u}
  linux-signed-image-4.13.0-43-generic{u} linux-signed-image-4.13.0-45-generic{u}
  snap-confine{u}
0 packages upgraded, 2 newly installed, 9 to remove and 152 not upgraded.
Need to get 5,768 kB of archives. After unpacking 19.1 MB will be used.
The following packages have unmet dependencies:
 libimgcnv : Depends: libavcodec56 (>= 10:2.6.9) which is a virtual package and is not provided by any available package.

             Depends: libavformat56 (>= 10:2.6.9) which is a virtual package and is not provided by any available package.

             Depends: libavutil54 (>= 10:2.6.9) which is a virtual package and is not provided by any available package.

             Depends: libgdcm2.4 which is a virtual package and is not provided by any available package.

             Depends: libswresample1 (>= 10:2.6.9) which is a virtual package and is not provided by any available package.

             Depends: libswscale3 (>= 10:2.6.9) which is a virtual package and is not provided by any available package.

 imgcnv : Depends: libavcodec56 (>= 10:2.6.9) which is a virtual package and is not provided by any available package.

          Depends: libavformat56 (>= 10:2.6.9) which is a virtual package and is not provided by any available package.

          Depends: libavutil54 (>= 10:2.6.9) which is a virtual package and is not provided by any available package.

          Depends: libgdcm2.4 which is a virtual package and is not provided by any available package.

          Depends: libswresample1 (>= 10:2.6.9) which is a virtual package and is not provided by any available package.

          Depends: libswscale3 (>= 10:2.6.9) which is a virtual package and is not provided by any available package.

The following actions will resolve these dependencies:

     Keep the following packages at their current version:
1)     imgcnv [Not Installed]
2)     libimgcnv [Not Installed]



Accept this solution? [Y/n/q/?] y
The following packages will be REMOVED:
  libappindicator1{u} libindicator7{u} libjpeg62{u} linux-signed-image-4.13.0-38-generic{u} linux-signed-image-4.13.0-39-generic{u} linux-signed-image-4.13.0-41-generic{u}
  linux-signed-image-4.13.0-43-generic{u} linux-signed-image-4.13.0-45-generic{u} snap-confine{u}
0 packages upgraded, 0 newly installed, 9 to remove and 152 not upgraded.
Need to get 0 B of archives. After unpacking 504 kB will be freed.
Do you want to continue? [Y/n/?] y


There's more output, but it appeared to install successfully.

(bqenv) austin@karlsplatz:~/bisque$ apt list | grep imgcnv

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

imgcnv/unstable 2.2.2-1 amd64
libimgcnv/unstable 2.2.2-1 amd64


Running bq-admin setup, I use all the default variables unless otherwise noted. 
Deploy all ... to public directory? N
I choose N to Change a site variable? 
N to enable mail delivery.

This time, I get errors after I choose the default for
Force initialization [N]?

and the first error in the output seems to trace back to issues with libimgcnv.

Force initialization  [N]?                                                                                                                                     
INFO:bq.config:DATABASE sqlite:///data/bisque.db
INFO:bq.config:SQLLite special handling NullPool timoout
ERROR:bq.service:Failed to load bisque service: ingest_service skipping
Traceback (most recent call last):
  File "/home/austin/bisque/bqcore/bq/core/service.py", line 158, in load_services
    service = x.load()
  File "/home/austin/bisque/bqenv/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2332, in load
    return self.resolve()
  File "/home/austin/bisque/bqenv/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2338, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/home/austin/bisque/bqserver/bq/ingest/controllers/ingest_server.py", line 19, in <module>
    from bq import image_service
  File "/home/austin/bisque/bqserver/bq/image_service/__init__.py", line 2, in <module>
    from api import *
  File "/home/austin/bisque/bqserver/bq/image_service/api.py", line 62, in <module>
    from controllers.service import ImageServiceController as LocalImageServer
  File "/home/austin/bisque/bqserver/bq/image_service/controllers/service.py", line 32, in <module>
    from .imgsrv import ImageServer, getOperations
  File "/home/austin/bisque/bqserver/bq/image_service/controllers/imgsrv.py", line 51, in <module>
    from .converters.converter_imgcnv import ConverterImgcnv
  File "/home/austin/bisque/bqserver/bq/image_service/controllers/converters/converter_imgcnv.py", line 53, in <module>
    imgcnvlib = ctypes.cdll.LoadLibrary(imgcnv_lib_name)
  File "/usr/lib/python2.7/ctypes/__init__.py", line 440, in LoadLibrary
    return self._dlltype(name)
  File "/usr/lib/python2.7/ctypes/__init__.py", line 362, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: libimgcnv.so: cannot open shared object file: No such file or directory


There are a few more tracebacks that end with that same OSError, and the end of the output:

Problem initializing preferences.. please use bq-admin preferences

Initialize your database with:
   $ bq-admin setup createdb

You can start bisque with:
   $ bq-admin server start
then point your browser to:
If you need to shutdown the servers, then use:
   $ bq-admin server stop
You can login as admin and change the default password.

Send Installation/Registration report [Y]? N
(bqenv) austin@karlsplatz:~/bisque$ 


I run bq-admin setup createdb just to see what happens, and it seems fine, so I run bq-admin server start, which appears to start h1, but visiting 0.0.0.0:8080 has the error page I saw before.

The issue still seems to be something to do with imgcnv and its location, but I can't be sure that I didn't do something wrong earlier. Any help would be much appreciated. 

kgk

unread,
Oct 4, 2018, 5:03:07 PM10/4/18
to bisque-...@googlegroups.com
imgcnv from the debian repository will not run on ubuntu.. too many differences (as you noted) in dependencies.

Please look here for a lis of prebuilts:



--
You received this message because you are subscribed to the Google Groups "bisque bioimage" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bisque-bioima...@googlegroups.com.
To post to this group, send email to bisque-...@googlegroups.com.
Visit this group at https://groups.google.com/group/bisque-bioimage.
For more options, visit https://groups.google.com/d/optout.


--
Kristian G. Kvilekval (kkvil...@gmail.com) 805-636-1599

mce...@ucsb.edu

unread,
Oct 4, 2018, 8:20:38 PM10/4/18
to bisque bioimage
Thanks for the reply, Kris. I downloaded  imgcnv_ubuntu16_2.4.3.tar.gz, and I put all the binaries in a /home/austin/.local/bin, which is in my PATH, but I'm hitting another issue now.

bq-admin appeared to find imgcnv, but then it is complained it can't find "libgdcmDICT.so.2.6", so I ran 

(bqenv) austin@karlsplatz:~/bisque$ sudo apt-get install libgdcm2.6


which appeared to solve that problem, because, during setup, bq-admin printed

Beginning install of ['bisque', 'engine'] with ['server'] 
CALLING  <function install_external_binaries at 0x7f31631786e0>
Fetch external binary files from Bisque development server [Y]? 
Matched section Linux-64bit-*
extjs.zip found locally
bioformats-pack.zip found locally
libmbgl-linux-64-large.a found locally
feature_extractors.zip found locally
ImarisConvert.tar.gz found locally
opencv-2.4.6.zip found locally
CALLING  <function install_dependencies at 0x7f3163178848>
2.4.3

Found imgcnv version 2.4.3


Imgcnv is installed and no-precompiled version exists. Using installed version
ImarisConvert is up to date
No pre-compiled version of openslide exists for your system
Please visit our mailing list https://groups.google.com/forum/#!forum/bisque-bioimage for help
Bioformats is up to date
CALLING  <function install_features at 0x7f3163178aa0>
Install feature extractors (Feature Server) [Y]? 


however, as I continued the setup, I still get the same OSError. 
OSError: libimgcnv.so: cannot open shared object file: No such file or directory

I thought maybe I should move the imgcnv binaries to bqenv/bin, but that did not fix the error either. Any idea where I should put these libimgcnv files so that everyone can find them?

kgk

unread,
Oct 4, 2018, 10:08:39 PM10/4/18
to bisque-...@googlegroups.com
Check these:
LD_LIBRARY_PATH 
ld.so.conf

you can also just copy them into /usr/local/lib

mce...@ucsb.edu

unread,
Oct 5, 2018, 7:42:29 PM10/5/18
to bisque bioimage
Awesome thank you. I'm not getting that error anymore. I copied the imgcnv .so files to /usr/local/lib and the executable to /usr/local/bin and then ran sudo ldconfig to update my library configuration files.

For reference, I found this page very helpful as I suppose I've never actually installed a library manually on Ubuntu before


Now, however, I am getting a message "Problem initializing preferences.." but I don't see a precise error message this time, just warnings and NO ACTION. Still, starting the server and pointing my browser to 0.0.0.0:8080 shows a server error. Here's the end of the output from bq-admin setup:

Initialize Preferences  [Y]? 
Force initialization  [N]? 
INFO:bq.config:DATABASE sqlite:///data/bisque.db
INFO:bq.config:SQLLite special handling NullPool timoout
INFO:bq.util.io_misc:BAD non-0 return code for ['ImarisConvert', '-v']
INFO:bq.util.io_misc:BAD non-0 return code for ['showinf', '-version', '-no-upgrade']
INFO:bq.image_service.server:Available converters: openslide (1.1.1), imgcnv (2.4.3)
INFO:bq.blobs.mounts:Loaded drivers OrderedDict([('local', {'top': 'file://$datadir/imagedir/', 'mounturl': 'file://$datadir/imagedir/$user/'})])
WARNING:bq.blobs.mounts:SQLITE does not support subtransactions: some mount service operations will fail
INFO:bq.blobs.plugins:Resource plugins: ['(bisque_pipeline, bq)', '(cellprofiler_pipeline, cppipe)', '(cellprofiler_pipeline, cp)', '(dream3d_pipeline, json)', '(imagej_pipeline, ijm)', '(table, csv)', '(table, hdf)', '(table, hdf5)', '(table, h5)', '(table, he5)', '(table, h5ebsd)', '(table, dream3d)', '(table, csv)', '(table, xls)', '(table, xlsx)', '(text, asc)', '(text, ascii)', '(text, zw)', '(text, hz)', '(text, txt)', '(text, utxt)', '(text, utf8)']
WARNING:bq.service:Following service were not found: set(['mnt'])
NO ACTION: System object initialized at <system created="2018-10-02T15:11:52.741963" name="system" owner="http://localhost/data_service/00-drw6CjeCb9YWooVrKRaEnm" permission="published" resource_uniq="00-ghjX9raxanhcAJP7agkhnj" ts="2018-10-02T15:11:52.741963" uri="http://localhost/data_service/00-ghjX9raxanhcAJP7agkhnj"/> 
Problem initializing preferences.. please use bq-admin preferences

Initialize your database with:
   $ bq-admin setup createdb

You can start bisque with:
   $ bq-admin server start
then point your browser to:
If you need to shutdown the servers, then use:
   $ bq-admin server stop
You can login as admin and change the default password.

Send Installation/Registration report [Y]? N
(bqenv) austin@karlsplatz:~/bisque$ bq-admin preferences
Usage: bq-admin preferences [init (db)|read (from db)|save (to db)]

bq-admin: error: argument must be init, read, save
(bqenv) austin@karlsplatz:~/bisque$ 

kgk

unread,
Oct 5, 2018, 7:50:39 PM10/5/18
to bisque-...@googlegroups.com
You can ignore it.  Or you can drop your database and re-initialize.

mce...@ucsb.edu

unread,
Oct 5, 2018, 8:16:43 PM10/5/18
to bisque bioimage
If I ignore the error and run bq-admin server start, my browser shows the Server Error page. If I bq-admin server stop and bq-admin setup createdb, I don't see any error messages; however, when I bq-admin server start and go to the page, I still see the Server Error page in my browser.

kgk

unread,
Oct 6, 2018, 11:29:09 AM10/6/18
to bisque-...@googlegroups.com
what is in the logs?

mce...@ucsb.edu

unread,
Oct 8, 2018, 2:40:05 PM10/8/18
to bisque bioimage
In bisque_8080.log, I saw serving on 0.0.0.0:8080, then after accessing, I get something along the lines of 

11:30:55,809 ERROR [bq.middleware] Error - <type 'exceptions.IOError'>: [Errno 2] No such file or     directory: './public/core/css/all_css.css'

which may be because I chose not to deploy in public directory. I run again and choose Y to deploy all static resources to public directory? which has the following output:

Deploy all static resources to public directory [Y]? 
Creating ./public
Making  ./public/core
Making  ./public/data_service
INFO:bq.util.io_misc:BAD non-0 return code for ['ImarisConvert', '-v']
INFO:bq.util.io_misc:BAD non-0 return code for ['showinf', '-version', '-no-upgrade']
INFO:bq.image_service.server:Available converters: openslide (1.1.1), imgcnv (2.4.3)
Making  ./public/ingest_service
Making  ./public/stats
Making  ./public/graph
Making  ./public/module_service
Making  ./public/client_service
Making  ./public/image_service
Making  ./public/dataset_service
Making  ./public/export
Problem loading registration = bq.registration.controllers.registration_service: 'User'
Making  ./public/usage
Making  ./public/import
Making  ./public/engine_service

Generating packaged JS and CSS files

WARNING:bq.util.io_misc:Could not remove "./public/core/css/all_css.css"
WARNING:bq.util.io_misc:Could not remove "./public/core/js/all_js.js"
INFO:minimatic:Combined -> ./public/core/css/all_css.css:
INFO:minimatic:Combined -> ./public/core/js/all_js.js:
CALLING  <function install_server_defaults at 0x7f0c3f37bf50>
Server config
Top level site variables are:
  bisque.admin_email=YourEmail@YourOrganization
  bisque.admin_id=admin
  bisque.organization=Your Organization
  bisque.paths.root=.
  bisque.server=http://0.0.0.0:8080
  bisque.title=Image Repository
Change a site variable [N]? 

I continue on selecting all defaults except enable mail delivery and send report.

After starting the server, I navigate to  http://127.0.0.1:8080/client_service/ and see a normal looking Bisque database! Thanks so much Kris.
Reply all
Reply to author
Forward
0 new messages