Docker images with Matlab?

[rob...@ucair.med.utah.edu]

I'm assuming that the Matlab hooks need to be built in at compilation time.

Do any of the Docker images have the Matlab hooks compiled into Gadgetron?

Would getting the Matlab feature working in the Docker images be simply a matter of recompiling with the appropriate MATLAB_HOME variable as well as whatever other Matlab compilation flags need to be turned on?

I'm currently working with the gadgetron/ubuntu_1404_cuda75 image.  Looking at /usr/local/share/gadgetron, it looks like only the base c++ and perhaps the python gadgets are linked in.

I attempted to simply drop in a copy of the matlab.xml in the /usr/local/share/gadgetron/config folder, but it complained about the missing Matlab related DLLs.  So, I'm guessing I need to recompile?

Thanks,
John.

[Michael Hansen]

Would this help:

https://github.com/gadgetron/gadgetron/wiki/Using-Matlab-with-Gadgetron-in-Docker

--
You received this message because you are subscribed to the Google Groups "Gadgetron" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gadgetron+...@googlegroups.com.
To post to this group, send email to gadg...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gadgetron/7ba62345-2c0e-433b-9467-e0f539d732ab%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[rob...@ucair.med.utah.edu]

I looked through that same link while setting up the Gadgetron docker. 

My understanding is that that link addresses two aspects about getting Matlab to work within Docker:

1. Accessing the host's installation of matlab from within a Docker container
   
2. How to handle x-windows within the Docker container so that Matlab GUI elements can be invoked

I think only the first aspect is directly relevant to getting Matlab working with Gadgetron itself.  It's certainly necessary that the Docker container have access to the host Matlab if Gadgetron is going to have access at all.

My question is whether Gadgetron needs to be compiled from source in order to enable the Gadgetron Matlab API.  I suspect that it does.  I hoped to avoid that much by using the Docker containers where someone else has generously done the hard work of compiling from source.

Thus, I wondered if any of the Docker Gadgetron images came with the Matlab hooks compiled in. 

It's been a while since I compiled Gadgetron from source.  I need to go back and read the instructions.  Perhaps it will be a simple matter of running cmake within the Docker container and turning on the Matlab hooks.  Presumably, everything else is already compiled within the source tree (Is the source within the container? I haven't checked.) and only the Matlab affected elements will need recompiling.

I suggest that it might be possible to distribute Docker containers with the Matlab hooks compiled in.  The user would still need to provide a valid Matlab installation on the host machine.  At least for linux machines, Matlab installations follow a standard directory structure.  As long as the Docker user mapped the host directory on to the same container directory (ex. /opt/LOCAL/matlab), the Gadgetron could be compiled assuming Matlab would always be at /opt/LOCAL/matlab.

I'll post an update here after I've had a chance to poke around the Docker Gadgetron installation.

John.

[Michael Hansen]

I think the problem here is that when compiling with the Matlab API, you compile against a specific version of Matlab, so if the Docker image is compiled against a certain version, you must have that exact version installed on your host for that image to work. The compiling from source inside the container is no big deal. All the tools are there, so if you provide access to the host matlab inside the container and simply go to /opt/code/gadgetron/build and do cmake and make, it will find Matlab and recompile the everything. It will take a few minutes but it shouldn't be too bad. 

Is it a major hurdle to redo the compile inside the container?

Michael

To view this discussion on the web visit https://groups.google.com/d/msgid/gadgetron/2d6503f8-2708-46bd-85a2-c80b03fca746%40googlegroups.com.

[rob...@ucair.med.utah.edu]

I'm currently working on recompiling within the container with Matlab hooks turned on.

As to your point, so is there some form of static linking going on against the Matlab installation that then is vulnerable to version changes?

Or, I might infer that even with dynamic linking, the Matlab API itself (as expressed in various *.h files and DLLs  in the Matlab installation) changes enough between versions that you have a moving target and have to compile against each version.  That's too bad.

Personally, I'm happy with c++ and python, but I have a number of colleagues who will suffer separation anxiety if Mother Matlab is not available.

John.

[Michael Hansen]

Even with dynamic linking this is very, very fragile. Sometimes some Matlab versions are compatible but most of the time they are not. So it it has to be compiled against the version of Matlab it needs to run with (unfortunately). All the Matlab stuff actually sucks a bit that way. I would like a better solution (like the way the Python integration works), but we have limited access to the APIs, etc. in Matlab. 

To view this discussion on the web visit https://groups.google.com/d/msgid/gadgetron/63e44db0-920d-4fcb-b6d7-80c6785331c8%40googlegroups.com.

[rob...@ucair.med.utah.edu]

I managed to rerun cmake and make and then make install ok.

It then took a minute or two to realize I might need to reset the system LD_LIBRARY_PATH to point to the dynamic libraries that matlab drags along in its installation.

I restarted gadgetron using the web interface, but I notice that gadgetron_cloud is still running the same process that started when I first launched the Docker container.  I'm wondering if it is integral to execution, but still running the old search path and missing the matlab DLLs.

Is there a command to restart the Gadgetron setup within the docker environment?  Is more necessary than restarting from the web interface?

When I run reconstruction against the Shepp phantom, I don't get much at all in the log - and no output results.

2016-08-04 01:15:54 INFO [main.cpp:200] Starting ReST interface on port 9080
2016-08-04 01:15:54 INFO [main.cpp:212] Starting cloudBus: localhost:8002
2016-08-04 01:15:54 INFO [main.cpp:260] Configuring services, Running on port 9002
2016-08-04 01:15:54 DEBUG [CloudBus.cpp:190] CloudBus connected to relay at  172.17.0.2:8002

Can I increase the logging level somehow? 

Thanks,
John.

[Michael Hansen]

Don't worry about the cloudbus relay. It is only used when you have multiple nodes that need to talk to each other and it has no Matlab dependencies that it needs.

It general to restart the Gadgetron in the container, you can just kill the process running the Gadgetron and the webapp should restart it. 

The log file you are showing is actually "empty". This is the log file you would get when the Gadgetron just starts, so I suspect that the Gadgetron actually crashed, which will cause the webapp to restart it and start a new log file. The old log file is saved in /tmp (in the Container) with a timestamp on it. Can you check if there is such a crash log and post that?

Michael

To unsubscribe from this group and stop receiving emails from it, send an email to gadgetron+unsubscribe@googlegroups.com.

To post to this group, send email to gadg...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gadgetron/9807b49c-de0d-4903-aac9-eeb3e1dfe160%40googlegroups.com.

[rob...@ucair.med.utah.edu]

Great! That helps.

I found the previous log and it looks like the matlab engine is not starting.

Looking at the start command in MatlabBufferGadget, I suspect I need to update the search path in the run time environment to find the matlab command.  Or, I should edit the start command in MatlabBufferGadget.h to include the path to the matlab executable.

Does that sound about right? 

I'll try updating the run time environment and restarting the Gadgetron process to see what happens.  If that doesn't work, I'll go back and recompile again.

John.

[rob...@ucair.med.utah.edu]

I added matlab to the search path, but I'm not sure how to get the gadgetron launch service to adopt the new path (or if that's the problem).

A couple things:

1. What scripts are run at the initial launch of Docker installations of Gadgetron?  I'd like to re-run those so that they inherit the updated environment.  I can't destroy and relaunch the container, because I'll lose all the matlab compilation I've built in to the current running container.  I observe a number of processes related to Gadgetron running in the background. 
   
2. When I launch matlab within the container at the command prompt, I do see a message about Java MWT being ignored because of the lack of a display.  Is the MWT a necessary component for running Gadgetron matlab and might that be my problem? 
   
3. I assumed the instructions about getting the matlab GUI working in the container were for users who wanted the matlab IDE, but I'm happy to do without it.  Is the GUI required to run Gadgetron matlab?
   

[Michael Hansen]

The two processes (which start other processes) that are run (by the "supervisor") application in the container are:

/opt/start_web.sh

/opt/start_jupyter.sh

As I said it is the "supervisor" application that manages this. A random search for how to stop and start supervisor yields this:

http://www.onurguzel.com/supervisord-restarting-and-reloading/

Be careful about stopping the supervisor, since it will stop the container. 

You can destroy the container if you would like. You just have to "commit" your changes first, effectively creating a new image. Then stop, and run a new container with the new image. 

https://docs.docker.com/engine/reference/commandline/commit/

I am not sure about the commands regarding the Java MWT. I have never run matlab in a container, so I really have very limites experience. I am CC'ing Oliver, who wrote the Wiki pages on this, he might have comments/suggestions. 

Oliver, do you have any hints for John?

Michael

To unsubscribe from this group and stop receiving emails from it, send an email to gadgetron+unsubscribe@googlegroups.com.

To post to this group, send email to gadg...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gadgetron/10ddcc13-65ba-4723-89a5-0e3d96c10f93%40googlegroups.com.

[rob...@ucair.med.utah.edu]

I'll look into the commit option to commit the current changes.

Regarding the argument passed to engOpen in MatlabBufferGadget.h, based on my reading of the Matlab manual I'm trying to decide whether I need to modify the source to include the full path to matlab and recompile.  I'm not certain the process invoked by engOpen will inherit the path to matlab from the environment.  If the argument is not NULL, the command is executed as a literal.  The manual hints that it might use the local PATH, given that using a NULL argument will then use "matlab" as the command.  I don't see "matlab" working alone unless we assume it's on the PATH.

Assuming I can get things working, I'll probably destroy the container, start completely from scratch, and do everything the "right" way before committing.  It would be nice if I could upload a working image, at least one that works for people with the same matlab version (R2013a).

[Michael Hansen]

I don't think you need to include the full path to matlab, but the command "matlab" has to be in your search path. That may mean making a symlink from say /usr/local/bin to wherever your Matlab installation is mounted in the image. 

I you do make an image (and upload it to Docker Hub (??)), be careful about any potential Matlab license violations in terms of distributing parts of Matlab and/or details of your license. 

To unsubscribe from this group and stop receiving emails from it, send an email to gadgetron+unsubscribe@googlegroups.com.

To post to this group, send email to gadg...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gadgetron/916d8a35-7292-47d0-9cd9-18b99cb52407%40googlegroups.com.

[Oliver Josephs]

Dear John,

You will need to recompile gadgetron within the docker image to link in the matlab stuff. But since the docker images include the entire build system and all dependencies this should not be very painful. You can COPY in an entire matlab installation in the Dockerfile or just the few files that are actually used. Here's a snippet from my installation:

#!/bin/bash

# copy the listed trees to current directory

cat <<- _EOF_ | xargs -i cp -R --parents {} .

/usr/local/MATLAB/R2016b/extern/include

/usr/local/MATLAB/R2016b/bin/glnxa64/libeng.so

/usr/local/MATLAB/R2016b/bin/glnxa64/libmat.so

/usr/local/MATLAB/R2016b/bin/glnxa64/libmex.so

/usr/local/MATLAB/R2016b/bin/glnxa64/libmx.so

/usr/local/MATLAB/R2016b/bin/glnxa64/libut.so

/usr/local/MATLAB/R2016b/java/jar/jmi.jar

/usr/local/MATLAB/R2016b/java/jar/util.jar

_EOF_

Then in the Dockerfile you'll need something like this:

#MATLAB

COPY usr /usr/

#GADGETRON

RUN cd /opt/code && \

    git clone https://github.com/gadgetron/gadgetron.git && \

    cd gadgetron && \

    mkdir build && \

    cd build && \

    export MATLAB_HOME=/usr/local/MATLAB/R2016b && \

    cmake ../ && \

    make -j $(nproc) && \

    make install && \

    /opt/code/gadgetron/docker/manifest --key .io.gadgetron.gadgetron.sha1 --value `git rev-parse HEAD` && \

    cp ${GADGETRON_HOME}/share/gadgetron/config/gadgetron.xml.example ${GADGETRON_HOME}/share/gadgetron/confi

g/gadgetron.xml

Oliver

[Michael Hansen]

And one more thing. I don't think we have been testing with that version of Matlab (pretty old) so you may run into other problems. 

[Oliver Josephs]

One gotcha is that you need tcsh to be installed for the matlab engine startup script to operate!

I'm happy to help you getting this working.

Oliver

2016-08-04 18:09 GMT+01:00 Michael Hansen <michael.sch...@gmail.com>:

And one more thing. I don't think we have been testing with that version of Matlab (pretty old) so you may run into other problems. 

[Michael Hansen]

Oliver, do you actually have a complete Dockerfile that you use. Should we add it to the repository?

On Thu, Aug 4, 2016 at 1:11 PM, Oliver Josephs <oliver....@gmail.com> wrote:

One gotcha is that you need tcsh to be installed for the matlab engine startup script to operate!

I'm happy to help you getting this working.

Oliver

[Oliver Josephs]

Yes. I have a complete recipe now. Can I tidy it up a little before submitting to the repo? But if John wants I don't mind sending him the (very) rough notes and associated scripts to try an make sense of.

Oliver

[rob...@ucair.med.utah.edu]

Given that the docker image I downloaded came with the full code tree, I assumed it was sufficient to simply run cmake again with matlab available.  Is that not the case?

I could git the whole code tree down again, but that would seem redundant given that it's all there in the Docker container.

[Michael Hansen]

Unless it is top secret. I would suggest just posting it here on the forum. Others might benefit.

On Thu, Aug 4, 2016 at 1:17 PM, Oliver Josephs <oliver....@gmail.com> wrote:

Yes. I have a complete recipe now. Can I tidy it up a little before submitting to the repo? But if John wants I don't mind sending him the (very) rough notes and associated scripts to try an make sense of.

Oliver

[rob...@ucair.med.utah.edu]

This one made a difference!  Now, it simply can't find it rather than failing to launch at all.

I suspect I now need to update the tcsh path.

John.

[Oliver Josephs]

Have sent John some additional notes which I will shortly upload to the Wiki and submit scripts to the repo.

Oliver

[rob...@ucair.med.utah.edu]

I used the link option to put a shortcut to matlab in /usr/local/bin.

Now I'm working through some programs the matlab script is looking for: awk and expr.  The matlab launch script can't find them, and so the engine start fails.

Both of these exist in /usr/bin for root, but not in /usr/local/bin.  I'm going to try to add /usr/bin to the default search PATH.

[rob...@ucair.med.utah.edu]

This thread is becoming a bit convoluted.  I suggest I start another once I solve my problem, given that readers won't know what solutions I tried.

Currently, the approach I've followed has been:

• nvidia-docker to download the gadgetron/ubuntu_1404_cuda75
• Mapping in my matlab installation tree to the container during launch
• Working with matlab R2013a.  I know it's old, but it's just a test case for now.
• Inside the docker container, running cmake within the /opt/code/gadgetron source directories to recompile and install gadgetron, now with matlab baked in.  I'm using the source tree that came with the docker container itself - no extra git.
  
• Added soft link to matlab script (from /usr/local/bin to the actual location on the docker mounted Matlab directory)
• running the Shepp_logan example with matlab.xml
• Currently working through the various "missing" parts of the matlab launch script.  This seems to be an issue of the process used to launch matlab not inheriting the system PATH.  'awk' and 'expr' are reported in the Gadgetron log as missing.  Yet they exist in /usr/bin.  When run from the command prompt as root, matlab launches without issue.  So the matlab script itself is taking some sort of alternate path during execution than what it takes when invoked from the command line.
  

I have not followed Oliver's notes for downloading via git into the Docker, recompiling the git download with matlab, or getting the x-windows aspect working.

John.

[rob...@ucair.med.utah.edu]

Oh, and I installed tcsh within Docker.  The engine launch of matlab won't work at all without it.

[rob...@ucair.med.utah.edu]

I could not manage to get the process that launches matlab from Gadgetron to recognize the default system PATH.  It seems to be inheriting a bare bones path.

• I removed the soft link I put in /usr/local/bin to matlab and placed a small csh script there wrapping the actual Mathworks matlab sh script.
  
• I then invoked the Mathworks matlab with sh -x in order to get verbose output.
• Letting gadgetron invoke matlab clearly reports a different path, missing /usr/bin.
• Forcing the path inside my wrapper script let Gadgetron matlab proceed further than previously

Now I'm currently working with:

2016-08-04 19:04:19 DEBUG [MatlabBufferGadget.h:58] >> [^HWarning: Name is nonexistent or not a directory: /share/ismrmrd/matlab]^H
[^H> In path at 110
  In addpath at 87]^H

This probably gives rise to the later error:
{^HUndefined variable "ismrmrd" or class "ismrmrd.xml.deserialize".

Error in BaseBufferGadget/init (line 21)
            g.xml = ismrmrd.xml.deserialize(xmlstr);
}^H

I suspect I am reaching the point where I'm going to have to carefully run the compilation that bakes in the matlab feature for Gadgetron.  It appears to be poking around incorrect directory paths for some of the files it wants.

I've been avoiding a full git download and compilation as that seems to reduce the utility of having a docker image to simplify things.

John.

^G2016-08-04 19:04:19 DEBUG [MatlabBufferGadget.h:128]

n Thursday, August 4, 2016 at 12:05:12 PM UTC-6, rob...@ucair.med.utah.edu wrote:

[rob...@ucair.med.utah.edu]

Success!

The remaining obstacle appeared to be getting the PATH set properly as used by the Mathworks matlab script.  To that end, I updated my matlab script to set up a proper environment (as measured by running env in the root account) before calling the Mathworks matlab script.

Once I prepared the environment prior to calling matlab, all tests worked.  The Shepp Logan example worked.  I also ran a successful reconstruction from my Siemens development environment (simulating how the scanner would call Gadgetron over the network).

I think that I will start from scratch and make careful notes and repost here.  I did not have to git the code, but made use of the source delivered with the docker image.

Thanks for the help and suggestions.
John.

[Michael Hansen]

Great, if you would like to add comments to the Wiki or work with Oliver on combining your notes, that would be great. We can also add a Dockerfile to the main repo to facilitate for others. 

Thanks for hanging in there!

--

You received this message because you are subscribed to the Google Groups "Gadgetron" group.

To unsubscribe from this group and stop receiving emails from it, send an email to gadgetron+unsubscribe@googlegroups.com.

To post to this group, send email to gadg...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gadgetron/d28d5e34-e9a3-4546-b505-44655fda3320%40googlegroups.com.

[rob...@ucair.med.utah.edu]

One of the challenges here appears to be that the matlab hooks require access to the libraries in the matlab installation directory in order to compile.  When I compiled  the Gadgetron matlab module, I updated the LD_LIBRARY_PATH to point to matlab's library installation to maintain consistency among all the DLLs shipped with matlab.

Putting Gadgetron aside for the moment, matlab ships with a lot of their own copies of familiar linux libraries.  Distinguishing between these and the host system's own copies is handled by the matlab script at run time.  The script adds paths to the start of the library search path so that matlab is happy.  It's copies of libraries are found before system copies.

With the Gadgetron setup, it's a bit more complicated.  After my compilation, the resulting libgadgetron_matlab.so file depends on libraries in the matlab installation hierarchy.  It needs to see those libraries in order to launch.  This requirement is true prior to calling matlab's path modifying script.

Yet I can't manually put the matlab library path on the system library path, because that might break other applications that use the native system's versions of libraries rather than those that ship with matlab.

Python, for example, shares some libraries with matlab (including the xml parsing libraries), but the native system python expects different versions.  I found this out when I tried to run my Gadgetron python gadgets and discovered that the python was looking in matlab's directory for libraries it should have taken from the system directories.  A version conflict ensued.

Removing the offending matlab path from the library search breaks Gadgetron matlab, but lets Gadgeron python work.  I'm still working on how I can get around this and have both at the same time.

John.

On Thursday, August 4, 2016 at 11:09:06 AM UTC-6, Oliver Josephs wrote:

[rob...@ucair.med.utah.edu]

Following https://help.ubuntu.com/community/EnvironmentVariables#File-location_related_variables

the solution so far has been to create a conf file for matlab, named zzzzzmatlab.conf (so as to occur last alphabetically and thus have the matlab path be last in the LD_LIBRARY_PATH search).

At the moment, both matlab and python gadgets are running. 

I'm still going to restart from scratch with a new nvidia-docker image and matlab compilation just to make sure I've identified all the pitfalls in the particular approach I've taken.

John.

[Oliver Josephs]

Yep. Dependency hell.

https://en.m.wikipedia.org/wiki/Dependency_hell

Oliver

[Michael Hansen]

Getting everything working at the same time, Matlab, Python, etc. is very tricky because of the potential library conflicts. Matlab also includes HDF5, which on some versions is in conflict with what ISMRMRD uses, etc, so there are several potential issues and pitfalls. I have yet to find a good way to mix and match Python and Matlab Gadgets because of the issues you mention, but it may be possible. The situation is usually though that people use one or the other, they typically don't mix. 

Matlab is not really a first class citizen in the Gadgetron. The Python support is much better and more elegant. Some things could be cleaned up with some work, but at the end of the day we are pretty limited by the closed development model and awkward deployment strategy for Matlab. 

If you come up with any great solutions, they would be much appreciated. 

[rob...@ucair.med.utah.edu]

Solution, Part 1: Complications

After much trial and error, I finally found a recipe to duplicate my Docker setup of last week with Gadgetron compiled with both Matlab and python.  At least, it works for very simple 2D demonstration slices or the Shepp-Logan phantom exercise.  I cannot say for certain that a more sophisticated use of python or Matlab might trigger some obscure DLL provided class and run into library conflicts

As mentioned in earlier posts, Matlab ships with their own versions of several DLLs already found on typical Linux host systems or used by other programs such as python.  To get around conflicts with the host system, Matlab wraps its executable in a script that front loads the various search paths (ex. LD_LIBRARY_PATH, PATH, and others) so that the Matlab versions of its DLLs are found first before the possibly different versions on the host system.

I'd like to compile Gadgetron against both Matlab and python with their possibly overlapping and conflicting sets of DLLs.  Compilation itself is not so problematic because the compiler doesn't really care that much about DLL versions as much as whether the referenced DLLs provide the requested class at all.  The conflicts arise at the time of execution when the DLL loader typically grabs the first matching DLL it finds on the search path.  

One solution, followed by Matlab, is to wrap the executable in a script that rearranges the DLL search path to present the correct DLLs first.  That might be possible with Gadgetron, but it seems pretty cumbersome to have to start and stop Gadgetron to rearrange the DLL search path when you wanted to switch from python to Matlab gadgets.

I should mention a few more complications not specifically related to Matlab or python, but which play a role in getting things working in an nvidia-docker Ubuntu based setup: when LD_LIBRARY_PATH and PATH are defined and how Ubuntu handles the default values for these search paths.

Running gadgetron/ubuntu_1404_cuda75 and then using exec to launch a bash shell within the container, I find that the DLL search path, LD_LIBRARY_PATH, is already defined:

• /usr/local/nvidia/lib:/usr/local/nvidia/lib64::/usr/local/lib:/usr/local/lib
  

Ubuntu lets the user define their own value for this variable, but also provides a default search path by way of the ld.so system.  Checking the values there (cat /etc/ld.so.conf.d/*.conf), in order:

/usr/local/cuda/lib

/usr/local/cuda/lib64

/usr/lib/x86_64-linux-gnu/libfakeroot

/usr/local/lib

/usr/local/nvidia/lib

/usr/local/nvidia/lib64

/lib/x86_64-linux-gnu

/usr/lib/x86_64-linux-gnu

/usr/lib/x86_64-linux-gnu/mesa

/lib32

/usr/lib32

/libx32

/usr/libx32

So, you see, the defined LD_LIBRARY_PATH you get upon launching a bash shell in in the container is both front loaded by nvidia paths and only contains a subset of the complete path defined in /etc/ld.so.conf.d/*.conf.

Running bash as I am, I may also be seeing the result of a default system bash init script defining LD_LIBRARY_PATH.  I'm not that familiar with Ubuntu, but as far as I can tell, in the docker image at least, there is no default bash value set for LD_LIBRARY_PATH.  Ubuntu seems to prever the ld.so system, except when the user wants to add on their own libraries to the search.  At that point, I believe LD_LIBRARY_PATH is prepended to the paths defined in /etc/ld.so.conf.d/*.conf.

It's my suspicion that the LD_LIBRARY_PATH is being set as an environmental variable in the Dockerfile somewhere between Nvidia's own creation of an Nvidia extended Ubuntu image and Gadgetron's base Dockerfile.  The fact that it front loads Nvidia paths makes me think Nvidia introduced it to their Dockerfile.

The mystery of where LD_LIBRARY_PATH is defined and that it's apparently incomplete is one complication.

A second complication occurred after successfully compiling Matlab (more on that later).  The problem is with the process environment that is inherited when Gadgetron uses Matlab's engOpen command to launch Matlab.  As coded in MatlabBufferGadget.h line 38, 

if (!(engine_ = engOpen("matlab -nodesktop -nosplash"))) 

the matlab script is invoked with no absolute path.  So, it needs to exist on the standard executable path of the process that engOpen starts.  The suggested solution is to run a soft link from /usr/local/bin to your Docker volume mapped Matlab script.  For example, if I mapped my host Matlab installation to /opt/matlab, then

ln -s /opt/matlab/bin/matlab /usr/local/bin/matlab

The problem I ran into was that, while /usr/local/bin WAS in the defined executable path of the process started by engOpen, there were a few necessary paths that were not, including /usr/bin.  The matlab wrapper script would immediately complain about missing awk and expr, both found in /usr/bin.  Running the matlab script myself from the bash shell would work just fine, because the executable path of the bash shell included /usr/bin.  So, it seems that engOpen is forking a process that inherits a partial executable search path.

[rob...@ucair.med.utah.edu]

Solution, Part 2: A few changes to get a working (?) combo Matlab/python Gadgetron

1. Install prerequisites for Gadgetron-Matlab compilation.  
1. I created a new Dockerfile based on gadgetron/ubuntu_1404_cuda75 and added a RUN command to install tcsh and libglu1-mesa-dev.  The tcsh is absolutely necessary to launch matlab.  I suspect the libglu is unecessary, but my 2015a Matlab was complaining (though not crashing) without it.
2. Additionally, I installed python-lxml (more on that later)
2. Build and run the new container with my host Matlab installation directory tree mapped to /opt/matlab inside the container
3. Use nvidia-docker to exec in to a bash shell
4. Rely on Ubuntu's library search management
1. Unset LD_LIBRARY_PATH
2. Add zzzzzzzzzzzzmatlab.conf to /etc/ld.so.conf.d/ with contents /opt/matlab/bin/glnxa64.  The Zs in the name force the matlab search path to be appended at the very end of the system search path.
3. ldconfig - reset the system library search path to cache the new path including matlab
5. Build Gadgetron.  cd /opt/code/gadgetron/build, cmake, make, make install
6. apt-get to force the reinstallation libexpat1-dev.  This is probably superstition on my part, but seemed to fix a Python problem down the road.
7. pkill gadgetron to force the launch of the new binary
8. Rather than creating a soft link to matlab in /usr/local/bin, I created a bash script that calls the matlab script.  This let me set up the proper environment, most importantly the executable search PATH (including the missing /usr/bin).  The LD_LIBRARY_PATH is less important as that tends to get clobbered by Matlab's own script with their prepended paths.
9. Tested simple Shepp Logan phantom test against matlab.  Success!
10. Unset local LD_LIBRARY_PATH variable if it's set and run a python reconstruction against the Shepp Logan phantom.  Success!

The big trick here is abandoning the use of LD_LIBRARY_PATH and putting the Matlab library search path in /etc/ld.so.conf.d with a *.conf file name that puts it at the end of the system search paths.  You can't use the LD_LIBRARY_PATH to do this because that gets prepended to the system paths.  In order for Python to work, we need the DLL search to check all standard system paths first before hitting Matlab DLLs.  

The Matlab path must be on the search list, but at the end.  If you don't provide Matlab's DLL path at all, two things happen.  First, the matlab script itself will still work.  Matlab launches because it sets up its own paths.  However, Gadgetron's call to matlab will fail.  It uses the libgadgetron_matlab.so DLL, which itself requires Matlab DLLs only found in the Matlab installation.

About the python libraries, python-lxml and libexpat1-dev.  Without these, I found that the Python gadget would crash looking for xml parsers in the DLL search path.  Ultimately, it would settle on libexpat within the Matlab distribution, which I didn't want.  I seemed to be missing some part of the xml parsing code base in my standard system, so I added the python-lxml and libexpat1-dev to the system and that got things working again.  Python would then find the necessary DLLs without ending up in the Matlab directories.

For those interested, I have a Dockerfile, matlab wrapper (for /usr/local/bin) and a script for running all the commands I listed above.  Could I attach those here?

[Michael Hansen]

Hi John,

Sorry for being late on commenting on this. It was quite a mouthful. In any case, we just discussed all of this on our Gadgetron Hangout (which BTW, you are most welcome to join, Fridays 9am EDT/EST, link posted here on the forum). It seems that you and Oliver between you have nailed most of the nasty Matlab problems and I would be most grateful if we could get all of this captured in a reproducible way. My preference is for all of this to be captured in a Dockerfile (possibly with a few ancillary scripts) so that people can generate a Matlab enabled container with a simple docker build command. Another motivation for doing this is that if we get it to that stage, we could set up a build slave that generates these images automatically whenever the Gadgetron itself has a passing build and that image could be pushed (automatically) to Dockerhub. That would mean that you (and other Matlab users) would always have an up to date image to start from. Oliver has volunteered to host the build slave if/when we get that far. 

The steps would be:

1. Consolidate all the build steps and scripts into a Dockerfile which uses say gadgetron/ubuntu_1404_cuda75 as a base. 

2. Add that Dockerfile (with any ancillary scripts) to a folder in ${GADGETRON_SOURCE}/docker/incremental and set up PR to get this merged in

3. Install buildbot slave on a machine at Oliver's

4. Add a buildbot master configuration on our build master, which is triggered when the Gadgetron has a successful build. 

5. Sit back and enjoy the splendor of automated continuous integration testing.

What do you think? Do you think you and Oliver could consolidate your build instructions and get a PR set up?

Thanks,

Michael

[John Roberts]

Michael,

I'd be happy to work with Oliver to consolidate our efforts on the matlab build.  As I am new to Docker, I'm sure there are ways my approach could be streamlined.  I'll send Oliver an email today.

Now that I understand the when of your Hangout meetings, I'll try to jump in at some point.  I checked in last week, but I was clearly late to the game, mid-day MDT.

John.