Build trouble on FreeBSD 12

[Andrew Morris]

Hello group,

Since I'd like to try SALSA with a set of measurements I have, and since I don't have a 64-bit Windows computer available to work with, I am trying to build it on the FreeBSD 12 machine that I use for assorted hobbyist-type things.

This has not gone too badly: I have had to adjust some of the paths in the CMake files to point to the system's installations of the dependencies, and I've gotten to the point of building the software, but the source ZIP appears to be missing two files so far:

• SparseMatrix.hpp. This is referenced at lsacommon/MatrixVector.hpp, line 29. I've been able to work around that by commenting out the "#define SPARSE 1", and apparently this will let it build, to some degree anyway.
• SunEarthSatGeometry.hpp. This is referenced in lsacommon/DATPoint.cpp at line 25. This one's not so easy to bypass as the other since there's no option.
  

It looks like both of these actually are headers from within GPSTk's source, and I may be able to link them through if they're the same, but are they? Or is this a sign that version 2.12 of GPSTk is mandatory (vs. 8.0.0, which is current in FreeBSD's ports tree)? There's nothing obvious to me in the readme that would indicate a need to take GPSTk's own headers into SALSA, but I'll also admit I may be missing something.

If I can get the build to work, I would eventually like to roll a port of this and make it easier for the next person.

[Clark Hughes]

Hi Andrew,

Thank you for your detailed explanation of what you've tried and where you're running into issues. I'll note that we've not tried building on BSD, so any lessons learned and pathfinding you achieve will be valuable to the community.  Both the files you note should be in the GPSTk, not in the salsa source zip, so I suspect you're correct that the issue may come down to a GPSTk version mismatch. Let me engage some of my more knowledgeable team members and see what guidance we can provide. It'll probably be early next week, so hang tight!  Thanks,

Clark

[Sarah Magliocca]

Hi Andrew, 

Our team took a closer look at your issue and we believe it is indeed due to your GPSTk version. We currently only support up to version 2.12, which can be downloaded from https://github.com/SGL-UT/GPSTk/tags. We do plan on upgrading to the latest GPSTk version in a future release. Please let us know if you run into issues with version 2.12. 

Lastly, be on the look out for a new SALSA release, which will be posted on our webpage soon. 

Thanks, 

Sarah

[Andrew Morris]

I suppose I owe the project an update on my attempts to build SALSA on FreeBSD. I can summarize by saying that I have not been successful so far, although it hasn't been a complete bust, and out of frustration I've resorted to installing SALSA 1.15 on Windows 10, on a Virtualbox instance, on a MacBook Pro. This basically works, although slowly.

Following is a fairly lengthy description of what I've done, in hopes it will be useful to anyone else wanting to do a build. I can't, of course, guarantee this will work for anyone else, but it might help them avoid needing to take the 1,000+ lines of notes that I took so far detailing all the dead ends I got myself into and then out of. Happy to diff my working directory against the published set and send a patch if that would be useful to show my work.

The environment that I wound up with was:

• GPSTk 2.12, as noted above.
• SWIG 2.0.12
• Doxygen 1.9.5
• CMake 3.24.0
• TexLive 20210325
• HDF5 1.12.2
• Base zlib
• QuaZIP-QT5 1.3
• Mesa 21.3.8. I'm using the graphics/mesa-dri and graphics/mesa-libs ports.
• Eigen 3.4.0
  
• QT5 5.15.5p165
• Marble 22.08.1 (which definitely caused trouble later)
  
• Probably a few other relevant dependencies and sub-dependencies I haven't counted.

GPSTk fought me a bit installing it manually, which I had to do since the Ports version is up to 8.0.0, mostly due to it not looking in the right places for its dependencies. I got that fixed, but this turned out to be a recurring issue once I got to actually building SALSA. I managed to get it straightened by setting environment variables as follows:

• $gpstk=$HOME/.local/gpstk
• $marble=/usr/local
• $eigen3=/usr/local/include/eigen3
• $QWT=/usr/local
• $CMAKE_PREFIX_PATH=/usr/local
  

Many of these result from FreeBSD's habit of putting third-party libraries and binaries in /usr/local rather than directly under /usr.

The provided CMake file searches for libqwt.so, which is actually installed for me as libqwt6.so.6. I had to tweak the CMake file to look for that instead. I think this may just be a difference between Debian and FreeBSD and how they handle installing libraries.

There is an unlisted dependency on the QT5 build tools. On FreeBSD, these are in devel/qt5-qmake (for qmake) and devel/qt5-buildtools (for moc and rcc).

The CMake process looks for "/usr/local/lib/marble/plugins/libAnnotatePlugin.so" and fails to find it. This traces back to it seeking the Marble plugins by prepending "${MARBLE_LIBRARY_DIRS}/marble/plugins/lib" to a list of plugins, which includes AnnotatePlugin. Those files are actually installed for me as AnnotatePlugin.so and so on, so I had to trim off the "lib" from the end of that prepend.

The CMake process locked up for me looking for an X server, and as I was building via SSH, it didn't have one. Opening an X session didn't actually help this at all, either. This was totally weird, but as it turns out, it was a simple problem. CMake was trying to run scripts/build/scrapePublicTestNames.py, but apparently it didn't grasp that's a Python script, so it just ran it as a shell script. It hits an "import" command, and it turns out that "import" for me is an X Windows utility to do a screen capture. (This comes with ImageMagick 6 according to my port records.) So, rather than importing a Python module, it was trying to capture an X screen, and since it's not formatted for the import command, widespread confusion results. The fix was to insert a shebang at the top of scrapePublicTestNames.py: "#!/usr/bin/env python3". This explicitly passes the script into Python where it operates properly.

CMake couldn't locate QuaZip. Partly this was it looking in /usr/lib rather than /usr/local/lib as noted above. Partly this was it looking for a filename I did not have. I have QuaZip installed as libquazip1-qt5.so and so on, and adding that to the CMake helped. CMake was also looking in /usr/include and /usr/local/include and the quazip subdirectories of each, but my QuaZip include is in /usr/local/include/QuaZip-Qt5-1.2/quazip. (I can't say I agree with that plan, but the port didn't ask me.) Tweaking the CMake file fixed that.

So far, this has all gone to get CMake to write the build files. Actually building the program introduced a couple of others.

Even with these edits, there were a few missed includes. In file src/lsacommon/expandpath.cpp, there is an include for <linux/limits.h> that (as you might expect) FreeBSD does not have. This turns out to be a simple enough one, since the FreeBSD limits.h itself already includes /usr/include/sys/syslimits.h, its equivalent to the Linux limits.h, so in short the include for <linux/limits.h> isn't needed.

In HistogramDialog.hpp, included file qwt_plot_histogram.h wasn't found. That comes down to needing the CMake file to set QWT_INCLUDE_DIRS to $QWT/include/qt5/qwt6 for how I have QWT installed. Similarly, CMake sets QMAKE_INCLUDE_DIRS to end in /quazip (for me, /usr/local/include/QuaZip-Qt5-1.2/quazip), but in src/gui/mapping.hpp the include is phrased as #include <quazip/JlCompress.h>. In other words, the include statement repeats the quazip directory. I think I fixed that by dropping the extra directory from the include, but it could probably also be fixed in CMake.

Now, I don't think it is a porting problem so much, but mapping.hpp also seems to omit a couple of necessary headers. Checking my working file, I had to add the following for various compilation errors:

#include <marble/GeoDataLineStyle.h>
#include <marble/GeoDataLinearRing.h>
#include <marble/GeoDataPolyStyle.h>
#include <marble/GeoDataLatLonAltBox.h>
#include <marble/GeoWriter.h>
#include <marble/GeoDataLookAt.h>

I'm not sure whether this reflects changes in the Marble API since version 16 or differences of installations.

Speaking of Marble, I didn't run the patch against my Marble version: since I was trying to build against current Marble, the patches would not quite work. (My notes aren't clear on the point, but I think they may not have worked against Marble 16.04 either, at least not the way I downloaded it.) The key problems this raised are that remoteIconLoader.h and FileManager.h aren't available without patching the Marble install. I did think through a couple of ways to fix this so they won't be required. For the file thing, I switched to GeoWriter::write, which appears to do the same thing and is in the published Marble API. For the remoteIconLoader calls, I just commented them out to get the build to work. I have not tested either of these so far, but they did remove the need for those two headers so the build could proceed.

At src/lsacommon/geoid.cpp:263, there is a call to

     dataset.read(undulationsBuffer,H5::PredType::NATIVE_FLOAT,memspace,filespace,H5P_DEFAULT);

that fails to compile because it doesn't like the last argument, complaining that it's an ambiguous conversion from int to const H5::DSetMemXferPropList. The HDF5 documents do list H5P_DEFAULT as legal for the Fortran implementation, but also say that the final argument can be omitted to let it default to, well, DEFAULT. I took that shortcut.

This more or less let the program be built. It segfaults, however. I have pages of notes on this, but at its heart the segfault comes in when mapping.cpp calls into sortPointsInFolders() and hits:

     std::sort(POSGFolder->begin(), POSGFolder->end(), Mapping::sortManner);

Calling POSGFolder->begin() seems to end up dereferencing a nearly-null pointer. My first impression is that this was a result of calling the function while the folder was empty, assuming that begin (and end) were uninitialized or undefined, or something like that. I fixed that by testing whether POSGFolder (and the other three being sorted here) is empty before going into the sort. This was enough to get the program to run, anyway, and it opens a window and looks fine. Opening a project file still crashes the program, however, because it calls sortPointsInFolders() again, and this time the folders aren't empty, so it goes through to the begin() method and segfaults. This problem appears to be in Marble and was reported in their bug tracker as bug number 458555. (I am somehow reassured to see that Sarah reported this originally.)

As limited as it is, I am at least able to get SALSA started at this point. It does report "ERROR - Unable to find . Unable to import instrumentation output." I haven't started trying to track that down and for the moment it does not seem to be affecting anything. The mapping widget does seem to work, at least: I'm able to open the base map and pan around the globe quite naturally. I am tempted to just comment out the sortPointsInFolders function to continue testing until KDE can address the bug report, but I want to put this aside for a little while.

Although I can get everything installed and apparently working on Windows 10, it is, as I noted, running in a virtual environment and is consequently quite slow to respond. I would still like to get this working on FreeBSD for better performance, so I don't think I'm ready to abandon the attempt.

[Connor Walker]

Andrew,

Thank you for sharing all of this information and feedback on how to build SALSA on FreeBSD. Hopefully I can answer some of your questions and provide guidance on how to finish out the build.

First, I'm glad you were able to figure out the environment variable and path issues. SALSA's linux support is primarily Debian so the work you've done will be a big help to those in the community building on other distributions.

We encountered the same Marble issues you did when attempting to upgrade from 16.04. We found the best course of action was to stay with Marble 16.04 (we are looking at upgrading in future releases). However, if using 16.04 is not an option you can comment out sortPointsInFolders() like you mentioned. While not recommended it is an ok solution that will not break SALSA. If you are not interested in exporting as a kml then you will notice no difference in performance but otherwise the kml points will be unsorted.

As for the error you're getting once you build SALSA I'm pretty sure this is a miniconda error. The two things you can do are first verify that you built the lsa_python environment. It should be located at $HOME/.local/salsa_python. If you did build it please make sure that you exported the SALSA_PYTHON_EXE variable e.g."export SALSA_PYTHON_EXE=$HOME/.local/salsa_python/bin/python". Since you are using a different OS you may have to modify the paths/location of the environment slightly. This should fix that error. Please let me know if you run into any additional errors or problems.

Thanks,

Connor