#28664 (Lattice Reduction Strategies in Sage binaries)

[Martin R. Albrecht]

Hi all,

At https://trac.sagemath.org/ticket/28664 I’m/we’re confronted with a bug where I’m not sure how to make progress.

Essentially, if you build Sage from source, all is well. But if you use the binaries provided then lattice reduction reverts to something really really slow by being unable to load the strategies that speed it up:

First problem:

sage: from fpylll import *
sage: BKZ.DEFAULT_STRATEGY
'/home/malb/software/SageMath/local/share/fplll/strategieparse error - unpreprocessing_blpruning_parameteError: gptr == nullpoints/default.json'

This should be a filename.

But even loading from a correct path fails:

sage: from fpylll import *
load_strategies_json("/home/malb/projects/lattices/fplll/strategies/default.json")[60]
Strategy< 60, (), 1.00-1.00>

The above should read `Strategy< 60, (40), 0.29-0.50>`

Any ideas?

Cheers,
Martin

--

_pgp: https://keybase.io/martinralbrecht
_www: https://malb.io

[Nils Bruin]

If you see a difference in behaviour between binary and source distributions, then perhaps it's due to

https://github.com/sagemath/binary-pkg/blob/master/binary_pkg/templates/relocate-once.py

From what I understand, a binary distribution is originally built on a rather long and distinctive path, so that a search-and_replace on absolute path names should work (and should have enough room to make the replacements). Perhaps this is what goes wrong? Perhaps only part of fpylll's files are picked up to make replacements on?

It may be worth comparing the files like bkz.so etc. on the binary install *before* anything was run and *after*. The mangled path names distinctly suggest that some path name rewrite is tried, but that it doesn't succeed (although the error might very well originate from your own C++ code -- that's not so easy for me to see).

[Martin R. Albrecht]

Hi Nils,

Thanks, this is helpful.

Is there a straight-forward way in which I can trigger the relocation, I’m trying to test what goes wrong, so:

1/ I built sage from source, all works as expected
2a/ I move the installation, Sage barks at me.
2b/ However, just starting python from the relocated binary and issuing

>>> from fpylll import *
>>> BKZ.DEFAULT_STRATEGY
'/bulk/home/malb/software/sage-long-path-for-binary-building/local/share/fplll/strategies/default.json'
>>> print load_strategies_json("/home/malb/projects/lattices/fplll/strategies/default.json")[60]
Strategy< 60, (40), 0.29-0.50>

seems to work.

3/ How do I get the relocation script to run to see if that indeed breaks the above code?

Sorry, if that should be obvious.

Cheers,
Martin

[Martin R. Albrecht]

Hi all,

A follow up: I ran

$ git clone https://github.com/sagemath/binary-pkg.git
$ cd binary-pkg
$ make bdist-sage-linux
$ cd ..
$ unp binary-pkg/dist/sage-9.0.beta6-Debian_GNU_Linux_9-x86_64.tar.bz2
$ cd SageMath
$ ./sage

(watch the patching going on)

sage: from fpylll import *

sage: BKZ.DEFAULT_STRATEGY_PATH
b'/bulk/home/malb/software/SageMath/local/share/fplll/strategie'
print(load_strategies_json(b"/home/malb/projects/lattices/fplll/strategies/default.json")[60])
Strategy< 60, (40), 0.29-0.50>

So the patching thing does seem to work when I do it locally.

Cheers,
Martin

[Nils Bruin]

On Tuesday, November 19, 2019 at 2:13:18 PM UTC-8, Martin R. Albrecht wrote:

So the patching thing does seem to work when I do it locally.

That means you have a pretty good lead. When you unpack the binary package (both the distributed, malfunctioning one and the locally built working one) should have a file relocate-once.py in them. The first time you run sage, this will be executed and it should then delete itself, so don't do that. Instead unpack them and copy the relocate-once.py files from the two different binary packages. If you "diff" them you might find a difference that gives a hint to what is going wrong.