Bazel build failing on fedora-toolbox (how to properly set up python env?)

[Ouro Boros]

Hi,

I am tryiung to install Beancount v3 using Bazel. I have gone through the installation instructions here and i have been stuck in `bazel build //bin:all` command.

These errors are kind of cryptic to me, and I have searched and found a lot of tensorflow problems. I couldn't figure out what exactly is the root of the issue.

The error is as follows (I have tried using the native fedora python build):

```

⬢[magus@toolbox beancount]$ bazel build //bin:all
INFO: Repository local_config_python instantiated at:
  /var/home/magus/ProgramFiles/Contabilidade/beancount/WORKSPACE:76:17: in <toplevel>
Repository rule python_configure defined at:
  /var/home/magus/.cache/bazel/_bazel_magus/bb3c731a895e7bb1e042dcf8348f9a5d/external/pybind11_bazel/python_configure.bzl:412:35: in <toplevel>
ERROR: An error occurred during the fetch of repository 'local_config_python':
   Traceback (most recent call last):
File "/var/home/magus/.cache/bazel/_bazel_magus/bb3c731a895e7bb1e042dcf8348f9a5d/external/pybind11_bazel/python_configure.bzl", line 409, column 36, in _python_autoconf_impl
_create_local_python_repository(repository_ctx)
File "/var/home/magus/.cache/bazel/_bazel_magus/bb3c731a895e7bb1e042dcf8348f9a5d/external/pybind11_bazel/python_configure.bzl", line 358, column 22, in _create_local_python_repository
_check_python_lib(repository_ctx, python_lib)
File "/var/home/magus/.cache/bazel/_bazel_magus/bb3c731a895e7bb1e042dcf8348f9a5d/external/pybind11_bazel/python_configure.bzl", line 241, column 14, in _check_python_lib
_fail("Invalid python library path: %s" % python_lib)
File "/var/home/magus/.cache/bazel/_bazel_magus/bb3c731a895e7bb1e042dcf8348f9a5d/external/pybind11_bazel/python_configure.bzl", line 27, column 9, in _fail
fail("%sPython Configuration Error:%s %s\n" % (red, no_color, msg))
Error in fail: Python Configuration Error: Invalid python library path:
ERROR: /var/home/magus/ProgramFiles/Contabilidade/beancount/WORKSPACE:76:17: fetching python_configure rule //external:local_config_python: Traceback (most recent call last):
File "/var/home/magus/.cache/bazel/_bazel_magus/bb3c731a895e7bb1e042dcf8348f9a5d/external/pybind11_bazel/python_configure.bzl", line 409, column 36, in _python_autoconf_impl
_create_local_python_repository(repository_ctx)
File "/var/home/magus/.cache/bazel/_bazel_magus/bb3c731a895e7bb1e042dcf8348f9a5d/external/pybind11_bazel/python_configure.bzl", line 358, column 22, in _create_local_python_repository
_check_python_lib(repository_ctx, python_lib)
File "/var/home/magus/.cache/bazel/_bazel_magus/bb3c731a895e7bb1e042dcf8348f9a5d/external/pybind11_bazel/python_configure.bzl", line 241, column 14, in _check_python_lib
_fail("Invalid python library path: %s" % python_lib)
File "/var/home/magus/.cache/bazel/_bazel_magus/bb3c731a895e7bb1e042dcf8348f9a5d/external/pybind11_bazel/python_configure.bzl", line 27, column 9, in _fail
fail("%sPython Configuration Error:%s %s\n" % (red, no_color, msg))
Error in fail: Python Configuration Error: Invalid python library path:
INFO: Repository com_google_protobuf instantiated at:
  /var/home/magus/ProgramFiles/Contabilidade/beancount/WORKSPACE:84:23: in <toplevel>
  /var/home/magus/ProgramFiles/Contabilidade/beancount/bazel/repositories.bzl:15:24: in beancount_dependencies
  /var/home/magus/ProgramFiles/Contabilidade/beancount/bazel/repositories.bzl:157:23: in _proto_dependencies
  /var/home/magus/ProgramFiles/Contabilidade/beancount/bazel/repositories.bzl:7:10: in maybe_http_archive
  /var/home/magus/.cache/bazel/_bazel_magus/bb3c731a895e7bb1e042dcf8348f9a5d/external/bazel_tools/tools/build_defs/repo/utils.bzl:233:18: in maybe
Repository rule http_archive defined at:
  /var/home/magus/.cache/bazel/_bazel_magus/bb3c731a895e7bb1e042dcf8348f9a5d/external/bazel_tools/tools/build_defs/repo/http.bzl:355:31: in <toplevel>
ERROR: /var/home/magus/ProgramFiles/Contabilidade/beancount/beancount/parser/BUILD:178:13: //beancount/parser:_parser_cc depends on @local_config_python//:python_headers in repository @local_config_python which failed to fetch. no such package '@local_config_python//': Python Configuration Error: Invalid python library path:
ERROR: Analysis of target '//bin:bean_check' failed; build aborted:
INFO: Elapsed time: 1.104s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded, 6 targets configured)
    currently loading: beancount/cparser

```

[Daniele Nicolodi]

On 31/01/24 18:31, Ouro Boros wrote:
> Hi,
>
> I am tryiung to install Beancount v3 using Bazel. I have gone through
> the installation instructions here

> <https://beancount.github.io/docs/installing_beancount_v3.html> and i

> have been stuck in `bazel build //bin:all` command.

The Bazel build for Beancount v3 builds some experimental C++ code that
is not yet used for anything else than a technology demonstration.
Martin run out of time for working on that and it is not really maintained.

If you just want to use Beancount v3 (and not hack on the C++ code) just
install it as a regular Python package. The v3 branch uses meson-python
to build the extension modules and pack them up in a Python wheel.

Cheers,
Dan

[Chary Chary]

Dan,

I thought that the main idea of beancount v3 was migrating some functionality to C++ to make the thing be faster.

If this is not being done in beancount v3, are there any  other improvements which make it worth using as an opposite to v2 at the moment?

[Daniele Nicolodi]

On 31/01/24 22:23, Chary Chary wrote:
> Dan,
>
> I thought that the main idea of beancount v3 was migrating some
> functionality to C++ to make the thing be faster.
>
> If this is not being done in beancount v3, are there any  other
> improvements which make it worth using as an opposite to v2 at the moment?

The C++ rewrite has seen significant progress but it has never landed in
a form where it can be used from a beancount application.

The major change in the v3 branch is that beancount.ingest has been
split of and significantly restructured into the beangulp project, that
beancount.query has been split of into the beanquery project and
significantly improved, that the deprecated bean-report and bean-web
functionality have been removed, and several other cleanups. A detailed
list of changes has not been compiled.

Cheers,
Dan

[Martin Blais]

On Sat, Feb 3, 2024 at 7:24 AM Daniele Nicolodi <dan...@grinta.net> wrote:

On 31/01/24 22:23, Chary Chary wrote:
> Dan,
>
> I thought that the main idea of beancount v3 was migrating some
> functionality to C++ to make the thing be faster.
>
> If this is not being done in beancount v3, are there any  other
> improvements which make it worth using as an opposite to v2 at the moment?

The C++ rewrite has seen significant progress but it has never landed in
a form where it can be used from a beancount application.

That's correct. It stalled on two things:

- not much free time to play with open source anymore (sad)

- a decent kludge I had to make passing protobufs between Python and C++ got taken away in a change in protobuf itself

I've pretty much decided to explore Rust as an alternative (not much progress, again, no time to play)

 

The major change in the v3 branch is that beancount.ingest has been
split of and significantly restructured into the beangulp project, that
beancount.query has been split of into the beanquery project and
significantly improved, that the deprecated bean-report and bean-web
functionality have been removed, and several other cleanups. A detailed
list of changes has not been compiled.

Cheers,
Dan

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/8e4ad0b7-e480-4cfd-866a-8f995dd424dd%40grinta.net.

[Chary Chary]

On Wednesday, January 31, 2024 at 8:11:39 PM UTC+1 dan...@grinta.net wrote:

If you just want to use Beancount v3 (and not hack on the C++ code) just
install it as a regular Python package. The v3 branch uses meson-python
to build the extension modules and pack them up in a Python wheel.

What if I want to install Beancount v3 for hacking, but not in C++, but in a python code with the goal to create a pull request later on.

Is there a "simplified" way to do this, without this Bazel?

P.S. I am using Windows

 

 

[Martin Blais]

You can still use the same build process (Makefiles + C) in the master branch. The C++ code is not enabled by default.

--

You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/4bb5c6cc-2dcf-4b80-b7b8-28fd770811a2n%40googlegroups.com.

[Chary Chary]

Martin,

thanks!

V3 seems to be working on Windows. 

just cloned from github

then  py setup.py build_ext -i

For some reason regex module was not auto installed, so I had to pip install it

when I run  py -3 -m pytest beancount  a lot of tests fail, but it looks like a lot of them (if not all) are related to this issue https://github.com/beancount/beancount/issues/222

Any way, in python 3.12 there is now new functionality, which allows to re-write the tests in a way that they will work on Windows as well:

The tempfile.NamedTemporaryFile function has a new optional parameter delete_on_close ( gh-58451)

But this is only in 3.12

We will have to wait until you drop support for anything below 3.12?

[Daniele Nicolodi]

On 27/02/24 15:38, Chary Chary wrote:
> Martin,
>
> thanks!
>
> V3 seems to be working on Windows.
>
> just cloned from github
> then py setup.py build_ext -i
>
> For some reason regex module was not auto installed

The reason is that you didn't ask for it to be installed:

python setup.py build_ext -i

only builds the extension modules, it does not install the package or
its dependencies.

Also, please note that the recommended way to install beancount from git
master is via the pyproject.toml based packaging definition, which uses
meson and meson-python for the build. Installing beancount from git
master with pip uses this build definition. The old setup.py based build
is kept for backward compatibility, but it is not tested (IIRC) and it
is likely to go away (for sure when setuptools will drop support for
setup.py based builds, which is probably not very far in the future).

> so I had to pip
> install it
>
> when I run py -3 -m pytest beancount  a lot of tests fail, but it looks
> like a lot of them (if not all) are related to this
> issue https://github.com/beancount/beancount/issues/222

The test suite assumes an unix-like system in many places. The temporary
file problem is mostly contained in a few test helpers that would not be
too difficult to fix. The other issues are more pervasive and it would
be quite a bit of work to fix them all. No one put in the work to do
that, so far.

> Any way, in python *3.12* there is now new functionality, which allows to re-write

> the tests in a way that they will work on Windows as well:
>
> The tempfile.NamedTemporaryFile function has a new optional
> parameter delete_on_close (gh-58451)
>
> But this is only in 3.12
>
> We will have to wait until you drop support for anything below 3.12?

The fix in Python 3.12 is good to have, but there are other ways to fix
this particular issue with the test suite.

Cheers,
Dan

[Chary Chary]

On Wednesday, February 28, 2024 at 9:08:19 PM UTC+1 dan...@grinta.net wrote:

The reason is that you didn't ask for it to be installed:

python setup.py build_ext -i

only builds the extension modules, it does not install the package or
its dependencies.

Dan, thanks. The whole package management thing of python is still on my to do list to learn.

 

Also, please note that the recommended way to install beancount from git
master is via the pyproject.toml based packaging definition, which uses
meson and meson-python for the build. Installing beancount from git
master with pip uses this build definition. The old setup.py based build
is kept for backward compatibility, but it is not tested (IIRC) and it
is likely to go away (for sure when setuptools will drop support for
setup.py based builds, which is probably not very far in the future).

Ok, I assume I had to do it like: 

py -m pip install .

This worked now on Windows, using the x64 Native Tools Developer Command Prompt

Thank you!

 

[Chary Chary]

On Wednesday, February 28, 2024 at 9:08:19 PM UTC+1 dan...@grinta.net wrote:

Also, please note that the recommended way to install beancount from git
master is via the pyproject.toml based packaging definition, which uses
meson and meson-python for the build. Installing beancount from git
master with pip uses this build definition. The old setup.py based build
is kept for backward compatibility, but it is not tested (IIRC) and it
is likely to go away (for sure when setuptools will drop support for
setup.py based builds, which is probably not very far in the future).

Dan,

sorry for coming back on this.

pip install . works well both on Linux and Windows, but it installs beancount as a package in /lib/site-packages

But my purpose is to install beancount for hacking / developing in python code. 

The command  py setup.py build_ext -i still works both on Windows and Linux and all tests pass afterwards (on Linux). So, I am good for now.

Still, can you please suggest the equivalent of    py setup.py build_ext -i, but for pyproject.toml / meson?

It looks like I am not the only one, who has such a question

 

Or may be general question: how do I install beancount v3 for hacking / developing of python code (not the advanced C++ fetures)?

 

[fin]

Chary Chary wrote:
> ------=_Part_36742_155774922.1709806590845
> Content-Type: multipart/alternative;
> boundary="----=_Part_36743_1688509485.1709806590845"
>
> ------=_Part_36743_1688509485.1709806590845
> Content-Type: text/plain; charset="UTF-8"
> Content-Transfer-Encoding: quoted-printable


the man page for pip says which i've been using for
years:

"
-e,--editable <path/url>
Install a project in editable mode (i.e. setuptools "develop
mode") from a local project path or a VCS url.
"

i use virtual environments to isolate various python
projects and installations from each other so there are
three steps to creating a clean version when i'm testing
strange things. i'm not sure for how long this will be
supported but i like how it's working for me)

one step is to create the virtual environment and then
the next step is to activate the environment and the third
step is to do all the package installs. here are the
scripts i use just in case they can help someone else...
(no warranty implied, contents may have settled during
shipping, etc.)...
===== step 1
#!/bin/sh
#
# create virtual environment

ENV_HOME="/home/me/env"
if test ! -d "$ENV_HOME" ; then
echo " directory $ENV_HOME does not exist!"
exit
fi

cd $ENV_HOME

if test ! -d "bc" ; then
echo " setting up virtual environment $ENV_HOME/bc"
python -m venv bc
fi
=====

step 2 is to run the activate command directly as for
some reason it may not work via script at times (maybe
i don't have that path in my environment yet...)

$ alias act_bc='cd ${HOME}/fin/beancount; source ${HOME}/env/bc/bin/activate'
$ act_bc

(to deactivate when done i just run the deactivate
command in whichever environment it is)

===== step 3
#!/bin/sh
#
# add some things to the virtual environment

ENV_HOME="/home/me/env"
if test ! -d "$ENV_HOME" ; then
echo " directory $ENV_HOME does not exist!"
exit
fi

cd $ENV_HOME

if test ! -d "bc" ; then
echo " directory $ENV_HOME/bc does not exist!"
exit
fi

pip3 install fastnumbers
pip3 install -e ${HOME}/src/github/beancount/beancount
pip3 install petl
pip3 install git+https://github.com/beancount/beanprice
pip3 install autobean-format
=====

hope this helps.

fin


> On Wednesday, February 28, 2024 at 9:08:19=E2=80=AFPM UTC+1 dan...@grinta.n=
> et wrote:
>
>
> Also, please note that the recommended way to install beancount from git=20
> master is via the pyproject.toml based packaging definition, which uses=20
> meson and meson-python for the build. Installing beancount from git=20
> master with pip uses this build definition. The old setup.py based build=20
> is kept for backward compatibility, but it is not tested (IIRC) and it=20
> is likely to go away (for sure when setuptools will drop support for=20
> setup.py based builds, which is probably not very far in the future).=20

>
>
> Dan,
>
> sorry for coming back on this.
>

> *pip install . *works well both on Linux and Windows, but it installs *bean=
> count=20
> *as a package in /lib/site-packages
>
> But my purpose is to install *beancount *for hacking / developing in python=
>=20
> code.=20
>
> The command *py setup.py build_ext -i* still works both on Windows and=20

> Linux and all tests pass afterwards (on Linux). So, I am good for now.
>

> Still, can you please suggest the equivalent of *py setup.py build_ext=
>=20
> -i, *but for pyproject.toml / meson?
>
> It looks like I am not the only one, who has such a question=20
><https://github.com/mesonbuild/meson/issues/6184>
>=20
> Or may be general question: how do I install beancount v3 for hacking /=20

> developing of python code (not the advanced C++ fetures)?

>=20
>
> --=20
> You received this message because you are subscribed to the Google Groups "=
> Beancount" group.
> To unsubscribe from this group and stop receiving emails from it, send an e=
> mail to beancount+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/=
> beancount/c939325a-2737-4916-beae-7dadbf4778c3n%40googlegroups.com.
>
> ------=_Part_36743_1688509485.1709806590845
> Content-Type: text/html; charset="UTF-8"
> Content-Transfer-Encoding: quoted-printable
>
><br /><br /><div><div dir=3D"auto">On Wednesday, February 28, 2024 at 9:08:=
> 19=E2=80=AFPM UTC+1 dan...@grinta.net wrote:</div><blockquote style=3D"marg=
> in: 0px 0px 0px 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-l=
> eft: 1ex;">
><br />Also, please note that the recommended way to install beancount from =
> git=20
><br />master is via the pyproject.toml based packaging definition, which us=
> es=20
><br />meson and meson-python for the build. Installing beancount from git=
>=20
><br />master with pip uses this build definition. The old setup.py based bu=
> ild=20
><br />is kept for backward compatibility, but it is not tested (IIRC) and i=
> t=20
><br />is likely to go away (for sure when setuptools will drop support for=
>=20
><br />setup.py based builds, which is probably not very far in the future).
><br /><br /></blockquote><div><br /></div><div>Dan,</div><div><br /></div><=
> div>sorry for coming back on this.</div><div><br /></div><div><b><font face=
>=3D"Times New Roman">pip install .</font> </b>works well both on Linux and =
> Windows, but it installs <b>beancount </b>as a package in=C2=A0<span style=
>=3D"background-color: rgb(227, 230, 232); color: rgb(12, 13, 14); font-fami=
> ly: ui-monospace, &quot;Cascadia Mono&quot;, &quot;Segoe UI Mono&quot;, &qu=
> ot;Liberation Mono&quot;, Menlo, Monaco, Consolas, monospace; font-size: 13=
> px; white-space-collapse: preserve;">/lib/site-packages</span></div><div><s=
> pan style=3D"background-color: rgb(227, 230, 232); color: rgb(12, 13, 14); =
> font-family: ui-monospace, &quot;Cascadia Mono&quot;, &quot;Segoe UI Mono&q=
> uot;, &quot;Liberation Mono&quot;, Menlo, Monaco, Consolas, monospace; font=
> -size: 13px; white-space-collapse: preserve;"><br /></span></div>But my pur=
> pose is to install <b>beancount </b>for hacking / developing in python code=
> .=C2=A0</div><div><br /></div><div>The command=C2=A0=C2=A0<font face=3D"Tim=
> es New Roman"><b>py setup.py build_ext -i</b>=C2=A0</font>still works both =
> on Windows and Linux and all tests pass afterwards (on Linux). So, I am goo=
> d for now.</div><div><br /></div><div>Still, can you please suggest the equ=
> ivalent of=C2=A0 =C2=A0=C2=A0<font face=3D"Times New Roman"><b>py setup.py =
> build_ext -i, </b>but for=C2=A0</font>pyproject.toml /=C2=A0<span style=3D"=
> font-family: &quot;Times New Roman&quot;;">meson?</span></div><div><span st=
> yle=3D"font-family: &quot;Times New Roman&quot;;"><br /></span></div><div><=
> span style=3D"font-family: &quot;Times New Roman&quot;;">It looks like I am=
> not the only one, who has <a href=3D"https://github.com/mesonbuild/meson/i=
> ssues/6184">such a question</a></span></div><div><div>=C2=A0</div><div>Or m=
> ay be general question: how do I install beancount v3 for hacking / develop=
> ing of python code (not the advanced C++ fetures)?</div><div>=C2=A0</div></=
> div>
>
><p></p>
>
> -- <br />
> You received this message because you are subscribed to the Google Groups &=
> quot;Beancount&quot; group.<br />
> To unsubscribe from this group and stop receiving emails from it, send an e=
> mail to <a href=3D"mailto:beancount+...@googlegroups.com">beancount=
> +unsub...@googlegroups.com</a>.<br />
> To view this discussion on the web visit <a href=3D"https://groups.google.c=
> om/d/msgid/beancount/c939325a-2737-4916-beae-7dadbf4778c3n%40googlegroups.c=
> om?utm_medium=3Demail&utm_source=3Dfooter">https://groups.google.com/d/msgi=
> d/beancount/c939325a-2737-4916-beae-7dadbf4778c3n%40googlegroups.com</a>.<b=
> r />
>
> ------=_Part_36743_1688509485.1709806590845--
>
> ------=_Part_36742_155774922.1709806590845--
>

[Chary Chary]

On Thursday, March 7, 2024 at 1:24:59 PM UTC+1 fin wrote:

the man page for pip says which i've been using for
years:

"
-e,--editable <path/url>
Install a project in editable mode (i.e. setuptools "develop
mode") from a local project path or a VCS url.
"

Thanks for looking at this, I did try the pip3 install -e . method. in virtual environment.

The installation works, but then unit tests do not work. Did you try running unit tests ( pytest beancount ) ? 

However when I use the good old py setup.py build_ext -i method, then pytests do work

 

[Daniele Nicolodi]

On 07/03/24 18:45, Chary Chary wrote:
> Thanks for looking at this, I did try the *pip3 install -e .* method. in

> virtual environment.
>
> The installation works, but then unit tests do not work. Did you try

> running unit tests ( *pytest beancount )* ?
>
> However when I use the good old *py setup.py build_ext -i *method, then
> pytests do work

A sane amount of RTFM would not be misplaced. It is not like Python
development and packaging are these obscure things that are discussed
nowhere on the internet.

beancount uses a source layout in which the tests are located next to
the source code being tested. However, the tests are not installed with
the code (because the users of beancount do not need to access the
testsuite and this makes the binary distribution significantly smaller).

With this setup, you need to point pytest to the root of the Python
package sources and use the --import-mode=importlib option to run the tests.

Cheers,
Dan

[Chary Chary]

On Thursday, March 7, 2024 at 8:31:27 PM UTC+1 dan...@grinta.net wrote:

package sources and use the --import-mode=importlib option to run the tests.

Cheers,
Dan

Dan, thank you for your help, I just could not make the pytests work no matter what

There are errors, that that the _parser module cannot be imported

But I found the following workaround, which uses pip install, rather than py setup.py build_ext -i

I will put is here, may be it will help someone

======================

setup virtual environment

In case of Windows go to the x64 Native Tools Command Prompt for VS 20XX

activate virtual environment

install beancount into virtual environment normal way pip install .  

go to   the virtual environment, locate folder  site-packages/beancount/parser

Find the file 

_parser.XXXXXX.so - for linux

_parser.XXXXXX.pyd - for window

Copy this file to the source code /beancount/parser

Now pip uninstall beancount

Run the tests normal way: pytest beancount

========================

So, for me it works for now, (both via py setup.py build_ext -i and as described above ) I can hack in the latest python code of the beancount. But I hope that one day somebody more clever will make the pip install -e . mode workable for pytests by either updating code or the documentation or both

 

[Daniele Nicolodi]

On 09/03/24 10:58, Chary Chary wrote:
> Dan, thank you for your help, I just could not make the pytests work no
> matter what
>

> There are errors, that that the *_parser *module cannot be imported

>
> But I found the following workaround, which uses pip install, rather

> than *py setup.py build_ext -i*

No work-around is required. You are doing something wrong. However,
unless you report exactly the commands you execute and the errors you
get, it is impossible to know what that is.

The setup I suggest works just fine for the CI jobs and in all the
systems where I tried:

$ cd /tmp
$ git clone g...@github.com:beancount/beancount.git
$ python3.12 -m venv beancount/venv
$ . beancount/venv/bin/activate
$ cd beancount/
$ python -m pip install -e .
$ python -m pip install pytest
$ pytest --import-mode=importlib beancount/
...
== 1090 passed, 22 skipped, 2 xfailed, 2 warnings in 14.18s ==

I suspect you are trying to run the tests on the beancount installed in
a virtual environment with the pytest executable installed for your
system Python. This is not going to work.

Cheers,
Dan

[Chary Chary]

Dan,

thank you very much for detailed step by step procedure!

I did exactly what you suggested, but still get errors. See attached

Regards.

[Daniele Nicolodi]

On 09/03/24 12:44, Chary Chary wrote:
> Dan,
>

> thank you very much for detailed step by step procedure!
>
> I did exactly what you suggested, but still get errors. See attached

Did you read and understand the error message you get?

FileNotFoundError: [Errno 2] No such file or directory:
'/tmp/pip-build-env-onlb9jfe/normal/bin/ninja'

As I proposed in an earlier message, reading the documentation would be
a good start:

https://mesonbuild.com/meson-python/how-to-guides/editable-installs.html#build-dependencies

The solution is to install ninja on your system, or in the Python
virtual environment:

apt-get install ninja-build

or

python -m pip install ninja

[Chary Chary]

On Saturday, March 9, 2024 at 1:42:01 PM UTC+1 dan...@grinta.net wrote:

As I proposed in an earlier message, reading the documentation would be
a good start:

https://mesonbuild.com/meson-python/how-to-guides/editable-installs.html#build-dependencies

The solution is to install ninja on your system, or in the Python
virtual environment:

apt-get install ninja-build

or

python -m pip install ninja

Dan,

thanks for pointing to the documentation. The solution at the end was to use pip install with the -no-build-isolation option

So, like that 

===============

git clone https://github.com/beancount/beancount.git

python3.12 -m venv beancount/venv

. beancount/venv/bin/activate

cd beancount

python -m pip install meson-python meson ninja

python -m pip install --no-build-isolation --editable .

python -m pip install pytest

pytest --import-mode=importlib beancount/

===================

 

[Daniele Nicolodi]

On 09/03/24 16:39, Chary Chary wrote:
> Dan,
>
> thanks for pointing to the documentation. The solution at the end was to

> use pip install with the --no-build-isolation option

No. The fix is installing ninja in the virtual environment. As I
demonstrated earlier, everything works just fine if you install without
--no-build-isolation. Building with --no-build-isolation is useful only
to the extent to fail immediately if the build dependencies are not
available.

[Chary Chary]

On Saturday, March 9, 2024 at 6:47:36 PM UTC+1 dan...@grinta.net wrote:

No. The fix is installing ninja in the virtual environment. As I
demonstrated earlier, everything works just fine if you install without
--no-build-isolation. Building with --no-build-isolation is useful only
to the extent to fail immediately if the build dependencies are not
available.

We must have some difference in behavior on our systems, which is interesting to investigate by itself

I use Ubuntu 22.04.4 LTS

I am not a Linux expert, but probably this behavior is to do with how files in the /tmp are recycled

I have tested this once again and confirm, that just installing ninja  does not help.

You can see attached complete logs where ninja  was installed, but test were still not working without --no-build-isolation

 And then started working with.

This also a recommended  setup as per the documentation you kindly pointed me out to

https://mesonbuild.com/meson-python/how-to-guides/editable-installs.html#editable-installs

The isolated build environment is deleted after the build is completed, resulting in failures when the package in rebuild on import. For these reasons, when installing packages in editable mode, it is recommended to disable build isolation passing the --no-build-isolation argument to pip.

Also it the logs you can see, that the error is that the directory /tmp/pip-build-env-t6qnb87w/normal/bin/ninja Cannot be  found. 

And this temporary directory indeed does not exist

If I install beancount in editable mode without --no-build-isolation, then the last part of the file _beancount_editable_loader.py in the site-packages looks like this

install(
    {'beancount'},
    '/home/evg/code/beancount/build/cp312',
    ['/tmp/pip-build-env-qc8r8vrg/normal/bin/ninja'],
    False,
)

So, it points to something which does not exist

However when  --no-build-isolation is activated, it then points to existing directory

install(
    {'beancount'},
    '/home/evg/code/beancount/build/cp312',
    ['/home/evg/code/beancount/venv/bin/ninja'],
    True,
)

Which kind of explains the behaviour.

[Daniele Nicolodi]

On 09/03/24 20:52, Chary Chary wrote:
>
>
> On Saturday, March 9, 2024 at 6:47:36 PM UTC+1 dan...@grinta.net wrote:
>
>
> No. The fix is installing ninja in the virtual environment. As I
> demonstrated earlier, everything works just fine if you install without
> --no-build-isolation. Building with --no-build-isolation is useful only
> to the extent to fail immediately if the build dependencies are not
> available.
>
>
> We must have some difference in behavior on our systems, which is
> interesting to investigate by itself
>
> I use Ubuntu 22.04.4 LTS
>
> I am not a Linux expert, but probably this behavior is to do with how

> files in the */tmp* are recycled

Are you creating your virtual environment in /tmp? It does not seem a
great idea, if you want it to survive a reboot. However, files in /tmp
should not be cleaned up (on the time scales involved here, anyway).

> I have tested this once again and confirm, that just installing ninja
> does not help.

How do you install ninja?

> You can see attached complete logs where ninja  was installed, but test

> were still not working without *--no-build-isolation*

>  And then started working with.
>
> This also a recommended  setup as per the documentation you kindly
> pointed me out to
>

> https://mesonbuild.com/meson-python/how-to-guides/editable-installs.html#editable-installs <https://mesonbuild.com/meson-python/how-to-guides/editable-installs.html#build-dependencies>
>
> /The isolated build environment is deleted after the build is completed,

> resulting in failures when the package in rebuild on import. For these
> reasons, when installing packages in editable mode, it is recommended to

> disable build isolation passing the --no-build-isolation argument to pip./

I know what that document says, I wrote it.

> Also it the logs you can see, that the error is that the directory

> */tmp/pip-build-env-t6qnb87w/normal/bin/ninja *Cannot be  found.

This points to the fact that ninja was installed into the temporary
build environment. This happens only if meson-python does not find a
suitable ninja on the system. How did you install ninja?

> However when *--no-build-isolation *is activated, it then points to

> existing directory
>
> install(
>     {'beancount'},
>     '/home/evg/code/beancount/build/cp312',

>     ['*/home/evg/code/beancount/venv/bin/ninja*'],

>     True,
> )
>
> Which kind of explains the behaviour.

Which also points to the fact that you installed ninja in your
development virtual environment. Sorry, I realize how the instructions I
gave you could have been misleading on this point.

[Chary Chary]

On Saturday, March 9, 2024 at 9:20:36 PM UTC+1 dan...@grinta.net wrote:

Are you creating your virtual environment in /tmp? It does not seem a
great idea, if you want it to survive a reboot. 

No, installation was done in my home directory 

How do you install ninja?

I actually installed it 2 time

First on the system

sudo apt-get install ninja-build

And then in a virtual environment as well

python -m pip install meson-python meson ninja

I know what that document says, I wrote it.

Very thorough documentation! 

[Filip Filmar]

On Wed, Jan 31, 2024 at 10:20 AM Ouro Boros <yuriba...@gmail.com> wrote:

I am tryiung to install Beancount v3 using Bazel. I have gone through the installation instructions here and i have been stuck in `bazel build //bin:all` command.

These errors are kind of cryptic to me, and I have searched and found a lot of tensorflow problems. I couldn't figure out what exactly is the root of the issue.

The error is as follows (I have tried using the native fedora python build):

Not sure why the build is using a local python config. I would recommend using a hermetic python setup instead, if possible.

F