"old school" beancount installation for development after setup.py is removed

[Chary Chary]

Hi everybody!

I just noticed, that the setup.py has been removed in this commit.

As a result this the  "old school" beancount installation for development described in the document  is not applicable any longer

==================================

I work with it is the "old school" way; I just build it locally and modify my shell's environment to find its libraries. You build it like this:

cd beancount
python3 setup.py build_ext -i  # or "make build"

and then both the PATH and PYTHONPATH environment variables need to be updated for it like this:

export PATH=$PATH:/path/to/beancount/bin
export PYTHONPATH=$PYTHONPATH:/path/to/beancount

===================================

Question: what is now the "old school" beancount installation for development on Windows?

[Martin Blais]

Fixed, thank you

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/beancount/7e73bf1e-fbca-42b1-9d2e-90a9aaacbe0fn%40googlegroups.com.

[Daniele Nicolodi]

You can do by hand what the Makefile 'build' target does:

meson setup --reconfigure build/
ninja -C build/
cp build/_parser*.pyd beancount/parser/

Note that for building from source flex and bison are now required. On
the CI infrastructure, we install them from MSYS2. I think that the
version from winflexbison https://github.com/lexxmark/winflexbison
should work too.

However, there is really no reason to stick to the "old school" way of
doing things. Just install beancount as an editable wheel. From the
source directory, execute:

python -m pip install meson ninja
python -m pip install --no-build-isolation -e .

This has the advantage that, if you work on the C code, the extension
modules are automatically rebuilt when they are imported. I find this
much more convenient than the alternatives.

Cheers,
Dan

[Daniele Nicolodi]

Sorry, I forgot one dependency here. It should be:

python -m pip install meson-python meson ninja

> python -m pip install --no-build-isolation -e .
>
> This has the advantage that, if you work on the C code, the extension
> modules are automatically rebuilt when they are imported. I find this
> much more convenient than the alternatives.

Beancount uses meson-python as build system. More details can be found
in the meson-python documentation https://mesonbuild.com/meson-python/

[Chary Chary]

On Wednesday, November 20, 2024 at 10:08:38 PM UTC+1 dan...@grinta.net wrote:

python -m pip install --no-build-isolation -e .

Dan, thanks.

this installation also seems to require bison. I tried to install it manually as well as the winflexbison, but I still get an error

But I also understand that bison is optional?

C:\_code\py\beancount>python -m pip install --no-build-isolation -e .
Obtaining file:///C:/_code/py/beancount
  Checking if build backend supports build_editable ... done
  Preparing editable metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Preparing editable metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [18 lines of output]
      + meson setup C:\_code\py\beancount C:\_code\py\beancount\build\cp312 -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --native-file=C:\_code\py\beancount\build\cp312\meson-python-native-file.ini
      The Meson build system
      Version: 1.6.0
      Source dir: C:\_code\py\beancount
      Build dir: C:\_code\py\beancount\build\cp312
      Build type: native build
      Project name: beancount
      Project version: 3.1.0.dev0
      Activating VS 17.11.6
      C compiler for the host machine: cl (msvc 19.41.34123 "Microsoft (R) C/C++ Optimizing Compiler Version 19.41.34123 for x64")
      C linker for the host machine: link link 14.41.34123.0
      Host machine cpu family: x86_64
      Host machine cpu: x86_64
      Program bison found: NO

      ..\..\meson.build:3:8: ERROR: Program 'bison' not found or not executable

      A full log can be found at C:\_code\py\beancount\build\cp312\meson-logs\meson-log.txt
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.

[notice] A new release of pip is available: 24.2 -> 24.3.1
[notice] To update, run: python.exe -m pip install --upgrade pip
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

C:\_code\py\beancount>

 

[Martin Blais]

Bison not optional anymore.

--

You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/beancount/96c94c32-48e9-49d3-843e-0551f558c139n%40googlegroups.com.

[Daniele Nicolodi]

On 20/11/24 22:41, Chary Chary wrote:
> Dan, thanks.
>
> this installation also seems to require bison. I tried to install it

> manually as well as the winflexbison <https://github.com/lexxmark/
> winflexbison>, but I still get an error

>
> But I also understand that bison is optional?

No, it is not optional anymore. As I wrote in my previous message, bison
and flex are required to build from source. Keeping the preprocessed
files in the repository is a bit of a pain.

I don't do much development on Windows, and I never used winflexbison,
thus I haven't tested how that would work to build Beancount. However,
glancing at the project README, it seems that it installs executables
named win_flex and win_bison, not flex and bison. If this is the case,
of course Meson cannot find the programs it is looking for, unless the
Meson build definition would need to be adapted to look for the
alternative names too. Someone that cares about building from source on
Windows would need to put in some work to figure things out.

Cheers,
Dan

[Daniele Nicolodi]

On 20/11/24 23:00, Daniele Nicolodi wrote:
> I don't do much development on Windows, and I never used winflexbison,
> thus I haven't tested how that would work to build Beancount. However,
> glancing at the project README, it seems that it installs executables
> named win_flex and win_bison, not flex and bison. If this is the case,
> of course Meson cannot find the programs it is looking for, unless the
> Meson build definition would need to be adapted to look for the
> alternative names too. Someone that cares about building from source on
> Windows would need to put in some work to figure things out.

After a very quick recognition, it seems that the executables name is
the only thing that should affect building with winflexbison installed
flex and bison. Among other projects, PostgreSQL can be build using
these instead that the MSYS equivalents. There are some weird things
about colliding temporary file names in win_flex, but there is only one
scanner definition in Beancount, so there are no possible race conditions.

Therefore, I am positive that the only change needed to build with
winflexbison is to turn these lines in meson.build

bison = find_program('bison', version: '>=3.8.0')
flex = find_program('flex', version: '>=2.6.4')

into

bison = find_program('bison', 'win_bison', version: '>=3.8.0')
flex = find_program('flex', 'win_flex', version: '>=2.6.4')

I'll appreciate if someone could test this.

Cheers,
Dan

[Chary Chary]

Dan,

thanks!

I confirm, that I was able to build beancount on Windows using winflexbison and changing   meson.build as you suggested

bison = find_program('bison', 'win_bison', version: '>=3.8.0')
flex = find_program('flex', 'win_flex', version: '>=2.6.4')

I will put suggestions to the documentation to describe exactly how to do it

Will you update the source code? Or shall I make a small PR?

Also, out of curiosity, how is the beancount windows .whl file build, which goes to the PyPI? I assume it is done using github actions? If yes, how doe github do it, if even we struggle?

[Martin Blais]

Yes please make a pr, that would be nice, thank you! 

Re whl, yes, it's a GitHub action. 

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/beancount/f23536d1-d8e8-45e6-8ced-280305fc0753n%40googlegroups.com.

[Daniele Nicolodi]

On 21/11/24 11:06, Chary Chary wrote:
> Also, out of curiosity, how is the beancount windows .whl file build,
> which goes to the PyPI? I assume it is done using github actions? If
> yes, how doe github do it, if even we struggle?

I'm not sure I understand the question.

There is really no struggle in building Beancount once the dependencies
are in place. Installing build tools on Windows sucks, but that is
hardly anything that we can fix on the Beancount side.

The "recipe" that build the release wheels is here
https://github.com/beancount/beancount/blob/master/.github/workflows/wheels.yaml
It looks complicated but it build redistributable wheels for three
platforms on a number of architectures, dealing with installing the
dependencies on all of them.

The "recipe" that simply builds Beancount and runs the tests is simpler
https://github.com/beancount/beancount/blob/master/.github/workflows/tests.yaml


As I wrote in another message, the CI infrastructure uses MSYS2 and its
package manager to install flex and bison because it is the easiest
thing to do given that MSYS is already installed on the CI virtual
machine images. There is really nothing unusual.

Cheers,
Dan

[Daniele Nicolodi]

On 21/11/24 13:15, Martin Blais wrote:
> Yes please make a pr, that would be nice, thank you!

I've opened a PR https://github.com/beancount/beancount/pull/873

Cheers,
Dan

[Chary Chary]

Dan,

thanks. I will study this