how to build and install python bindings for ledger 3.0 on debian wheezy

[c b]

Hi,

I just built ledger 3.0 from sources on a Debian wheezy system with prefix=/opt/local/

I'm a python noob and  cannot figure out how to enable python bindings or where they will be installed

Any pointers will be appreciated.

Thanks,

cb

[Tim Crews]

cb:

Once you've successfully build ledger with the  USE_PYTHON setting enabled, that's as enabled as the Python bindings get.  The bindings are built into the ledger executable itself; there is nothing else to install other than ledger.

You access the Python bindings by executing ledger with the python sub-command, i.e.

    ledger python

You will then be placed in a Python environment that has access to the Ledger Python bindings.  If you have a Python program that needs to access the Ledger bindings, run it like:

    ledger python my_program.py

It is not currently possible to import ledger in a standalone Python program or module without running the ledger python command first.

In the example Python code found at http://www.ledger-cli.org/3.0/doc/ledger3.html#Extending-with-Python, note that the calls to xacts and posts should be followed by parentheses, i.e. the example should actually read:

        import ledger
     
        for xact in ledger.read_journal("sample.dat").xacts():
            for post in xact.posts():
                print "Transferring %s to/from %s" % (post.amount, post.account)

I hope this is enough to get you started. 

Tim Crews

[Martin Blais]

Is there any specific reason that ledger can't be built as a Python extension module?
Something specific at static initialization?

> --
>
> ---
> You received this message because you are subscribed to the Google Groups "Ledger" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to ledger-cli+...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Tim Crews]

On Monday, June 17, 2013 9:00:27 AM UTC-7, Martin Blais wrote:

Is there any specific reason that ledger can't be built as a Python extension module?
Something specific at static initialization?

Martin:

I may have been wrong about the claim that you cannot directly import ledger from a standalone Python script.  John Wiegley indicated otherwise in this post:  https://groups.google.com/d/msg/ledger-cli/1Jeizqr9A7M/Irum7QIfHvIJ .  In that post, he said that you would have to add something to your PYTHONPATH to make it work.  I don't know what this would be.  There is nothing that looks like a ledger Python package or extension module in the ledger source tree or build output (i.e. no directory named "ledger" with a file named __init__.py, and no DLL file with the name "ledger.dll").

Tim Crews

[John Wiegley]

>>>>> Martin Blais <goo...@furius.ca> writes:

> Is there any specific reason that ledger can't be built as a Python
> extension module? Something specific at static initialization?

Ledger *does* build as Python module, but until you "make install", you can
just do a Python import. That's why I added the "python" subcommand to
Ledger: so that the tests could import in the currently built Ledger module
before installation.

After installation, just do "import ledger".

John

[c b]

Thanks  for the pointer Tim,

I tried your recommendation and am able to run the snippet above with ledger python my_program.py.

-cb

[c b]

Hi John,

Thanks for your response.

I did a "make install" and see the following files installed in my prefix location (/opt/local)

/opt/local

          /bin/ledger

          /include/ledger/<a bunch of header files>

          /lib/libledger.a 

          /share

                /doc

                /man

I tried including /opt/local/lib/ into PYTHON_PATH, but couldn't do an 'import ledger' in python

Does that look right?

Thanks,

cb          

[Martin Blais]

It's PYTHONPATH not PYTHON_PATH.

[c b]

Thanks Martin,

I tried updating and exporting PYTHONPATH too. It doesn't work, either. 

Here's what I get

>>> import ledger

Traceback (most recent call last):

  File "<stdin>", line 1, in <module>

ImportError: No module named ledger

>>> exit

[c b]

Still doesn't work.. but forgot to mention that I am using Python 2.7.3

[John Wiegley]

>>>>> c b <24x7...@gmail.com> writes:

> /opt/local
> /bin/ledger
> /include/ledger/<a bunch of header files>
> /lib/libledger.a
> /share
> /doc
> /man

Where is pyledger.so? If you built with Python support, it should be
somewhere on your system.

John

[Martin Blais]

Try this:

python
>>> import sys
>>> print sys.path

This should like the directories in your actual runtime path.
The directory where the module you're attempting to import should be there.
(If not, that's a problem.)

[c b]

I don't see pyledger.so, anywhere on my system

However, I do see  the following files in ./src/CMakeFiles/libledger.dir/

  py_account.cc.o  

  py_balance.cc.o    

  py_expr.cc.o    

  pyinterp.cc.o  

  py_journal.cc.o  

  py_post.cc.o     

  py_times.cc.o  

  py_value.cc.o

  py_amount.cc.o   

  py_commodity.cc.o  

  py_format.cc.o  

  py_item.cc.o   

  pyledger.cc.o    

  py_session.cc.o  

  py_utils.cc.o  

  py_xact.cc.o

Adding this folder to PYTHONPATH doesn't help either.

Thanks,

sanjay

[Tim Crews]

On Monday, June 17, 2013 10:06:15 PM UTC-7, John Wiegley wrote:

>>>>> c b <24x7...@gmail.com> writes:

Where is pyledger.so?  If you built with Python support, it should be
somewhere on your system.

John

John,

I don't see anything that would produce this library.  In src/CMakeLists.txt, the HAVE_BOOST_PYTHON flag just causes all of the py_.cc files to be appended to the list of source files that are built into either libledger or ledger.  I don't see anything building a pyledger library.

Did this maybe get lost in the shuffle during the CMake conversion?  Looking at the git history, it looks like it's been this way from the very beginning of the CMake-based build system.

Tim Crews

[c b]

Hi Tim and John,

Thanks for looking into this. I dug around a little bit in the CMake documentation and changed src/CMakeLists.txt as follows.

$ git diff

diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt

index 2d6b22f..1ca4362 100644

--- a/src/CMakeLists.txt

+++ b/src/CMakeLists.txt

@@ -243,6 +243,7 @@ add_pch_rule(${PROJECT_BINARY_DIR}/system.hh LEDGER_SOURCES main.cc global.cc)

 

 if(BUILD_LIBRARY)

   add_library(libledger ${LEDGER_SOURCES} ${PROJECT_SOURCE_DIR}/lib/sha1.cpp)

+  add_library(pyledger SHARED ${LEDGER_SOURCES} ${PROJECT_SOURCE_DIR}/lib/sha1.cpp)

   set_target_properties(libledger PROPERTIES OUTPUT_NAME ledger)

 

   add_executable(ledger main.cc global.cc)

I now get a libpyledger.so shared library built, but adding it to PYTHONPATH and trying import ledger  in a python script still results in an error.

Any tips on how to proceed are appreciated.

Thanks,

cb

[c b]

Hi,

I made the following changes

$ git diff

diff --git a/CMakeLists.txt b/CMakeLists.txt

index fc2ef2b..bea5db2 100644

--- a/CMakeLists.txt

+++ b/CMakeLists.txt

@@ -62,7 +62,7 @@ endif()

 # Set BOOST_ROOT to help CMake to find the right Boost version

 find_package(Boost 1.46.0

-  REQUIRED date_time filesystem system iostreams regex unit_test_framework

+  REQUIRED date_time filesystem system iostreams regex unit_test_framework

   ${BOOST_PYTHON})

 include_directories(SYSTEM ${Boost_INCLUDE_DIRS})

diff --git a/src/pyledger.cc b/src/pyledger.cc

index ee7b99c..069823f 100644

--- a/src/pyledger.cc

+++ b/src/pyledger.cc

@@ -39,7 +39,7 @@ namespace ledger {

   extern void initialize_for_python();

 }

-BOOST_PYTHON_MODULE(ledger)

+BOOST_PYTHON_MODULE(libpyledger)

 {

   using namespace ledger;

I got a libpyledger.so, but upon doing an import libpyledger in python, I get the following error

ImportError: ./libpyledger.so: undefined symbol: __gmpq_add

Any ideas about this? I have absolutely no idea about this since I'm not a C++ programmer.

Thanks,

cb

[Tim Crews]

On Friday, June 21, 2013 9:44:04 PM UTC-7, c b wrote:

I got a libpyledger.so, but upon doing an import libpyledger in python, I get the following error

ImportError: ./libpyledger.so: undefined symbol: __gmpq_add

Any ideas about this? I have absolutely no idea about this since I'm not a C++ programmer.

Thanks,

cb

That's a promising development.  I'm surprised that simply changing the name in the BOOST_PYTHON_MODULE macro had this effect.

__gmpq_add is  one of the functions from the GMP library (one of ledger's dependencies), so it looks like this library did not get linked into libpyledger.so.  Unfortunately, that's actually a CMake issue, not a C++ issue, and CMake is almost a complete mystery to me.  Nevertheless, I'll risk saying something ignorant in the hopes it might help you along.

In the top-level CMakeLists.txt, it defines a macro named add_ledger_library_dependencies, and you can see it adding MPFR_LIB and GMP_LIB dependencies to the passed-in target.  This macro is used in src/CMakeLists.txt to set "ledger" as the target that these dependencies are added to.

So, I think you need to use similar directives to add the same dependencies to your python module library.  In src/CMakeLists.txt, after you use add_library to create a pyledger library, I think you should try

   target_link_libraries(pyledger, ${MPFR_LIB}
   target_link_libraries(pyledger, ${GMP_LIB}

Tim Crews

[c b]

Thanks Tim!

That did it!

After targetting MPFR_LIB and GMP_LIB to pyledger, I get libpyledger.so  that imports cleanly.

However, I now get a RunTimeError on read_journal when I try the following code

Type "help", "copyright", "credits" or "license" for more information.

>>> import libpyledger as ledger

>>> ledger.read_journal("sample.dat")

Traceback (most recent call last):

  File "<stdin>", line 1, in <module>

RuntimeError: No default scope in which to read journal file '"/tmp/sample.dat"'

Using the same sample.dat file with the ledger binary works fine.

Any ideas?

Thanks,

cb

[Stefano Zacchiroli]

[ resent, as the first one apparently didn't get through ]

On Fri, Jun 28, 2013 at 10:46:34PM -0700, c b wrote:
> That did it! After targetting MPFR_LIB and GMP_LIB to pyledger, I get
> libpyledger.so that imports cleanly.
>
> However, I now get a RunTimeError on read_journal when I try the
> following code

Hi cb, did you have any luck in making it work in the end?

I haven't been helping out much thus far (sorry about that!), but I've
followed this discussion with interest as it's related to solving this
bug http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=704980 in the
Debian ledger packaging.

Also, no matter whether the result you reached is final or not, I wonder
if the changes you found out as needed has been integrated into ledger
main development branch, so that other could benefit from them.

With many thanks for having looked into this!
Cheers.
--
Stefano Zacchiroli . . . . . . . za...@upsilon.cc . . . . o . . . o . o
Ma�tre de conf�rences . . . . . http://upsilon.cc/zack . . . o . . . o o
Former Debian Project Leader . . @zack on identi.ca . . o o o . . . o .
� the first rule of tautology club is the first rule of tautology club �

[Stefano Zacchiroli]

On Fri, Jun 28, 2013 at 10:46:34PM -0700, c b wrote:

> That did it! After targetting MPFR_LIB and GMP_LIB to pyledger, I get
> libpyledger.so that imports cleanly.
>
> However, I now get a RunTimeError on read_journal when I try the
> following code

[janp...@vanbest.org]

Hi Cb, John,

On Saturday, June 29, 2013 7:46:34 AM UTC+2, c b wrote:

RuntimeError: No default scope in which to read journal file '"/tmp/sample.dat"'

Using the same sample.dat file with the ledger binary works fine.

I'm having the same issue, and cannot find out what is happening. John, what is the difference between running 'ledger python' and doing an 'import ledger' from a regular python script or shell? Is there some way to set the default scope from Python (I guess that should be in scope_t::default_scope)?

Jan-Pascal

[John Wiegley]

>>>>> janpascal <janp...@vanbest.org> writes:

> I'm having the same issue, and cannot find out what is happening. John, what
> is the difference between running 'ledger python' and doing an 'import
> ledger' from a regular python script or shell? Is there some way to set the
> default scope from Python (I guess that should be in
> scope_t::default_scope)?

There shouldn't be much difference at all; this error doesn't ring a bell. It
sounds like a bug in the Python support to me.

John

[janp...@vanbest.org]

Hi John,

On Thursday, October 24, 2013 8:31:42 PM UTC+2, John Wiegley wrote:

There shouldn't be much difference at all; this error doesn't ring a bell.  It
sounds like a bug in the Python support to me.

Any suggestions on how to start debugging this?

[John Wiegley]

>>>>> janpascal <janp...@vanbest.org> writes:

> Any suggestions on how to start debugging this?

I don't have any at the moment, actually; I haven't looked at the Python
support code in quite a long time at this point.

John

[Alexis]

Hi everyone,
I created a pull request, which solves this issue.

I am uncertain whether this is the best way to do it, maybe someone with
a deeper understanding can comment...

Interestingly running a script with `ledger python` is much faster then
running a script which imports ledger with python. Any idea what may
cause the longer execution time?


Cheers,
Alexis