bean-price has now moved to https://github.com/beancount/beanprice

[Martin Blais]

Hi,

As discussed prior on this mailing-list, bean-price has now moved away from the beancount repository to a new repository under the same umbrella organization:

https://github.com/beancount/beanprice

The code has been removed from beancount/master, yet it lingers in beancount/v2 (where the plan is to minimize disruption).

All new tickets related to bean-price will be redirected to this new repository; existing tickets in Beancount will be migrated.

New price sources are welcome into the new repo. Barrier to entry is a bit lower (since I'm not planning to do all the maintenance work myself), basically if you can write unit tests for a new price source that call both the latest and historical functions (without actually making requests to the network), compliant with the API and the style guide, your new price source is likely welcome, send a PR. This may be a good time to consolidate some of your custom price sources into one place if you want.

Note that in terms of code dependencies bean-price still depends on beancount.ops.find_prices, where routines to enumerate commodities for price enumeration remain. I plan to keep maintaining code the enumerates prices required from a Beancount ledger there, as they are purely processing Beancount directives. If they need to be extended for bean-price, patches are welcome. I'm not too sure how importing from the v3 Bazel build will work yet, but I've kept the BUILD files for now, as they will perhaps become necessary when the C++ build becomes the norm there.

Note that the new beanprice repo could be extended to work for Ledger and HLedger as well; after all, the output is just text to be inserted in one's ledger file, and only a change of output syntax.

[Daniele Nicolodi]

On 25/07/2020 11:14, Martin Blais wrote:
> Hi,
> As discussed prior on this mailing-list, bean-price has now moved away
> from the beancount repository to a new repository under the same
> umbrella organization:
>
> https://github.com/beancount/beanprice
>
> The code has been removed from beancount/master, yet it lingers in
> beancount/v2 (where the plan is to minimize disruption).
> All new tickets related to bean-price will be redirected to this new
> repository; existing tickets in Beancount will be migrated.

Would it make sense to have v2 depend on the new project instead that
having the code in two places? beancount v2 setup.py could restrict the
version to (for example) 'beanprice < 2.0' so that incompatible changes
can be pushed to another branch.

This only requires publishing beanprice to PyPI and declaring the
dependency in setup.py (and merge the patch that removes beanprice from
master to the v2 branch). I can do that if there is agreement about this
way forward.

I am assuming that beanprice still install a bean-price executable.

Cheers,
Dan

[Kirill Goncharov]

>New price sources are welcome into the new repo. Barrier to entry is a bit lower (since I'm not planning to do all the maintenance work myself), basically if you can write unit tests for a new price source that call both the latest and historical functions (without actually making requests to the network), compliant with the API and the style guide, your new price source is likely welcome, send a PR. This may be a good time to consolidate some of your custom price sources into one place if you want.

This might not be a good idea, because keeping different sources in one repo would require additional coordination and increase burden on the maintainers. Personally, I will continue to maintain my sources separately.

[Martin Blais]

I thought about removing it from v2, and doing the same for report/web, but I kind-of feel that until v3 has at least a "beta" level base to work from, it's probably best not to mess too much with v2, current users should be able to just keep working without having to make changes.

 

I am assuming that beanprice still install a bean-price executable.

Yes. 

[Daniele Nicolodi]

On 25/07/2020 13:14, Martin Blais wrote:
> On Sat, Jul 25, 2020 at 2:23 PM Daniele Nicolodi <dan...@grinta.net

> Would it make sense to have v2 depend on the new project instead that
> having the code in two places?  beancount v2 setup.py could restrict the
> version to (for example) 'beanprice < 2.0' so that incompatible changes
> can be pushed to another branch.
>
> This only requires publishing beanprice to PyPI and declaring the
> dependency in setup.py (and merge the patch that removes beanprice from
> master to the v2 branch). I can do that if there is agreement about this
> way forward.
>
>
> I thought about removing it from v2, and doing the same for report/web,
> but I kind-of feel that until v3 has at least a "beta" level base to
> work from, it's probably best not to mess too much with v2, current
> users should be able to just keep working without having to make changes.

Unless I am missing something, there would not be any user visible
change other than one more package downloaded and installed when 'pip
install beancount' is run.

Cheers,
Dan

[Martin Blais]

That's assuming people aren't running from source.

I assume many users here run directly from source code and it's probably a minority who pip install.

I don't really know TBH.

I think of v2 as "frozen" at this point, more or less, new development will occur on v3. As long as the changes are trivially mergeable to v3 and introduce only very minimal changes (e.g. bug fixes), I probably will do them on the v2 branch, but anything interesting will happen on v3. There's no point in breaking things on v2. Let's break things in v3 (and make progress).

[Stefano Zacchiroli]

This announcement by Martin made me look again into bean-price (at the
moment I'm using horrible hand-written scrapers for my bank websites).

What I'd be interested in is (1) a service that allow to fetch prices
based on ISINs (as in:
https://en.wikipedia.org/wiki/International_Securities_Identification_Number
) and (2) a bean-price data source for it.

(2) would be trivial to add, but I haven't found a solution for (1) yet.
It seems that neither Google Finance nor Yahoo Finance offer that
service, although I found that very surprising.

Does anyone know of a (free) service allowing to lookup current prices
by ISIN?

Thanks in advance,
Cheers

> --
> You received this message because you are subscribed to the Google Groups "Beancount" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/CAK21%2BhO98B71fjdp6RmH5z2e9-aSGntrJjnzceSesEkavavurA%40mail.gmail.com.

--
Stefano Zacchiroli . za...@upsilon.cc . upsilon.cc/zack . . o . . . o . o
Computer Science Professor . CTO Software Heritage . . . . . o . . . o o
Former Debian Project Leader & OSI Board Director . . . o o o . . . o .
« the first rule of tautology club is the first rule of tautology club »

[Kirill Goncharov]

>I assume many users here run directly from source code and it's probably a minority who pip install.

I think that the opposite is true.

According to this site, beancount package has ~50 downloads on average per day: https://pypistats.org/packages/beancount , many of these downloads are probably coming from Fava installations.

I also use only packaged stable version.

[Patrick Ruckstuhl]

Hi Stefano,

It's unfortunately quite painful to find free pricing information for
stocks and etf.

I'm using https://www.alphavantage.co/ for some of it, but it got worse
lately (the don't have Swiss Traded Stocks/ETF anymore).

What I now did was switch to take the prices from my broker (which
provides an api) but this only works with credentials.


For currencies and crypto currencies it's easier as there are more
public apis available for that.


My price sources are here https://github.com/tarioch/beancounttools


Regards,

Patrick

[Martin Blais]

What about this: In version 2.4 (we're now at 2.3.1), a minor version, I moved the reporting code, the web app, the price fetching script, and possibly the ingest library out of the main repo. Is that reasonable? I don't think so.

What could make sense, is to make that a major revision (v3) - just v2 with the tools removed - and the new work in C++ would eventuawlly become v4. I feel like waiting until the new version has at least some meat before bothering people currently using v2.

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/7e4f68fc-af1d-4cd7-9a45-2e29bbafea71n%40googlegroups.com.

[Kirill Goncharov]

I agree, moving these components out of the main repo should result in a major version change. The idea of dividing the whole process in two stages (v3 and v4) seems reasonable too.

[Daniele Nicolodi]

On 25/07/2020 20:24, Martin Blais wrote:
> That's assuming people aren't running from source.
> I assume many users here run directly from source code and it's probably
> a minority who pip install.

That's does not make much of a difference, also running 'python setup.py
install' or 'python setup.py develop' installs dependencies. The only
case in which this does not work is when someone builds the project in
place with 'python setup.py build' and runs the binaries from the source
directory. However, if someone makes this work, I don't think installing
one extra package is much work.

Cheers,
Dan

[Pranesh Prakash]

On Sunday, 26 July, 2020 at 2:06:50 pm UTC+5:30 Stefano Zacchiroli wrote:

Does anyone know of a (free) service allowing to lookup current prices
by ISIN?

I'm not sure if necro-bumping is frowned down upon in this mailing list. If so, my apologies.  I just came across this and thought I'd share some comments, since I didn't see a direct response to this question.

At one point I was trying to do the same: I use ISIN as the commodity symbol for all my mutual funds, ETFs, stocks, and bonds, and I was trying to automate downloading of prices via an API.

I eventually realized this wasn't quite that simple (though there is a script for this, that I've linked to at the end).  

A single ISIN can translate into multiple ticker symbols in multiple exchanges.  An ISIN only identifies the security but not the exchange.  The same security might be traded in multiple exchanges across different countries valued in different currencies and at slightly different prices (the price differences wouldn't be too large, else there'd be opportunities for arbitrage).  (And while there is a standard code for exchanges (called MIC, or market identification code), I don't know of many who actually use that.)

At one point, I was looking at converting ISIN into ticker symbols from the exchange I wanted information from, and then doing the look-up, but I realized that if you need to be specifying the exchange, one might as well directly get information from the exchange if it provides it.  That's what I ended up doing with small scripts like, which get information from the primary source:

https://github.com/the-solipsist/scripts/blob/master/ledger_get-indian-stock-prices_bse.sh

https://github.com/the-solipsist/scripts/blob/master/ledger_get-indian-mf-prices.sh

and so on.

There are tools (such as OpenFIGI) to get ticker symbols from ISIN.  So you could convert from ISIN into the ticker and then use sources like Alphavantage or Yahoo Finance.  The tool pricehist supports both, and can output to beancount format:

https://gitlab.com/chrisberkhout/pricehist

And there are also sources (such as Morningstar's API) which given an ISIN and an exchange, can provide you data.  Here's a Github project that makes use of Morningstar's API with ISIN:

https://github.com/LunaticMuch/msdownloader

I hope that helps.

Cheers,

Pranesh

[redst...@gmail.com]

Nice, comprehensive rundown, thank you Pranesh. It would be helpful to have your post be easily referenced in a place like the plaintextaccounting wiki or elsewhere. Would you consider contributing your post to that wiki?

[Stefano Zacchiroli]

On Sun, Apr 10, 2022 at 09:12:22AM -0700, Pranesh Prakash wrote:
> I'm not sure if necro-bumping is frowned down upon in this mailing list. If
> so, my apologies. I just came across this and thought I'd share some
> comments, since I didn't see a direct response to this question.

This is super-useful and following up to old threads helps in preserving
contexts for future searches. Thanks a lot!

--
Stefano Zacchiroli . za...@upsilon.cc . upsilon.cc/zack _. ^ ._
Full professor of Computer Science o o o \/|V|\/
Télécom Paris, Polytechnic Institute of Paris o o o </> <\>
Co-founder & CTO Software Heritage o o o o /\|^|/\
Former Debian Project Leader & OSI Board Director '" V "'

[redst...@gmail.com]

Added here. Feel free to edit: https://github.com/plaintextaccounting/plaintextaccounting/wiki/Looking-up-current-prices-by-ISIN