Currently having a problem with beanprice

[Samuel W. Flint]

For the following call to bean-price (and for similar):

bean-price -e "USD:google/AAPL"

I get the following error:

ERROR : Could not fetch for job: DatedPrice(base='AAPL', quote='USD', date=None, sources=[PriceSource(module=<module 'beancount.prices.sources.google' from '/nix/store/gmvvcvbg7dlrd8c13c9sp0jqk63cc3pj-beancount-2.0b15/lib/python3.6/site-packages/beancount/prices/sources/google.py'>, symbol='AAPL', invert=False)])

Is there anything that I could possibly do?

Thanks,

Sam

--
Samuel W. Flint
4096R/266596F4
(9477 D23E 389E 40C5 2F10 DE19 68E5 318E 2665 96F4)
λs.(s s) λs.(s s)

[Justus Pendleton]

See the bug report I filed a week ago. Google quotes are gone for good, with no workaround I know. A new price source needs to be written.

[yegle]

I think this is doable using Google App Script, similar to this PR https://github.com/bitly/oauth2_proxy/pull/497

Sample Google App Script:

```
function GetStockPrice() {
  var files = DriveApp.searchFiles("name = 'beancount_get_stock_price' and mimeType ='" + MimeType.GOOGLE_SHEETS + '"');
  var cell;
  if (files === null) {
    cell = SpreadsheetApp.open(files.next()).getActiveCell();
  } else {
    cell = SpreadsheetApp.create('beancount_get_stock_price').getActiveCell();
  }
  cell.setFormula('=GoogleFinance("GOOG", "price")');
  Logger.log(cell.getValue());
}
```

On Fri, Dec 8, 2017 at 5:17 AM, Justus Pendleton <just...@gmail.com> wrote:

See the bug report I filed a week ago. Google quotes are gone for good, with no workaround I know. A new price source needs to be written.

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscribe@googlegroups.com.
To post to this group, send email to bean...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/02cd79f2-98c5-4ffd-9983-f6d88f84c663%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Yuchen Ying
https://about.me/yegle

[Martin Blais]

Both sources gone indeed. Makes me a bit sad.

It would be real nice to find a good replacement source of free delayed market data quotes to pull from.

yegle@'s App Script is a good idea re. Google Finance quotes. 

Another similar idea, which perhaps would require a bit less setup by the user having to click for permissions and enabling APIs and creating this "app script" and such would be to "programmatically" create a Sheets docs using the Sheets API and insert =GOOGLEFINANCE() calls in the spreadsheet and let it evaluate all the cells at once. This could be done with a Python script, not entirely unlike this:

https://bitbucket.org/blais/beancount/src/de867919655ed57158579b6d8d6dce98ead0706c/beancount/tools/sheets_upload.py?at=default&fileviewer=file-view-default

Basically have the script

- create a new sheets doc in your account (automatically)

- insert formulas

- wait until it updates the cell values (how to know if it's done? Not sure)

- download the values locally

- delete the doc

But maybe that's a bit of a Rube Goldberg machine...

Perhaps ultimately the best thing to do would be to write price fetchers straight from the exchanges websites, e.g. NASDAQ, NYSE, etc.

http://www.nasdaq.com/symbol/vti

For our pedestrian and very personal finance usage delayed data - which is what Y. and G. were providing anyway - scraped off the web page is totally fine IMO; I'd ignore the expensive API and just scrape NASDAQ or NYSE perhaps.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/CAFL5w3VXd26FVsou-%2BbDDxLeWE8ad2Q3Jw20%2BboUfhpcG%2BJ4_A%40mail.gmail.com.

[Martin Blais]

BTW... just occurred to me... maybe this could be a watershed moment.

Think about it: ALL those old Perl and Python libraries out there are fetching data from mainly Yahoo and Google APIs.

Now that these are gone, they're ALL going to break, as well as a ton of software using those.

Maybe it's time to create a brand new project, to support a very many data sources...

Say, in Python3, with type annotations, and a really simple and nice API...

Just an idea.

[jus...@ego-pulse.com]

I written a Morningstar price fetcher which I've been using for a few weeks: https://github.com/hoostus/beancount-price-sources

One nice benefit (for me) is they aggregate multiple exchanges, especially non-US ones.

[Martin Blais]

That's great! 

Added to the contribs list

On Sun, Dec 10, 2017 at 8:21 AM, <jus...@ego-pulse.com> wrote:

I written a Morningstar price fetcher which I've been using for a few weeks: https://github.com/hoostus/beancount-price-sources

One nice benefit (for me) is they aggregate multiple exchanges, especially non-US ones.

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscribe@googlegroups.com.
To post to this group, send email to bean...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/0d9f77bf-b3e2-4da6-b599-e3b85062661e%40googlegroups.com.

[redst...@gmail.com]

Thanks for this, very useful! A couple questions:

1) How do I get bean-price to use this by default? I don't use commodity directives in my sources

2) Is there a way to automatically determine exchange and ticker with morningstar so I don't have to manually find it for each ticker I have?

Thanks again for sharing this!

[Martin Blais]

The commodity directive allows you to specify the price source.

e.g.

2001-05-24 commodity XSP

  name: "iShares S&P 500 Index Fund (CAD Hedged)"

  assets: "Stock"

  price: "CAD:google/TSE:XSP"

  export: "TSE:XSP"

  quote: CAD

So instead of "google/" here you'd use the name of the morningstar Python library.

--

You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscribe@googlegroups.com.
To post to this group, send email to bean...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/18853291-34f2-4efd-adee-e77edd66dd87%40googlegroups.com.

[Justus Pendleton]

On Monday, January 15, 2018 at 8:51:51 AM UTC+7, redst...@gmail.com wrote:

1) How do I get bean-price to use this by default? I don't use commodity directives in my sources 

You have to use commodity directives.

 

2) Is there a way to automatically determine exchange and ticker with morningstar so I don't have to manually find it for each ticker I have?

Nope. The simple (and obvious when you think about it) answer is that tickers aren't globally unique, they are simply unique to an exchange. (Because why would exchanges care if a symbol is globally unique or not as long as it is unique on their exchange?) Since tickers aren't globally unique, how do you "automatically" determine an exchange and ticker?

Take "VEU" as an example. Do you mean the Vanguard FTSE All World ex-US ETF that is listed on ARCX in the US? Or the one listed on the ASX in Australia? Or the ones that Vanguard is about to list on exchanges in Europe now that they've expanded abroad?

Is SPY the SPY on ARCX or the ASX?

Is GOOGL ("Alphabet Inc A") the one on XNAS (quoted in USD) or XMIL (quoted in Euros)?

Is GE the one on the NYSE or the one on XMIL (GE Gefran SpA)?

And so on. If your answer is "well, I'm an American so it should default to picking all the things an American would pick....", I'm willing to accept a --american patch but I'm Australian. :)

[redst...@gmail.com]

On Monday, January 15, 2018 at 7:13:37 PM UTC-8, Justus Pendleton wrote:

On Monday, January 15, 2018 at 8:51:51 AM UTC+7, redst...@gmail.com wrote: 

2) Is there a way to automatically determine exchange and ticker with morningstar so I don't have to manually find it for each ticker I have?

Nope. The simple (and obvious when you think about it) answer is that tickers aren't globally unique, they are simply unique to an exchange. (Because why would exchanges care if a symbol is globally unique or not as long as it is unique on their exchange?) Since tickers aren't globally unique, how do you "automatically" determine an exchange and ticker?

Take "VEU" as an example. Do you mean the Vanguard FTSE All World ex-US ETF that is listed on ARCX in the US? Or the one listed on the ASX in Australia? Or the ones that Vanguard is about to list on exchanges in Europe now that they've expanded abroad?

I was simply looking for a drop-in replacement for  the erstwhile yahoo/google price lookups. I just about realized yesterday that tickers aren't globally unique. So what you have makes sense. I'm assuming a --country= patch could resolve the exchange when possible, but manual commodity entries will suffice for myself for now. Thanks again!

[Martin Blais]

On Mon, Jan 15, 2018 at 10:13 PM, Justus Pendleton <just...@gmail.com> wrote:

On Monday, January 15, 2018 at 8:51:51 AM UTC+7, redst...@gmail.com wrote:

1) How do I get bean-price to use this by default? I don't use commodity directives in my sources 

You have to use commodity directives.

 

2) Is there a way to automatically determine exchange and ticker with morningstar so I don't have to manually find it for each ticker I have?

Nope. The simple (and obvious when you think about it) answer is that tickers aren't globally unique, they are simply unique to an exchange. (Because why would exchanges care if a symbol is globally unique or not as long as it is unique on their exchange?) Since tickers aren't globally unique, how do you "automatically" determine an exchange and ticker?

Take "VEU" as an example. Do you mean the Vanguard FTSE All World ex-US ETF that is listed on ARCX in the US? Or the one listed on the ASX in Australia? Or the ones that Vanguard is about to list on exchanges in Europe now that they've expanded abroad?

Is SPY the SPY on ARCX or the ASX?

Is GOOGL ("Alphabet Inc A") the one on XNAS (quoted in USD) or XMIL (quoted in Euros)?

Is GE the one on the NYSE or the one on XMIL (GE Gefran SpA)?

That's just one dimension of uniqueness; the other is where do you fetch the price?

As in, do you fetch the price of ARCX:SPY from Google or Yahoo?

So basically, the right thing to do here, since each user has a limited number of commodities, is simply to use the Commodity directives and be explicit about where you want your prices to come from.

 

And so on. If your answer is "well, I'm an American so it should default to picking all the things an American would pick....", I'm willing to accept a --american patch but I'm Australian. :)

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscribe@googlegroups.com.
To post to this group, send email to bean...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/bfa83ffb-2676-4439-ba13-5ec486bb4ce8%40googlegroups.com.

[Martin Blais]

On Sun, Jan 14, 2018 at 9:28 PM, <redst...@gmail.com> wrote:

Thanks, and sorry if I didn't make this clear: I prefer not to use commodity directives, so everything can be automatically picked up by using --undeclared with bean-price.

It seems like the price source has been hardcoded to default to yahoo if I'm not mistaken. So perhaps I'd have to do something like do some parsing to automatically produce commodity directives, and then get prices on those?

There's no option to change the default at the moment, but one could easily be added. I think it should.

 

On Sunday, January 14, 2018 at 6:12:26 PM UTC-8, Martin Blais wrote:

The commodity directive allows you to specify the price source.

e.g.

2001-05-24 commodity XSP

  name: "iShares S&P 500 Index Fund (CAD Hedged)"

  assets: "Stock"

  price: "CAD:google/TSE:XSP"

  export: "TSE:XSP"

  quote: CAD

So instead of "google/" here you'd use the name of the morningstar Python library.

On Sun, Jan 14, 2018 at 8:51 PM, <redst...@gmail.com> wrote:

Thanks for this, very useful! A couple questions:

1) How do I get bean-price to use this by default? I don't use commodity directives in my sources

2) Is there a way to automatically determine exchange and ticker with morningstar so I don't have to manually find it for each ticker I have?

Thanks again for sharing this!

On Sunday, December 10, 2017 at 5:21:07 AM UTC-8, jus...@ego-pulse.com wrote:

I written a Morningstar price fetcher which I've been using for a few weeks: https://github.com/hoostus/beancount-price-sources

One nice benefit (for me) is they aggregate multiple exchanges, especially non-US ones.

--
You received this message because you are subscribed to the Google Groups "Beancount" group.

To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.

To post to this group, send email to bean...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/18853291-34f2-4efd-adee-e77edd66dd87%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscribe@googlegroups.com.
To post to this group, send email to bean...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/e377cf48-fb23-47ce-8f7c-d55cef5ed5be%40googlegroups.com.

[Martin Blais]

https://bitbucket.org/blais/beancount/issues/210/add-option-to-set-default-price-source-for

[keesj...@google.com]

I'm having some trouble getting the morningstar source to work. I'm probably doing something stupid, but can't really find what, so maybe someone can give a pointer.

I installed the library by cloning https://github.com/hoostus/beancount-price-sources.git and then running python setup.py install.

But when I try to run the examples I get an error:

$ bean-price -e 'USD:hoostus_sources.morningstar/etfs:XASX:VEU'

usage: bean-price [-h] [-e] [-v] [-d DATE] [-i] [-u] [-c] [-a] [-s] [-n]

                  [--cache CACHE_FILENAME] [--no-cache] [--clear-cache]

                  sources [sources ...]

bean-price: error: Invalid source "USD:hoostus_sources.morningstar/etfs:XASX:VEU"

Any ideas?

cheers,

Kees

https://bitbucket.org/blais/beancount/issues/210/add-option-to-set-default-price-source-for

[Martin Blais]

Are you running python3 setup.py install?

Are you using the same version of Python which runs bean-price?

To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscribe@googlegroups.com.

To post to this group, send email to bean...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/2b62ee12-ddc3-4093-97be-aadb21387567%40googlegroups.com.

[Kees]

Yeah, both are python3. I got it working. It seemed to have to do with the underscore in the name. I looked at the bean-price code and the regex doesn't seem to allow for underscores in the module name.