pricehist: a tool to fetch historical prices

[Chris Berkhout]

Hi all,

I've written a command-line tool to fetch daily historical prices. It currently supports 5 sources of data and will generate output in 4 formats, including ledger/hledger.

https://gitlab.com/chrisberkhout/pricehist

The ledger output looks like this:

pricehist fetch ecb EUR/USD -s 2021-06-25 -o ledger

P 2021-06-25 00:00:00 EUR 1.195 USD

P 2021-06-28 00:00:00 EUR 1.191 USD

P 2021-06-29 00:00:00 EUR 1.1888 USD

P 2021-06-30 00:00:00 EUR 1.1884 USD

but details of the formatting can be controlled with options:

pricehist fetch ecb EUR/USD -s 2021-06-25 -o ledger --fmt-base € --fmt-quote $ --fmt-symbol left --fmt-time '' --fmt-datesep /

P 2021/06/25 € $1.195

P 2021/06/28 € $1.191

P 2021/06/29 € $1.1888

P 2021/06/30 € $1.1884

The closest alternative I'm aware of is bean-price, which requires input from a Beancount file in addition to command-line options, and has a fixed output format.

If pricehist might be useful for you please try it and let me know what you think. I'm still improving error handling and some other things but the main functionality is done.

Cheers,
Chris

[Andreas Pauley]

Hi Chris,

Thanks for releasing this!

The pipx installation was seamless, and I've been fetching some Yahoo and ECB prices for the last few days (since your announcement on the ledger group). It works really well.

It is great that it can fetch historical prices from previous years - the previous time I searched for a similar tool I found one which could only display the most recent price.

Kind regards,

Andreas

--

You received this message because you are subscribed to the Google Groups "hledger" group.

To unsubscribe from this group and stop receiving emails from it, send an email to hledger+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/hledger/297d2d51-3f72-40e7-9d49-9c360163a75an%40googlegroups.com.

[Simon Michael]

Chris, thanks a lot! This is my new favourite price fetcher, by far.

(Almost) no need to set up special accounts or keys!

And I like very much that you included information about each of the sources.

Of course, something has to go wrong the first time. The numbers I see below (ignore the expected errors) seem wrong - any ideas ?

10:07:43 ~$ for s in alphavantage coindesk coinmarketcap ecb yahoopricehist; do echo $s:; pricehist fetch $s ADA/USD -s 2021-06-30 -e 2021-07-01 -o ledger; done

alphavantage:

CRITICAL The environment variable ALPHAVANTAGE_API_KEY is empty. Get a free API key from https://www.alphavantage.co/support/#api-key, export ALPHAVANTAGE_API_KEY="YOUR_OWN_API_KEY" and retry.

coindesk:

WARNING Available data covers the interval [2021-06-30--2021-06-30], which ends 1 day earlier than requested.

P 2021-06-30 00:00:00 ADA 35053.4183 USD

coinmarketcap:

WARNING No data found for the interval [2021-06-30--2021-07-01].

ecb:

P 2021-06-30 00:00:00 ADA 1.1884 USD

P 2021-07-01 00:00:00 ADA 1.1884 USD

yahoopricehist:

usage: pricehist fetch SOURCE PAIR [-h] [-t TYPE] [-s DATE | -sx DATE] [-e DATE | -ex DATE] [-o beancount|csv|gnucash-sql|ledger] [--invert] [--quantize INT] [--fmt-base SYM] [--fmt-quote SYM] [--fmt-time TIME] [--fmt-decimal CHAR] [--fmt-thousands CHAR] [--fmt-symbol rightspace|right|leftspace|left] [--fmt-datesep CHAR] [--fmt-csvdelim CHAR]

pricehist fetch: error: argument SOURCE: invalid choice: 'yahoopricehist' (choose from 'alphavantage', 'coindesk', 'coinmarketcap', 'ecb', 'yahoo')

10:07:54 ~$ for s in alphavantage coindesk coinmarketcap ecb yahoopricehist; do echo $s:; pricehist fetch $s TSLA/USD -s 2021-06-30 -e 2021-07-01 -o ledger; done

alphavantage:

CRITICAL The environment variable ALPHAVANTAGE_API_KEY is empty. Get a free API key from https://www.alphavantage.co/support/#api-key, export ALPHAVANTAGE_API_KEY="YOUR_OWN_API_KEY" and retry.

coindesk:

WARNING Available data covers the interval [2021-06-30--2021-06-30], which ends 1 day earlier than requested.

P 2021-06-30 00:00:00 TSLA 35053.4183 USD

coinmarketcap:

WARNING No data found for the interval [2021-06-30--2021-07-01].

ecb:

P 2021-06-30 00:00:00 TSLA 1.1884 USD

P 2021-07-01 00:00:00 TSLA 1.1884 USD

yahoopricehist:

usage: pricehist fetch SOURCE PAIR [-h] [-t TYPE] [-s DATE | -sx DATE] [-e DATE | -ex DATE] [-o beancount|csv|gnucash-sql|ledger] [--invert] [--quantize INT] [--fmt-base SYM] [--fmt-quote SYM] [--fmt-time TIME] [--fmt-decimal CHAR] [--fmt-thousands CHAR] [--fmt-symbol rightspace|right|leftspace|left] [--fmt-datesep CHAR] [--fmt-csvdelim CHAR]

pricehist fetch: error: argument SOURCE: invalid choice: 'yahoopricehist' (choose from 'alphavantage', 'coindesk', 'coinmarketcap', 'ecb', 'yahoo')

[Simon Michael]

On Jul 1, 2021, at 10:12 AM, Simon Michael <si...@joyful.com> wrote:

On Jul 1, 2021, at 3:22 AM, Chris Berkhout <chrisb...@gmail.com> wrote:

Hi all,

I've written a command-line tool to fetch daily historical prices. It currently supports 5 sources of data and will generate output in 4 formats, including ledger/hledger.

https://gitlab.com/chrisberkhout/pricehist

The ledger output looks like this:

pricehist fetch ecb EUR/USD -s 2021-06-25 -o ledger

P 2021-06-25 00:00:00 EUR 1.195 USD

P 2021-06-28 00:00:00 EUR 1.191 USD

P 2021-06-29 00:00:00 EUR 1.1888 USD

P 2021-06-30 00:00:00 EUR 1.1884 USD

but details of the formatting can be controlled with options:

pricehist fetch ecb EUR/USD -s 2021-06-25 -o ledger --fmt-base € --fmt-quote $ --fmt-symbol left --fmt-time '' --fmt-datesep /

P 2021/06/25 € $1.195

P 2021/06/28 € $1.191

P 2021/06/29 € $1.1888

P 2021/06/30 € $1.1884

The closest alternative I'm aware of is bean-price, which requires input from a Beancount file in addition to command-line options, and has a fixed output format.

If pricehist might be useful for you please try it and let me know what you think. I'm still improving error handling and some other things but the main functionality is done.

Chris, thanks a lot! This is my new favourite price fetcher, by far.

(Almost) no need to set up special accounts or keys!

And I like very much that you included information about each of the sources.

Of course, something has to go wrong the first time. The numbers I see below (ignore the expected errors) seem wrong - any ideas ?

Sorry, a better paste:

10:14:31 ~$ for s in alphavantage coindesk coinmarketcap ecb yahoo; do printf "\n$s:\n"; pricehist fetch $s TSLA/USD -s 2021-06-29 -e 2021-07-01 -o ledger; done

alphavantage:

CRITICAL The environment variable ALPHAVANTAGE_API_KEY is empty. Get a free API key from https://www.alphavantage.co/support/#api-key, export ALPHAVANTAGE_API_KEY="YOUR_OWN_API_KEY" and retry.

coindesk:

WARNING Available data covers the interval [2021-06-29--2021-06-30], which ends 1 day earlier than requested.

P 2021-06-29 00:00:00 TSLA 35918.88 USD

P 2021-06-30 00:00:00 TSLA 35053.4183 USD

coinmarketcap:

WARNING Available data covers the interval [2021-06-30--2021-06-30], which starts 1 day later and ends 1 day earlier than requested.

P 2021-06-30 00:00:00 TSLA 684.024836145 USD

ecb:

P 2021-06-29 00:00:00 TSLA 1.1888 USD

P 2021-06-30 00:00:00 TSLA 1.1884 USD

P 2021-07-01 00:00:00 TSLA 1.1884 USD

yahoo:

P 2021-06-29 00:00:00 TSLA 680.760010 USD

P 2021-06-30 00:00:00 TSLA 679.700012 USD

P 2021-07-01 00:00:00 TSLA 677.919983 USD

10:14:57 ~$ for s in alphavantage coindesk coinmarketcap ecb yahoo; do printf "\n$s:\n"; pricehist fetch $s ADA/USD -s 2021-06-29 -e 2021-07-01 -o ledger; done

alphavantage:

CRITICAL The environment variable ALPHAVANTAGE_API_KEY is empty. Get a free API key from https://www.alphavantage.co/support/#api-key, export ALPHAVANTAGE_API_KEY="YOUR_OWN_API_KEY" and retry.

coindesk:

WARNING Available data covers the interval [2021-06-29--2021-06-30], which ends 1 day earlier than requested.

P 2021-06-29 00:00:00 ADA 35918.88 USD

P 2021-06-30 00:00:00 ADA 35053.4183 USD

coinmarketcap:

WARNING Available data covers the interval [2021-06-30--2021-06-30], which starts 1 day later and ends 1 day earlier than requested.

P 2021-06-30 00:00:00 ADA 1.34259020 USD

ecb:

P 2021-06-29 00:00:00 ADA 1.1888 USD

P 2021-06-30 00:00:00 ADA 1.1884 USD

P 2021-07-01 00:00:00 ADA 1.1884 USD

yahoo:

WARNING No data found for the interval [2021-06-29--2021-07-01].

The coindesk and ecb numbers seem wrong in both cases. I guess unsupported commodities return junk data right now.

[Simon Michael]

One more, for a fiat currency:

10:21:17 ~$ for s in alphavantage coindesk coinmarketcap ecb yahoo; do printf "\n$s:\n"; pricehist fetch $s EUR/USD -s 2021-06-29 -e 2021-07-01 -o ledger; done

alphavantage:

CRITICAL The environment variable ALPHAVANTAGE_API_KEY is empty. Get a free API key from https://www.alphavantage.co/support/#api-key, export ALPHAVANTAGE_API_KEY="YOUR_OWN_API_KEY" and retry.

coindesk:

WARNING Available data covers the interval [2021-06-29--2021-06-30], which ends 1 day earlier than requested.

P 2021-06-29 00:00:00 EUR 35918.88 USD

P 2021-06-30 00:00:00 EUR 35053.4183 USD

coinmarketcap:

Traceback (most recent call last):

  File "/opt/homebrew/bin/pricehist", line 8, in <module>

    sys.exit(cli())

  File "/opt/homebrew/lib/python3.9/site-packages/pricehist/cli.py", line 64, in cli

    result = fetch(series, source, output, args.invert, args.quantize, fmt)

  File "/opt/homebrew/lib/python3.9/site-packages/pricehist/fetch.py", line 12, in fetch

    series = source.fetch(series)

  File "/opt/homebrew/lib/python3.9/site-packages/pricehist/sources/coinmarketcap.py", line 53, in fetch

    for item in data["data"]["quotes"]:

KeyError: 'quotes'

ecb:

P 2021-06-29 00:00:00 EUR 1.1888 USD

P 2021-06-30 00:00:00 EUR 1.1884 USD

P 2021-07-01 00:00:00 EUR 1.1884 USD

[Chris Berkhout]

Hey there,

Thanks for the feedback Andreas & Simon!

Simon, thanks for doing those checks! They show two issues I'm working on now:
1) coinmarketcap (and others) don't gracefully handle a response with no data (coinmarketcap doesn't do EUR/USD).

2) ecb and coindesk always use EUR and BTC respectively as the base, but will output the requested base without checking that they match.

I opened issues for those in GitLab: https://gitlab.com/chrisberkhout/pricehist/-/issues

I should have them resolved soon.

The message about AlphaVantage explains itself, and the warnings about date coverage are expected because of some sources not having weekend prices and today's price not always being available. The next version won't warn if the only thing missing is today's price.

If you find other things please don't hesitate to open a GitLab issue.

Cheers,

Chris

On Thu, 1 Jul 2021 at 22:23, Simon Michael <si...@joyful.com> wrote:

Chris, thanks a lot! This is my new favourite price fetcher, by far.

(Almost) no need to set up special accounts or keys!

And I like very much that you included information about each of the sources.

Of course, something has to go wrong the first time. The numbers I see below (ignore the expected errors) seem wrong - any ideas ?

...

[Simon Michael]

Awesome! I look forward to using it.

--
You received this message because you are subscribed to the Google Groups "hledger" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hledger+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/hledger/CAPekLyErVKKcGij8opVfq7dDuYWLjALR3M4%2B6GxRx_xkXqt4Cw%40mail.gmail.com.