Help Needed with bean-price for Stock Prices

[Errol Lee]

Hi everyone,

I’m new to Beancount (and honestly, I’m amazed by the amount of functionality it offers—wish I had discovered it sooner!). I’m trying to set up bean-price to fetch stock prices, but I’ve run into some issues, particularly with Yahoo and Alphavantage as sources.

Here’s what I’ve tried so far:

1. Yahoo:

It seems like Yahoo has blocked API access for fetching stock prices. When I try fetching prices for AAPL, I get the following error:

❯ bean-price -e 'USD:yahoo/AAPL'                 

ERROR   : Error fetching AAPL: Status 401: {'code': 'Unauthorized', 'description': 'User is unable to access this feature - https://bit.ly/yahoo-finance-api-feedback'}

ERROR   : Could not fetch for job: DatedPrice(base='AAPL', quote='USD', date=None, sources=[PriceSource(module=<module 'beancount.prices.sources.yahoo' from '/opt/homebrew/lib/python3.11/site-packages/beancount/prices/sources/yahoo.py'>, symbol='AAPL', invert=False)])

2. Alphavantage:

Despite being listed as a source in the GitHub documentation, I can’t seem to get it working. Here’s the error I encounter:

❯ bean-price -e 'USD:alphavantage/AAPL'          

Traceback (most recent call last):

  File "/opt/homebrew/lib/python3.11/site-packages/beancount/prices/price.py", line 178, inimport_source

    __import__(default_name)

ModuleNotFoundError: No module named 'beancount.prices.sources.alphavantage'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "/opt/homebrew/lib/python3.11/site-packages/beancount/prices/price.py", line 182, inimport_source

    __import__(module_name)

ModuleNotFoundError: No module named 'alphavantage'

Ultimately, I’d like to fetch prices for some Australian stocks in my portfolio (e.g., ASX:VDHG, ASX:VGS, etc.).

Does anyone have advice on:

• Workarounds for Yahoo?

• How to enable or properly configure Alphavantage as a source?

• Any other sources I can use for Australian stocks?

Thanks in advance for your help! Any guidance would be much appreciated.

[Patrick Ruckstuhl]

Hi Errol,

Did you install beanprice separately?

https://github.com/beancount/beanprice

I'm using alphavantage on a daily basis.

You will need to set an environment variable ALPHAVANTAGE_API_KEY with a free api key from them and use it like

USD:alphavantage/price:AAPL:USD

Regards,
Patrick

[Errol Lee]

Hi Patrick,

Thank you for your guidance! I thought I did, but I must've done something wrong earlier. I've reinstalled and can now successfully retrieve prices using Yahoo. 

For example:

❯ bean-price -e 'AUD:yahoo/VDHG.AX'

2024-12-27 price VDHG.AX                             69.53 AUD

I’ve defined my commodities in a commodities.bean file. For instance, VDHG is defined as follows:

2000-01-01 commodity VDHG

  name: "Vanguard Diversified High Growth ETF"

  asset-class: "Equity"

  symbol: "VDHG"

  price: "AUD:yahoo/VDHG.AX"

However, when I run the following command, it seems to execute successfully, but I don’t see the prices updated in any of my files:

❯ bean-price commodities.bean

Ideally, I’d like the latest prices to be saved automatically in a prices.bean file. Is there something I’m missing in my setup or an additional step I need to take to ensure the prices are recorded in the desired file?

Thanks again for your help!

Best regards,

Errol

[Patrick Ruckstuhl]

Hi Errol,

if I remember correctly it depends not on the commodities but if you have actually any balances using the instruments.

For updating a file, this is what I use. I have a prices folder with a file per instrument.

#!/usr/bin/env python3

from beanprice import price
from beancount import loader
from beancount.parser import printer

entries, errors, options = loader.load_file('main.beancount')

priceJobs = price.get_price_jobs_at_date(entries, inactive=True)
prices = []
for job in priceJobs:
    try:
        prices.append(price.fetch_price(job))
    except Exception as e:
        print("failed to fetch, ", job, e)

prices, ignoredEntries = price.filter_redundant_prices(prices, entries)
for newPrice in prices:
    with open('prices/' + newPrice.currency.lower() + '.beancount', 'a') as priceFile:
        priceFile.write(printer.format_entry(newPrice))

Regards,

Patrick

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/beancount/1e797d83-4d9c-4951-8609-a20781fb5164n%40googlegroups.com.

[Errol Lee]

Thanks for the input. I was hoping to use one of the functions that came with beanprice. These steps in the documentation is what I was trying to replicate. Seems like you've written your own script.

Now that I can use bean-price to fetch the individual stocks, I'll probably try to do the same..

[Vasily]

Hi Errol,

To expand a bit on Patrick's reply.
Indeed, beanprice won't save values to file automatically but you don't have to necessarily write a script (well, it's actually still a shell scripting but a bit simpler). You can append result to an existing beancount file included in your ledger as in:

bean-price -i -c commodities.bean >> prices.bean

Note the -i and -c parameters that make sure values are fetched a little more "deterministically" (check out bean-price --help to see what they actually mean).
Instead of using include prices.bean in the main ledger you can include files using a wildcard, like include "prices/*.bean" and then store prices in the separate files like:

bean-price -i -c commodities.bean >> "prices/price-$(date +'%Y-%m-%d').bean"

You can store them splitted by date as in this example but you can also come up with the way to split by instrument/stock if necessary (well, here I'd also probably opt for using Python instead of bash).

I've automated this process a bit in Lazy Beancount: https://lazy-beancount.xyz/docs/stage1_totals/currencies/#conversion-rates, maybe it'd work for you off the shelf as well. It's nothing too fancy, however, just literally doing what's described here with a UI on top.

Also I've recently stumbled across https://gitlab.com/chrisberkhout/pricehist and looks like it can solve a couple more problems with price fetching (e.g. fetching prices for time intervals) but I haven't tested it yet personally.

Best regards,
Vasily

суббота, 28 декабря 2024 г. в 15:40:15 UTC+1, Errol Lee:

[Errol Lee]

Thanks Vasily for the tips. I've wrote a simple script that meet my requirements for now (I don't have many holdings). I've run this manually now by changing the ticker and output file values - given I only have 5 holdings. This works for me.

Will also check out these other commands and packages. Thanks!

import subprocess

import re

from datetime import datetime, timedelta

def fetch_and_process_price(ticker: str, date: str) -> str:

# Run the bash command

result = subprocess.run(['bean-price', '-e', f'--date={date}', ticker], capture_output=True, text=True)

# Check if the command was successful

if result.returncode == 0:

# Process the output

output = result.stdout

# Use regex to remove ".AX" from any text before it

output = re.sub(r'(\w+)\.AX', r'\1', output)

# Remove multiple spaces before the dollar value

output = ' '.join(output.split())

return f"{output}"

else:

print(f"Error: {result.stderr}")

return ""

def append_to_file(content: str, filename: str):

with open(filename, 'a') as file:

file.write(content + '\n')

# Generate dates for every Wednesday for the last 3 years

def generate_wednesdays():

end_date = datetime.now()

start_date = end_date - timedelta(weeks=3*52)

current_date = start_date

wednesdays = []

while current_date <= end_date:

if current_date.weekday() == 2: # Wednesday

wednesdays.append(current_date.strftime('%Y-%m-%d'))

current_date += timedelta(days=1)

return wednesdays

# Example usage

dates = generate_wednesdays()

ticker = 'AUD:yahoo/ETH-AUD'

for date in dates:

price_entry = fetch_and_process_price(ticker, date)

if price_entry:

append_to_file(price_entry, 'ETH-AUD.bean')