Transitioning from Quicken
766 views
Skip to first unread message
edwin...@gmail.com
Mar 5, 2016, 7:16:12 PM3/5/16
to Beancount
For the past few years, I've been searching for something to replace Quicken. I'm not a big fan of Intuit, but haven't found anything better. I've tried GnuCash and Moneydance and they did not handle downloading data from banks well. Nor did they handle the import of more than 15 years of data from Quicken well. While not strictly required, it'd would be really nice if I can import most, if not all, of my data, but not being able to download my account data well is really what stopped me from using either.
I just recently discovered this class of plain text accounting software (Ledger, Beancount) and it really appeals to me, more so I think because I am a programmer. I have high hopes Beancount will be the replacement I've been looking for, even if I have to script and occasionally fix the account downloads. But there is surprisingly little about transitioning from Quicken to Beancount on this group. Maybe this audience just knew better and avoided Quicken in the first place? But I'd be interested in hearing about how anyone made the transition. I'm definitely going to try in the coming weeks/months as time allows.
I have a few questions though:
1. For importing existing data from Quicken, is importing a QFX the best way? Or importing individual QIF files per account? Any stories about how well it worked or didn't work?
2. What does your periodic data download process look like? One of my attempts to replace Quicken even involved writing a program that will automatically download account data from all my accounts, but I was stymied by certain banks' javascript-based logins. For Beancount, do you manually log into all your accounts and manually gather all the OFX files and then initiate something that'll slurp it all into Beancount? Or were you able to get solve the problem of auto-logging into all bank accounts in order to download/scrape data?
I hope to use Beancount alongside Quicken for a period of time in the near future and make the switchover when I'm confident Beancount will work for me (and confident that I can work Beancount). I see a lot of posts about how to use Beancount here, which is great to see and reinforces my decision to try Beancount, but would also love to see stories about how people made a switchover from Quicken (or other software) to Beancount! Thanks all for listening.
Martin Blais
Mar 5, 2016, 7:35:02 PM3/5/16
to Beancount
On Sat, Mar 5, 2016 at 4:16 PM, <edwin...@gmail.com> wrote:
For the past few years, I've been searching for something to replace Quicken. I'm not a big fan of Intuit, but haven't found anything better.
I quite like their TurboTax product.
I've tried GnuCash and Moneydance and they did not handle downloading data from banks well. Nor did they handle the import of more than 15 years of data from Quicken well. While not strictly required, it'd would be really nice if I can import most, if not all, of my data, but not being able to download my account data well is really what stopped me from using either.
What comes out of Quicken?
If it's an open format, it should be possible to write an importer for it into Beancount.
I just recently discovered this class of plain text accounting software (Ledger, Beancount) and it really appeals to me, more so I think because I am a programmer. I have high hopes Beancount will be the replacement I've been looking for, even if I have to script and occasionally fix the account downloads. But there is surprisingly little about transitioning from Quicken to Beancount on this group. Maybe this audience just knew better and avoided Quicken in the first place? But I'd be interested in hearing about how anyone made the transition. I'm definitely going to try in the coming weeks/months as time allows.
I have a very basic QIF importer that uses a modified version of qifparse.
I can share that with you if you like.
I also have a working OFX/QFX parser that I've used for years.
These things used to be embedded in LedgerHub, but for a number of reasons I've decided to kill that project and move the useful generic workflow bits in beancount.ingest.
See http://furius.ca/ledgerhub/doc/postmortem for details.
I have a few questions though:1. For importing existing data from Quicken, is importing a QFX the best way? Or importing individual QIF files per account? Any stories about how well it worked or didn't work?
Never tried.
I don't have example data to work off of, I'd like to include something like this eventually.
One person proposed to share their data so I can write the importer, I haven't followed up on it yet.
2. What does your periodic data download process look like?
One of my attempts to replace Quicken even involved writing a program that will automatically download account data from all my accounts, but I was stymied by certain banks' javascript-based logins. For Beancount, do you manually log into all your accounts and manually gather all the OFX files and then initiate something that'll slurp it all into Beancount?
Yes. I've never solved this problem. I think it's a huge PIA and requires constant manicuring when things fail or when they change (over a bunch of accounts, they change every couple of months/quarters). I don't feel it's worth my time... I just log in manually and download all the files, following some notes I wrote in the past for each account, takes <5 minutes. Then I run identify/extract/file.
Or were you able to get solve the problem of auto-logging into all bank accounts in order to download/scrape data?
It should be solvable for some accounts using the network part of the OFX standard.
I hope to use Beancount alongside Quicken for a period of time in the near future and make the switchover when I'm confident Beancount will work for me (and confident that I can work Beancount).
Indeed, best way to proceed IMO. Write a converter and treat your Quicken file as the pristine source for a while. Once you're feeling convinced, change it over.
(Note: It would be really fun to be able to work it the other way around, e.g. given a Beancount input file, generate Quicken-compatible QFX that can be imported.)
I see a lot of posts about how to use Beancount here, which is great to see and reinforces my decision to try Beancount, but would also love to see stories about how people made a switchover from Quicken (or other software) to Beancount! Thanks all for listening.
I'd love to hear such a story too!
--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To post to this group, send email to bean...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/7bc23c98-5b78-41f8-984d-13e234abaced%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Ed Wingate
Mar 5, 2016, 7:59:52 PM3/5/16
to Beancount
On Saturday, March 5, 2016 at 4:35:02 PM UTC-8, Martin Blais wrote:
What comes out of Quicken?If it's an open format, it should be possible to write an importer for it into Beancount.
There are options in Quicken to export everything as a QFX or each individual accounts as QIF. Not sure if the QFX includes old closed account data or not. Sounds like I can try the OFX/QFX importer you talk about below, or if necessary, the QIF importer.
I have a very basic QIF importer that uses a modified version of qifparse.I can share that with you if you like.I also have a working OFX/QFX parser that I've used for years.
2. What does your periodic data download process look like?
Thanks for this reference. Gives me a good idea of what I'm in for.
Or were you able to get solve the problem of auto-logging into all bank accounts in order to download/scrape data?It should be solvable for some accounts using the network part of the OFX standard.
You can initiate it from your own software at home? You don't need to be another bank or institution to use it?
I also want to say your documentation in Google Docs is excellent. Better than some, if not most, commercial software!
Martin Blais
Mar 5, 2016, 8:29:55 PM3/5/16
to Beancount
On Sat, Mar 5, 2016 at 4:59 PM, Ed Wingate <edwin...@gmail.com> wrote:
On Saturday, March 5, 2016 at 4:35:02 PM UTC-8, Martin Blais wrote:What comes out of Quicken?If it's an open format, it should be possible to write an importer for it into Beancount.There are options in Quicken to export everything as a QFX or each individual accounts as QIF. Not sure if the QFX includes old closed account data or not. Sounds like I can try the OFX/QFX importer you talk about below, or if necessary, the QIF importer.
Yep, but you'll almost assuredly have to write at least some code.
NOTE: You really need to use my fork of qifparse; the original is broken. It fixes a couple of bugs. I sent a pull request way back but AFAIK it was never merged in.
I have a very basic QIF importer that uses a modified version of qifparse.I can share that with you if you like.I also have a working OFX/QFX parser that I've used for years.2. What does your periodic data download process look like?Thanks for this reference. Gives me a good idea of what I'm in for.Or were you able to get solve the problem of auto-logging into all bank accounts in order to download/scrape data?It should be solvable for some accounts using the network part of the OFX standard.You can initiate it from your own software at home? You don't need to be another bank or institution to use it?
AFAIK this should be possible, but I've never tried it myself.
I also want to say your documentation in Google Docs is excellent. Better than some, if not most, commercial software!
Hey thanks a lot! :-)
You're welcome to add in suggestions or comments in the margins on the bits you find aren't clear.
ch...@gstaff.org
Mar 5, 2016, 9:55:09 PM3/5/16
to Beancount
On Saturday, March 5, 2016 at 7:16:12 PM UTC-5, Ed Wingate wrote:
I hope to use Beancount alongside Quicken for a period of time in the near future and make the switchover when I'm confident Beancount will work for me (and confident that I can work Beancount). I see a lot of posts about how to use Beancount here, which is great to see and reinforces my decision to try Beancount, but would also love to see stories about how people made a switchover from Quicken (or other software) to Beancount! Thanks all for listening.
I did something similar last month.
* Exported 21 years of Quicken data into a single QIF
* Imported into MoneyDance because I can't convince DW to use a text-based system. Fixed a bunch of categories that have drifted over the years and other mistakes we'd made.
* Moneydance is now the one true source
* Export a single QIF from MoneyDance
* Wrote a QIF parser and converter to generate a Beancount file
* Use fava, Beancheck, and all the awesome Beancount tools to find more errors
Martin, I'd like to 2nd Ed's comment about the docs. They're excellent!
Martin Blais
Mar 5, 2016, 10:40:41 PM3/5/16
to Beancount
Thank you :-)
(What is "DW"?)
This is the way to go IMO: Choose some system as the one true source and automate all outgoing conversions. This is also what I suggest to Ledger users kicking the tires on Beancount. There's no need to take a giant leap of faith.
How interesting you're using the QIF format for conversion. It's nice and simple though, simpler than OFX. You know it's deprecated, right? Apparently Quicken won't import it anymore. They only import QFX, which is OFX + some extensions, and only if some money has been paid to them (see Wikipedia), I just learned this.
In any case, I'm convinced your Quicken -> QIF and QIF -> Beancount converters could be useful to others! :-)
ch...@gstaff.org
Mar 5, 2016, 11:06:35 PM3/5/16
to Beancount
(What is "DW"?)
A term I picked up on some personal finance sites. DW = dear wife, DH = dear husband.
How interesting you're using the QIF format for conversion. It's nice and simple though, simpler than OFX. You know it's deprecated, right? Apparently Quicken won't import it anymore. They only import QFX, which is OFX + some extensions, and only if some money has been paid to them (see Wikipedia), I just learned this.
Agreed, QIF is very simple. A bit ambiguous in spots, but easier to convert than plenty of other old formats.
GnuCash's notes were very helpful https://github.com/Gnucash/gnucash/blob/master/src/import-export/qif-imp/file-format.txt
I didn't know it was deprecated. I'm OK with that since I never plan on moving back to Quicken.
Even less so, now that they've been sold.
Moneydance still imports and exports it, so good enough.
In any case, I'm convinced your Quicken -> QIF and QIF -> Beancount converters could be useful to others! :-)
I'll see what I can generalize - like most converters, there's a lot of "just for me" conversions in there.
There's probably some useful code trying to find its way out of a sea of special cases. :-)
Ed Wingate
Mar 6, 2016, 4:14:12 AM3/6/16
to Beancount
So I installed Beancount and went through the Tutorial & Example. When I bean-check the example.beancount generated by bean-example, I get a lot of "Transaction does not balance: (0.01 USD)", all of them coming from Hooli Payroll transactions. One such is pasted below. Is the output from bean-example just out of date and not in sync with the capabilities of bean-check?
/root/example/example.beancount:4337: Transaction does not balance: (0.01 USD)
2016-02-25 * "Hooli" | "Payroll"
Assets:US:BofA:Checking 1350.61 USD
Assets:US:Vanguard:Cash 1200.00 USD
Assets:US:Federal:PreTax401k -1200.00 IRAUSD
Expenses:Taxes:Y2016:US:Federal:PreTax401k 1200.00 IRAUSD
Income:US:Hooli:Salary -4615.38 USD
Income:US:Hooli:GroupTermLife -24.32 USD
Expenses:Health:Life:GroupTermLife 24.32 USD
Expenses:Health:Dental:Insurance 2.90 USD
Expenses:Health:Medical:Insurance 27.38 USD
Expenses:Health:Vision:Insurance 42.30 USD
Expenses:Taxes:Y2016:US:Medicare 106.62 USD
Expenses:Taxes:Y2016:US:Federal 1062.92 USD
Expenses:Taxes:Y2016:US:State 365.08 USD
Expenses:Taxes:Y2016:US:CityNYC 174.92 USD
Expenses:Taxes:Y2016:US:SDI 1.12 USD
Expenses:Taxes:Y2016:US:SocSec 281.54 USD
Assets:US:Hooli:Vacation 5 VACHR
Income:US:Hooli:Vacation -5 VACHR
Martin Blais
Mar 6, 2016, 4:17:25 AM3/6/16
to Beancount
Interesting.
I can't seem to reproduce this.
(Moreover, I have an unit test that automatically checks for this as well.)
Could you share one of the failing generated files?
--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To post to this group, send email to bean...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/d2711816-a428-4aa3-b52a-0a72ed4d254c%40googlegroups.com.
Ed Wingate
Mar 6, 2016, 4:51:35 AM3/6/16
to Beancount
On Sunday, March 6, 2016 at 1:17:25 AM UTC-8, Martin Blais wrote:
Interesting.I can't seem to reproduce this.(Moreover, I have an unit test that automatically checks for this as well.)Could you share one of the failing generated files?
I'm running:
Ubuntu 14.04.2 LTS, Trusty Tahr
Python 3.4.3
Kernel: Linux 4.2.6-1-pve #1 SMP Wed Dec 9 10:49:55 CET 2015 x86_64 x86_64 x86_64 GNU/Linux
Also, output from running bean-example:
# bean-example > example.beancount
INFO : Generating Salary Employment Income
INFO : Generating Expenses from Banking Accounts
INFO : Generating Regular Expenses via Credit Card
INFO : Generating Credit Card Expenses for Trips
INFO : Generating Credit Card Payment Entries
INFO : Generating Tax Filings and Payments
INFO : Generating Opening of Banking Accounts
INFO : Generating Transfers to Investment Account
INFO : Generating Prices
INFO : Generating Employer Match Contribution
INFO : Generating Retirement Investments
INFO : Generating Taxes Investments
INFO : Generating Expense Accounts
INFO : Generating Equity Accounts
INFO : Generating Balance Checks
INFO : Outputting and Formatting Entries
INFO : Contextualizing to Realistic Names
INFO : Writing contents
INFO : Validating Results
<string>:3217: Transaction does not balance: (0.01 USD)
2014-01-02 * "Hooli" | "Payroll"
Assets:US:BofA:Checking 1350.61 USD
Assets:US:Vanguard:Cash 1200.00 USD
Assets:US:Federal:PreTax401k -1200.00 IRAUSD
Expenses:Taxes:Y2014:US:Federal:PreTax401k 1200.00 IRAUSD
Income:US:Hooli:Salary -4615.38 USD
Income:US:Hooli:GroupTermLife -24.32 USD
Expenses:Health:Life:GroupTermLife 24.32 USD
Expenses:Health:Dental:Insurance 2.90 USD
Expenses:Health:Medical:Insurance 27.38 USD
Expenses:Health:Vision:Insurance 42.30 USD
Expenses:Taxes:Y2014:US:Medicare 106.62 USD
Expenses:Taxes:Y2014:US:Federal 1062.92 USD
Expenses:Taxes:Y2014:US:State 365.08 USD
Expenses:Taxes:Y2014:US:CityNYC 174.92 USD
Expenses:Taxes:Y2014:US:SDI 1.12 USD
Expenses:Taxes:Y2014:US:SocSec 281.54 USD
Assets:US:Hooli:Vacation 5 VACHR
Income:US:Hooli:Vacation -5 VACHR
[snipped more "Transaction does not balance" blocks]
Daniël Bos
Mar 6, 2016, 6:13:12 AM3/6/16
to bean...@googlegroups.com
Very strange indeed. I can reproduce this issue with my distro's Python-3.4.3 and beancount-2.0b7. (I had to manually upgrade my distro's httplib2 by the way)
However, when I use Python-3.5.1 with exactly the same beancount source, the problem goes away.
--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To post to this group, send email to bean...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/a3935d86-9719-470a-b5cd-6a85431ee4bb%40googlegroups.com.
ch...@gstaff.org
Mar 6, 2016, 9:15:19 AM3/6/16
to Beancount
Python version 3.4.3 with cdecimal has the problem Ed reported.
Python version 3.4.3 without cdecimal works fine.
ch...@gstaff.org
Mar 6, 2016, 9:22:19 AM3/6/16
to Beancount
To narrow down the issue I used a seed of 1.
Here's what bean-example produces without cdecimal.
2014-01-01 * "Opening Balance for checking account"
Assets:US:BofA:Checking 3496.22 USD
Equity:Opening-Balances -3496.22 USD
Here's what bean-example produces with cdecimal.
2014-01-01 * "Opening Balance for checking account"
Assets:US:BofA:Checking 3496.08 USD
Equity:Opening-Balances -3496.08 USD
I'm guessing a bug in cdecimal. :)
ch...@gstaff.org
Mar 6, 2016, 11:24:08 AM3/6/16
to Beancount, ch...@gstaff.org
Here's a smaller reproducible example:
from beancount.core.number import D
import decimal
a = D(120000)
bw = a / 26
print(bw - D(1200))
with decimal.localcontext() as ctx:
ctx.prec = 6
print(bw - D(1200))
Python version 3.4.3 with cdecimal prints
3415.384615384615384615384615
3415.384615384615384615384615
Python version 3.4.3 without cdecimal prints
3415.384615384615384615384615
3415.38
The version installed by pip3 is 2.3, while the latest version is 2.4.2. Might be a bug in mpdecimal.
I'll try upgrading.
Ed Wingate
Mar 6, 2016, 4:52:56 PM3/6/16
to Beancount
On Sunday, March 6, 2016 at 6:15:19 AM UTC-8, ch...@gstaff.org wrote:
Python version 3.4.3 with cdecimal has the problem Ed reported.Python version 3.4.3 without cdecimal works fine.
I upgraded to Python 3.5.1 and the problem went away as well.
Martin Blais
Mar 8, 2016, 12:14:24 AM3/8/16
to Beancount
3.5 with cdecimal?
(Just confirming that the problem was fixed.)
--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To post to this group, send email to bean...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/51fec791-1c38-46e8-b152-08c49e7686c3%40googlegroups.com.
Ed Wingate
Mar 8, 2016, 2:26:12 AM3/8/16
to Beancount
On Monday, March 7, 2016 at 9:14:24 PM UTC-8, Martin Blais wrote:
3.5 with cdecimal?(Just confirming that the problem was fixed.)
Yes, with cdecimal.
Ed Wingate
Mar 9, 2016, 5:27:40 PM3/9/16
to Beancount
On Saturday, March 5, 2016 at 4:35:02 PM UTC-8, Martin Blais wrote:
I have a very basic QIF importer that uses a modified version of qifparse.I can share that with you if you like.
Martin,
I'm at the point in my Beancount journey that I want to try importing my Quicken data. Could you share the QIF importer you mentioned here with me?
I also have a working OFX/QFX parser that I've used for years.
Could you share this also? Is it just a parser, or is there a surrounding importer as well?
Thanks,
Ed
Martin Blais
Mar 10, 2016, 1:34:40 AM3/10/16
to Beancount
Here's a quick copy:
http://furius.ca/tmp/generic-importers/These aren't clean enough for inclusion into Beancount yet, and they do not have tests.
Note: When I say those importer codes is "ugly", this is an example of what I mean.
- I've used the OFX converter for a variety of credit cards.
- I've used the QIF for a single account with little transactions - it's not going to be very useful to you.
My plans:
- I'll clean up the OFX one properly, add tests, and include that in Beancount.
- I probably won't do that with the QIF one... QIF is deprecated, and I don't have much test data to do it.
- I'll write a configurable CSV importer.
So in the end, Beancount will contain a CSV and an OFX importers, mainly as working examples.
In addition, I'm currently writing an example of how to configure and write your own importer.
Look for the source code here (WARNING: this will be evolving over the coming week):
--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To post to this group, send email to bean...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/0b289815-1f1b-4d5c-91a1-85efc877424a%40googlegroups.com.
Ed Wingate
Mar 17, 2016, 2:41:37 AM3/17/16
to Beancount
On Wednesday, March 9, 2016 at 10:34:40 PM UTC-8, Martin Blais wrote:
Here's a quick copy:http://furius.ca/tmp/generic-importers/
So I discovered Quicken exports not QFX (a superset of OFX) like I originally thought, but QXF (an encrypted proprietary format). The OFX importer obviously won't work. Quicken does export QIF, so it looks like I'll have to write my own QIF importer/converter.
Martin Blais
Mar 17, 2016, 10:41:51 PM3/17/16
to Beancount
The old qif importer available here should get you started:
http://furius.ca/tmp/generic-importers/It's VERY basic.
Note: you will need to install my fork of qifparse:
(the version upstream is broken)
The spec is here:
I think it might be worthwhile to include one of these in Beancount at some point.
--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.
To post to this group, send email to bean...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/ea13fe17-7b50-4aa0-8d4b-a78001f5cf45%40googlegroups.com.
Ed Wingate
Mar 18, 2016, 3:10:50 AM3/18/16
to Beancount
On Thursday, March 17, 2016 at 7:41:51 PM UTC-7, Martin Blais wrote:
The old qif importer available here should get you started:http://furius.ca/tmp/generic-importers/
Thanks, I'm also looking at the ofx importer here: https://bitbucket.org/blais/beancount/src/559b6413c9eb0960767b553958ed981ae702e77f/src/python/beancount/ingest/importers/ofx.py
Should I try to write my qif importer to subclass importer.ImporterProtocol rather than compat.Importer like the old qif importer?
This qif importer I want to write is conceptually doing the same thing as the importers in the beancount.ingest module, is that correct?
Martin Blais
Mar 18, 2016, 10:20:13 AM3/18/16
to Beancount
On Fri, Mar 18, 2016 at 3:10 AM, Ed Wingate <edwin...@gmail.com> wrote:
On Thursday, March 17, 2016 at 7:41:51 PM UTC-7, Martin Blais wrote:The old qif importer available here should get you started:http://furius.ca/tmp/generic-importers/Thanks, I'm also looking at the ofx importer here: https://bitbucket.org/blais/beancount/src/559b6413c9eb0960767b553958ed981ae702e77f/src/python/beancount/ingest/importers/ofx.pyShould I try to write my qif importer to subclass importer.ImporterProtocol rather than compat.Importer like the old qif importer?
Yes.
compat.Importer is only there for those who have used LedgerHub in the past and I'll be removing this transitional base class and move most of its functionality into mixins soon.
This qif importer I want to write is conceptually doing the same thing as the importers in the beancount.ingest module, is that correct?
Yes.
Reply all
Reply to author
Forward
0 new messages