Beancount on PyPi

[Martin Michlmayr]

2.0rc1 is almost a year old. Would it be possible to cut an rc2 since
there have been a lot of changes in the meantime?

This would help those not installing from mercurial directly.

Thanks,

Reference:
https://pypi.org/project/beancount/#history
--
Martin Michlmayr
http://www.cyrius.com/

[Martin Blais]

How would you feel about me baking an official 2.0 stamp on the current tip?

I was waiting to finish the doc integration to do this and adopt semantic versioning.

I don't think I will wait to close all the tickets before I make numbered releases. 

It's been pretty stable and I don't have enough bandwidth these days.

Here's what I see is next in terms of "small stuff" that would be part of near upcoming 2.x releases:

- Integrate Dominik's docs as a static copy.

- Adding coherent cmdline options to disable the caches (pickle cache, price source cache, implement import cache for costly conversions).

- Remove the Holding class (less code); replace all its functionality by aggregation of positions. Rewrite all the reports with SQL queries only.

- Promote list_holdings.py (the script I use myself) to the main codebase and share an example Google Sheets doc to track assets (that's what I've been doing for years, works great).

- Figure out what to do about speed (we need something like a 3x speedup IMO).

- Do something about price sources now that G and Y finance are gone (perhaps just provide a helpers library instead of a tool, not sure)

This is small-ish stuff that gets on my own nerves regularly.

Here's some stuff that's being requested regularly by others that I also would like to work on:

- Adding options to override / take priority over inference for precision and allow you to set the num digits to render directly.

- Take a solid look at the CSV importer contributions waiting in PRs.

- Implement AVERAGE booking.

- Implement query assertions to have a good budget story / take time to check out Fava's custom directive for that, maybe integrate.

- Supporting UTF8

and of course, slowly work my way through all those tickets and pending PRs.

There's work for three dozen lifetimes (I don't suggest wading through the TODO file, you'll need courage and it's organized like a 14 year old's bedroom). Overall I'd like to reduce the amount of code (deprecate Holdings, perhaps delete bean-web (because Fava, if it can be baked), factor out SQL query client to a configurable row-based query tool that works on any data source) and focus on doing more with a smaller body of code (e.g. compute returns, support complex risk calculations (I hold options), implement pivot table in SQL for e.g. monthly reports, make the shell so powerful it's the only way to generate reports, simplify the tags/links schema, etc.).

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscribe@googlegroups.com.
To post to this group, send email to bean...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/20180323140005.4y4rquuqigi7mgts%40jirafa.cyrius.com.
For more options, visit https://groups.google.com/d/optout.

[Dominik Aumayr]

> I don't think I will wait to close all the tickets before I make numbered releases.
> It's been pretty stable and I don't have enough bandwidth these days.
>
> Here's what I see is next in terms of "small stuff" that would be part of near upcoming 2.x releases:

> This is small-ish stuff that gets on my own nerves regularly.

I think that laying out the "roadmap" as clearly as you did below does help contributors to pick things that they then know that you are also interested in, and work on those things.

Personally I think now that the project feels like gaining more and more followers/users the amount of PRs, issues, etc. likely won't decline.

Automatically converting the docs to HTML format is a very good step for improving discoverability (HTML can be search-indexed, etc.), as well as keeping to Google Docs as the "UI"/mechanism for editing them keeps the bar low for contributions to the docs.

As for the code, moving to git (and github.com/beancount/beancount? We already "own" that org) would further lower the barriers to help/contribute for many more people. Please (you, dear reader) do not take this as a starting-point for another lengthy discussion about this. I just want to friendly remind you, Martin, what possible ways of lowering the amount of work beancount takes up could be.

- Dominik

> To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.

> To post to this group, send email to bean...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/20180323140005.4y4rquuqigi7mgts%40jirafa.cyrius.com.
> For more options, visit https://groups.google.com/d/optout.
>
>

> --
> You received this message because you are subscribed to the Google Groups "Beancount" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.

> To post to this group, send email to bean...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/CAK21%2BhM_sHSBRpGVhi-GTeYd-yCKwdjePpigCsSdXRDQK3afiw%40mail.gmail.com.

[Stefano Zacchiroli]

On Sat, Mar 24, 2018 at 01:30:09AM -0400, Martin Blais wrote:
> How would you feel about me baking an official 2.0 stamp on the
> current tip? I was waiting to finish the doc integration to do this
> and adopt semantic versioning.
>
> I don't think I will wait to close all the tickets before I make
> numbered releases. It's been pretty stable and I don't have enough
> bandwidth these days.

The current version feels indeed stable enough to be stamped 2.0. But.
Pushing a new release to PyPi will probably make the tip version (as
opposed to the last rc1) get some more exposure from users that are not
using tip. So it might make sense to release current tip as rc2 to PyPi,
wait 2 weeks (to see if anything important shows up), then stamp 2.0.


My (semantically versioned) 0.0.2 EUR,
Cheers
--
Stefano Zacchiroli . za...@upsilon.cc . upsilon.cc/zack . . o . . . o . o
Computer Science Professor . CTO Software Heritage . . . . . o . . . o o
Former Debian Project Leader & OSI Board Director . . . o o o . . . o .
« the first rule of tautology club is the first rule of tautology club »

[Martin Michlmayr]

* Dominik Aumayr <dom...@aumayr.name> [2018-03-24 11:10]:

> As for the code, moving to git (and github.com/beancount/beancount?
> We already "own" that org) would further lower the barriers to
> help/contribute for many more people. Please (you, dear reader) do
> not take this as a starting-point for another lengthy discussion
> about this. I just want to friendly remind you, Martin, what
> possible ways of lowering the amount of work beancount takes up
> could be.

I was actually going to ask Martin about this in private (in private
because I didn't want to start a bike-shedding discussion).

I've submitted some minor pull requests and while hg is not too hard
if you know git, remembering the right hg commands and signing up for
bitbucket is a small barrier that imho could easily be avoided by
moving to GitHub.

If Martin says no, we should accept that without further discussion.
But if it's a maybe or yes, I'd be happy to look into the migration
(unless someone has experience with this already).

There is a script that can be used to import bitbucket issues into
GitHub:
https://github.com/jeffwidman/bitbucket-issue-migration/blob/master/migrate.py

And the mercurial repo is mirrored already.

[Martin Michlmayr]

* Martin Blais <bl...@furius.ca> [2018-03-24 01:30]:

> How would you feel about me baking an official 2.0 stamp on the current tip?

I was actually planning to send an email to ask about the status of
2.0, what needs to be done and how people can help. :)

> I don't think I will wait to close all the tickets before I make numbered
> releases.
> It's been pretty stable and I don't have enough bandwidth these days.
>
> Here's what I see is next in terms of "small stuff" that would be part of
> near upcoming 2.x releases:

Personally, I'd make another rc release now, prioritize some items for
2.0 (as you did), encourage people to submit fixes for 2.0 and make
2.0 in a few months. This assumes that you can make some time to
review patches, accept pull requests and possibly fix some of the
harder issues yourself.

Of course, you could release 2.0 now and do a 2.x in a few months
but I feel some stuff is missing from what I'd call a "2.0". I'm
still exploring beancount but for example UTF-8 support in account
names is crucial for some users (like our French friends :).

So maybe do an rc now and set a date for the 2.0 release (3 months
from now?) to give people a chance to get fixes/features in without
delaying 2.0 for too long.

> - Do something about price sources now that G and Y finance are gone
> (perhaps just provide a helpers library instead of a tool, not sure)

Someone on this list posted the following in December
https://github.com/hoostus/beancount-price-sources
which supports Morningstar and OpenExchange.
(I also saw anotehr OpenExchange implementation on GitHub.)

There's a pull request for Quandl.

I haven't used an price source yet but maybe these would be
alternatives.

Thanks for creating beancount, btw!

[Zhuoyun Wei]

2018-03-24 06:46:31 Stefano Zacchiroli <za...@upsilon.cc>:

> On Sat, Mar 24, 2018 at 01:30:09AM -0400, Martin Blais wrote:
> > How would you feel about me baking an official 2.0 stamp on the
> > current tip? I was waiting to finish the doc integration to do this
> > and adopt semantic versioning.
> >
> > I don't think I will wait to close all the tickets before I make
> > numbered releases. It's been pretty stable and I don't have enough
> > bandwidth these days.
>
> The current version feels indeed stable enough to be stamped 2.0. But.
> Pushing a new release to PyPi will probably make the tip version (as
> opposed to the last rc1) get some more exposure from users that are not
> using tip. So it might make sense to release current tip as rc2 to PyPi,
> wait 2 weeks (to see if anything important shows up), then stamp 2.0.
>
>

I agree with Stefano. We have so many changes since RC1, we should stamp
another "release candidate" before 2.0.

If Martin is okay with moving the repository to GitHub, as Dominik and
many others suggest, we could release RC2 directly to GitHub. This could
greatly speed up the progress towards 2.0 because of more contributors.

--
Zhuoyun Wei

[Martin Blais]

On Sat, Mar 24, 2018 at 11:27 AM, Martin Michlmayr <t...@cyrius.com> wrote:

* Martin Blais <bl...@furius.ca> [2018-03-24 01:30]:
> How would you feel about me baking an official 2.0 stamp on the current tip?

I was actually planning to send an email to ask about the status of
2.0, what needs to be done and how people can help. :)

> I don't think I will wait to close all the tickets before I make numbered
> releases.
> It's been pretty stable and I don't have enough bandwidth these days.
>
> Here's what I see is next in terms of "small stuff" that would be part of
> near upcoming 2.x releases:

Personally, I'd make another rc release now, prioritize some items for
2.0 (as you did), encourage people to submit fixes for 2.0 and make
2.0 in a few months.  This assumes that you can make some time to
review patches, accept pull requests and possibly fix some of the
harder issues yourself.

Of course, you could release 2.0 now and do a 2.x in a few months
but I feel some stuff is missing from what I'd call a "2.0".  I'm
still exploring beancount but for example UTF-8 support in account
names is crucial for some users (like our French friends :).

So maybe do an rc now and set a date for the 2.0 release (3 months
from now?) to give people a chance to get fixes/features in without
delaying 2.0 for too long.

Well the slowing down for 2.0 release was something like 3-6 months ago. 

I haven't changed a whole lot since, mainly doing bug fixes.

I feel that "waiting period" has been spent.

Maybe it's time.

 

> - Do something about price sources now that G and Y finance are gone
> (perhaps just provide a helpers library instead of a tool, not sure)

Someone on this list posted the following in December
https://github.com/hoostus/beancount-price-sources
which supports Morningstar and OpenExchange.
(I also saw anotehr OpenExchange implementation on GitHub.)

There's a pull request for Quandl.

Yes, I'm aware. It does require to add a dependency on an external library though.

 

I haven't used an price source yet but maybe these would be
alternatives.

Thanks for creating beancount, btw!

Thanks!

 

--
Martin Michlmayr
http://www.cyrius.com/

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscribe@googlegroups.com.
To post to this group, send email to bean...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/20180324152726.war4cucvzomxn7zr%40jirafa.cyrius.com.

[Saša Janiška]

Martin Blais <bl...@furius.ca> writes:

Hello Martin,

> - Implement query assertions to have a good budget story / take time
> to check out Fava's custom directive for that, maybe integrate.

Can you provide some more info about it? Is that feature provide support
for 'envelope budgeting' as it is available in (h)ledger?

> - Supporting UTF8

Any rough estimate when it might happen?


Sincerely,
Gour

--
It is far better to discharge one's prescribed duties, even though
faultily, than another's duties perfectly. Destruction in the course
of performing one's own duty is better than engaging in another's duties,
for to follow another's path is dangerous.

[Jason Chu]

On Fri, Mar 23, 2018 at 10:30 PM Martin Blais <bl...@furius.ca> wrote:

How would you feel about me baking an official 2.0 stamp on the current tip?

I was waiting to finish the doc integration to do this and adopt semantic versioning.

I don't think I will wait to close all the tickets before I make numbered releases. 

It's been pretty stable and I don't have enough bandwidth these days.

Here's what I see is next in terms of "small stuff" that would be part of near upcoming 2.x releases:

- Integrate Dominik's docs as a static copy.

- Adding coherent cmdline options to disable the caches (pickle cache, price source cache, implement import cache for costly conversions).

- Remove the Holding class (less code); replace all its functionality by aggregation of positions. Rewrite all the reports with SQL queries only.

- Promote list_holdings.py (the script I use myself) to the main codebase and share an example Google Sheets doc to track assets (that's what I've been doing for years, works great).

- Figure out what to do about speed (we need something like a 3x speedup IMO).

The patch for Inventory as a dict will speed up anything to do with inventories...

https://bitbucket.org/blais/beancount/issues/191/beancountcoreinventory-should-be-a-dict

- Do something about price sources now that G and Y finance are gone (perhaps just provide a helpers library instead of a tool, not sure)

This is small-ish stuff that gets on my own nerves regularly.

Here's some stuff that's being requested regularly by others that I also would like to work on:

- Adding options to override / take priority over inference for precision and allow you to set the num digits to render directly.

- Take a solid look at the CSV importer contributions waiting in PRs.

- Implement AVERAGE booking.

- Implement query assertions to have a good budget story / take time to check out Fava's custom directive for that, maybe integrate.

- Supporting UTF8

and of course, slowly work my way through all those tickets and pending PRs.

There's work for three dozen lifetimes (I don't suggest wading through the TODO file, you'll need courage and it's organized like a 14 year old's bedroom). Overall I'd like to reduce the amount of code (deprecate Holdings, perhaps delete bean-web (because Fava, if it can be baked), factor out SQL query client to a configurable row-based query tool that works on any data source) and focus on doing more with a smaller body of code (e.g. compute returns, support complex risk calculations (I hold options), implement pivot table in SQL for e.g. monthly reports, make the shell so powerful it's the only way to generate reports, simplify the tags/links schema, etc.).

On Fri, Mar 23, 2018 at 10:00 AM, Martin Michlmayr <t...@cyrius.com> wrote:

2.0rc1 is almost a year old.  Would it be possible to cut an rc2 since
there have been a lot of changes in the meantime?

This would help those not installing from mercurial directly.

Thanks,

Reference:
https://pypi.org/project/beancount/#history

--
Martin Michlmayr
http://www.cyrius.com/

--
You received this message because you are subscribed to the Google Groups "Beancount" group.

To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.

To post to this group, send email to bean...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/20180323140005.4y4rquuqigi7mgts%40jirafa.cyrius.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Beancount" group.

To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.

To post to this group, send email to bean...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/CAK21%2BhM_sHSBRpGVhi-GTeYd-yCKwdjePpigCsSdXRDQK3afiw%40mail.gmail.com.

[Martin Michlmayr]

* Martin Michlmayr <t...@cyrius.com> [2018-03-24 16:14]:

> I've submitted some minor pull requests and while hg is not too hard

Ok, I take that back. hg is quite a bit if pain, actually. I just
spent 30 minutes trying to figure out how to update a pull request
after getting all confused by hg and multiple heads on a branch...

[Martin Blais]

That's unfair. The fact that you're unfamiliar with it doesn't mean it's a pain.

Perhaps it's a pain for you now, but imagine the pains of a Mercurial user wrangling some of the crazy problems which occur frequently with Git (much worse).

I'm not going to once again go over the technical reasons for this, but Mercurial is in many ways a success over Git, which is why some of the larger integrations have chosen it over Git (e.g. look at what Facebook has done, and I'm aware of other such large integrations and have discussed the details with some of the developers carrying them out at a conference, they're convincing technical reasons). Don't let your OSS perspective skew your vision. Github might be more popular, but so was VHS.

--
You received this message because you are subscribed to the Google Groups "Beancount" group.

To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscribe@googlegroups.com.

To post to this group, send email to bean...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/20180327183754.jhikyhomm2iu2moe%40jirafa.cyrius.com.

[Martin Blais]

Definitely not planning github, and also don't believe that's the impediment to more contributors.

The larger impediment to faster and more/faster contributions is writing good unit tests.

Writing complete and thorough unit tests is the best way to ensure a quickly merged PR.

[Martin Blais]

On Sat, Mar 24, 2018 at 2:00 AM, Jason Chu <xen...@gmail.com> wrote:

On Fri, Mar 23, 2018 at 10:30 PM Martin Blais <bl...@furius.ca> wrote:

How would you feel about me baking an official 2.0 stamp on the current tip?

I was waiting to finish the doc integration to do this and adopt semantic versioning.

I don't think I will wait to close all the tickets before I make numbered releases. 

It's been pretty stable and I don't have enough bandwidth these days.

Here's what I see is next in terms of "small stuff" that would be part of near upcoming 2.x releases:

- Integrate Dominik's docs as a static copy.

- Adding coherent cmdline options to disable the caches (pickle cache, price source cache, implement import cache for costly conversions).

- Remove the Holding class (less code); replace all its functionality by aggregation of positions. Rewrite all the reports with SQL queries only.

- Promote list_holdings.py (the script I use myself) to the main codebase and share an example Google Sheets doc to track assets (that's what I've been doing for years, works great).

- Figure out what to do about speed (we need something like a 3x speedup IMO).

The patch for Inventory as a dict will speed up anything to do with inventories...

https://bitbucket.org/blais/beancount/issues/191/beancountcoreinventory-should-be-a-dict

Yep, aware of that one. I didn't want to merge something that affects beancount.core this much before the release.

Now that 2.0.0 is stamped, I can look at it.

Thanks for submitting it!

 

- Do something about price sources now that G and Y finance are gone (perhaps just provide a helpers library instead of a tool, not sure)

This is small-ish stuff that gets on my own nerves regularly.

Here's some stuff that's being requested regularly by others that I also would like to work on:

- Adding options to override / take priority over inference for precision and allow you to set the num digits to render directly.

- Take a solid look at the CSV importer contributions waiting in PRs.

- Implement AVERAGE booking.

- Implement query assertions to have a good budget story / take time to check out Fava's custom directive for that, maybe integrate.

- Supporting UTF8

and of course, slowly work my way through all those tickets and pending PRs.

There's work for three dozen lifetimes (I don't suggest wading through the TODO file, you'll need courage and it's organized like a 14 year old's bedroom). Overall I'd like to reduce the amount of code (deprecate Holdings, perhaps delete bean-web (because Fava, if it can be baked), factor out SQL query client to a configurable row-based query tool that works on any data source) and focus on doing more with a smaller body of code (e.g. compute returns, support complex risk calculations (I hold options), implement pivot table in SQL for e.g. monthly reports, make the shell so powerful it's the only way to generate reports, simplify the tags/links schema, etc.).

On Fri, Mar 23, 2018 at 10:00 AM, Martin Michlmayr <t...@cyrius.com> wrote:

2.0rc1 is almost a year old.  Would it be possible to cut an rc2 since
there have been a lot of changes in the meantime?

This would help those not installing from mercurial directly.

Thanks,

Reference:
https://pypi.org/project/beancount/#history

--
Martin Michlmayr
http://www.cyrius.com/

--
You received this message because you are subscribed to the Google Groups "Beancount" group.

To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscribe@googlegroups.com.

To post to this group, send email to bean...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/20180323140005.4y4rquuqigi7mgts%40jirafa.cyrius.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Beancount" group.

To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscribe@googlegroups.com.

To post to this group, send email to bean...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/CAK21%2BhM_sHSBRpGVhi-GTeYd-yCKwdjePpigCsSdXRDQK3afiw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "Beancount" group.

To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscribe@googlegroups.com.

To post to this group, send email to bean...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/CAFFHUgs_rgYhAuOtoYgBpAXtD0Zx3WiyU-f7Dgz-OwZhqCv-1g%40mail.gmail.com.

[Martin Blais]

On Tue, Mar 27, 2018 at 4:54 AM, Saša Janiška <go...@atmarama.com> wrote:

Martin Blais <bl...@furius.ca> writes:

Hello Martin,

> - Implement query assertions to have a good budget story / take time
> to check out Fava's custom directive for that, maybe integrate.

Can you provide some more info about it? Is that feature provide support
for 'envelope budgeting' as it is available in (h)ledger?

> - Supporting UTF8

Any rough estimate when it might happen?

No idea (it's a labor of love), but I know lots of European friends want this, thanks for reminding me, I'm aware it's one of the most sought-after features.

I need to spend some quality time with flex, or write my own lexer (not hard), or look at reasonably lightweight alternatives.

[Zhuoyun Wei]

2018-03-27 20:52:48 Martin Blais <bl...@furius.ca>:

> That's unfair. The fact that you're unfamiliar with it doesn't mean it's a pain.
> Perhaps it's a pain for you now, but imagine the pains of a Mercurial user wrangling some of the crazy problems which
> occur frequently with Git (much worse).
>
> I'm not going to once again go over the technical reasons for this, but Mercurial is in many ways a success over Git,
> which is why some of the larger integrations have chosen it over Git (e.g. look at what Facebook has done, and I'm
> aware of other such large integrations and have discussed the details with some of the developers carrying them out at
> a conference, they're convincing technical reasons). Don't let your OSS perspective skew your vision. Github might be
> more popular, but so was VHS.

I am not trying to start a flame war, but I would like to point out that
choosing a popular tool/platform attracts more developers.

A friend of mine lives in Canada and often jokes about the mandatory
French usage in Canada when chatting with me. Say, if the lead developer
of an OSS is from Québec and believes that French is better than
English when coding, writing commit messages and discussing in mailing
list -- this project may not get as much contributors as another project
which chooses English as the main language.

I am not judging which language (English vs. French) is better, nor am I
going to say Git is better than Mercurial. I just would like to point
out: **if** we move the project to a popular tool/platform, the project
may attract more contributors. Even if the tool/platform may be
obsolete like VHS in the future, it is popular right now, isn't it?

Just my humble 0.02 Canadian Dollars.

--
Zhuoyun Wei

[Martin Blais]

On Tue, Mar 27, 2018 at 10:30 PM, Zhuoyun Wei <wzy...@wzyboy.org> wrote:

2018-03-27 20:52:48 Martin Blais <bl...@furius.ca>:
> That's unfair. The fact that you're unfamiliar with it doesn't mean it's a pain.
> Perhaps it's a pain for you now, but imagine the pains of a Mercurial user wrangling some of the crazy problems which
> occur frequently with Git (much worse).
>
> I'm not going to once again go over the technical reasons for this, but Mercurial is in many ways a success over Git,
> which is why some of the larger integrations have chosen it over Git (e.g. look at what Facebook has done, and I'm
> aware of other such large integrations and have discussed the details with some of the developers carrying them out at
> a conference, they're convincing technical reasons). Don't let your OSS perspective skew your vision. Github might be
> more popular, but so was VHS.

I am not trying to start a flame war, but I would like to point out that
choosing a popular tool/platform attracts more developers.

"Attracting more developers" is not an explicit goal of the project, though "attracting more developers who write very long and thoroughly thought-out suites of unit tests for very small and contained changes worked and reworked again from all the findings found from aforementioned laboriously written unit tests covering most of the cases" is quite useful. Do you believe moving to github would result in more unit testing?

[Stefano Zacchiroli]

First of all, thanks a lot for tagging/releasing Beancount 2.0, I'm
really excited about this (even if it's the same code base than a few
days ago, I know, but I can't help it!)

On Tue, Mar 27, 2018 at 10:52:48PM -0400, Martin Blais wrote:
> "Attracting more developers" is not an explicit goal of the project, though
> "attracting more developers who write very long and thoroughly thought-out
> suites of unit tests for very small and contained changes worked and
> reworked again from all the findings found from aforementioned laboriously
> written unit tests covering most of the cases" is quite useful. Do you
> believe moving to github would result in more unit testing?

I totally agree that one platform or another does nothing to *actively*
encourage developing more unit tests.

And I'm certainly not a fan of GitHub --- proprietary platform, forces
non-free JavaScript onto users, etc., etc., you know, my "usual FOSS
things" :) But FWIW the way I think about this is not in terms of
actively encouraging contributions, but rather not *discouraging* it.
Say a user has developed a bunch of test suites and want to contribute
them. And say that user is like Martin M., fluent with Git/GitHub, but
almost never used hg/bitbucket. He might give up sending you the tests
at the first attempt of rebasing/rewriting a PR that fails, unless he
has additional spare time and motivation. And at that point you would
have lost a contribution that you consider valuable.

That said, I don't particularly care, Bitbucket is as bad as a platform
as GitHub in terms of software freedom, and I'm happily using
git-remote-hg locally anyway, so the two alternatives are really the
same to me.

Cheers.

[Zhuoyun Wei]

2018-03-28 09:59:48 Stefano Zacchiroli <za...@upsilon.cc>:

> I'm happily using git-remote-hg locally anyway

I never knew something like this is possible. I tried the project -- it
works like a charm.

https://github.com/felipec/git-remote-hg

--
Zhuoyun Wei

[Martin Blais]

On Wed, Mar 28, 2018 at 3:59 AM, Stefano Zacchiroli <za...@upsilon.cc> wrote:

First of all, thanks a lot for tagging/releasing Beancount 2.0, I'm
really excited about this (even if it's the same code base than a few
days ago, I know, but I can't help it!)

Like I said, there's really nothing much special about today, it's like a birth date on the years after birth.

The main impact of tagging 2.0.0 is that knowing that a large number of people will stick with the release I can probably start taking more risks again.

On Tue, Mar 27, 2018 at 10:52:48PM -0400, Martin Blais wrote:
> "Attracting more developers" is not an explicit goal of the project, though
> "attracting more developers who write very long and thoroughly thought-out
> suites of unit tests for very small and contained changes worked and
> reworked again from all the findings found from aforementioned laboriously
> written unit tests covering most of the cases" is quite useful. Do you
> believe moving to github would result in more unit testing?

I totally agree that one platform or another does nothing to *actively*
encourage developing more unit tests.

And I'm certainly not a fan of GitHub --- proprietary platform, forces
non-free JavaScript onto users, etc., etc., you know, my "usual FOSS
things" :) But FWIW the way I think about this is not in terms of
actively encouraging contributions, but rather not *discouraging* it.
Say a user has developed a bunch of test suites and want to contribute
them. And say that user is like Martin M., fluent with Git/GitHub, but
almost never used hg/bitbucket. He might give up sending you the tests
at the first attempt of rebasing/rewriting a PR that fails, unless he
has additional spare time and motivation. And at that point you would
have lost a contribution that you consider valuable.

That said, I don't particularly care, Bitbucket is as bad as a platform
as GitHub in terms of software freedom, and I'm happily using
git-remote-hg locally anyway, so the two alternatives are really the
same to me.

This comes as no surprise, but your radical Leftist POV is always welcome on this list no matter what.

It's our political diversity initiative :-)

The thing is, if reading the one or two short pages on doing a mercurial checkout is too much, I want to say it's almost a Good Thing that there's a little bit of a hurdle. It ensures you're not going to waste your time making a submission that will never get merged. How can the contribution be expected to be valuable if that's all the effort someone is willing to put in?  You speak as if a little bit of untested code is worth anything. It's not. Let me explain.

Here's how I go about adding new code to Beancount:

1- I have an idea, or I need something that's not there

2- I code something up, it works (in the one case I'm using it for)

3- So I write a couple of unit tests for the new idea

4- Very quickly (and invariably) I realize some flaws in my approach, it breaks some other thing or assumption, other test far away from it start to fail, and/or it's not general enough, or while testing some corner case I realize something I hadn't thought about which invalidates the whole idea

5- So I go back re-design, fixup a lot of other stuff, and write more unit tests

- I iterate like that 4-5 more times, until it looks good (or I abandon the idea because it's not general enough after I realize I was wrong).

What I thought would take 10 minutes ends up taking 4 or 5 hours of development time.

(BTW that's one of the reasons I love plugins, usually they're isolated and are less likely to lead down that road.)

Most people's contributions stops at (2). That's a problem. So then if I want that contribution - and to keep things stable, you DO want to be able to do your accounting without facing bugs, don't you? - I have to go write the unit tests myself, and INEVITABLY it either breaks some other stuff or it needs improvement or even rewriting and 3, 4, 5, and iterate. (2) takes almost none of the time, it's not really "valuable." It's really super ridiculously easy to hack something that works in your one use case without testing it than going the extra 10 miles further to fix everything upstream to finish the job right.  All the time is spent in 3-4-5-iterate.  THAT's where the value is. The artifact (the code) is not valuable; the testing and running of it is. The more we run that codebase successfully and shape it to solve our problem, the more value it acquires. (That's one reason a really loose and wildly unchecked language like Python often wins over more pedantic approaches - we move fast, and the code more quickly morphs around the problem's "shape", that optimization often more than makes up for correctness.)

Learning to checkout, commit and push a Mercurial PR? Honestly that should take VASTLY less time than coding up the idea and doing the work necessary to merge it in. If that hurdle is enough to deter someone... maybe it just keeps a lot of quickly coded, untested "drive-by contributions" away, all the better.  If you make a contribution and there's no test... well you're forcing  me to either (a) go write the tests myself for your contribution (in other words, I end up doing most of the work), or (b) ignore it and work on the stuff that I care about. (And it's not like I'm in need of ideas... visit the TODO file, there's 5000 lines of ideas there. I'm pretty sure I'll never be able to do it all. My problem is time. I like to learn what users' pain points are to address those when I don't rub against the (e.g. UTF8) but new ideas are a dime a dozen...)

Now... I'll say it again: the quickest, fastest way to any contribution getting merged is to write thorough set of corresponding unit tests, and then iterate, and then address merge comments and iterate on them until it's good enough. It's a lot of work... but it's the work that I put in myself. (There's a reason it's reasonably stable and that you're here.) I know I sound a bit harsh, but really, I'm very generous with my time and I promise I will be helpful in the process of working through PRs. And I have the advantage of having written most of this, so I'm really familiar with it, I can provide some helpful guidance. If you really want something in... and you're willing to put in the work to write tests for "most" (say, 80%) of the corner cases and to automate checks so it won't break in the future, and are willing to maintain the feature when new issues do arise (they will... otherwise I have to do it?), and iterate with me until it's well tested, and are open to the fact that maybe, just maybe, in the process we will both realize it might be better to drop the feature after all because it breaks too many things (this happens, occasionally, sorry Michael about the @@)... then I think it's possible to make good contributions. Will it take more of your time than you think when you set out? Yep, almost certainly. And here cue that Willy Wonka meme...

https://memegenerator.net/img/instances/81584686/oh-you-cant-learn-4-or-5-mercurial-commands-thats-sad.jpg

[Stefano Zacchiroli]

On Thu, Mar 29, 2018 at 08:30:30PM -0400, Martin Blais wrote:
> You speak as if a little bit of untested code is worth anything. It's
> not. Let me explain.

Oh, no, I agree it's not worth it. And it's great that you, as Beancount
maintainer, have high standards for code acceptance that encompass: (1)
not breaking existing tests, and (2) having thorough unit tests for the
new code being contributed.

But it seems to me that that is almost completely unrelated to the
choice of hosting platform, isn't it? Aren't you in fact just saying
that what you want is continuous integration (CI) integrated with the
contribution work-flow for proposed patches?

Both GitLab and GitHub have integrated CI offerings, and IME they go a
long way in avoiding wasting maintainer time in "complaining" about
breaking existing tests. You make the CI run on incoming patches, if
existing tests get broken by it, submitters get immediate feedback about
it and can iterate by themselves to fix that, without any need of your
intervention. And, in fact, you can do the same for missing tests. Just
enable the nose (or equivalent) code coverage plugin and make it fail if
the coverage is not up to a given standard or threshold, and there too
you automatically send the ball back in the camp of code contributors if
they don't show up with tests.

I don't know if BitBucket has any CI integration, but I'd be surprised
if it doesn't. Aside from that aspect, this seem unrelated to the "lower
barriers for contribution due to what is well-known out there". (But is
an interesting discussion anyway!)

Cheers

[Patrick Ruckstuhl]

Did some searching and came up with


https://semaphoreci.com

As an option. It supports bitbucket and is free for open source.

Regards,
Patrick

[Martin Blais]

CI only makes it easier to run the test, it doesn't write the tests.

Running the tests is easy ("make test" locally)

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscribe@googlegroups.com.
To post to this group, send email to bean...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/20180331090248.GD13350%40upsilon.cc.

[Stefano Zacchiroli]

On Sat, Mar 31, 2018 at 12:45:12PM -0400, Martin Blais wrote:
> CI only makes it easier to run the test, it doesn't write the tests.
> Running the tests is easy ("make test" locally)

My point was that it gives actionable incentives to write the tests to
wannabe contributors.

[Martin Michlmayr]

Since CI came up, I looked into this today. I ran into a number of
test failures:

> Issue #263: test test_read_encrypted_file fails when $GNUPGHOME is not set

Martin accepted the fix already

> Issue #264: test test_bake_archive__known fails when zip is not installed

I made sure that zip (and gnupg) were installed in my test
environment. This needs to be documented (Martin mentioned converting
the code to use Python modules instead).

> Issue #265: tests beancount.prices.sources.oanda_test.TestOandaGetHistorical fail because of timezone

I haven't fixed this but I hope the comments I added show where the
problem is.

> Issue #266: test beancount.plugins.forecast_test.TestExampleForecast fails

This test was time dependent. I submitted a PR.

I also submitted a Travis file:
https://bitbucket.org/blais/beancount/pull-requests/68

With this Travis configuration, I can do CI Travis builds automatically.
I used GitHub using a beancount git repo converted from Mercurial with
hg-fast-export, but if there's Travis for BitBucket it should work the
same way since it's normal Travis.

[Martin Blais]

On Sun, Apr 1, 2018 at 1:36 PM, Martin Michlmayr <t...@cyrius.com> wrote:

Since CI came up, I looked into this today.  I ran into a number of
test failures:

> Issue #263: test test_read_encrypted_file fails when $GNUPGHOME is not set

Martin accepted the fix already

Thx! :-)

 

> Issue #264: test test_bake_archive__known fails when zip is not installed

I made sure that zip (and gnupg) were installed in my test
environment.  This needs to be documented (Martin mentioned converting
the code to use Python modules instead).

Yes, the modules for zipping and tarring are available from the stdlib, that's the right thing to do to get rid of that dependency.

Easy change.

 

> Issue #265: tests beancount.prices.sources.oanda_test.TestOandaGetHistorical fail because of timezone

I haven't fixed this but I hope the comments I added show where the
problem is.

Based on past experience this is probably tripping a bug that doesn't show up in New York. This has happened to me a lot in the past when I was developing on Beancount from conferences and stuff, where it would suddenly start to fail. I'll look at it.

> Issue #266: test beancount.plugins.forecast_test.TestExampleForecast fails

This test was time dependent.  I submitted a PR.

I also submitted a Travis file:
https://bitbucket.org/blais/beancount/pull-requests/68

With this Travis configuration, I can do CI Travis builds automatically.
I used GitHub using a beancount git repo converted from Mercurial with
hg-fast-export, but if there's Travis for BitBucket it should work the
same way since it's normal Travis.

I think they have something called "Pipelines" but I need to enable it.

Thanks for This,

 

--
Martin Michlmayr
http://www.cyrius.com/

--
You received this message because you are subscribed to the Google Groups "Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscribe@googlegroups.com.
To post to this group, send email to bean...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/20180401173636.vdxoptxq5ffobirf%40jirafa.cyrius.com.

[Jakob Schnitzer]

I think one thing CI adds is a certain reproducibility. I can remember
having the full Beancount test suite pass exactly once (after I
submitted a PR fixing some broken tests). Having a test suite that
doesn't fully pass (fully passing on just the main developer's computer
doesn't count IMHO), when new to a project this to me sends a message
'this project doesn't care too much about tests'. From the mailing list
and the quality of code and tests I of course know this to be false -
having a frequent CI build could still help with obtaining (and keeping)
a more reproducible test suite (and make the standard of tests clear to
potential contributors).

With regards to linting, I'm not sure what the standard is (`make lint`)
fails with a staggering list of 'errors'. Again, having a CI build would
more clearly communicate the expected standard with regards to linting.

On Sat, Mar 31, 2018 at 12:45:12PM -0400, Martin Blais wrote:

>> email to beancount+...@googlegroups.com.

>> To post to this group, send email to bean...@googlegroups.com.
>> To view this discussion on the web visit https://groups.google.com/d/
>> msgid/beancount/20180331090248.GD13350%40upsilon.cc.
>> For more options, visit https://groups.google.com/d/optout.
>>
>

>--
>You received this message because you are subscribed to the Google Groups "Beancount" group.

>To unsubscribe from this group and stop receiving emails from it, send an email to beancount+...@googlegroups.com.

>To post to this group, send email to bean...@googlegroups.com.

>To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/CAK21%2BhPNHd3EUyDN%2B6Wsqx127qXu8bZXM0xiKLepVNN8yRRCsQ%40mail.gmail.com.

[Martin Blais]

I love CI. CI is good. I'll try enabling Atlassian's pipelines thing, see if it works. In the past I've worked with buildbot. We should indeed have a working build status on a minimal Docker install somewhere.

email to beancount+unsubscribe@googlegroups.com.

To post to this group, send email to bean...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/beancount/20180331090248.GD13350%40upsilon.cc.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Beancount" group.

To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscribe@googlegroups.com.

To post to this group, send email to bean...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/CAK21%2BhPNHd3EUyDN%2B6Wsqx127qXu8bZXM0xiKLepVNN8yRRCsQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Beancount" group.

To unsubscribe from this group and stop receiving emails from it, send an email to beancount+unsubscribe@googlegroups.com.

To post to this group, send email to bean...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/20180331174637.GA6277%40js-arch.

[Martin Blais]

https://bitbucket.org/blais/beancount/issues/269/enable-a-ci-solution-try-atlassians

[mpl...@gmail.com]

Martin hi,

Sorry for bringing it up again :) Thank you for all the work you did, I love beancount and will keep using it not matter what version control systems you will be using.

Please consider one point, it's not only about attracting developers, it's also about discoverability for users too.

When i was looking for plain text accounting tool first thing i did i went on github and searched for accounting. Ledger and hleadger came up, did more reading around and chose ledger. I ignored beancount totally cause i assumed that any worthy tool should have at least some community behind it.

So i've chose ledger and was using it for 1 months. Than somehow i came to reading about beancount, found this google group and realized that beacount have community (in fact bigger than hledger and more active than ledger), etc. I tried it. loved it and switched to it.

So having project on github could've saved me 1 months :) and not having to learn new system again. (though leadger and beancount are similar, they still differ quite a lot for a newbie) 

To summarize: discoverability and users are important for a project development too. It's not only developers.

[Alen Šiljak]

On Tuesday, 27 March 2018 20:37:56 UTC+2, Martin Michlmayr wrote:

* Martin Michlmayr <t...@cyrius.com> [2018-03-24 16:14]:
> I've submitted some minor pull requests and while hg is not too hard

Ok, I take that back.  hg is quite a bit if pain, actually.  I just
spent 30 minutes trying to figure out how to update a pull request
after getting all confused by hg and multiple heads on a branch...

Heheheh, this pretty much summarizes it. Since (the other) Martin mentioned it, VHS is unfortunately the one that took over the world.

Don't get me wrong, I had a Beta VCR when I was a kid and was even more vocal about Mercurial adoption in favour of Git. But that was almost 10 years ago.

Nowadays, it's the tools that make the process fluent. Whether Mercurial has technical benefits or not, Git is everywhere. I'm pretty sure (even if I don't know it for a fact) that today's kids learn Git at school and probably think Mercurial comes from a Biology class, not Computer Programming. Anyways, back to the tools - SmartGit (well, it also has support for Hg but that is being phased out soon) and VS Code, for example, make it quite convenient and painless to use Git as underlying source repository. Pull requests and everything else, made popular by GitHub, is today a standard. Because they made it convenient.

Having said that, I'm also on GitLab side and create all my new repositories there. Nowadays it is about CI integration, not even the repositories any more.

I'm not pushing for anything here but just wanted to share my 2 Euro Cents, for what it's worth.

Especially since I touched upon this topic in another post so there's no need to continue there (neither probably here).

Cheers!

[Martin Blais]

The next major version might be on GitHub.

Major versions are a good time to reconsider everything (e.g. implementation language, breaking syntax changes) so that might be in the cards, who knows?