Making wazzap scale
38 views
Skip to first unread message
Christopher Allan Webber
Nov 14, 2012, 11:38:05 AM11/14/12
to magit
Hello all,
I'm a magit user and fan. When reviewing code in mediagoblin, for the
first 9 months of the project, I used magit-wazzap to keep track of all
the code I needed to review.
However, about 500 branches locally later, it takes so long to load and
refresh it's simply unusable. I know you can mark branches as ignored
in magit... however, there's not really an easy way to do so. I've been
thinking about maybe writing a small major mode that shows the inverse
of magit-wazzap: all branches that have no new commits in them so I can
mass mark them as ignorable. Or is there some other way to speed up my
current problems?
Thoughts?
- Chris
I'm a magit user and fan. When reviewing code in mediagoblin, for the
first 9 months of the project, I used magit-wazzap to keep track of all
the code I needed to review.
However, about 500 branches locally later, it takes so long to load and
refresh it's simply unusable. I know you can mark branches as ignored
in magit... however, there's not really an easy way to do so. I've been
thinking about maybe writing a small major mode that shows the inverse
of magit-wazzap: all branches that have no new commits in them so I can
mass mark them as ignorable. Or is there some other way to speed up my
current problems?
Thoughts?
- Chris
Rémi Vanicat
Nov 15, 2012, 2:03:37 AM11/15/12
to Christopher Allan Webber, magit
believe a solution would be to not update the buffer when one ignore a
branch, but to just remove it, of visually mark it is now ignored, and
wait for the user to explicitly update the buffer before doing it. It
would make ignoring branch far less painful
--
Rémi Vanicat
Christopher Allan Webber
Nov 22, 2012, 12:41:12 PM11/22/12
to Rémi Vanicat, magit
Rémi Vanicat writes:
> I didn't use wazzup anymore, but I've already think about it, and I
> believe a solution would be to not update the buffer when one ignore a
> branch, but to just remove it, of visually mark it is now ignored, and
> wait for the user to explicitly update the buffer before doing it. It
> would make ignoring branch far less painful
This would be helpful, though I think I need more than this. For
> I didn't use wazzup anymore, but I've already think about it, and I
> believe a solution would be to not update the buffer when one ignore a
> branch, but to just remove it, of visually mark it is now ignored, and
> wait for the user to explicitly update the buffer before doing it. It
> would make ignoring branch far less painful
example, it's only easy to see something should be marked as ignorable
if it still shows up on my wazzap list. I'm writing this email in the
car, but for a repository with 336 branches it takes 7.5 seconds to run
merge-base on each of them, which effectively for all the branches that
don't show up and aren't explicitly ignored, is something that happens.
On my desktop I know I have about 4 times as many branches as
this... it's just way too much to manage.
I've been thinking of writing a new mode which is basically like an
enhanced wazzap mode for people who are crazy like me and just have tons
and tons of branches to go through and review. I'm thinking of calling
it magit-reviewer. Basically, it would have these features:
- Be able to mark things as ignored (like already exits in
magit-wazzap) but in two differen ways: ignored unconditionally, or
ignored under the premise that there's nothing new in this branch to
look at anymore (if that changes, in the branch-reviewing mode that
will be flagged for reconsideration)
- Be able to mark things as being tracked. Sometimes I have a queue of
things I'm reviewing, and I just want to review those and be able to
move things in and out of that review queue. This will make things a
lot faster also... refreshing a whitelist based wazzap type interface
as opposed to a blacklist type should be much faster because there's
less to check for.
- The ability to change your filter: view just tracked stuff? View
everything? Check for branches that might have new things that
aren't tracked and possibly move them onto the list?
- Should split things visually into said categories of tracked,
ignored, has-new-but-not-tracked, etc.
Is this interesting to anyone else? I'll probably be writing it
regardless... I just can't keep up with my workload without it. Would
be interesting to know if it would be helpful to anyone else though.
- cwebb
Rémi Vanicat
Nov 24, 2012, 5:17:51 AM11/24/12
to Christopher Allan Webber, magit
> Rémi Vanicat writes:
>
>> I didn't use wazzup anymore, but I've already think about it, and I
>> believe a solution would be to not update the buffer when one ignore a
>> branch, but to just remove it, of visually mark it is now ignored, and
>> wait for the user to explicitly update the buffer before doing it. It
>> would make ignoring branch far less painful
>
> This would be helpful, though I think I need more than this. For
> example, it's only easy to see something should be marked as ignorable
> if it still shows up on my wazzap list. I'm writing this email in the
> car, but for a repository with 336 branches it takes 7.5 seconds to run
> merge-base on each of them, which effectively for all the branches that
> don't show up and aren't explicitly ignored, is something that happens.
> On my desktop I know I have about 4 times as many branches as
> this... it's just way too much to manage.
>
> I've been thinking of writing a new mode which is basically like an
> enhanced wazzap mode for people who are crazy like me and just have tons
> and tons of branches to go through and review. I'm thinking of calling
> it magit-reviewer. Basically, it would have these features:
>
[...]
>
>> I didn't use wazzup anymore, but I've already think about it, and I
>> believe a solution would be to not update the buffer when one ignore a
>> branch, but to just remove it, of visually mark it is now ignored, and
>> wait for the user to explicitly update the buffer before doing it. It
>> would make ignoring branch far less painful
>
> This would be helpful, though I think I need more than this. For
> example, it's only easy to see something should be marked as ignorable
> if it still shows up on my wazzap list. I'm writing this email in the
> car, but for a repository with 336 branches it takes 7.5 seconds to run
> merge-base on each of them, which effectively for all the branches that
> don't show up and aren't explicitly ignored, is something that happens.
> On my desktop I know I have about 4 times as many branches as
> this... it's just way too much to manage.
>
> I've been thinking of writing a new mode which is basically like an
> enhanced wazzap mode for people who are crazy like me and just have tons
> and tons of branches to go through and review. I'm thinking of calling
> it magit-reviewer. Basically, it would have these features:
>
I tend to agree that wazzup as it is now is not usable in some common
case (when you want to review lot of branch), so I hope that your
extension will improve things
--
Rémi Vanicat
Christopher Allan Webber
May 11, 2013, 3:07:19 PM5/11/13
to Rémi Vanicat, magit
By the way, I built this tool some time ago, and never really
documented/released it.
Here it is:
http://gitorious.org/magit-review/
Note: I don't know if this works on present versions of magit. I'm
running an older version, and it works there... commit
45712322c1bd713a4172976bc40bb0d92366e410 specifically!
It's a lot like magit-wazzap, except that you move things in and out of
states (like: ignore this, there wasn't anything new last time I
checked, or "put this on my review queue") and filters (which show
*only* branches that match the specified states). The metadata is
stored in .git/info/magit-review.
A bit of expanded docs from the README.org file:
-----
* About magit-review
** Motivation
For a long time I used magit-wazzap to handle all my code reviews.
magit-wazzup is awesome... who doesn't want a buffer where they can
see all branches with new commits for them to check out at once?
But I found that as my project grew, magit-wazzup failed to scale for
me:
- It was too slow. My project had about 500 or so branches and
magit-wazzup would check *all* of them if they had new commits and
format them all for display. magit-wazzup had an interface to mark
something as "ignored" but it didn't really work for me because the
branches that should have been ignored didn't show up for me to
ignore them anyway!
- I also found magit-wazzup's ignore tool annoying because I didn't
have a nice way to double check later that I really wanted things
that were ignored to stay ignored.
- Sometimes I wanted to mark a few branches as "I need to review
these" and jump to a limited view of wazzup so I could just focus
on the branches I knew needed attention.
If these irritations sound familiar to you, you might like
magit-review (if not, you might want to just stick with magit-wazzup;
it's admittedly slightly simpler). And if you already like
magit-wazzup, you'll be happy to see that magit-review basically works
the same way, with just a few small enhancements.
** magit-review's strategy
magit-review works pretty much the same as magit-wazzup except that it
adds two features: states and filters. You can mark a branch with
some sort of state (magit-review will serialize this so that it's
remembered) and apply a filter so that only branches that match that
state actually show up. For more information on how to actually make
use of that strategy, read on.
----
You can find the code here:
g...@github.com:cwebber/magit-review.git
or here:
http://gitorious.org/magit-review/
(Note: I normally use gitorious but I am trying to get an understanding
of what it's missing in comparison to GitHub :))
documented/released it.
Here it is:
http://gitorious.org/magit-review/
Note: I don't know if this works on present versions of magit. I'm
running an older version, and it works there... commit
45712322c1bd713a4172976bc40bb0d92366e410 specifically!
It's a lot like magit-wazzap, except that you move things in and out of
states (like: ignore this, there wasn't anything new last time I
checked, or "put this on my review queue") and filters (which show
*only* branches that match the specified states). The metadata is
stored in .git/info/magit-review.
A bit of expanded docs from the README.org file:
-----
* About magit-review
** Motivation
For a long time I used magit-wazzap to handle all my code reviews.
magit-wazzup is awesome... who doesn't want a buffer where they can
see all branches with new commits for them to check out at once?
But I found that as my project grew, magit-wazzup failed to scale for
me:
- It was too slow. My project had about 500 or so branches and
magit-wazzup would check *all* of them if they had new commits and
format them all for display. magit-wazzup had an interface to mark
something as "ignored" but it didn't really work for me because the
branches that should have been ignored didn't show up for me to
ignore them anyway!
- I also found magit-wazzup's ignore tool annoying because I didn't
have a nice way to double check later that I really wanted things
that were ignored to stay ignored.
- Sometimes I wanted to mark a few branches as "I need to review
these" and jump to a limited view of wazzup so I could just focus
on the branches I knew needed attention.
If these irritations sound familiar to you, you might like
magit-review (if not, you might want to just stick with magit-wazzup;
it's admittedly slightly simpler). And if you already like
magit-wazzup, you'll be happy to see that magit-review basically works
the same way, with just a few small enhancements.
** magit-review's strategy
magit-review works pretty much the same as magit-wazzup except that it
adds two features: states and filters. You can mark a branch with
some sort of state (magit-review will serialize this so that it's
remembered) and apply a filter so that only branches that match that
state actually show up. For more information on how to actually make
use of that strategy, read on.
----
You can find the code here:
g...@github.com:cwebber/magit-review.git
or here:
http://gitorious.org/magit-review/
(Note: I normally use gitorious but I am trying to get an understanding
of what it's missing in comparison to GitHub :))
Tom Davis
May 11, 2013, 10:42:31 PM5/11/13
to Christopher Allan Webber, Rémi Vanicat, magit
It sounded cool, so I tried it out. Certainly appears to work with my
version of magit, which is master as of two days ago. Thanks!
version of magit, which is master as of two days ago. Thanks!
Óscar Fuentes
May 12, 2013, 12:55:21 PM5/12/13
to ma...@googlegroups.com
> But I found that as my project grew, magit-wazzup failed to scale for
> me:
> - It was too slow. My project had about 500 or so branches and
> magit-wazzup would check *all* of them if they had new commits and
> format them all for display. magit-wazzup had an interface to mark
> something as "ignored" but it didn't really work for me because the
> branches that should have been ignored didn't show up for me to
> ignore them anyway!
Tried with a large repo (Emacs). After invoking magit-review Emacs
> me:
> - It was too slow. My project had about 500 or so branches and
> magit-wazzup would check *all* of them if they had new commits and
> format them all for display. magit-wazzup had an interface to mark
> something as "ignored" but it didn't really work for me because the
> branches that should have been ignored didn't show up for me to
> ignore them anyway!
freezes while git/magit-review inspects the repo. After several minutes,
I canceled the operation by pressing C-g. This is the same problem
wazzup has, which is unusable on this type of repos.
Does magit-review provide a method for improving this?
Christopher Allan Webber
May 12, 2013, 11:24:15 PM5/12/13
to Óscar Fuentes, ma...@googlegroups.com
slowed down drastically by having a ton of branches to review (I have
around 500ish) but that's not accounting for slowness based on history
length itself.
I wonder if you've done "git gc --agressive" if that impacts
performance? Anyway, it's still using the *general* trick that
git-wazzup is, which does involve trying to find a common head. What
magit-review does differently is that you're able to move branches over
to a state where you don't see them anymore, such as the ignore
branches, or a subset of branches to focus on, such as the tracking
ones. But other than that, I don't suspect it has anything else that
helps.
(There is one way where it may be able to become faster in this regard:
it may be able to cache data on which branches have new commits and a
common head, but of course the next time you run a git pull on master
this info won't be useful anymore...)
- Chris
Óscar Fuentes
May 13, 2013, 6:38:37 AM5/13/13
to ma...@googlegroups.com
Christopher Allan Webber
<cwe...@dustycloud.org> writes:
>> Tried with a large repo (Emacs). After invoking magit-review Emacs
>> freezes while git/magit-review inspects the repo. After several minutes,
>> I canceled the operation by pressing C-g. This is the same problem
>> wazzup has, which is unusable on this type of repos.
>>
>> Does magit-review provide a method for improving this?
>
> I've found that magit-review helps for MediaGoblin, where things are
> slowed down drastically by having a ton of branches to review (I have
> around 500ish) but that's not accounting for slowness based on history
> length itself.
Yes, the problem here is history length. Quite a few of the older
<cwe...@dustycloud.org> writes:
>> Tried with a large repo (Emacs). After invoking magit-review Emacs
>> freezes while git/magit-review inspects the repo. After several minutes,
>> I canceled the operation by pressing C-g. This is the same problem
>> wazzup has, which is unusable on this type of repos.
>>
>> Does magit-review provide a method for improving this?
>
> I've found that magit-review helps for MediaGoblin, where things are
> slowed down drastically by having a ton of branches to review (I have
> around 500ish) but that's not accounting for slowness based on history
> length itself.
branches have been merged rarely, if at all.
> I wonder if you've done "git gc --agressive" if that impacts
> performance?
after `git gc --agressive' and canceled wazzup after 20 minutes or so.
> Anyway, it's still using the *general* trick that
> git-wazzup is, which does involve trying to find a common head. What
> magit-review does differently is that you're able to move branches over
> to a state where you don't see them anymore, such as the ignore
> branches, or a subset of branches to focus on, such as the tracking
> ones. But other than that, I don't suspect it has anything else that
> helps.
you have to reach the initial display of magit-review/wazzup, which
takes forever.
Apart from that, for the case of emacs.git, even ignoring almost all
branches, I'm afraid that one month worth of commits would be enough to
slow down wazzup to annoying levels just showing the difference with
origin/master.
One thing that could work is to show the branches, mark those that
diverged but do not gather the commits on advance, do so just when the
user requires it, on a similar whay magit-status does: it shows the
modified files but inspects and shows the difference only when the user
ask for it.
Óscar Fuentes
May 13, 2013, 6:41:11 AM5/13/13
to ma...@googlegroups.com
Óscar Fuentes <o...@wanadoo.es> writes:
> One thing that could work is to show the branches, mark those that
> diverged but do not gather the commits on advance, do so just when the
> user requires it, on a similar whay magit-status does: it shows the
> modified files but inspects and shows the difference only when the user
> ask for it.
Oh, and one thing that does definitively wrong in wazzup is to show
> One thing that could work is to show the branches, mark those that
> diverged but do not gather the commits on advance, do so just when the
> user requires it, on a similar whay magit-status does: it shows the
> modified files but inspects and shows the difference only when the user
> ask for it.
history differences for tags. I think that tags should have no place in
wazzup.
Christopher Allan Webber
May 13, 2013, 10:53:18 AM5/13/13
to Óscar Fuentes, ma...@googlegroups.com
magit-review. Patches welcome, seriously!
I'm also interested in other optimization routes. I agree that there's
no need to show the branches until you expand them. Would you be
interested in helping me with this?
- cwebb
Óscar Fuentes
May 14, 2013, 7:46:33 AM5/14/13
to Christopher Allan Webber, ma...@googlegroups.com
>> Oh, and one thing that does definitively wrong in wazzup is to show
>> history differences for tags. I think that tags should have no place in
>> wazzup.
>
> I agree on tags. I'd be interested in getting that removed from
> magit-review. Patches welcome, seriously!
Maybe this already implemented the feature?
>> history differences for tags. I think that tags should have no place in
>> wazzup.
>
> I agree on tags. I'd be interested in getting that removed from
> magit-review. Patches welcome, seriously!
https://github.com/magit/magit/pull/468
> I'm also interested in other optimization routes. I agree that there's
> no need to show the branches until you expand them. Would you be
> interested in helping me with this?
commitment.
Reply all
Reply to author
Forward
0 new messages