Diff could also show the changes within lines

[Tom]

When doing diffs it often occurs to me the diff output could be
more readable if it showed the actual changes within lines.

Something like this (I mean the concept, the actual implemenation shown
may not be the best output which can be achieved), so you don't have to
search for the changes within the line:

http://stackoverflow.com/questions/1265949/multiple-changes-in-one-line-with-diff-
tool/1266114#1266114

It doesn't even necessarily need support from the external diff tool,
because diffs are usually not huge, so the highlighting of the changes
could be done in lisp by parsing the diff output.

Has anyone tried improving diff like this? Is there a package which does
this?

[Drew Adams]

> Has anyone tried improving diff like this? Is there a package
> which does this?

ediff

E.g. `M-x ediff-buffers'

[Tom]

Okay, but I mean output from git/mercurial/etc. when I get
a file history from emacs, or diff between versions, etc.

These commands invoke diff, and I like the single buffer diff
output, I don't need ediff showing diffs in two windows, etc.
that's why I think improving the regular diff output could
be useful.

[chandan r]

Tom <adatg...@gmail.com> writes:

> When doing diffs it often occurs to me the diff output could be
> more readable if it showed the actual changes within lines.
>
> Something like this (I mean the concept, the actual implemenation shown
> may not be the best output which can be achieved), so you don't have to
> search for the changes within the line:
>
> http://stackoverflow.com/questions/1265949/multiple-changes-in-one-line-with-diff-
> tool/1266114#1266114
>
> It doesn't even necessarily need support from the external diff tool,
> because diffs are usually not huge, so the highlighting of the changes
> could be done in lisp by parsing the diff output.
>

> Has anyone tried improving diff like this? Is there a package which does
> this?

Add the following to your Emacs init file:

(add-hook 'diff-mode-hook
(lambda () (diff-auto-refine-mode 1)))

[Tom]

chandan r <chandanrmail <at> gmail.com> writes:
>
> Add the following to your Emacs init file:
>
> (add-hook 'diff-mode-hook
> (lambda () (diff-auto-refine-mode 1)))
>

Wow, I'm impressed. Builtin Emacs features can still surprise me after
all these years. Thanks.

[Tom]

Looks like it does not work with DVC diff, so it needs to be disabled for
that:

(unless (or (eq this-command 'dvc-diff)
(eq this-command 'dvc-generic-refresh))
(diff-auto-refine-mode 1))

[Doug Lewan]

Tom,

If you're working with software that VC supports (RCS, CVS, there are external packages for others too), then you can also do M-x ediff-revision.

,Doug

> -----Original Message-----
> From: help-gnu-emacs-bounces+dougl=shuberttic...@gnu.org
> [mailto:help-gnu-emacs-bounces+dougl=shuberttic...@gnu.org] On
> Behalf Of Drew Adams
> Sent: Friday, 2012 October 05 10:40
> To: 'Tom'; help-gn...@gnu.org
> Subject: RE: Diff could also show the changes within lines
>
> > Has anyone tried improving diff like this? Is there a package
> > which does this?
>

> ediff
>
> E.g. `M-x ediff-buffers'
>

[Tom]

Tom <adatgyujto <at> gmail.com> writes:
>

> (diff-auto-refine-mode 1)
>

Turns out it only refines one hunk (the current) and the others only
if you use hunk navigation commands (diff-auto-refine-mode does this which
is on by default in Emacs 24). So to refine all hunks automatically,
do this in the diff hook:

(save-excursion
(goto-char (point-min))
(while (not (eobp))
(diff-hunk-next))))

[Stefan Monnier]

> Looks like it does not work with DVC diff, so it needs to be disabled for
> that:

If you describe the problem, maybe it can be fixed instead.


Stefan

[Tom]

It seems dvc-diff calls the diff hooks early, when the diff
buffer is still empty (I guess it's because it starts an external async
process to get the diff ouput), so diff refine cannot find the diff
headers in the buffer.

[Sebastien Vauban]

Hello Tom, Stefan and all,

I'd like to get that feature as well. However, the above code causes me
2 problems:

- only the last hunk is getting refined, both in VC dir's buffer, or when
reading (with Gnus) an email with a patch file in it;

- occasionally, an error message "no hunk found" when reading an email with a
patch file in it.

I patched your code to try and avoid those problems, by:

- adding a condition-case nil... error construct
- using save-excursion
- using an explicit goto-char min
- ...

Here is my current code:

--8<---------------cut here---------------start------------->8---
;; enable Diff Auto Refine mode (highlight the changes with better
;; granularity)
(add-hook 'diff-mode-hook
(lambda ()
;; refine all hunks automatically
(when (>= (diff-end-of-hunk) 1)
;; avoid "No hunk found" error

(goto-char (point-min))
(while (not (eobp))
(diff-hunk-next)))

(goto-char (point-min))))
--8<---------------cut here---------------end--------------->8---

... but that does not help...

I still have the two above mentioned problems...

Any idea?

Best regards,
Seb

--
Sebastien Vauban

[Tom]

Sebastien Vauban <wxhgmqzgwmuf@...> writes:

>
> I'd like to get that feature as well. However, the above code causes me
> 2 problems:
>
> - only the last hunk is getting refined, both in VC dir's buffer, or when
> reading (with Gnus) an email with a patch file in it;
>

Currently I use this code added to the diff mode hook:

(defun my-refine-all-diff-hunks ()
(interactive)
(condition-case nil

(save-excursion
(goto-char (point-min))
(while (not (eobp))
(diff-hunk-next)))

(error nil)))


It works well for me, it highlights the changes in all hunks
automatically. I use it only for diffing before checkin.

But. It works only because the VC mercurial implementation calls
the backend synchronously. In case of DVC it doesn't work for
me, because DVC calls the backend asynchronously, so when the diff
hook is run the output is not there yet and I did not find a hook
which is called when the async diff command is finished

You may want to check how the diff output is produced in your use
cases. If it is done asynchronously then the diff hook is not a
suitable place to call the highlighting code from.

[Oleksandr Gavenko]

On 2012-11-10, Tom wrote:

> Currently I use this code added to the diff mode hook:
>
> (defun my-refine-all-diff-hunks ()
> (interactive)
> (condition-case nil

> (save-excursion
> (goto-char (point-min))
> (while (not (eobp))

> (diff-hunk-next)))
> (error nil)))
>
> It works well for me, it highlights the changes in all hunks
> automatically. I use it only for diffing before checkin.

I found that with code (more precisely 'diff-refine-hunk') hung up Emacs on
large chunks (10-20 KiB).

So I just use "C-c C-b" instead when I need refine-hunk.

--
Best regards!

[Tom]

Oleksandr Gavenko <gavenkoa <at> gmail.com> writes:

>
> I found that with code (more precisely 'diff-refine-hunk') hung up Emacs on
> large chunks (10-20 KiB).
>
> So I just use "C-c C-b" instead when I need refine-hunk.
>

I should have added that I have an additional check in my
diff hook, which calls the automatic highlighting of hunks
only if the diff is below a certain size, so I have
automatic highlighting for the majority of my diffs which
are small and for the big ones I toggle the highlight manually
if needed.

[Sebastien Vauban]

Hi Tom,

I'm not sure how to check for that, but I could reduce all my `.emacs' file to
an example file which shows the problem on the current development snapshot of
the future Emacs 24.3 (from 2012-10-22):

--8<---------------cut here---------------start------------->8---
;;; emacs-minimal.el

(setenv "LC_ALL" "C")

;; enable Diff Auto Refine mode (highlight the changes with better
;; granularity)

(defun my/diff-refine-all-hunks ()

(interactive)
(condition-case nil
(save-excursion
(goto-char (point-min))
(while (not (eobp))
(diff-hunk-next)))
(error nil)))

(add-hook 'diff-mode-hook
'my/diff-refine-all-hunks)

(message "%s" (emacs-version))
(sit-for 1)
--8<---------------cut here---------------end--------------->8---

You can see, in the screencast on http://screencast.com/t/JOlsyWHh, that:

- the cursor goes to the end of the diff

- only the last diff is refined (see the string " for Diff" which should be
highlighted in the first hunk)

[Tom]

Sebastien Vauban <wxhgmqzgwmuf@...> writes:

>
> You can see, in the screencast on http://screencast.com/t/JOlsyWHh, that:
>
> - the cursor goes to the end of the diff
>
> - only the last diff is refined (see the string " for Diff" which should be
> highlighted in the first hunk)
>

Try it manually. Go to the diff buffer and call M-x my/diff-refine-all-hunks
there.

Does it highlight all the hunks then?

[Sebastien Vauban]

Hi Tom,

No, it doesn't, as you can see on http://screencast.com/t/5NHwKwF7c9h.

What does that mean, then -- excepting that it looks coherent with what is put
in the hook?

[Tom]

Sebastien Vauban <wxhgmqzgwmuf@...> writes:
> >
> > Does it highlight all the hunks then?
>
> No, it doesn't, as you can see on http://screencast.com/t/5NHwKwF7c9h.
>
> What does that mean, then -- excepting that it looks coherent with what is put
> in the hook?
>

The diff highlighting function simply calls a builtin feature repeatedly:

(goto-char (point-min))
(while (not (eobp))
(diff-hunk-next)))

diff-hunk-next is bound to TAB, so you can try going to the diff
buffer, go to the beginning of the buffer and start pressing TAB.

Every TAB press should jump to the next hunk and highlight the
diffs within lines in that hunk. It is done by default.

Here's the relevant code from diff-mode.el:

;; Define diff-{hunk,file}-{prev,next}
(easy-mmode-define-navigation
diff-hunk diff-hunk-header-re "hunk" diff-end-of-hunk diff-restrict-view
(if diff-auto-refine-mode
(condition-case-unless-debug nil (diff-refine-hunk) (error nil))))


The last two lines show that if diff-auto-refine-mode is t
(which is by default) then it should do the highlighting.
If it doesn't the you may want to ask the emacs developers about it.

[Sebastien Vauban]

Hi Tom,

Tom wrote:
> Sebastien Vauban <wxhgmqzgwmuf@...> writes:
>>> Does it highlight all the hunks then?
>>
>> No, it doesn't, as you can see on http://screencast.com/t/5NHwKwF7c9h.
>>
>> What does that mean, then -- excepting that it looks coherent with what is put
>> in the hook?
>
> The diff highlighting function simply calls a builtin feature repeatedly:
>
> (goto-char (point-min))
> (while (not (eobp))
> (diff-hunk-next)))
>
> diff-hunk-next is bound to TAB, so you can try going to the diff buffer, go
> to the beginning of the buffer and start pressing TAB.
>
> Every TAB press should jump to the next hunk and highlight the diffs within
> lines in that hunk. It is done by default.

Going to the beginning of the diff buffer and hitting TAB, does indeed moves
over every hunk, in turn, and refines the differences at the same time.

> Here's the relevant code from diff-mode.el:
>
> ;; Define diff-{hunk,file}-{prev,next}
> (easy-mmode-define-navigation
> diff-hunk diff-hunk-header-re "hunk" diff-end-of-hunk diff-restrict-view
> (if diff-auto-refine-mode
> (condition-case-unless-debug nil (diff-refine-hunk) (error nil))))
>
> The last two lines show that if diff-auto-refine-mode is t (which is by
> default) then it should do the highlighting. If it doesn't the you may want
> to ask the emacs developers about it.

The variable diff-auto-refine-mode is as well t on my side. But... that does
not help.

BTW, how do you explain that your cursor is not at the end of the buffer, in
your case, after the refine process?

[Tom]

Sebastien Vauban <wxhgmqzgwmuf@...> writes:
>
> Going to the beginning of the diff buffer and hitting TAB, does indeed moves
> over every hunk, in turn, and refines the differences at the same time.
>

My function does the same, so I can't explain why it
does not work for you when you call it from M-x.

I guess you also use Emacs 24, so it's not about an old emacs version
behaving differently.

>
> BTW, how do you explain that your cursor is not at the end of the buffer, in
> your case, after the refine process?
>

I use save-excursion in the function which puts the cursor back to
where it was before the function was called.

I don't know why it is at the end of the buffer for you. There are
some emacs gurus on this list. Hopefully some of them will chime in
and helps you to solve this mistery, because I'm out of ideas.

[Sebastien Vauban]

Tom wrote:
> Sebastien Vauban <wxhgmqzgwmuf@...> writes:
>>
>> Going to the beginning of the diff buffer and hitting TAB, does indeed
>> moves over every hunk, in turn, and refines the differences at the same
>> time.
>
> My function does the same, so I can't explain why it does not work for you
> when you call it from M-x.

That's very weird, indeed.

> I guess you also use Emacs 24, so it's not about an old emacs version
> behaving differently.

Yep, sort of future Emacs 24.3, that is:

GNU Emacs 24.2.50.1 (i386-mingw-nt5.1.2600) of 2012-10-22 on DANI-PC

>> BTW, how do you explain that your cursor is not at the end of the buffer,
>> in your case, after the refine process?
>
> I use save-excursion in the function which puts the cursor back to where it
> was before the function was called.

I had overlooked it. Of course.

> I don't know why it is at the end of the buffer for you. There are some
> emacs gurus on this list. Hopefully some of them will chime in and helps you
> to solve this mistery, because I'm out of ideas.

Thanks for your help so far...

Last thing: can you post your complete chunk of code regarding this? I mean
with your additional check which calls the automatic highlighting of hunks
only if the diff is below a certain size? Thanks a lot...

[Tom]

Sebastien Vauban <wxhgmqzgwmuf@...> writes:

>
> Last thing: can you post your complete chunk of code regarding this? I mean
> with your additional check which calls the automatic highlighting of hunks
> only if the diff is below a certain size? Thanks a lot...

It's quite straightforward:


(add-hook 'diff-mode-hook 'my-diff-stuff)


(defun my-diff-stuff ()
(unless (or (eq this-command 'dvc-diff) ; does not work with DVC
(eq this-command 'dvc-generic-refresh)
(> (buffer-size) 20000))
(my-refine-all-diff-hunks)))

(defun my-refine-all-diff-hunks ()
(interactive)
(condition-case nil
(save-excursion

(goto-char (point-min))
(while (not (eobp))
(diff-hunk-next)))

(error nil)))

[Sebastien Vauban]

FYI, have a look at http://debbugs.gnu.org/cgi/bugreport.cgi?bug=12894.

[Tom]

Sebastien Vauban <wxhgmqzgwmuf@...> writes:
>
> FYI, have a look at http://debbugs.gnu.org/cgi/bugreport.cgi?bug=12894.
>

That explains it. I'm on Emacs 24.1 and it does not have this
scheduling code, so that's why it works for me. I didn't think
they changed this recently.

Anyway, you can try calling diff-refine-hunk explicitly after
diff-hunk-next then:

(condition-case nil
(save-excursion
(goto-char (point-min))
(while (not (eobp))
(diff-hunk-next)

(diff-refine-hunk)))
(error nil)))

I can't try it, because I use 24.1, but it may work.

[Sebastien Vauban]

Hi Tom,

I wasn't sure how to adapt the code. Thanks for trying...

And the results of the jury are:
- it works for the refining of *all* hunks
- it does not work wrt the position of the cursor, that is it's at the end of
the buffer...

Thanks for solving already half of the problem!

[Tom]

Sebastien Vauban <wxhgmqzgwmuf@...> writes:

>
> And the results of the jury are:
> - it works for the refining of *all* hunks

Cool. I think the last hunk is actually processed twice, once when
you call the refine function explicitly and once when the timer mentioned
in the bug report explanation kicks in. But it's not a big deal, probably
does not add a noticable delay.

> - it does not work wrt the position of the cursor, that is it's at the end of
> the buffer...
>

I have no idea what causes this. You can try adding a (goto-char (point-min))
call to the end of the refine all function or if it does not work then you
can try using a null timer for this, so it is called after the command is
finished, and hopefully after that code is finished which moves the cursor:

(run-at-time 0.0 nil (lambda () (goto-char (point-min))))

[Sebastien Vauban]

Tom,

Tom wrote:
> Sebastien Vauban <wxhgmqzgwmuf@...> writes:
>>
>> And the results of the jury are:
>> - it works for the refining of *all* hunks
>
> Cool. I think the last hunk is actually processed twice, once when you call
> the refine function explicitly and once when the timer mentioned in the bug
> report explanation kicks in. But it's not a big deal, probably does not add
> a noticable delay.
>
>> - it does not work wrt the position of the cursor, that is it's at the end
>> of the buffer...
>
> I have no idea what causes this. You can try adding a (goto-char
> (point-min)) call to the end of the refine all function

Did not work.

> or if it does not work then you can try using a null timer for this, so it
> is called after the command is finished, and hopefully after that code is
> finished which moves the cursor:
>
> (run-at-time 0.0 nil (lambda () (goto-char (point-min))))

Like this?

--8<---------------cut here---------------start------------->8---
(defun my/diff-refine-all-hunks ()
(interactive)

(condition-case nil
(save-excursion
(goto-char (point-min))
(while (not (eobp))
(diff-hunk-next)
(diff-refine-hunk)))
(error nil))

(run-at-time 0.0 nil (lambda () (goto-char (point-min)))))
--8<---------------cut here---------------end--------------->8---

Yep! This does work perfectly: all hunks refined, and pointer at beginning of
buffer...

Thanks a lot for your more than precious help!

[Tom]

Sebastien Vauban <wxhgmqzgwmuf@...> writes:

>
> Yep! This does work perfectly: all hunks refined, and pointer at beginning of
> buffer...
>
> Thanks a lot for your more than precious help!
>

No problem. I'll also need it when I upgrade emacs, so I helped
my future self as well. :)


BTW, highlighting within the lines is helpful in most of the cases, but
sometimes when the diff is very convoluted it can get in the way. To remedy
this I added a key (f7) which when pressed in the diff buffer toggles
the in-line highlight on and off.

I put it here in case someone else also finds it useful:



(add-hook 'diff-mode-hook 'my-diff-stuff)

(setq my-diff-refine-change-color "yellow1")

(defun my-toggle-diff-refine-change-color()
(interactive)
(if (face-background 'diff-refine-change)
(set-face-background 'diff-refine-change nil)
(set-face-background 'diff-refine-change my-diff-refine-change-color)))


(defun my-diff-stuff ()
(set-face-background 'diff-refine-change my-diff-refine-change-color)
(local-set-key (kbd "<f7>") 'my-toggle-diff-refine-change-color)

(unless (or (eq this-command 'dvc-diff)

[Stefan Monnier]

> That explains it. I'm on Emacs 24.1 and it does not have this
> scheduling code, so that's why it works for me. I didn't think
> they changed this recently.

> Anyway, you can try calling diff-refine-hunk explicitly after
> diff-hunk-next then:

> (condition-case nil
> (save-excursion
> (goto-char (point-min))
> (while (not (eobp))

> (diff-hunk-next)
> (diff-refine-hunk)))
> (error nil)))

> I can't try it, because I use 24.1, but it may work.

If you let-bind diff-auto-refine-mode to nil in the above code, not only
you'll be able to test it on Emacs-24.1 (and earlier), but you'll avoid
refining the hunks redundantly when diff-hunk-next does it.


Stefan

[Tom]

Stefan Monnier <monnier <at> iro.umontreal.ca> writes:
>

> If you let-bind diff-auto-refine-mode to nil in the above code, not only
> you'll be able to test it on Emacs-24.1 (and earlier), but you'll avoid
> refining the hunks redundantly when diff-hunk-next does it.

Good suggestion. Thank you.

What I did not get in the new diff-mode implementation is why it
uses run-at-time instead of run-with-idle-timer?

run-at-time is for specifying a timer which runs at a specific time
while run-with-idle-timer runs it when emacs becomes idle. Isn't
using run-with-idle-timer is more appropriate (makes the code clearer)
in this case?

And why the code uses 0.0 instead of just 0? Is there a difference?

[Sebastien Vauban]

Hi Tom,

"Sebastien Vauban" wrote:
> Tom wrote:
>> Sebastien Vauban <wxhgmqzgwmuf@...> writes:
>>>
>>> And the results of the jury are:
>

> (defun my/diff-refine-all-hunks ()
> (interactive)
> (condition-case nil
> (save-excursion
> (goto-char (point-min))
> (while (not (eobp))
> (diff-hunk-next)
> (diff-refine-hunk)))
> (error nil))
> (run-at-time 0.0 nil (lambda () (goto-char (point-min)))))
>

> Yep! This does work perfectly: all hunks refined, and pointer at beginning of
> buffer...
>
> Thanks a lot for your more than precious help!

After a couple of days using this, I've observed a very tricky collateral
damage in Gnus...

Let's assume I have received 5 mails:

--8<---------------cut here---------------start------------->8---
* 18 Sun 14:25 0.2k Foo Mail #1 Hello
18 Sun 14:25 0.4M Bar Mail #2 How are you?
18 Sun 17:58 6.9k Baz Mail #3 [diff] Fix bug
18 Sun 18:06 4.1k Baz Mail #4 [diff] Fix doc
18 Sun 18:27 3.1k Baz Mail #5 I've done it
--8<---------------cut here---------------end--------------->8---

* = position of the cursor

... and I want first to read the 3rd one. I position the cursor on that one...

--8<---------------cut here---------------start------------->8---
18 Sun 14:25 0.2k Foo Mail #1 Hello
18 Sun 14:25 0.4M Bar Mail #2 How are you?
* 18 Sun 17:58 6.9k Baz Mail #3 [diff] Fix bug
18 Sun 18:06 4.1k Baz Mail #4 [diff] Fix doc
18 Sun 18:27 3.1k Baz Mail #5 I've done it
--8<---------------cut here---------------end--------------->8---

... press RET, get a new buffer created on the mid-right pane, and read it.

--8<---------------cut here---------------start------------->8---
* 18 Sun 14:25 0.2k Foo Mail #1 Hello |
18 Sun 14:25 0.4M Bar Mail #2 How are you? |
18 Sun 17:58 6.9k Baz Mail #3 [diff] Fix bug | (contents of 3rd email)
18 Sun 18:06 4.1k Baz Mail #4 [diff] Fix doc |
18 Sun 18:27 3.1k Baz Mail #5 I've done it |
--8<---------------cut here---------------end--------------->8---

Then, as I always do, I simply press "n" to read the "next one" (4th)... but
Gnus displays me the 1st one!

Why? Because the cursor has been positioned on the first line of my *summary*
buffer (on the left), after displaying the contents of the "Fix bug" email (on
the right).

Why? Because diff-mode is called (as that email contains a diff), and some
some effect -- which I don't understand -- is that *the cursor is repositioned
onto the first line of the summary buffer* (while the code is supposed to be
executed in the other window, the one with the *contents* of the mail).

So, here, to reproduce the problem, I have to:

- have emails with diffs in them (otherwise, diff-mode won't be called)

- skip reading the first email (so that, clicking "n" does open the "next"
unread email... from the current cursor position, which is wrong here...)

I know describing a dynamic behavior is not obvious per email. So, did you
understand the description of the problem, at least?

Do you understand why this is occurring, or have some helpful ideas?

[Stefan Monnier]

>> If you let-bind diff-auto-refine-mode to nil in the above code, not only
>> you'll be able to test it on Emacs-24.1 (and earlier), but you'll avoid
>> refining the hunks redundantly when diff-hunk-next does it.
> Good suggestion. Thank you.
> What I did not get in the new diff-mode implementation is why it
> uses run-at-time instead of run-with-idle-timer?
> run-at-time is for specifying a timer which runs at a specific time
> while run-with-idle-timer runs it when emacs becomes idle. Isn't
> using run-with-idle-timer is more appropriate (makes the code clearer)
> in this case?

run-at-time timers don't run while Elisp is running either, they only
run when Elisp is being interrupted (e.g. during redisplay, or while
running process filters). So the difference is pretty slim (more
specifically, there's only a difference in the case where diff-hunk-next
is run from code which later calls something like sit-for or redisplay).

> And why the code uses 0.0 instead of just 0?

I can't remember.

> Is there a difference?

No, no difference.


Stefan

[Tom]

Sebastien Vauban <wxhgmqzgwmuf@...> writes:
>
> Why? Because diff-mode is called (as that email contains a
> diff), and some some effect -- which I don't understand -- is
> that *the cursor is repositioned onto the first line of the
> summary buffer* (while the code is supposed to be executed in
> the other window, the one with the *contents* of the mail).

I don't use Gnus, but from your decription it seem this line

(run-at-time 0.0 nil (lambda () (goto-char (point-min)))))

causes the problem. It is executed after the current command is
finished and I guess Gnus puts back the cursor to the summary
buffer, so this delayed (goto-char (point-min)) is executed
there.

An obvious fix is to execute the delayed goto-char only if
the cursor is in a diff buffer after the command, that is
don't do it in the Gnus summary buffer:

(run-at-time 0.0 nil (lambda ()

(if (eq major-mode 'diff-mode)
(goto-char (point-min)))))