vimdiff-like highlighting within the same file?
Andrei Popescu
I'm trying to add new features to vim's handling of .po files. How could
I highlight the differences between the current msgid and the previous
one?
#, fuzzy
#| msgid ""
#| "The following disk access storage devices (DASD) are available. Please "
#| "select each device you want to use one at a time."
msgid ""
"The following direct access storage devices (DASD) are available. Please "
"select each device you want to use one at a time."
msgstr ""
"Următoarele Dispozitive de stocare cu acces la disc (DASD) sunt disponibile. "
"Vă rugăm să alegeți pe rând fiecare dispozitiv pe care doriți să-l folosiți."
I'd be grateful for any hints or RTFM pointing to the right FM, because
I have no idea where to start.
Regards,
Andrei
--
If you can't explain it simply, you don't understand it well enough.
(Albert Einstein)
Marc Weber
> I have no idea where to start.
I don't know details. I'd guess that the highlighting is implemented
somewhere in Vims C code. But I don't know for sure.
You can always write a script which creates two files you can diff then.
If you add file:line locations you can jump to the original source file
using gf fast. (Jump back by using ctrl-^). Maybe this is a bearable workaround.
Example
tmp-file1:
goto: your.po:20
msgid ""
"The following disk access storage devices (DASD) are available. Please "
"select each device you want to use one at a time."
> msgid ""
goto: your.po:20
tmp-file2:
> "The following direct access storage devices (DASD) are available. Please "
> "select each device you want to use one at a time."
Then you can diff both tmp files.
It should be easy to create those tmp files using Vim script.
Marc Weber
Christian Brabandt
> I'm trying to add new features to vim's handling of .po files. How could
> I highlight the differences between the current msgid and the previous
> one?
I don't understand your question. Can you elaborate, on what the file
looks like and where the previous message id comes from? Please show a
sample file, with which we can see your problem.
regards,
Christian
Marc Weber
Excerpts from Christian Brabandt's message of Wed Jun 30 11:12:03 +0200 2010:
> I don't understand your question. Can you elaborate, on what the file
> looks like and where the previous message id comes from? Please show a
> sample file, with which we can see your problem.
He gave an example. Note that two sentences start with "The following"
He wants to diff both ignoring the '#| ' in the first sentence.
So both sentences which should be diffed are in the same file. That's
how I understood the task
#, fuzzy
#| msgid ""
#| "The following disk access storage devices (DASD) are available. Please "
#| "select each device you want to use one at a time."
msgid ""
"The following direct access storage devices (DASD) are available. Please "
"select each device you want to use one at a time."
msgstr ""
"Următoarele Dispozitive de stocare cu acces la disc (DASD) sunt disponibile. "
"Vă rugăm să alegeți pe rând fiecare dispozitiv pe care doriți să-l folosiți."
Marc Weber
Andrei Popescu
What I posted was an excerpt of a .po file. Here is a full "string" with
comments:
[blank line]
#. Type: select
#. Description
#. :sl5:
#: ../s390-dasd.templates:1002
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ other stuff, not interesting in this case
#, fuzzy
^^^^^^^^ fuzzy flag, indicates the original (usually English) msgid
(string) has changed
#| msgid ""
#| "The following disk access storage devices (DASD) are available. Please "
#| "select each device you want to use one at a time."
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ old English msgid
msgid ""
"The following direct access storage devices (DASD) are available. Please "
"select each device you want to use one at a time."
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ new English msgid
msgstr ""
"Următoarele Dispozitive de stocare cu acces la disc (DASD) sunt disponibile. "
"Vă rugăm să alegeți pe rând fiecare dispozitiv pe care doriți să-l folosiți."
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ translation
[blank line]
"strings" are always separated by (at least) one blank line and a .po
file usually contains a lot of them (hundreds or more), which is why I
won't post a complete file. A translator must not, ever, touch the
"other stuff" and the current msgid.
After editing the translation the fuzzy flag must be removed to indicate
the translation is ok (handled by po.vim) and the previous msgid becomes
useless and is removed as well (I have a patch for po.vim).
It would be of great help to the translator to have the differences
between the previous and current msgid highlighted, especially in long
strings with only small changes.
Christian Brabandt
Thanks, that makes it clearer to me.
I would use the NarrowRegion plugin[1]. Make sure, it uses vertical
split windows, (:let g:nrrw_rgn_vert = 1), set nowinfixwidth in each
narrowed window (:set nowinfixwidth), resize each window to your desired
width and diff each narrowed window (:diffthis). You can then
interactively merge the differences (see :h copy-diff) and when
finished, simply write the Narrowed window. Be sure to read the
documentation of the plugin (:h NrrwRgn.txt)
But then again, I am a little bit biased, as I am the author of the
plugin and there might be better ways to do it.
[1] http://www.vim.org/scripts/script.php?script_id=3075
regards,
Christian
Andrei Popescu
>
> I would use the NarrowRegion plugin[1]. Make sure, it uses vertical
> split windows, (:let g:nrrw_rgn_vert = 1), set nowinfixwidth in each
> narrowed window (:set nowinfixwidth), resize each window to your desired
> width and diff each narrowed window (:diffthis). You can then
> interactively merge the differences (see :h copy-diff) and when
> finished, simply write the Narrowed window. Be sure to read the
> documentation of the plugin (:h NrrwRgn.txt)
I'll try it out, but seems a little too much for just *showing* the
differences (the translator must *not* touch the current msgid and the
previous msgid is just a convenience to easily spot changes).
Marc Weber
> On Mi, 30 iun 10, 12:30:15, Christian Brabandt wrote:
> >
> > I would use the NarrowRegion plugin[1]. Make sure, it uses vertical
> > split windows, (:let g:nrrw_rgn_vert = 1), set nowinfixwidth in each
> > narrowed window (:set nowinfixwidth), resize each window to your desired
> > width and diff each narrowed window (:diffthis). You can then
> > interactively merge the differences (see :h copy-diff) and when
> > finished, simply write the Narrowed window. Be sure to read the
> > documentation of the plugin (:h NrrwRgn.txt)
>
> I'll try it out, but seems a little too much for just *showing* the
> differences (the translator must *not* touch the current msgid and the
> previous msgid is just a convenience to easily spot changes).
Can't you just provide two files? The old and the new one? Most VCS
systems do that anyway
#| msgid ""
#| "The following disk access storage devices (DASD) are available. Please "
#| "select each device you want to use one at a time."
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ old English msgid
You can get rid of those comments: :g/^$|/d
Then you can diff old and new files directly
Then translater will see as well what changed.
Moreover they can see if a id changed but the translation was not
changed yet.
That's even better because translators want to pay attention to
translations which didn't change but whos id changed.
Scripting up the solution I proposed can be done in several minutes.
However I'm not sure wether it serves you best?
Do your translators know Vim ? Or do they use it because of the syntax
highlighting? I"m asking because there are existing gui solutions.
Marc Weber
Andrei Popescu
>
> Can't you just provide two files? The old and the new one? Most VCS
> systems do that anyway
previous .po file, but this is not always the case. Especially bigger
projects will only provide .po files through some web interface.
> #| msgid ""
> #| "The following disk access storage devices (DASD) are available. Please "
> #| "select each device you want to use one at a time."
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ old English msgid
>
> You can get rid of those comments: :g/^$|/d
> Then you can diff old and new files directly
Not really. The workflow is like this (best illustrated with a sample):
#: ../file.c: 123
msgid "translatable string"
msgstr "translation of the string"
1. programmer changes the translatable string in the program source code
and uses automated tools to update the .po. During this update the
changed msgid is completely replaced with the new one (might involve
wrapping changes)
2. fuzzy flag is set for the respective string
3. (optional) for the benefit of translators the old msgid is *added* to
the file and marked as such with the '#|'
#: ../file.c: 123
#, fuzzy
#| msgid "translatable string"
msgid ""
"This is the new translatable string, too big to fit on one line, which"
"is why the line is wrapped"
msgstr "translation of the string"
a) the translator opens the new .po file and updates the translation
b) removes fuzzy flag (and the previous msgid if present) to indicate
that the translation is now ok
#: ../file.c: 123
msgid ""
"This is the new translatable string, too big to fit on one line, which"
"is why the line is wrapped"
msgstr ""
"new translation ....................................................."
"................................................."
(of course, this example shows a big change, were highlighting the
changes is rather unnecessary, but I think you get the point and why the
usual diff tools are not very useful)
> Then translater will see as well what changed.
> Moreover they can see if a id changed but the translation was not
> changed yet.
The fuzzy flag already shows that.
> Scripting up the solution I proposed can be done in several minutes.
> However I'm not sure wether it serves you best?
Because the previous msgid is not needed in the translated .po file I
thought of pre-processing the .po file[1]. Unfortunately I don't have
the programing skills for that either :(
[1] http://nuvreauspam.ro/2010/05/6-translate-tool-needed/
> Do your translators know Vim ? Or do they use it because of the syntax
> highlighting? I"m asking because there are existing gui solutions.
The translator is me :) I already tried the GUI tools, but I still
prefer vim, even without this feature.
I'm also sure, that any other translators using vim will be very
grateful for such a feature, which is why I plan on submitting the
feature as patch to the ftplugin or the syntax file for .po (whatever
makes more sense).
Marc Weber
> thought of pre-processing the .po file[1]. Unfortunately I don't have
> the programing skills for that either :(
Can you describe this preprocessing?
What do you expect from the highlighting?
one line:
"Hello World"
two lines
"Hello"
"World"
should they "differ" ?
file 1:
#| msgid "foo bar"
msgid "foo bar changed"
if you created a second file from that:
#| msgid "foo bar"
msgid "foo bar" << duplicate the msgid found in comments
can you diff both files then? Is this what you're looking for?
Can you illustrate which example you want to be highlighted?
Marc Weber
Jürgen Krämer
Hi,
maybe the following can be a start. Open your .po file and execute these
commands (the fourth command is one long line):
:%y
:vert new
:put!
:%s/^\(#| msgid ""\n\(\_.\{-\}\n\)msgid ""\n\)\(\_.\{-\}\n\)\(msgstr ""\)/\=submatch(1).substitute(submatch(2), '#| ', '', 'g').submatch(4)/
:nohls
:diffthis
:wincmd w
:diffthis
Modifications are now highlighted in the new string -- but not in the old
string.
Regards,
Jï¿œrgen
--
Sometimes I think the surest sign that intelligent life exists elsewhere
in the universe is that none of it has tried to contact us. (Calvin)
Andrei Popescu
>
> maybe the following can be a start. Open your .po file and execute these
> commands (the fourth command is one long line):
>
> :%y
> :vert new
> :put!
> :%s/^\(#| msgid ""\n\(\_.\{-\}\n\)msgid ""\n\)\(\_.\{-\}\n\)\(msgstr ""\)/\=submatch(1).substitute(submatch(2), '#| ', '', 'g').submatch(4)/
> :nohls
> :diffthis
> :wincmd w
:wincmd o
> :diffthis
>
> Modifications are now highlighted in the new string -- but not in the old
> string.
Cool ;)
Efraim Yawitz
:virtbuf bufname filename startline endline
to create a virtual buffer, diffing it with another real or virtual buffer would be easy.
I'm wondering if this isn't already possible with some of the vimscript commands. If not, I may look into how to do it in the C code.
Ephraim
Charles Campbell
http://www.vim.org/scripts/script.php?script_id=3075 .
Regards,
Chip Campbell
Adam Monsen
http://stackoverflow.com/questions/3231759/how-can-i-visualize-per-character-differences-in-a-unified-diff-file
(same link, tiny: http://tinyurl.com/363f8he )
Adam Monsen
Someone posted a response to that question. Apparently there's another editor
that does highlight intra-line differences in a unified diff file. It's
interesting, it works... I tried it out. Having used Vim for about 10 years, I'm
having trouble adjusting to the crazy keystrokes of that other editor.
Sorry, just a little kidding around there when I say "other editor". Anyway, I
brought this up because maybe that would be a useful working demo on which to
base a Vim intra-file intra-line highlighting script.
Seems like the major bits of functionality in such a script might be
* confirm current buffer is a unified diff file, if so, proceed
* identify diff lines
* identify parts of lines that differ
(maybe with the help of "diff" or " program?)
* alter highlighting so intra-line differences stand out
* bind some keystroke to jump to each difference
I'm fairly certain Vimscript could do all this, but I'm stumped on the third
bullet... anyone have ideas on how to do this?
Hmm, another idea would be using Python... that might be more straightforward.
Charles Campbell
plugin. A procedure, after having installed NarrowRegion:
* edit buffer
* select lines with V
* :NR (this will make a "narrow region" buffer holding just those lines)
* select other lines with V
* :NR
* in each narrow-region window, type :diffthis
Regards,
Chip Campbell
Adam Monsen
> A procedure, after having installed NarrowRegion:
>
> * edit buffer
> * select lines with V
> * :NR (this will make a "narrow region" buffer holding just those lines)
> * select other lines with V
> * :NR
> * in each narrow-region window, type :diffthis
Thanks, Chip!
I have indeed tried that, and it works fine.
But, IMHO,
* it is cumbersome (unless scripted/mapped, but see below)
* it takes the diff out of context
What if you have a patch with 30 hunks? Yes, I could map a small function to
create the narrow regions while cycling through each diff, but why can't Vim
just go ahead and highlight all intra-line differences for a whole diff file?
Andy Wokula
Attached script defines a :DiffMsgId command, which attempts to
overlay diff highlighting to the current section using matchadd().
E.g. it works on the following section:
#| msgid ""
#| "The following disk access storage devices (DASD) are available. Please "
#| "select each device you want to use one at a time."
msgid ""
"The following direct access storage devices (DASD) are available. Please"
"select each device you want to use one at a time."
msgstr ""
"Urmatoarele Dispozitive de stocare cu acces la disc (DASD) sunt disponibile."
"Va rugam sa alege?i pe rând fiecare dispozitiv pe care dori?i sa-l folosi?i."
Not yet tested with UTF-8
--
Andy
Christian Brabandt
On Do, 19 Aug 2010, Adam Monsen wrote:
> > May I suggest that you look into Christian Brabandt's NarrowRegion plugin.
> > A procedure, after having installed NarrowRegion:
> >
> > * edit buffer
> > * select lines with V
> > * :NR (this will make a "narrow region" buffer holding just those lines)
> > * select other lines with V
> > * :NR
> > * in each narrow-region window, type :diffthis
[…]
>
> But, IMHO,
> * it is cumbersome (unless scripted/mapped, but see below)
How would you like the plugin to handle that automatically. If you make
good suggestions, I might implement that.
regards,
Christian
Adam Monsen
> good suggestions, I might implement that.
Do you mean the narrow region plugin? Actually, that plugin may not be
involved at all.
Here's a use case for what I'm envisioning.
(1) one plain text file in unified diff format is opened in vim
(2) vim auto-detects that this is a unified diff file
(3) for lines that have changed, the *parts* of lines that have
changed are specifically highlighted (similar to the highlighting
provided when two files are opened with vimdiff)
Here's a mockup:
* http://yfrog.com/j1vimintrafilediffhighligp
* http://imagebin.ca/view/LmlbQoR.html
Bonus use case step:
(4) provide keybindings that can
(a) cycle through differences
(b) cycle through intra-line differences
Does that make sense?
Christian Brabandt
Well, Andy provided a short script for po files, that attempted to
overlay diff-highlighting to changed regions. Something like this might
be possible with unified diff files, although I am not sure how to get
the right lines to compare with.
So I went the other way and hacked my NrrwRgn Plugin to provide the :NUD
(Narrow Unified Diff, it does not really make sense, to use :NUD in any
other filetype) command, that opens a chunk in diff mode in a separate
window.
:NUD will open the chunk on which the cursor is currently in 2 new split
windows in diff mode and whenever you move around in your patch file,
use :NUD to update your windows. (I find it easy enough to navigate in
such a patch file [/^@@ to move to the next chunk], so I did not include
a mapping to move to the next chunk, besides, such a mapping would not
fit well into the NarrowRegion plugin anyway, but would need to be in a
filetype specific plugin).
This is an experimental feature, but please try it out. Unfortunately
vim.org seems to have database problems currently, but you can also get
it from http://github.com/chrisbra/NrrwRgn (the NrrwRgn.vba file
contains everything). Granted, this is not exactly what you asked for,
but this seems to work well enough.
regards,
Christian
Adam Monsen
I did, and I'm impressed! Nice work! This is actually a novel way to tweak a
diff, if one wanted to do such a thing. The only issue I ran into was when I did
":only!" in the window with the original .diff file I got a bunch of errors.
I'm still hacking on my "highlight only" version, but it's slow going.
Christian Brabandt
On Do, 26 Aug 2010, Adam Monsen wrote:
> > This is an experimental feature, but please try it out.
>
> I did, and I'm impressed! Nice work! This is actually a novel way to
> tweak a diff, if one wanted to do such a thing. The only issue I ran
> into was when I did ":only!" in the window with the original .diff
> file I got a bunch of errors.
Nice to hear it is actually useful. I am looking into your error (I only
seldomly use :only, so this is a bug I have never run into).
>
> I'm still hacking on my "highlight only" version, but it's slow going.
Please share!
regards,
Christian
--
Was zugunsten des Staates begonnen wird, geht oft zuungunsten der Welt
aus.
-- Karl Kraus
Adam Monsen
>
> Please share!
I don't really have anything useful to share at this point, but I can
tell you where I'm at. I'll start at the beginning so folks can join
in even if they've missed earlier posts in this thread.
I want improved syntax highlighting for unified diff/patch files in
Vim. I've mentioned this in various forms earlier, but I think that
statement hits it more accurately and succinctly.
I'm trying to use Vim script identify areas in unified diffs showing
changes in lines (as opposed to additions or deletions) and use a more
striking highlighting mode to point out, say, when a single letter or
space changes in a line. This is quite difficult to spot visually in a
unified diff. Vimdiff finds and highlights these types of changes
quite nicely, emacs diff mode does as well, and "git diff
--color-words" (now called --word-diff, I think) also does a great
job. I think I need to use or emulate the intra-line difference
location algorithm(s) from those tools. I deliberately left out wdiff,
whose output I don't particularly like. Of those listed, only emacs
diff mode actually does the intra-line word difference highlighting
for a single unified diff file (the others require both the old and
new files).
Once I've identified changes in hunks, I plan on using something like
the following snippet to highlight specific parts of lines (thanks to
bairui on #vim for the pointer on \%v):
:highlight DiffRegion ctermbg=red guibg=red
:let m = matchadd("DiffRegion", '\%6l\%9v.*\%14v')
:let n = matchadd("DiffRegion", '\%7l\%9v.*\%16v')
I'm stuck trying to identify changes in words or spaces.
I started by writing a line-by-line scanner ( http://pastey.net/140210
) that would detect when it started seeing the "old" ('^-') and "new"
('^+') parts of a change hunk, but then I realized that I'd rather
just manipulate pointers to those different parts of hunks and walk
through the parts in parallel... and I'm kinda lost here.
Another approach I played with a little was using a regex (
http://pastey.net/140211 ) to get to each hunk, something like this:
'\nindex.*\n--- .*\n+++ .*\n@@.*@@.*$'
And this is about the point that I've decided it's going to be quite a
tricky task to actually get all the use cases covered for intra-line
differences.
Here's sort of a collection of use cases, in one unified diff.
tmp.diff: http://pastey.net/140214
old : http://pastey.net/140212
new : http://pastey.net/140213
To see how other tools highlight stuff, I've been doing stuff like:
- vimdiff old new
- git diff old new
- git diff --color-words old new
- emacs tmp.diff (then ALT-n once emacs starts up)
Anyway, I'm probably going to set this stuff down for a while. It
would be fun to have a little in-person hack session on this,
especially with someone at least mildly talented in Vim script and/or
general scanning/parsing algorithms. Vim script might not even be
what's blocking me... for some reason I'm also really getting stuck
breaking down/identifying/solving the problem about how to identify
which parts of a line have changed. Tokenizing/scanning/whatever you
call it is partly subjective--see the new "git --word-diff-regex"
command line argument, and discussion around same on the git mailing
list. In fact, the git solution might be the most elegant and flexible
algorithm to emulate.
That's all for now, cheers,
-Adam
James Cole
Hi,
> ...I'm trying to use Vim script identify areas in unified diffs showing
> striking highlighting mode to point out, say, when a single letter or
> space changes in a line.
>
> To see how other tools highlight stuff, I've been doing stuff like:
> - vimdiff old new
> - git diff old new
> - git diff --color-words old new
> - emacs tmp.diff (then ALT-n once emacs starts up)
>
>
> for some reason I'm also really getting stuck
> breaking down/identifying/solving the problem about how to identify
> which parts of a line have changed. Tokenizing/scanning/whatever you
> call it is partly subjective--see the new "git --word-diff-regex"
> command line argument, and discussion around same on the git mailing
> list. In fact, the git solution might be the most elegant and flexible
> algorithm to emulate.
>
expert on all-things-diffing, but I thought that Neil Fraser's
diff_match_patch (which I have no affilation with; available here:
http://code.google.com/p/google-diff-match-patch/) might be of
interest. It handles intra-line changes nicely, and the souce code is
available to look at. According to its page:
"This library implements Myer's diff algorithm (http://
neil.fraser.name/software/diff_match_patch/myers.pdf) which is
generally considered to be the best general-purpose diff. A layer of
pre-diff speedups and post-diff cleanups surround the diff algorithm,
improving both performance and output quality.
This library also implements a Bitap matching algorithm (http://
neil.fraser.name/software/diff_match_patch/bitap.ps) at the heart
of a flexible matching and patching strategy (http://neil.fraser.name/
writing/patch/)."
Regards,
James.
Adam Monsen
Awesome! Thanks for the pointer, James!
The Python version of diff-match-patch seems to work pretty dang well.
I did a few quick tests with diff_main() and diff_cleanupSemantic().
The API is simple, clear, and predictable! This could be used in a Vim
script if Python support is compiled into Vim.
Porting diff-match-patch to Vim script is another option... that might
be kinda fun.
Hmm, that gives me another idea... Vim also has/uses some diffing
algorithm since Vimdiff highlights differences in lines. We could
change Vim itself to expose whatever diffing logic vimdiff uses so Vim
scripts could use it. Maybe even using an API similar to
diff-match-patch.