search replace in repeat pattern
Tarlika Elisabeth Schmitz
I have a csv file with the following content:
country X
aa
bb
cc
dd
country Y
ee
ff
which I would like to convert to
X aa
X bb
X cc
X dd
Y ee
Y ff
Can this be done with search and replace? I have got a search pattern
but don't manage to churn out multiple lines.
--
Best Regards,
Tarlika Elisabeth Schmitz
Tim Chase
>
> country X
> aa
> bb
> cc
> dd
> country Y
> ee
> ff
>
> which I would like to convert to
>
> X aa
> X bb
> X cc
> X dd
> Y ee
> Y ff
>
> Can this be done with search and replace? I have got a search pattern
> but don't manage to churn out multiple lines.
It might be doable in a more complex s&r, but I'd do it in two
passes:
1) bring the "country ___" line down to each non-country line:
:v/^country /?country ?t-|s/country //|j
2) delete the country lines that are now superfluous:
:g/^country /d
Step #1 is a bit packed:
:v/ on every line that doesn't match
country the literal text "country "
/ perform this set of actions:
?country ? look backwards to the most recent
line containing "country "
t and copy it
- to the line before this match
("-" is short-hand for ".-1")
| and then
s/ substitute
country the first "country " on the line
// replacing with nothing
| and then
j join this line (the actual country data)
with the next line (the one to which we
are prepending, e.g. "cc")
Hope this helps both solve the problem and give understanding to
my solution.
-tim
Tarlika Elisabeth Schmitz
>>
>> country X
>> aa
>> bb
>> cc
>> dd
>> country Y
>> ee
>> ff
>>
>> which I would like to convert to
>>
>> X aa
>> X bb
>> X cc
>> X dd
>> Y ee
>> Y ff
>
>
> :v/^country /?country ?t-|s/country //|j
>
>
>2) delete the country lines that are now superfluous:
>
> :g/^country /d
Thank you, Tim, for solution and explanation. This dioes exactly what I
need.
>Step #1 is a bit packed:
>
> :v/ on every line that doesn't match
> [...]
Just to get me started on the more complex commands such as these:
where can I look up commands like :v and :g ?
sc
Schmitz wrote:
> Just to get me started on the more complex commands such as
> these: where can I look up commands like :v and :g ?
both are documented in repeat.txt, accessible via
:help :v
and
:help :g
sc
Tim Chase
> need.
>
>> Step #1 is a bit packed:
>>
>> :v/ on every line that doesn't match
>> [...]
>
> Just to get me started on the more complex commands such as these:
> where can I look up commands like :v and :g ?
The :g and :v commands are documented as "sc" mentioned. The
trick is knowing how to string them together with other Ex
commands to do what you want. The format for both is the same:
:g/pattern/ex_command(s)_to_perform_on_each_matching_line
:v/pattern/ex_command(s)_to_perform_on_each_nonmatching_line
So in your case, I wanted to perform an action (copy from above,
twiddle and join) on each line that didn't match my pattern.
The commands are assembled from the Ex commands cataloged at
:help ex-cmd-index
and made more flexible by the ability to enter powerful ranges
(relative and absolute)
:help :range
and chain them together with the bar
:help :|
-tim
Tarlika Elisabeth Schmitz
Tim Chase wrote:
>> Just to get me started on the more complex commands such as these:
>> where can I look up commands like :v and :g ?
>
>The :g and :v commands are documented as "sc" mentioned. The
>trick is knowing how to string them together with other Ex
>commands to do what you want.
Gosh! The possibilities are endless.
Many thanks to you and sc for putting me on the right track.
As a matter of interest, I have another repeat expression, but within
one line. I have a working solution but it does not seem very elegant:
line ends and finishes with two bars:
|| dfjdfjd || dfjdkfjsd || sds ||
In reality the repeat pattern is longer.
I need to insert some text after the bars:
||X dfjdfjd ||X dfjdkfjsd ||X sds ||
In reality X is fairly long.
My substitution:
:s/^||\(.*\)||\(.*\)||\(.*\)||$/||X \1||X \2||X \3||/
I have mapped this in nomral mode:
:nmap :s/ ... / ... /<Enter>
For some reason, I need to escape the bars now.
Is there a better way of doing this?
Tim Chase
Indeed, the power of global (":g" and ":v") commands along with
the substitute (":s") are two of the things that make vim my
favorite editor. Many hours of my time (and my coworkers' time)
have been saved by them.
> line ends and finishes with two bars:
> || dfjdfjd || dfjdkfjsd || sds ||
> In reality the repeat pattern is longer.
>
> I need to insert some text after the bars:
> ||X dfjdfjd ||X dfjdkfjsd ||X sds ||
> In reality X is fairly long.
>
>
> My substitution:
> :s/^||\(.*\)||\(.*\)||\(.*\)||$/||X \1||X \2||X \3||/
This sounds like "replace every '||' with '|| X' as long as
either (1) it's not the last one or (2) it's got some character
immediately after it (which the last one shouldn't)"
For #1, that would be something like either of these:
s/||\_$\@<!/& X/g
s/||\_$\@<!/|| X/g
Both assert that an end-of-line doesn't happen after the "||"
that gets replaced.
For #2, that would be something like either of these
s/||\ze./& X/g
s/||\(.\)/|| X\1/g
The first form stops the replacement before the "any" (".")
character, while the second one snags the "any" character and
then puts it back in the replacement as "\1"
Both can be thrown off if you have trailing characters (most
likely whitespace) after your last "||" so it might be good
ensure that by pre-processing with
:%s/||\s\+$/||
"replace two pipes followed by one or more whitespace at the EOL,
and replace it with just two pipes (omitting the trailing
whitespace)"
> I have mapped this in nomral mode:
> :nmap :s/ ... / ... /<Enter>
>
> For some reason, I need to escape the bars now.
When used in a mapping command, you need to either escape the
bars or use "<bar>" (5 character) as in
:nnoremap :s/^<bar><bar> ...
I prefer the latter because it's a bit more readable to me, but
that's just my taste. You can read about it at
:help map_bar
Hope this helps,
-tim
Jan-Herbert Damm
Tim Chase wrote on 21.11.09:
> 1) bring the "country ___" line down to each non-country line:
>
> :v/^country /?country ?t-|s/country //|j
> [...]
> Step #1 is a bit packed:
>
> :v/ on every line that doesn't match
> country the literal text "country "
> / perform this set of actions:
> ?country ? look backwards to the most recent
> line containing "country "
I think i don't understand this: why do you look *backward* for an occurence of
"country" after you find a line that doesn't contain "country"?
Intuitively i would walk forward through the occurences of "country".
> t and copy it
> - to the line before this match
> ("-" is short-hand for ".-1")
and why do you do this instead of simply deleting *country* in place?
> | and then
> s/ substitute
> country the first "country " on the line
> // replacing with nothing
> | and then
> j join this line (the actual country data)
> with the next line (the one to which we
> are prepending, e.g. "cc")
thank you for enlightening a simple mind.
jan
Tarlika Elisabeth Schmitz
Jan-Herbert Damm wrote:
>> 1) bring the "country ___" line down to each non-country line:
>>
>> :v/^country /?country ?t-|s/country //|j
>> [...]
>> ?country ? look backwards to the most recent
>> line containing "country "
>
>I think i don't understand this: why do you look *backward* for an
>occurence of "country" after you find a line that doesn't contain
>"country"?
country GB
London
Birmingham
country GER
Berlin
-->
GB London
GB Birmingham
...
When you are on a line without "country" you need to find the country
preceding it.
Tim Chase
>>
>> :v/^country /?country ?t-|s/country //|j
>> [...]
>> Step #1 is a bit packed:
>>
>> :v/ on every line that doesn't match
>> country the literal text "country "
>> / perform this set of actions:
>> ?country ? look backwards to the most recent
>> line containing "country "
>
> I think i don't understand this: why do you look *backward* for an occurence of
> "country" after you find a line that doesn't contain "country"?
Tarlika already answered this (something like "because the
germane country-code is above instead of below"), but skipped the
2nd half (even though it's reasoning falls from her answer)
>> t and copy it
>> - to the line before this match
>> ("-" is short-hand for ".-1")
>
> and why do you do this instead of simply deleting *country* in place?
When you have a "country" line, you don't know how many
non-country lines will follow it, so you have to keep it around
for any further copying-to-subsequent-non-country-lines. So from
her (tweaked by adding Bonn and completing the GER output
section) example:
country GB
London
Birmingham
country GER
Berlin
Bonn
-->
GB London
GB Birmingham
GER Berlin
GER Bonn
It finds London, then copies the "country GB" line before it,
deletes the word "country" and then joins "GB" and "London" to
make "GB London". It then finds Birmingham which also needs to
copy down the previous "country" line and perform the same
sequence of operations. If we had deleted it in-place, it
wouldn't be there to copy down. If we had moved it instead of
deleting it, then we would undo the work we did to put it befor
London. The alternative would still be two passes but something like
1) for each "country" line, join it with the next one and
delete "country "
:g/^country /j|s/country /
2) for each harder-to-identify-non-country-line (harder
because we no longer have "country" on the line to identify
Birmingham and Bonn), copy down the country code from some
harder-to-identify-country-line.
:g/magic-regexp/magic-ex-commands
Hope this helps shed some light on my thought-process :)
-tim
Tarlika Elisabeth Schmitz
Tim Chase wrote:
>> line ends and finishes with two bars:
>> || dfjdfjd || dfjdkfjsd || sds ||
>> -->
>> ||X dfjdfjd ||X dfjdkfjsd ||X sds ||
To be more precise, I know that
- line begins with ||
- line ends with
- there might be a succession of double bars (e.g. ||||||), in which
case X must not be inserted.
>For #1, that would be something like either of these:
>
> s/||\_$\@<!/& X/g
works beautifully!
Presumably, \_$\@<! means "anything but end of line" ?
I can envisage the need for operating this substitution on every other
line of a selected range!
||X dfjdfjd ||X dfjdkfjsd ||X sds ||
|| dfjdfjd || dfjdkfjsd || sds ||
||X dfjdfjd ||X dfjdkfjsd ||X sds ||
|| dfjdfjd || dfjdkfjsd || sds ||
>When used in a mapping command, you need to either escape the
>bars or use "<bar>" (5 character) ...
>
>I prefer the latter because it's a bit more readable to me
agree!
Jan-Herbert Damm
Tim Chase wrote on 23.11.09:
> country GB
> London
> Birmingham
> country GER
> Berlin
> Bonn
> -->
> GB London
> GB Birmingham
> GER Berlin
> GER Bonn
>
> It finds London, then copies the "country GB" line before it,
> deletes the word "country" and then joins "GB" and "London" to
> make "GB London". It then finds Birmingham which also needs to
> copy down the previous "country" line and perform the same
> sequence of operations. If we had deleted it in-place, it
> wouldn't be there to copy down. If we had moved it instead of
> deleting it, then we would undo the work we did to put it befor
> London.
Oh, ok. Beautiful logic.
> Hope this helps shed some light on my thought-process :)
Unfortunately I always have to squeeze considerably to shape my brain into this
procedural(?) thinking.
jan
Tim Chase
> case X must not be inserted.
>
>> For #1, that would be something like either of these:
>>
>> s/||\_$\@<!/& X/g
>
> works beautifully!
I think this fails your last test about "||||||", but as this is
a new detail, I claim leniency :)
To meet your "||||||" test, it should work with
s/||\(|\|\_$\)\@!/& X/g
(I've shifted to the "\@!" which seems to work where the "\@<!"
doesn't...kinda surprised me that didn't work). I tried it on a
line that looked like
|| abc || def |||||| ghi ||
that changed it into
|| X abc || X def |||||| X ghi ||
which I understand was what you wanted.
> Presumably, \_$\@<! means "anything but end of line" ?
close...it means "an EOL can't match here" (the subtle difference
being in the case of nothing..."anything" requires a "something"
and fails on a "nothing"; whereas "nothing" can be satisfied by
the negative "no-EOL-here" assertion) You can read more about it at
:help /\@<!
:help /\@!
> I can envisage the need for operating this substitution on every other
> line of a selected range!
>
> ||X dfjdfjd ||X dfjdkfjsd ||X sds ||
> || dfjdfjd || dfjdkfjsd || sds ||
> ||X dfjdfjd ||X dfjdkfjsd ||X sds ||
> || dfjdfjd || dfjdkfjsd || sds ||
Doing things on odd/even lines requires reaching into Vim's
control structures, something like
:g/^/if line('.') % 2 | s/||\(|\|\_$\)\@!/& X/g | endif
(":g/^/" is a common notation for "do the following set of Ex
commands on every line" because every line has a
"beginning-of-line" and thus matches) It can be generalized with
any arbitrary modulo:
:g/^/if line('.') % n == 1 | actions_on_nth_line | endif
where the "n" is the number of lines you want to skip.
-tim
Tarlika Elisabeth Schmitz
Tim Chase wrote:
>
>> - there might be a succession of double bars (e.g. ||||||), in which
>> case X must not be inserted.
>
>I think this fails your last test about "||||||", but as this is
>a new detail, I claim leniency :)
>
>To meet your "||||||" test, it should work with
>
> s/||\(|\|\_$\)\@!/& X/g
played around with it, tested and mapped!
>(I've shifted to the "\@!" which seems to work where the "\@<!"
>doesn't...kinda surprised me that didn't work).
I didn't understand the subtle differences
> :help /\@<!
> :help /\@!
but, more importantly, I know where to look now. It wouldn't have
occurred to me that those expressions were in the help file.
It's really helpful that you're posting the help command.
>> I can envisage the need for operating this substitution on every
>> other line of a selected range!
>
>
>Doing things on odd/even lines requires reaching into Vim's
>control structures, something like
>
> :g/^/if line('.') % 2 | s/||\(|\|\_$\)\@!/& X/g | endif
Implemented!
Tim Chase
>> doesn't...kinda surprised me that didn't work).
>
> I didn't understand the subtle differences
>
>> :help /\@<!
>> :help /\@!
I'm not really sure I completely understand the differences
myself...I generally use the "\@<!" form for most things, but it
didn't work here, and changing them solved my problem. Hurray
for trial-and-error. :)
> but, more importantly, I know where to look now. It wouldn't have
> occurred to me that those expressions were in the help file.
> It's really helpful that you're posting the help command.
Vim's help is top-notch. Just about every facet you can think of
is contained within. Finding it can be a bit more troublesome,
especially if you know what you want, but don't know how Vim
implements it. Using ":helpgrep" and tab/^D expansion after
typing ":help subject can help get you pointed in the right
direction.
:help :helpgrep
:help {subject}
-tim
Tarlika Elisabeth Schmitz
Tim Chase wrote:
>Vim's help is top-notch. Just about every facet you can think of
>is contained within. Finding it can be a bit more troublesome,
>especially if you know what you want, but don't know how Vim
>implements it. Using ":helpgrep" and tab/^D expansion after
>typing ":help subject can help get you pointed in the right
>direction.
>
> :help :helpgrep
> :help {subject}
I think the only way to get into vim is to learn a bit each time a need
arises.
The support I got the last couple of days has really given me an
incentive to make better use of vim. I spent a good few hours
yesterday playing around with various suggested solutions and that'll
take some time to recoup! But I am already getting the benefit of it.
--
Many thanks,
Tarlika Elisabeth Schmitz