/some test/ {
if (foo != "bar") {
if (baz != ""){
array1[xyz] = N
if (N > max_N) max_N = N
}
xyz = temp[1]
# Create
print "," > "something.sql"
split(description,temp,parse_descriptions[file])
print "\t-- "temp[1] > "something.sql"
printf "\t%s %s", xyz, pg_type > "something.sql"
# Load
insert = insert",\n\t"xyz
fields[M]=xyz
M++
N = 1
}
field_to_XX_field[xyz,N] = field
N++
}
position the cursor on the '{' that ends the first line. Then do =%. You get
/some test/ {
if (foo != "bar") {
if (baz != ""){
array1[xyz] = N
if (N > max_N) max_N = N
}
xyz = temp[1]
# Create
print "," > "something.sql"
split(description,temp,parse_descriptions[file])
print "\t-- "temp[1] > "something.sql"
printf "\t%s %s", xyz, pg_type > "something.sql"
# Load
insert = insert",\n\t"xyz
fields[M]=xyz
M++
N = 1
}
field_to_XX_field[xyz,N] = field
N++
}
The line 'N = 1' is indented by an extra space, incorrectly, and the
closing '}' is also incorrectly indented. Don't focus on the code
itself or its correctness. I modified it to make it simpler for the
purpose of this example and also for privacy. The point is that it is
syntactically correct awk and the behavior I'm showing you here is
identical to the indentation I get with the unmodified, semantically
correct code.
This is one of a number of problems I've had with indentation. Looking
on the web for the usual discussions of emacs vs. vim, it's not hard
to find the observation that one of emacs' advantages is more accurate
indentation (emacs indents this code snippet correctly).
Unfortunately, my own experience is confirming this. I prefer vim in a
lot of ways, but when programming, indentation is a critical tool and,
for me, if I can't trust the editor to do it correctly, it's not of
much use to me. In the work I'm currently doing, I've been forced to
use emacs in viper mode because of this issue.
I'm not trying to start an editor religious war here. I'm simply
making an observation about something that I think is an important
area where vim seems somewhat weak. vim and awk have been around for a
long time. it ought to be able to indent awk code correctly and I
would suggest that fixing indentation problems should take priority
over adding new features.
(By the way, you get the above behavior with no ~/.vimrc and all
options set to their defaults. The behavior remains the same when you
turn on autoindent or cindent. This is with a vim built from source
up-to-date via git as of last night. Version is 7.3.315 running on a
Slackware 13.37 64-bit system.)
/Don
[...]
I guess that the R indent script may work better for awk code. The awk
script was my starting point to develop the R one. Could you please
set the file type as "r" before indenting the code and tell us whether
the indentation becomes correct? As you certainly know, to set the
file type you should do:
:set ft=r
Note: Bram added the indent/r.vim to Vim's runtime recently (today, I think).
Best regards,
Jakson Aquino
Vim's indentation of most languages (except C) is done by plugins.
These reside in $VIMRUNTIME/indent and are contributed and
maintained by a number of different authors. In the case of awk,
the indentation plugin is $VIMRUNTIME/indent/awk.vim. At the top of
that file is the author's name, e-mail address, and an invitation to
contact him with bug reports. He would be the person to contact
first with a bug report.
I have cc'd him on this reply.
I took a look at the plugin and think I found the problem and a
solution. The plugin tries to identify continuing lines and indents
the subsequent line an additional amount. It tries to identify
continuing lines by looking for certain patterns such unfinished
arithmetic statements including lines ending with '+'.
Unfortunately, its pattern for '+' at the end of the line also
matches '++', so it thinks that "N = 1" and the last '}' are
continuation lines.
This patch seems to fix that problem for the particular cases of the
++ and -- operators.
--------------------------------------------------------------------
*** awk.vim.orig 2011-05-16 14:55:32.553959000 -0700
--- awk.vim 2011-09-15 12:48:40.004332000 -0700
***************
*** 188,193 ****
--- 188,195 ----
function! s:Seems_continuing( line )
" Unfinished lines
+ if a:line =~ '\(--\|++\)\s*$'
+ return 0
if a:line =~ '[\\,\|\&\+\-\*\%\^]\s*$'
return 1
endif
--------------------------------------------------------------------
If you apply that patch, make sure to do so to a copy of awk.vim in
your ~/.vim/indent directory rather than to the original in
$VIMRUNTIME/indent.
Regards,
Gary
I tried this with the same build of vim as I used for the example in
my original post, current as of last night (on the East Coast of the
US). Setting the filetype to R does indeed produce the correct result,
but after a 3-second delay. Doing the same in emacs/viper is
essentially instantaneous.
Now, honesty compells me to mention that I am running this on a
home-brew micro-ITX machine with an Intel D510MO motherboard, which
includes a dual-core 1.6 Ghz Atom processor. Each core is equivalent
in performance to about an 800 Mhz Core 2 processor, so about 3x
slower than the Intel processors we routinely encounter. So while you
could argue that this would be 3x less a problem on a normal
processor, I think you need to look at your algorithm. I tried it on a
300-line section of awk code and it took 15 or 20 seconds on this
machine, so this gets too slow pretty quickly as the number of lines
increases That's going to be annoying even on a normal machine.
emacs/viper took about a second to indent this same section.
Thanks for your response. Despite the performance issue, which I'm
confident can be fixed, your R version is a definite improvement over
the awk version.
/Don
>
> Best regards,
>
> Jakson Aquino
>
> --
> You received this message from the "vim_dev" maillist.
> Do not top-post! Type your reply below the text you are replying to.
> For more information, visit http://www.vim.org/maillist.php
>
Thanks very much. I'll give it a try and report back.
/Don
Yes, your patch fixes the issue I reported. Thanks very much.
/Don
> Vim's indentation of most languages (except C) is done by plugins.
> These reside in $VIMRUNTIME/indent and are contributed and
> maintained by a number of different authors. In the case of awk,
> the indentation plugin is $VIMRUNTIME/indent/awk.vim. At the top of
> that file is the author's name, e-mail address, and an invitation to
> contact him with bug reports. He would be the person to contact
> first with a bug report.
>
> I have cc'd him on this reply.
I communicated with Eric about the need to fix awk indent plugin six years ago. Recently, Eric sent me a new version of the plugin.
He's obviously looking at it again and trying to fix it.
Unfortunately, even the new version does not work well on my large AWK programs. I use the standard style from "The AWK Programming Language" book by Aho, Kernighan, and Weinberger.
I did not have motivation to further pursue the change because:
(a) I rarely write in AWK these days,
(b) I wrote my plugin which perfectly indents over 1000 lines of AWK.
I just tried Donald's program and it indents it perfectly.
Donald, can you try it and see if it works better on your other AWK programs. It's attached.
Best regards,
Zvezdan Petkovic
Thanks for sending this. Unfortunately, it does not work correctly on
all of my code. I've also found parts of my code where the plugin
supplied with vim does not work correctly even with Gary Johnson's
patch. If I have a chance, I will send examples tomorrow, but I need
to get my work done (as opposed to fighting with an editor) and will
have to continue with emacs/viper, since I don't presently have a vim
awk indenter that works correctly in all cases and is acceptably fast.
But I do very much appreciate the efforts of everyone to help with
this and I will continue to try to find some time to document the
problems.
/Don
>
> Best regards,
>
> Zvezdan Petkovic
> If I have a chance, I will send examples tomorrow, but I need
> to get my work done (as opposed to fighting with an editor)
No hurry. The work is more important.
FWIW, I'll be at a conference the next two days.
I probably will not be able to reply promptly.
Zvezdan
I agree that the indent/r.vim script is very slow, but I don't know
how to improve the speed. For each line to be indented, the script has
to calculate what was the indent level of the last command, but the
last command is not always the line above. So the algorithm goes
slowly upwards until finding the beginning of the last command. I
think it would be necessary to take a different approach.
Best regards,
Jakson
I haven't read your code, but perhaps remember the indent level of the
'last command' as you make your forward pass over the indentation
region?
Having said that, I will observe that I've found problems with the
emacs/viper awk indentation. They took awhile to turn up, but there
are problems there, too. Your indenter, slow though it may be, is the
most consistently correct of any of them and I'm now using it, which
allows me to return to vim, which, as I said earlier, I prefer.
/Don
>
> Best regards,
>
> Jakson
Thanks for having a look at this. You already have my example.
/Don
>
> Regards,
>
> Erik.
Yes, I have already thought about this. Instead of going backwards
looking for the beginning of the command the script could first go
backwards looking for any character in column 1 or the beginning of
the file and, then, goes forward, remembering the current indent
level. While going forward, it would be possible to keep track of the
indentation level. This could be faster, but I don't have the
necessary spare time to implement this alternate algorithm.
Best regards,
Jakson
I can certainly understand the spare time problem :-)
Erik --
Just to be sure I am interpreting your several messages correctly, you
sent a message prior to the one to which I'm replying -- it began "I
think good news ..." and had an awk.vim attached. Your subsequent
message (above) says that found a problem with "the latest official
one" and that another version was coming, so I am not testing the
"good news" version. Presumably you are still working on that, as I
haven't seen it yet (I'm not trying to rush you; I'm just trying to
make sure I've got this straight)?
/Don
>
>
> Erik.
It works on the example I gave you, but it does not work on this
(which was indented by the R indenter):
file != previous_file && previous_file in create_pivoted_postamble {
# Finish up field we just worked on
if (previous_field != "") {
periods_per_field[previous_field] = N
if (N > max_N) max_N = N
}
# Create
print "," > "create_pivoted.sql"
print create_pivoted_postamble[previous_file] > "create_pivoted.sql"
# Load
for (i=1; i<max_N; i++) {
printf "%s)\nselect\n%s", insert,
sprintf(load_pivoted_select_preamble[previous_file],i) >
"load_pivoted.sql"
for (j=1; j<M; j++)
if (i<periods_per_field[fields[j]])
printf ",\n\t%s", field_to_SI_field[fields[j],i] > "load_pivoted.sql"
else
printf ",\n\t%s", "NULL" > "load_pivoted.sql"
printf "\n%s\n", sprintf(load_pivoted_postamble[previous_file],i) >
"load_pivoted.sql"
}
# Duplicate elimination
print ";\n" > "duplicate_elimination_pivoted.sql"
}
Running the new awk indenter produces:
file != previous_file && previous_file in create_pivoted_postamble {
# Finish up field we just worked on
if (previous_field != "") {
periods_per_field[previous_field] = N
if (N > max_N) max_N = N
}
# Create
print "," > "create_pivoted.sql"
print create_pivoted_postamble[previous_file] > "create_pivoted.sql"
# Load
for (i=1; i<max_N; i++) {
printf "%s)\nselect\n%s", insert,
sprintf(load_pivoted_select_preamble[previous_file],i) >
"load_pivoted.sql"
for (j=1; j<M; j++)
if (i<periods_per_field[fields[j]])
printf ",\n\t%s", field_to_SI_field[fields[j],i] > "load_pivoted.sql"
else
printf ",\n\t%s", "NULL" > "load_pivoted.sql"
printf "\n%s\n", sprintf(load_pivoted_postamble[previous_file],i) >
"load_pivoted.sql"
}
# Duplicate elimination
print ";\n" > "duplicate_elimination_pivoted.sql"
> I think I have this issue is fixed, as well as every other pending
> complaint.
Not all issues yet. :-)
> May I ask all involved in this thread to evaluate?
Gladly.
> You can find the indenter here: http://dl.dropbox.com/u/26176183/awk.vim
Thanks for providing this.
> Tested with vim 7.3 and 6.3, on several awk programs. It works fine on
> 7.3. It fails on vim 6.3 because imho cursor( v:lnum, 1) does not work
> (at least during indent).
>
> Hope you like it.
First, the good news: It works much better than before.
The following doesn't work.
1. This may be a matter of preference but I think the continued lines
are not aligned correctly. For example, the code I aligned as this:
vcard_tr_palm = "ADR;WORK ADR;HOME ADR " \
"TEL;WORK TEL;FAX " \
...
is aligned by awk.vim as this:
vcard_tr_palm = "ADR;WORK ADR;HOME ADR " \
"TEL;WORK TEL;FAX " \
...
IOW, I believe that continued lines should be aligned either:
(a) fixed amount (e.g. one shiftwidth), or
(b) under the first line part to the right of equal sign
The awk.vim indents it exactly under the = sign.
Solution: If you move it two spaces further from = sign it would get
aligned under the first line text.
Another example:
param_regex = (options["v2"] || options["v2e"]) ? ptext_regex : \
name_regex "=" pvalue_regex "(," pvalue_regex ")*"
2. This remaining ones are definitely bugs.
This code (my alignment):
if ($i !~ ldif_regex &&
$i !~ /^mozilla_AimScreenName:[:<]? */) {
print "Ignoring badly formed line: " $i >"/dev/stderr"
continue
}
if (attr != "") {
if (base64) value = decode(value)
parse_and_store_ldif_value(attr, value)
}
...
is aligned by awk.vim as this.
if ($i !~ ldif_regex &&
$i !~ /^mozilla_AimScreenName:[:<]? */) {
print "Ignoring badly formed line: " $i >"/dev/stderr"
continue
}
if (attr != "") {
if (base64) value = decode(value)
parse_and_store_ldif_value(attr, value)
}
...
So, everything is indented wrongly additional four spaces because
the condition of the if statement is broken in two lines.
The indentation is relative to the second line of the condition.
The closing brace "}" is wrongly indented, and then everything below
that brace as well (the next if statement).
It seems that every continued condition in my code was indented
wrongly as shown above. Some of them were broken after && and some
after || operator.
Solution: Indent relative to the line where if statement starts.
Similarly for other statements (e.g., while, for, ...)
3. The following is not indented correctly (my alignment first):
if (!(a in ab_a)) {
if (toupper(a) in ab_a)
move_value(a, toupper(a))
else
delete person[a]
}
It was indented by awk.vim as follows:
if (!(a in ab_a)) {
if (toupper(a) in ab_a)
move_value(a, toupper(a))
else
delete person[a]
}
The line below if without the brace is not indented relative to if.
Solution: Indent it relative to if.
Similarly for if statement without braces surrounded by
while or for statement with the braces.
4. In a long if-else if-else if ... else statement:
} else if (backend == "mutt" || backend == "mailrc") {
a = attr " " value
} else { # mh
a = value
}
awk.vim indents as follows:
} else if (backend == "mutt" || backend == "mailrc") {
a = attr " " value
} else { # mh
a = value
}
The statement inside else is not indented correctly (perhaps because
of that # mh comment? or because it's a cuddled "} else {"
[standard style])
Solution: Indent it even if the comment follows cuddled "} else {".
5. The following code gets incorrect indentation and after that
everything is so broken that looking at the diff becomes useless.
My indentation first. (The standard awk style is to declare input
arguments for the function first, then several spaces or TAB, then
the local variables. Since this function has a lot of local
variables, they couldn't all fit on a line with that TAB separator.)
function format_alias(fmt, quote, sep,
i, n, q, a, m, s, name, email) {
if (!("n" in person && "sn" in person && "nickname" in person) &&
!("member" in person && "nickname" in person)) {
if ("email" in person) {
print "Missing name, surname, or nickname for " \
person["email"]
}
return
}
...
}
This was indented by awk.vim as follows:
function format_alias(fmt, quote, sep,
i, n, q, a, m, s, name, email) {
if (!("n" in person && "sn" in person && "nickname" in person) &&
!("member" in person && "nickname" in person)) {
if ("email" in person) {
print "Missing name, surname, or nickname for " \
person["email"]
}
return
}
...
}
Obviously, this is just a more pronounced example of continued
parenthesized expression (e.g., condition or argument/variable list)
gone wrong. Notice also that a continued condition in the first
if statement is indented 7 spaces instead of a full shiftwidth.
Is that because of "!"? It should be simply aligned as shown in
the original.
Solution: Indent based on the line that starts the function
(with the function keyword).
Erik, I hope this helps.
I'm quite sure that if you fix the continued parenthesized line
handling everything will pretty much fall into place.
The alignment of continued assignments should be an easy fix.
I don't know what is causing that cuddled else problem,
but that would be the only remaining thing to fix.
If you don't have my large awk program any more,
I can send it off the list.
Best regards,
Zvezdan
The command "call" is missing in lines 179 and 181.
Best regards,
--
Jakson
> May I ask for volunteers for testing again.
This is very close.
It definitely fixes issues I reported the last time.
There is a regression bug in handling this:
if (!(attr in urls))
value = escape_txt_chars(value)
# more code here.
It turns it into this:
if (!(attr in urls))
value = escape_txt_chars(value)
# more code here.
This happens also for else if (...), else, for, while, followed by a single statement. With or without a blank line following the statement, the indent is increased by an extra shiftwidth.
I've sent you my large awk program off the list so you can test easier.
Best regards,
Zvezdan
I'll be back
Erik.
> Can you have a look again. Let's see what surprises will pop up.
The indent now works perfectly on several of my large scripts including the one I sent you. The only difference is the alignment of continuation lines in the original. I personally stopped aligning things long time ago and prefer your choice of a fixed indentation amount relative to the previous line. Perhaps the amount of indent could be made configurable similar to C-indent (0.5s, 1s, etc.), but, it's not necessary.
Those are good news.
I was really happy to see it work on the large awk program so well.
I didn't have time to look deeply over all of my awk programs.
A quick comparison of some smaller programs has uncovered couple of issues.
1. It seems that side comments are still confusing the indent.
For example:
if (comment && $0 ~ comment_expr || in_comment) {
if ($0 ~ /:[ ]*$/) { # the last line ends in : not :\
in_comment = 0
} else {
in_comment = 1
}
$0 = "#" $0
}
was indented as:
if (comment && $0 ~ comment_expr || in_comment) {
if ($0 ~ /:[ ]*$/) { # the last line ends in : not :\
in_comment = 0
} else {
in_comment = 1
}
$0 = "#" $0
}
If the side comment is deleted it indents correctly.
2. A backslash on the line that is **not** a line continuation, confuses
the indent and it treats the next line as a continuation.
For example:
split($1, line, /[ \t]/) # line: *** 321,52 ****
split(line[2], start, /,/)
was indented as:
split($1, line, /[ \t]/) # line: *** 321,52 ****
split(line[2], start, /,/)
Again, if the side comment is removed, the line is indented
correctly. So, perhaps this will be fixed by fixing 1.
3. In a quick, one-off try, this:
if (c > 0) name = substr(narr[1], 1, c-1)
else name = narr[1]
if (k > 1) {
...
}
...
was indented as:
if (c > 0) name = substr(narr[1], 1, c-1)
else name = narr[1]
if (k > 1) {
...
}
...
We could argue that this style is bad. I would not use it.
But there is awk code written in this style out there and the indent
seems to be confused by the one-liners.
As I get some time I'll look and compare with my other programs, but I probably will not have time for that before the weekend.
Best regards,
Zvezdan
Eric,
I'm so sorry, but I just could not find the time to test.
I promise, as soon as I get some free time, I'll look into it.
It may take another week or two.
Zvezdan
The attached patch fix a minor bug in the script.
--
Jakson
Hello,
-- <snip> --
> Thanks for pointing that one out again. I have applied your patch
>
> Erik.
It would also be nice if the script would either 'unlet' the variables again, or
limit their scope ':h internal-variables'. See attached patch as example.
It also fixes a comment.
--
Regards,
Thilo
4096R/0xC70B1A8F
721B 1BA0 095C 1ABA 3FC6 7C18 89A4 A2A0 C70B 1A8F
I thought variables inside functions were implicitly in function scope
(i.e. you don't need to specify the "l:" prefix) and you don't need to
:unlet them because this will happen implicitly when the function
returns. Did I misunderstand? Oh my then I have a lot of code to fix :-)
- Peter
Hello Peter,
Thanks for the hint. I looked again into documentation and it seems you are right.
,----[ :h local-variable ]--------
*local-variable* *l:var*
Inside functions local variables are accessed without prepending anything.
But you can also prepend "l:" if you like. However, without prepending "l:"
you may run into reserved variable names. For example "count". By itself it
refers to "v:count". Using "l:count" you can have a local variable with the
same name.
`---------------------------------------------
Still i think being explicit instead of implicit is a good habit for writing
near any code and also the documentation mentions that you have to be careful
about variable naming schemes without the 'l:' . Especially when it comes down
to official runtime files.
My personal hope and intention is to help to make runtime files the same code
quality as vim itself.