awk indentation

891 views
Skip to first unread message

Donald Allen

unread,
Sep 15, 2011, 2:51:45 PM9/15/11
to vim_dev
If you have following in a vim buffer, file named, say, foo.awk

/some test/ {
if (foo != "bar") {
if (baz != ""){
array1[xyz] = N
if (N > max_N) max_N = N
}
xyz = temp[1]
# Create
print "," > "something.sql"
split(description,temp,parse_descriptions[file])
print "\t-- "temp[1] > "something.sql"
printf "\t%s %s", xyz, pg_type > "something.sql"
# Load
insert = insert",\n\t"xyz
fields[M]=xyz
M++
N = 1
}
field_to_XX_field[xyz,N] = field
N++
}

position the cursor on the '{' that ends the first line. Then do =%. You get

/some test/ {
if (foo != "bar") {
if (baz != ""){
array1[xyz] = N
if (N > max_N) max_N = N
}
xyz = temp[1]
# Create
print "," > "something.sql"
split(description,temp,parse_descriptions[file])
print "\t-- "temp[1] > "something.sql"
printf "\t%s %s", xyz, pg_type > "something.sql"
# Load
insert = insert",\n\t"xyz
fields[M]=xyz
M++
N = 1
}
field_to_XX_field[xyz,N] = field
N++
}


The line 'N = 1' is indented by an extra space, incorrectly, and the
closing '}' is also incorrectly indented. Don't focus on the code
itself or its correctness. I modified it to make it simpler for the
purpose of this example and also for privacy. The point is that it is
syntactically correct awk and the behavior I'm showing you here is
identical to the indentation I get with the unmodified, semantically
correct code.

This is one of a number of problems I've had with indentation. Looking
on the web for the usual discussions of emacs vs. vim, it's not hard
to find the observation that one of emacs' advantages is more accurate
indentation (emacs indents this code snippet correctly).
Unfortunately, my own experience is confirming this. I prefer vim in a
lot of ways, but when programming, indentation is a critical tool and,
for me, if I can't trust the editor to do it correctly, it's not of
much use to me. In the work I'm currently doing, I've been forced to
use emacs in viper mode because of this issue.

I'm not trying to start an editor religious war here. I'm simply
making an observation about something that I think is an important
area where vim seems somewhat weak. vim and awk have been around for a
long time. it ought to be able to indent awk code correctly and I
would suggest that fixing indentation problems should take priority
over adding new features.

(By the way, you get the above behavior with no ~/.vimrc and all
options set to their defaults. The behavior remains the same when you
turn on autoindent or cindent. This is with a vim built from source
up-to-date via git as of last night. Version is 7.3.315 running on a
Slackware 13.37 64-bit system.)

/Don

Jakson Alves de Aquino

unread,
Sep 15, 2011, 4:16:18 PM9/15/11
to vim...@googlegroups.com
On Thu, Sep 15, 2011 at 3:51 PM, Donald Allen <donald...@gmail.com> wrote:
> If you have following in a vim buffer, file named, say, foo.awk
>
> /some test/ {
> if (foo != "bar") {
> if (baz != ""){
> array1[xyz] = N
> if (N > max_N) max_N = N
> }
> xyz = temp[1]
> # Create
> print "," > "something.sql"
> split(description,temp,parse_descriptions[file])
> print "\t-- "temp[1] > "something.sql"
> printf "\t%s %s", xyz, pg_type > "something.sql"
> # Load
> insert = insert",\n\t"xyz
> fields[M]=xyz
> M++
> N = 1
> }
> field_to_XX_field[xyz,N] = field
> N++
> }
>
> position the cursor on the '{' that ends the first line. Then do =%. You get

[...]

I guess that the R indent script may work better for awk code. The awk
script was my starting point to develop the R one. Could you please
set the file type as "r" before indenting the code and tell us whether
the indentation becomes correct? As you certainly know, to set the
file type you should do:

:set ft=r

Note: Bram added the indent/r.vim to Vim's runtime recently (today, I think).

Best regards,

Jakson Aquino

Gary Johnson

unread,
Sep 15, 2011, 4:18:36 PM9/15/11
to vim_dev, Erik Janssen

Vim's indentation of most languages (except C) is done by plugins.
These reside in $VIMRUNTIME/indent and are contributed and
maintained by a number of different authors. In the case of awk,
the indentation plugin is $VIMRUNTIME/indent/awk.vim. At the top of
that file is the author's name, e-mail address, and an invitation to
contact him with bug reports. He would be the person to contact
first with a bug report.

I have cc'd him on this reply.

I took a look at the plugin and think I found the problem and a
solution. The plugin tries to identify continuing lines and indents
the subsequent line an additional amount. It tries to identify
continuing lines by looking for certain patterns such unfinished
arithmetic statements including lines ending with '+'.
Unfortunately, its pattern for '+' at the end of the line also
matches '++', so it thinks that "N = 1" and the last '}' are
continuation lines.

This patch seems to fix that problem for the particular cases of the
++ and -- operators.

--------------------------------------------------------------------
*** awk.vim.orig 2011-05-16 14:55:32.553959000 -0700
--- awk.vim 2011-09-15 12:48:40.004332000 -0700
***************
*** 188,193 ****
--- 188,195 ----

function! s:Seems_continuing( line )
" Unfinished lines
+ if a:line =~ '\(--\|++\)\s*$'
+ return 0
if a:line =~ '[\\,\|\&\+\-\*\%\^]\s*$'
return 1
endif
--------------------------------------------------------------------

If you apply that patch, make sure to do so to a copy of awk.vim in
your ~/.vim/indent directory rather than to the original in
$VIMRUNTIME/indent.

Regards,
Gary

Donald Allen

unread,
Sep 15, 2011, 4:56:45 PM9/15/11
to vim...@googlegroups.com

I tried this with the same build of vim as I used for the example in
my original post, current as of last night (on the East Coast of the
US). Setting the filetype to R does indeed produce the correct result,
but after a 3-second delay. Doing the same in emacs/viper is
essentially instantaneous.

Now, honesty compells me to mention that I am running this on a
home-brew micro-ITX machine with an Intel D510MO motherboard, which
includes a dual-core 1.6 Ghz Atom processor. Each core is equivalent
in performance to about an 800 Mhz Core 2 processor, so about 3x
slower than the Intel processors we routinely encounter. So while you
could argue that this would be 3x less a problem on a normal
processor, I think you need to look at your algorithm. I tried it on a
300-line section of awk code and it took 15 or 20 seconds on this
machine, so this gets too slow pretty quickly as the number of lines
increases That's going to be annoying even on a normal machine.
emacs/viper took about a second to indent this same section.

Thanks for your response. Despite the performance issue, which I'm
confident can be fixed, your R version is a definite improvement over
the awk version.

/Don

>
> Best regards,
>
> Jakson Aquino
>
> --
> You received this message from the "vim_dev" maillist.
> Do not top-post! Type your reply below the text you are replying to.
> For more information, visit http://www.vim.org/maillist.php
>

Donald Allen

unread,
Sep 15, 2011, 4:58:44 PM9/15/11
to vim...@googlegroups.com, Erik Janssen

Thanks very much. I'll give it a try and report back.

/Don

Donald Allen

unread,
Sep 15, 2011, 5:29:37 PM9/15/11
to vim...@googlegroups.com, Erik Janssen
On Thu, Sep 15, 2011 at 4:18 PM, Gary Johnson <gary...@spocom.com> wrote:

Yes, your patch fixes the issue I reported. Thanks very much.

/Don

Zvezdan Petkovic

unread,
Sep 15, 2011, 10:43:28 PM9/15/11
to vim...@googlegroups.com, Erik Janssen

On Thu, Sep 15, 2011 at 4:18 PM, Gary Johnson <gary...@spocom.com> wrote:

> Vim's indentation of most languages (except C) is done by plugins.
> These reside in $VIMRUNTIME/indent and are contributed and
> maintained by a number of different authors. In the case of awk,
> the indentation plugin is $VIMRUNTIME/indent/awk.vim. At the top of
> that file is the author's name, e-mail address, and an invitation to
> contact him with bug reports. He would be the person to contact
> first with a bug report.
>
> I have cc'd him on this reply.


I communicated with Eric about the need to fix awk indent plugin six years ago. Recently, Eric sent me a new version of the plugin.
He's obviously looking at it again and trying to fix it.

Unfortunately, even the new version does not work well on my large AWK programs. I use the standard style from "The AWK Programming Language" book by Aho, Kernighan, and Weinberger.

I did not have motivation to further pursue the change because:

(a) I rarely write in AWK these days,
(b) I wrote my plugin which perfectly indents over 1000 lines of AWK.

I just tried Donald's program and it indents it perfectly.

Donald, can you try it and see if it works better on your other AWK programs. It's attached.

Best regards,

Zvezdan Petkovic

awk.vim

Donald Allen

unread,
Sep 16, 2011, 12:05:04 AM9/16/11
to vim...@googlegroups.com, Erik Janssen

Thanks for sending this. Unfortunately, it does not work correctly on
all of my code. I've also found parts of my code where the plugin
supplied with vim does not work correctly even with Gary Johnson's
patch. If I have a chance, I will send examples tomorrow, but I need
to get my work done (as opposed to fighting with an editor) and will
have to continue with emacs/viper, since I don't presently have a vim
awk indenter that works correctly in all cases and is acceptably fast.

But I do very much appreciate the efforts of everyone to help with
this and I will continue to try to find some time to document the
problems.

/Don

>
> Best regards,
>
>        Zvezdan Petkovic

Zvezdan Petkovic

unread,
Sep 16, 2011, 12:57:56 AM9/16/11
to vim...@googlegroups.com, Erik Janssen
On Sep 16, 2011, at 12:05 AM, Donald Allen wrote:

> If I have a chance, I will send examples tomorrow, but I need
> to get my work done (as opposed to fighting with an editor)

No hurry. The work is more important.
FWIW, I'll be at a conference the next two days.
I probably will not be able to reply promptly.

Zvezdan

Jakson Alves de Aquino

unread,
Sep 16, 2011, 9:06:36 AM9/16/11
to vim...@googlegroups.com
On Thu, Sep 15, 2011 at 5:56 PM, Donald Allen <donald...@gmail.com> wrote:
> I tried this with the same build of vim as I used for the example in
> my original post, current as of last night (on the East Coast of the
> US). Setting the filetype to R does indeed produce the correct result,
> but after a 3-second delay. Doing the same in emacs/viper is
> essentially instantaneous.
[...]

> Thanks for your response. Despite the performance issue, which I'm
> confident can be fixed, your R version is a definite improvement over
> the awk version.

I agree that the indent/r.vim script is very slow, but I don't know
how to improve the speed. For each line to be indented, the script has
to calculate what was the indent level of the last command, but the
last command is not always the line above. So the algorithm goes
slowly upwards until finding the beginning of the last command. I
think it would be necessary to take a different approach.

Best regards,

Jakson

erik

unread,
Sep 17, 2011, 7:21:40 AM9/17/11
to vim_dev

Hello all,

I will take a look at the problems with the original indenter, plz
allow a few days I have to go through the learning curve again.
Some example code using the aho kerninghan weinberger style
appreciated. I am no big fan but I agree it should work.
I joined vim_dev for the purpose, but only one e-mail a day.

Regards,

Erik.

Donald Allen

unread,
Sep 17, 2011, 8:18:40 AM9/17/11
to vim...@googlegroups.com

I haven't read your code, but perhaps remember the indent level of the
'last command' as you make your forward pass over the indentation
region?

Having said that, I will observe that I've found problems with the
emacs/viper awk indentation. They took awhile to turn up, but there
are problems there, too. Your indenter, slow though it may be, is the
most consistently correct of any of them and I'm now using it, which
allows me to return to vim, which, as I said earlier, I prefer.

/Don

>
> Best regards,
>
> Jakson

Donald Allen

unread,
Sep 17, 2011, 8:19:43 AM9/17/11
to vim...@googlegroups.com

Thanks for having a look at this. You already have my example.

/Don

>
> Regards,
>
> Erik.

Jakson Alves de Aquino

unread,
Sep 17, 2011, 12:39:54 PM9/17/11
to vim...@googlegroups.com
On Sat, Sep 17, 2011 at 9:18 AM, Donald Allen <donald...@gmail.com> wrote:
> On Fri, Sep 16, 2011 at 9:06 AM, Jakson Alves de Aquino
> <jalv...@gmail.com> wrote:
>> I agree that the indent/r.vim script is very slow, but I don't know
>> how to improve the speed. For each line to be indented, the script has
>> to calculate what was the indent level of the last command, but the
>> last command is not always the line above. So the algorithm goes
>> slowly upwards until finding the beginning of the last command. I
>> think it would be necessary to take a different approach.
>
> I haven't read your code, but perhaps remember the indent level of the
> 'last command' as you make your forward pass over the indentation
> region?

Yes, I have already thought about this. Instead of going backwards
looking for the beginning of the command the script could first go
backwards looking for any character in column 1 or the beginning of
the file and, then, goes forward, remembering the current indent
level. While going forward, it would be possible to keep track of the
indentation level. This could be faster, but I don't have the
necessary spare time to implement this alternate algorithm.

Best regards,

Jakson

Donald Allen

unread,
Sep 17, 2011, 2:42:46 PM9/17/11
to vim...@googlegroups.com
On Sat, Sep 17, 2011 at 12:39 PM, Jakson Alves de Aquino

<jalv...@gmail.com> wrote:
> On Sat, Sep 17, 2011 at 9:18 AM, Donald Allen <donald...@gmail.com> wrote:
>> On Fri, Sep 16, 2011 at 9:06 AM, Jakson Alves de Aquino
>> <jalv...@gmail.com> wrote:
>>> I agree that the indent/r.vim script is very slow, but I don't know
>>> how to improve the speed. For each line to be indented, the script has
>>> to calculate what was the indent level of the last command, but the
>>> last command is not always the line above. So the algorithm goes
>>> slowly upwards until finding the beginning of the last command. I
>>> think it would be necessary to take a different approach.
>>
>> I haven't read your code, but perhaps remember the indent level of the
>> 'last command' as you make your forward pass over the indentation
>> region?
>
> Yes, I have already thought about this. Instead of going backwards
> looking for the beginning of the command the script could first go
> backwards looking for any character in column 1 or the beginning of
> the file and, then, goes forward, remembering the current indent
> level. While going forward, it would be possible to keep track of the
> indentation level. This could be faster, but I don't have the
> necessary spare time to implement this alternate algorithm.


I can certainly understand the spare time problem :-)

erik

unread,
Sep 21, 2011, 3:51:36 PM9/21/11
to vim_dev
All,

I think good news, bit frustrating though.

I have done my work and I was finding out how to commit my patches
when I noted Bram has already applied several patches from this thread
and also others. The version in Mercurial is essentially the same as
my version.

I tested on awk scripts from Donald Allen and Zvezdan Petkovic and it
works fine.

I kindly ask you to get the latest awk.vim and try it for yourself. It
should work. Let me know if it does not work and supply example code,
I can take a look next week.

I also looked at the R indenter, it's impressive, but I couldn't see
how I could be of value supplying ideas about speeding up. I'd say
stick at the updated awk indenter for awk.

Happy awk-ing, I am doing it by far not enough these days (python has
advantages)

Regards,

Erik.



On 17 sep, 20:42, Donald Allen <donaldcal...@gmail.com> wrote:
> On Sat, Sep 17, 2011 at 12:39 PM, Jakson Alves de Aquino
>
>
>
>
>
>
>
>
>
> <jalve...@gmail.com> wrote:

erik

unread,
Sep 21, 2011, 4:29:16 PM9/21/11
to vim_dev
No, I should correct myself, I overlooked one diff. The latest
official one is missing an essential change suggested by Philipp
Edelmann a few months ago, it may still miscount braces. I will send
you another one to test.


Erik.

Donald Allen

unread,
Sep 23, 2011, 9:33:12 AM9/23/11
to vim...@googlegroups.com

Erik --

Just to be sure I am interpreting your several messages correctly, you
sent a message prior to the one to which I'm replying -- it began "I
think good news ..." and had an awk.vim attached. Your subsequent
message (above) says that found a problem with "the latest official
one" and that another version was coming, so I am not testing the
"good news" version. Presumably you are still working on that, as I
haven't seen it yet (I'm not trying to rush you; I'm just trying to
make sure I've got this straight)?

/Don

>
>
> Erik.

erik

unread,
Sep 25, 2011, 3:27:08 PM9/25/11
to vim_dev
Hi Don,

I didn't attach the script (knowingly) to the 'I think good news'
posting on vim_dev. I sent it by e-mail directly to you and Zvezdan.
That's the one to use.

I keep my fingers crossed

Erik.

Donald Allen

unread,
Sep 25, 2011, 3:50:06 PM9/25/11
to vim...@googlegroups.com

It works on the example I gave you, but it does not work on this
(which was indented by the R indenter):

file != previous_file && previous_file in create_pivoted_postamble {
# Finish up field we just worked on
if (previous_field != "") {
periods_per_field[previous_field] = N


if (N > max_N) max_N = N
}

# Create
print "," > "create_pivoted.sql"
print create_pivoted_postamble[previous_file] > "create_pivoted.sql"
# Load
for (i=1; i<max_N; i++) {
printf "%s)\nselect\n%s", insert,
sprintf(load_pivoted_select_preamble[previous_file],i) >
"load_pivoted.sql"
for (j=1; j<M; j++)
if (i<periods_per_field[fields[j]])
printf ",\n\t%s", field_to_SI_field[fields[j],i] > "load_pivoted.sql"
else
printf ",\n\t%s", "NULL" > "load_pivoted.sql"
printf "\n%s\n", sprintf(load_pivoted_postamble[previous_file],i) >
"load_pivoted.sql"
}
# Duplicate elimination
print ";\n" > "duplicate_elimination_pivoted.sql"
}

Running the new awk indenter produces:

file != previous_file && previous_file in create_pivoted_postamble {
# Finish up field we just worked on
if (previous_field != "") {
periods_per_field[previous_field] = N


if (N > max_N) max_N = N
}

# Create
print "," > "create_pivoted.sql"
print create_pivoted_postamble[previous_file] > "create_pivoted.sql"
# Load
for (i=1; i<max_N; i++) {
printf "%s)\nselect\n%s", insert,
sprintf(load_pivoted_select_preamble[previous_file],i) >
"load_pivoted.sql"
for (j=1; j<M; j++)
if (i<periods_per_field[fields[j]])
printf ",\n\t%s", field_to_SI_field[fields[j],i] > "load_pivoted.sql"
else
printf ",\n\t%s", "NULL" > "load_pivoted.sql"
printf "\n%s\n", sprintf(load_pivoted_postamble[previous_file],i) >
"load_pivoted.sql"
}
# Duplicate elimination
print ";\n" > "duplicate_elimination_pivoted.sql"

erik

unread,
Sep 26, 2011, 7:16:44 AM9/26/11
to vim_dev
Hi Don,

To prevent any misunderstanding, I see this point where it goes wrong:

for (j=1; j<M; j++)
if (i<periods_per_field[fields[j]])
printf ",\n\t%s",
field_to_SI_field[fields[j],i] > "load_pivoted.sql"
else
printf ",\n\t%s", "NULL" > "load_pivoted.sql"


If I missed something let me know.
I will look into it.

Erik.

erik

unread,
Sep 30, 2011, 8:45:30 AM9/30/11
to vim_dev
All,

I think I have this issue is fixed, as well as every other pending
complaint. May I ask all involved in this thread to evaluate?

You can find the indenter here: http://dl.dropbox.com/u/26176183/awk.vim

Tested with vim 7.3 and 6.3, on several awk programs. It works fine on
7.3. It fails on vim 6.3 because imho cursor( v:lnum, 1) does not work
(at least during indent).

Hope you like it.

Erik

Zvezdan Petkovic

unread,
Oct 1, 2011, 12:03:40 AM10/1/11
to vim...@googlegroups.com
On Sep 30, 2011, at 8:45 AM, erik wrote:

> I think I have this issue is fixed, as well as every other pending
> complaint.


Not all issues yet. :-)


> May I ask all involved in this thread to evaluate?


Gladly.


> You can find the indenter here: http://dl.dropbox.com/u/26176183/awk.vim


Thanks for providing this.


> Tested with vim 7.3 and 6.3, on several awk programs. It works fine on
> 7.3. It fails on vim 6.3 because imho cursor( v:lnum, 1) does not work
> (at least during indent).
>
> Hope you like it.

First, the good news: It works much better than before.

The following doesn't work.

1. This may be a matter of preference but I think the continued lines
are not aligned correctly. For example, the code I aligned as this:

vcard_tr_palm = "ADR;WORK ADR;HOME ADR " \
"TEL;WORK TEL;FAX " \
...

is aligned by awk.vim as this:

vcard_tr_palm = "ADR;WORK ADR;HOME ADR " \
"TEL;WORK TEL;FAX " \
...

IOW, I believe that continued lines should be aligned either:

(a) fixed amount (e.g. one shiftwidth), or
(b) under the first line part to the right of equal sign

The awk.vim indents it exactly under the = sign.

Solution: If you move it two spaces further from = sign it would get
aligned under the first line text.

Another example:

param_regex = (options["v2"] || options["v2e"]) ? ptext_regex : \
name_regex "=" pvalue_regex "(," pvalue_regex ")*"


2. This remaining ones are definitely bugs.
This code (my alignment):

if ($i !~ ldif_regex &&
$i !~ /^mozilla_AimScreenName:[:<]? */) {
print "Ignoring badly formed line: " $i >"/dev/stderr"
continue
}
if (attr != "") {
if (base64) value = decode(value)
parse_and_store_ldif_value(attr, value)
}
...

is aligned by awk.vim as this.

if ($i !~ ldif_regex &&
$i !~ /^mozilla_AimScreenName:[:<]? */) {
print "Ignoring badly formed line: " $i >"/dev/stderr"
continue
}
if (attr != "") {
if (base64) value = decode(value)
parse_and_store_ldif_value(attr, value)
}
...

So, everything is indented wrongly additional four spaces because
the condition of the if statement is broken in two lines.
The indentation is relative to the second line of the condition.
The closing brace "}" is wrongly indented, and then everything below
that brace as well (the next if statement).

It seems that every continued condition in my code was indented
wrongly as shown above. Some of them were broken after && and some
after || operator.

Solution: Indent relative to the line where if statement starts.
Similarly for other statements (e.g., while, for, ...)


3. The following is not indented correctly (my alignment first):

if (!(a in ab_a)) {
if (toupper(a) in ab_a)
move_value(a, toupper(a))
else
delete person[a]
}

It was indented by awk.vim as follows:

if (!(a in ab_a)) {
if (toupper(a) in ab_a)
move_value(a, toupper(a))
else
delete person[a]
}

The line below if without the brace is not indented relative to if.

Solution: Indent it relative to if.
Similarly for if statement without braces surrounded by
while or for statement with the braces.


4. In a long if-else if-else if ... else statement:

} else if (backend == "mutt" || backend == "mailrc") {
a = attr " " value
} else { # mh
a = value
}

awk.vim indents as follows:

} else if (backend == "mutt" || backend == "mailrc") {
a = attr " " value
} else { # mh
a = value
}

The statement inside else is not indented correctly (perhaps because
of that # mh comment? or because it's a cuddled "} else {"
[standard style])

Solution: Indent it even if the comment follows cuddled "} else {".


5. The following code gets incorrect indentation and after that
everything is so broken that looking at the diff becomes useless.

My indentation first. (The standard awk style is to declare input
arguments for the function first, then several spaces or TAB, then
the local variables. Since this function has a lot of local
variables, they couldn't all fit on a line with that TAB separator.)

function format_alias(fmt, quote, sep,
i, n, q, a, m, s, name, email) {
if (!("n" in person && "sn" in person && "nickname" in person) &&
!("member" in person && "nickname" in person)) {
if ("email" in person) {
print "Missing name, surname, or nickname for " \
person["email"]
}
return
}
...
}

This was indented by awk.vim as follows:

function format_alias(fmt, quote, sep,
i, n, q, a, m, s, name, email) {
if (!("n" in person && "sn" in person && "nickname" in person) &&
!("member" in person && "nickname" in person)) {
if ("email" in person) {
print "Missing name, surname, or nickname for " \
person["email"]
}
return
}
...
}

Obviously, this is just a more pronounced example of continued
parenthesized expression (e.g., condition or argument/variable list)
gone wrong. Notice also that a continued condition in the first
if statement is indented 7 spaces instead of a full shiftwidth.
Is that because of "!"? It should be simply aligned as shown in
the original.

Solution: Indent based on the line that starts the function
(with the function keyword).


Erik, I hope this helps.

I'm quite sure that if you fix the continued parenthesized line
handling everything will pretty much fall into place.
The alignment of continued assignments should be an easy fix.
I don't know what is causing that cuddled else problem,
but that would be the only remaining thing to fix.

If you don't have my large awk program any more,
I can send it off the list.

Best regards,

Zvezdan

erik

unread,
Oct 2, 2011, 3:24:35 PM10/2/11
to vim_dev
Zvezdan,

Thanks for your time to make your elaborate reply. This makes a
valuable set of code for regression testing.

1.During the work this item was bothering me as well. My preference
goes to option
(a), so that's what I'll do.

2. :) I like this one. A continued statement, type 1, inside what the
indenter
considers a continued statement, type 2. I scanned my remaining awk
code, it
is not much, and I was surprised to see I never do this, while in C++
I break if's
all the time.

3. This is caused by new code to catch if statements that complete in
a single line. Easy fix.

4. Well... what a disappointment. It is supposed to support } else
{ and ignore comments now.

5. My quick judgement is it is (are) another occurence(s) of issue 2.
(so what you say)


Coming week is a busy one for me, so i'll look into it after that.
(but my plan is not to wait 6 years before the next reply like last
time :))

Erik.

erik

unread,
Oct 10, 2011, 5:41:46 PM10/10/11
to vim_dev
All,

A new attempt. It is a near-rewrite

It handles correctly now all cases from Zvezdan as well as some more
code of my own.
May I ask for volunteers for testing again.

You can find the indenter here: http://dl.dropbox.com/u/26176183/awk.vim


Fingers crossed :)

Erik.

Jakson Alves de Aquino

unread,
Oct 10, 2011, 6:02:43 PM10/10/11
to vim...@googlegroups.com
On Mon, Oct 10, 2011 at 6:41 PM, erik <ejan...@itmatters.nl> wrote:
> A new attempt. It is a near-rewrite
>
> It handles correctly now all cases from Zvezdan as well as some more
> code of my own.
> May I ask for volunteers for testing again.
>
> You can find the indenter here: http://dl.dropbox.com/u/26176183/awk.vim

The command "call" is missing in lines 179 and 181.

Best regards,

--
Jakson

Zvezdan Petkovic

unread,
Oct 11, 2011, 10:07:54 PM10/11/11
to vim...@googlegroups.com
On Oct 10, 2011, at 5:41 PM, erik wrote:

> May I ask for volunteers for testing again.


This is very close.
It definitely fixes issues I reported the last time.

There is a regression bug in handling this:

if (!(attr in urls))
value = escape_txt_chars(value)
# more code here.

It turns it into this:

if (!(attr in urls))
value = escape_txt_chars(value)
# more code here.

This happens also for else if (...), else, for, while, followed by a single statement. With or without a blank line following the statement, the indent is increased by an extra shiftwidth.

I've sent you my large awk program off the list so you can test easier.

Best regards,

Zvezdan

erik

unread,
Oct 12, 2011, 3:22:30 PM10/12/11
to vim_dev

Does it never stop... :)
Thanks Zvezdan and Jakson for supplying all the reference code
Schedule doesn't really permit all the work, so it'll be silent again
for at least a week.

I'll be back

Erik.

Donald Allen

unread,
Oct 12, 2011, 3:24:15 PM10/12/11
to vim...@googlegroups.com
I've been away for a bit, unable to pay attention to this. I will test the latest as well and get back ...

/Don 

I'll be back

Erik.

erik

unread,
Oct 24, 2011, 6:06:10 PM10/24/11
to vim_dev
All,

Finally had time to look at it again. I put an updated version here:

http://dl.dropbox.com/u/26176183/awk.vim


Sorry it took so long, it took me three evenings to fix and I don't
have many for the purpose.

Still tidying up to do but imho it indents correctly all the code I
have now, including a large script from Zvezdan
It's a bit subjective, depending on your personal preferences. I
simplified the multi-line expression handing, it is
consistent but not what everyone wants I am afraid (most diffs
generated on that aspect).

Can you have a look again. Let's see what surprises will pop up.

Erik.

Zvezdan Petkovic

unread,
Oct 25, 2011, 1:10:33 AM10/25/11
to vim...@googlegroups.com
On Oct 24, 2011, at 6:06 PM, erik wrote:

> Can you have a look again. Let's see what surprises will pop up.

The indent now works perfectly on several of my large scripts including the one I sent you. The only difference is the alignment of continuation lines in the original. I personally stopped aligning things long time ago and prefer your choice of a fixed indentation amount relative to the previous line. Perhaps the amount of indent could be made configurable similar to C-indent (0.5s, 1s, etc.), but, it's not necessary.

Those are good news.
I was really happy to see it work on the large awk program so well.

I didn't have time to look deeply over all of my awk programs.
A quick comparison of some smaller programs has uncovered couple of issues.

1. It seems that side comments are still confusing the indent.
For example:

if (comment && $0 ~ comment_expr || in_comment) {
if ($0 ~ /:[ ]*$/) { # the last line ends in : not :\
in_comment = 0
} else {
in_comment = 1
}
$0 = "#" $0
}

was indented as:

if (comment && $0 ~ comment_expr || in_comment) {
if ($0 ~ /:[ ]*$/) { # the last line ends in : not :\
in_comment = 0
} else {
in_comment = 1
}
$0 = "#" $0
}

If the side comment is deleted it indents correctly.

2. A backslash on the line that is **not** a line continuation, confuses
the indent and it treats the next line as a continuation.
For example:

split($1, line, /[ \t]/) # line: *** 321,52 ****
split(line[2], start, /,/)

was indented as:

split($1, line, /[ \t]/) # line: *** 321,52 ****
split(line[2], start, /,/)

Again, if the side comment is removed, the line is indented
correctly. So, perhaps this will be fixed by fixing 1.

3. In a quick, one-off try, this:

if (c > 0) name = substr(narr[1], 1, c-1)
else name = narr[1]
if (k > 1) {
...
}
...

was indented as:

if (c > 0) name = substr(narr[1], 1, c-1)
else name = narr[1]
if (k > 1) {
...
}
...

We could argue that this style is bad. I would not use it.
But there is awk code written in this style out there and the indent
seems to be confused by the one-liners.


As I get some time I'll look and compare with my other programs, but I probably will not have time for that before the weekend.

Best regards,

Zvezdan

erik

unread,
Oct 25, 2011, 3:06:31 PM10/25/11
to vim_dev
You're quick. Consider a career as beta tester :)

I don't need to look in the code to confirm this is indeed the case.
I'll extend my testcases and rework. be patient.

Erik.


On 25 okt, 07:10, Zvezdan Petkovic <zpetko...@acm.org> wrote:
> On Oct 24, 2011, at 6:06 PM, erik wrote:
>
> > Can you have a look again. Let's see what surprises will pop up.
>
> Theindentnow works perfectly on several of my large scripts including the one I sent you.  The only difference is the alignment of continuation lines in the

erik

unread,
Oct 26, 2011, 4:19:17 PM10/26/11
to vim_dev
New version on the same location: http://dl.dropbox.com/u/26176183/awk.vim

I fixed the issues you mentioned as well as the fact that the indenter
ignored "else if" (if it worked before it was accidental)
And the code gets smaller every update, I assume that is a good sign

Greetz

Erik

erik

unread,
Nov 7, 2011, 5:05:10 PM11/7/11
to vim_dev
Bram,

Though I lack feedback on the final version I think the indenter is
now worth committing
How should I proceed, I find no info on vim.org (or I am overlooking
it like indent errors :( )

Erik.

Zvezdan Petkovic

unread,
Nov 8, 2011, 7:57:19 PM11/8/11
to vim...@googlegroups.com
On Nov 7, 2011, at 5:05 PM, erik wrote:
> Though I lack feedback on the final version ...

Eric,

I'm so sorry, but I just could not find the time to test.
I promise, as soon as I get some free time, I'll look into it.
It may take another week or two.

Zvezdan

erik

unread,
Nov 9, 2011, 9:18:47 AM11/9/11
to vim_dev

> I'm so sorry, but I just could not find the time to test.


No problem! I have been doing other things too.
Searching the net for 'awk' and 'vim' surfaces some unhappiness
though, and it feels a bit I did that... so now that I finally have a
good-enough fix I want it out


Erik.

Jakson Alves de Aquino

unread,
Nov 9, 2011, 7:53:21 PM11/9/11
to vim...@googlegroups.com
On Wed, Oct 26, 2011 at 5:19 PM, erik <ejan...@itmatters.nl> wrote:
> New version on the same location: http://dl.dropbox.com/u/26176183/awk.vim

The attached patch fix a minor bug in the script.

--
Jakson

awk.vim.patch

erik

unread,
Nov 11, 2011, 3:49:22 PM11/11/11
to vim_dev
Jakson,

Thanks for pointing that one out again. I have applied your patch

Erik.

On 10 nov, 01:53, Jakson Alves de Aquino <jalve...@gmail.com> wrote:
> On Wed, Oct 26, 2011 at 5:19 PM, erik <ejans...@itmatters.nl> wrote:
> > New version on the same location:http://dl.dropbox.com/u/26176183/awk.vim
>
> The attached patch fix a minor bug in the script.
>
> --
> Jakson
>
>  awk.vim.patch
> < 1 KWeergevenDownloaden

Thilo Six

unread,
Nov 12, 2011, 9:23:21 AM11/12/11
to vim...@vim.org
erik wrote the following on 11.11.2011 21:49

Hello,

-- <snip> --


> Thanks for pointing that one out again. I have applied your patch
>
> Erik.

It would also be nice if the script would either 'unlet' the variables again, or
limit their scope ':h internal-variables'. See attached patch as example.

It also fixes a comment.

--
Regards,
Thilo

4096R/0xC70B1A8F
721B 1BA0 095C 1ABA 3FC6 7C18 89A4 A2A0 C70B 1A8F

awk.vim.patch

Peter Odding

unread,
Nov 12, 2011, 10:25:16 AM11/12/11
to vim...@googlegroups.com
> It would also be nice if the script would either 'unlet' the variables again, or
> limit their scope ':h internal-variables'. See attached patch as example.

I thought variables inside functions were implicitly in function scope
(i.e. you don't need to specify the "l:" prefix) and you don't need to
:unlet them because this will happen implicitly when the function
returns. Did I misunderstand? Oh my then I have a lot of code to fix :-)

- Peter

Thilo Six

unread,
Nov 12, 2011, 10:50:41 AM11/12/11
to vim...@vim.org
Peter Odding wrote the following on 12.11.2011 16:25

Hello Peter,

Thanks for the hint. I looked again into documentation and it seems you are right.

,----[ :h local-variable ]--------

*local-variable* *l:var*
Inside functions local variables are accessed without prepending anything.
But you can also prepend "l:" if you like. However, without prepending "l:"
you may run into reserved variable names. For example "count". By itself it
refers to "v:count". Using "l:count" you can have a local variable with the
same name.
`---------------------------------------------

Still i think being explicit instead of implicit is a good habit for writing
near any code and also the documentation mentions that you have to be careful
about variable naming schemes without the 'l:' . Especially when it comes down
to official runtime files.
My personal hope and intention is to help to make runtime files the same code
quality as vim itself.

erik

unread,
Nov 14, 2011, 4:03:06 PM11/14/11
to vim_dev

>
>                                                 *local-variable* *l:var*
> Inside functions local variables are accessed without prepending anything.
> But you can also prepend "l:" if you like.  However, without prepending "l:"
> you may run into reserved variable names.  For example "count".  By itself it
> refers to "v:count".  Using "l:count" you can have a local variable with the
> same name.


I tend to agree with Thilo that prefixing locals with l: is mandatory
for defensive coding.
I actually used names like 'lnum', and I had problems before
forgetting to declare variables in the indent script.
Combine it and you hit variables with wider scipe. Every abbreviated
name seems to already have a meaning anyway

For readability I don't like it very much. I would prefer having to
prefix everything else with s:, v: whatever and keep the locals clean
so one can focus on the algorithm.


The updated script can be found at the link from the earlier posts.

Erik.
Reply all
Reply to author
Forward
0 new messages