How to replace pattern with shell command's output using matched pattern

[Lifepillar]

Use case at hand: replace Unix timestamps with date/time in a log file:

[1630720618] unbound[63495:0] info: 127.0.0.1 foo.bar.com AAAA IN
[1630720618] unbound[63495:1] info: 127.0.0.1 foo.bar.com A IN
...

The shell command I'd apply is `date -r <timestamp>`. How would you
perform the substitution?

The desired output is:

[Sat Sep 4 03:56:58 CEST 2021] unbound[63495:0] info: 127.0.0.1 foo.bar.com AAAA IN
[Sat Sep 4 03:56:58 CEST 2021] unbound[63495:1] info: 127.0.0.1 foo.bar.com A IN
...

Thanks,
Life.

[Tim Chase]

This should do the trick

%s/\[\zs\d\+\ze\]/\=substitute(system('date -r '.submatch(0)), '\n',
'', '')

It took a bit of testing to get right since GNU `date` expects a file
for -r where FreeBSD `date` uses it like you wanted, so I had to test
on the right OS. :-)

It's not particularly speedy, so if you have thousands of them, you're
invoking a command for every single one of them, but it's vastly
faster than doing each by hand.

It should also allow you to alter the format if you prefer (some
shell commands mung "%" with the current filename, but in this case,
system() doesn't, so you're safe)

-tim

[Andreas Perstinger]

On 04.09.21 11:15, Lifepillar wrote:
> Use case at hand: replace Unix timestamps with date/time in a log file:
>
> [1630720618] unbound[63495:0] info: 127.0.0.1 foo.bar.com AAAA IN
> [1630720618] unbound[63495:1] info: 127.0.0.1 foo.bar.com A IN
> ...
>
> The shell command I'd apply is `date -r <timestamp>`. How would you
> perform the substitution?

:%s_^\[\(\d\+\)_\="[" . systemlist("date -r " . submatch(1))[0]_

Alternatively, without using an external command:

:%s_^\[\(\d\+\)_\="[" . strftime("%c", submatch(1))_

Bye, Andreas

[Tim Chase]

On 2021-09-04 14:07, Andreas Perstinger wrote:
> Alternatively, without using an external command:
>
> :%s_^\[\(\d\+\)_\="[" . strftime("%c", submatch(1))_

This is much better than the suggestion I gave, keeping it all
internal to vim. To clean it up, I'd tweak it to

:%s/\[\zs\d\+\ze\]/\=strftime("%c", submatch(0))

-tim

[rwmit...@gmail.com]

I'm just curious, is the file being otherwise edited in vim?  I usually just use less to view logs, in which case, a command line filter would be more efficient (for me).

If you're writing a report, makes sense to keep it in vim.

[Lifepillar]

On 2021-09-04, rwmit...@gmail.com <rwmit...@gmail.com> wrote:
> I'm just curious, is the file being otherwise edited in vim? I usually
> just use less to view logs, in which case, a command line filter would be
> more efficient (for me).

Thanks that's a good suggestion in general. In this case, yes, I am
going to perform more editing.

> On Saturday, September 4, 2021 at 11:23:28 AM UTC-4 Tim Chase wrote:
>
>> On 2021-09-04 14:07, Andreas Perstinger wrote:
>> > Alternatively, without using an external command:
>> >
>> > :%s_^\[\(\d\+\)_\="[" . strftime("%c", submatch(1))_
>>
>> This is much better than the suggestion I gave, keeping it all
>> internal to vim. To clean it up, I'd tweak it to
>>
>> :%s/\[\zs\d\+\ze\]/\=strftime("%c", submatch(0))

Learning Vim is a never-ending experience: I didn't know (or remember)
about submatch()!

Can you explain why \zs... \ze is to be preferred to capturing with \(
and \)?

Thanks,
Life.

[Tim Chase]

On 2021-09-04 18:43, Lifepillar wrote:
>> > :%s_^\[\(\d\+\)_\="[" . strftime("%c", submatch(1))_

> ⋮

> >> :%s/\[\zs\d\+\ze\]/\=strftime("%c", submatch(0))
>
> Learning Vim is a never-ending experience: I didn't know (or
> remember) about submatch()!
>
> Can you explain why \zs... \ze is to be preferred to capturing with
> \( and \)?

In this case, it was mostly to prevent the need to add the "[" back
in at the begining.

In this particular case, assuming every line had the timestamp at the
beginning, it could really have just skipped the "["/"]" aspects and
just replaced the first run of digits on each line without the need
to capture anything:

:%s/\d\+/\=strftime("%c", submatch(0))

but I find using the \zs and \ze makes it clear what I expect to
match/replace.

tl;dr: my weird personal preferences? ;-)

-tim