How to automate Zap Gremlins

[Chad Baker]

I work in tech support and I have to search log files for things like "error". When I get log files from Windows computers, the Mac Console won't find any matches. I search for "Error" and it says 0 results because it sees E¿r¿r¿o¿r not Error. If I zap the gremlins, then it will match all instances of "Error". So I basically have to run every Windows log through this process manually:

Open Log File

Reopen using encoding > UTF-8

Click ok to warning about damaged file

Text > Zap Gremlins

OK to Gremlins dialog

File > Save

This is a lot of manual steps for the dozens of logs I have to sift through. Is there any way I can use BBEdit and maybe something like Automator to either automatically Zap Gremlins on every log file that goes into a certain folder, or create a context menu Service, or a droplet, that will run through this sequence so I can get to work quicker?

Thanks!

[Rich Siegel]

On 3/4/20 at 5:04 PM, bbe...@googlegroups.com ('Chad Baker' via

BBEdit Talk) wrote:

>I work in tech support and I have to search log files for
>things like "error". When I get log files from Windows
>computers, the Mac Console won't find any matches. I search for
>"Error" and it says 0 results because it sees E¿r¿r¿o¿r not Error.

The file is encoded as UTF-16 without a BOM; so try choosing
either "Unicode (UTF-16, No BOM)" or "Unicode (UTF-16 Little
Endian, No BOM)" for "Reopen Using Encoding". (It's *probably*
the latter if it came from a Windows machine, so try that first.)

That should save you some steps.

R.
--
Rich Siegel Bare Bones Software, Inc.
<sie...@barebones.com> <https://www.barebones.com/>

Someday I'll look back on all this and laugh... until they
sedate me.

[Chad Baker]

The file opens fine without the ¿ gremlins in BBEdit without any intervention on my part. And I can search it fine in BBEdit. What Console does differently , that I like, is that when I search it will filter the file so that it only shows log lines that have a matching word, and hides the rest. So when I search "error" it will hide 10,000 lines of irrelevant log content and just show me the 5 lines of errors. If BBEdit has a mode where it can hide lines that don't contain a match, then that would work too. I could dump Console and just use BBE for my log analysis tool. 

If I follow your suggestion and use Reopen Using Encoding Unicode (UTF-16, No BOM) it looks fine in BBEdit, but it doesn't actually change the file so Console will search it correctly. For that, I still have to do the steps I mentioned and actually do a destructive edit that removes the ¿'s  then Save. Unless I'm just being dense, which is a possibility that I apologize in advance for. 

[Steve deRosier]

A few hints:

1. You can open using the UTF-16 encoding and then use the little drop
down in the file's footer to change the encoding and resave it.
2. You can use the Find dialog (cmd-f, use the "Find All" button) to
bring up a results browser on "error" and then you can just jump from
one to the other, while retaining context.
3. You can use Text->Process Lines Containing... to either delete all
non "error" lines or create a new file just with the "error" lines.

- Steve

> --
> This is the BBEdit Talk public discussion group. If you have a feature request or need technical support, please email "sup...@barebones.com" rather than posting here. Follow @bbedit on Twitter: <https://twitter.com/bbedit>
> ---
> You received this message because you are subscribed to the Google Groups "BBEdit Talk" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to bbedit+un...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/bbedit/4b43e52c-94a2-4b9d-9c8f-6b16782b343e%40googlegroups.com.

[Chad Baker]

Thanks, I'll give those a try. 

-Chad

> To unsubscribe from this group and stop receiving emails from it, send an email to bbe...@googlegroups.com.

[TJ Luoma]

On Wed, Mar 4, 2020 at 5:36 PM ‘Chad Baker’ via BBEdit Talk
<bbe...@googlegroups.com> wrote:

> The file opens fine without the ¿ gremlins in BBEdit without any intervention on my part. And I
> can search it fine in BBEdit. What Console does differently , that I like, is that when I search
> it will filter the file so that it only shows log lines that have a matching word, and hides the
> rest. So when I search “error” it will hide 10,000 lines of irrelevant log content and just show
> me the 5 lines of errors. If BBEdit has a mode where it can hide lines that don’t contain a match,
> then that would work too. I could dump Console and just use BBE for my log analysis tool

I don’t know if BBEdit has such a mode, but there’s always `grep`
which will just show matching lines.

fgrep -i error input.log

You could used that on the command line or as a text filter in BBEdit,
I suppose.

> If I follow your suggestion and use Reopen Using Encoding Unicode (UTF-16, No BOM) it looks fine in
> BBEdit, but it doesn’t actually change the file so Console will search it correctly. For that, I
> still have to do the steps I mentioned and actually do a destructive edit that removes the ¿’s
> then Save.

Well, at the risk of getting banned from the list ;-)
I’m going to offer another non-BBEdit solution.

I don't have a UTF-16 file handy to test, but it should work something
like this:

iconv --from-code=UTF-16 --to-code=UTF-16LE input.log > input-16LE.log

That will save the log to a new file which can be opened in BBEdit or elsewhere.

(There’s also the `-c` option, which means “When this option is given,
characters that cannot be converted are silently discarded, instead of
leading to a conversion error.”)

If you want to combine those two commands, you could try this:

iconv --from-code=UTF-16 --to-code=UTF-16LE input.log | fgrep -i error

That convert the file and pipe the resulting text to `fgrep` which
will then filter out everything except the lines which contain the
word 'error'

Tj