Fwd: (Case 417198) Need to remove blank lines

[Bruce Van Allen]

Hi All,

Here’s Rich’s explanation of BBEdit’s behavior.

I’m still not clear why Christopher seems to get different results.

HTH

Forwarded message:

> From: Bare Bones Software Technical Support <sup...@barebones.com>
> To: Bruce Van Allen <b...@cruzio.com>
> Subject: Re: (Case 417198) Need to remove blank lines
> Date: Thu, 14 Jan 2021 16:04:01 +0000
>
> Bruce,
>
> Thanks for the additional detail.
>
> To begin, a couple of definitions to clarify my subsequent remarks:
>
> - an "empty" line is a line which contains no characters between its
> beginning and its end.
>
> - a "blank" line is a line which contains only whitespace characters
> between its beginning and end.
>
> To continue: "Process Lines Containing" is not a suitable tool for
> removing empty lines from a document. The emphasis is on "Containing",
> and empty lines don't contain anything. The operation will explicitly
> skip empty lines (it won't attempt a literal match, and because the
> line's contents are zero length, the regular-expression engine won't
> bother searching).
>
> Furthermore, when Process Lines Containing examines a line's contents,
> those contents by definition do not include the line break character
> at the end of the line.
>
> You *can* use "Process Lines Containing" to remove _blank_ lines from
> a document, but there are none in the example that you provided.
>
> In order to remove empty *and* blank lines for a document, a Grep
> replacement will do the job:
>
> search for (grep): ^\s*$\n
> replace with: (leave empty)
>
> This is suitable because, unlike Process Lines Containing, Replace All
> examines the entire contents of the document and not the inner
> contents of each line in isolation.
>
> Note that the "\n" at the end of the search pattern is necessary,
> because without it, the replace operation would replace only the
> *contents* of the empty/blank line with nothing. This would turn blank
> lines into empty lines bout would not actually remove either.
>
> I hope this helps, but please let us know if you have any other
> questions.
>
> Regards,
>
> Rich Siegel
> Bare Bones Software, Inc.
>
> -----Original Message-----
> From: "Bruce Van Allen" <b...@cruzio.com>
> Reply-To: "Bruce Van Allen" <b...@cruzio.com>
> Date: Wed, 13 Jan 2021 20:03:03 -0800
> To: "Bare Bones Software Technical Support" <sup...@barebones.com>
> Subject: Re: (Case 417198) Need to remove blank lines
>
>> On 13 Jan 2021, at 19:03, Bare Bones Software Technical Support
>> wrote:
>>
>>> May I trouble you to describe the issue you're having, and provide a
>>> Zipped copy of a text file which is not behaving correctly?
>>
>> I was responding to a post to the BBEdit g-group list from someone
>> who
>> needed advice about removing blank lines from a text file.
>>
>> The original post was from Howard <leadwi...@gmail.com> at 2:28
>> PM,
>> Jan 13, 2021.
>>
>> The OP’s sample text is below - note that in the G-group email it
>> didn’t seem to have the blank lines they’d described.
>>
>> Anyway, members of the group began posting conflicting results with
>> using “Process Lines Containing…” to remove “blank” lines;
>>
>> One person said using the regex ^$ didn’t work; another said using
>> ^\h*$ DID work, with the optional horizontal space added. More
>> conflicting results were posted.
>>
>> So I tried it, running BBEdit version 13.5.5 build 415089 on macOS v.
>> 11.1.
>>
>> My test file is attached. I used the OP’s text but added blank
>> lines.
>>
>> When I try “Process Lines Containing…” with Grep checked
>> (screenshot attached) and the regex ^\s*$ — no lines are removed.
>> Same
>> with ^\h*$. But group member Christopher Stone posted that it works
>> for
>> him. So I reposted and asked people what versions they were using. I
>> included their responses in my message to support.
>>
>> Sorry that wasn’t more clear.
>>
>> OP’s sample text:
>> How can I remove all the blank lines? After doing that, I want to be
>> able
>> to paste the data back into a spreadsheet for further manipulation.
>>
>> Rk Name Player Matching W L W-Lpct ERA GS CG SHO SV IP H R ER HR BB
>> SO
>> WHIP
>> SB CS WPA Tm
>> 1 Max Scherzer\scherma01 Max Scherzer 84 56 8 0.875 1.56 84 8 4 0
>> 584.1
>> 294
>> 107 101 53 89 787 0.66 16 5 21.1 WSN
>> 2 Justin Verlander\verlaju01 Justin Verlander 80 49 11 0.817 1.72 80
>> 5 3
>> 0
>> 554.0 269 114 106 59 88 693 0.64 9 7 18.9 DETHOU
>> 3 Zack Greinke\greinza01 Zack Greinke 79 51 6 0.895 1.63 79 1 1 0
>> 529.0
>> 306
>> 98 96 37 68 524 0.71 11 11 17.5 LADARIHOU
>> 4 Clayton Kershaw\kershcl01 Clayton Kershaw 75 48 7 0.873 1.38 75 6 5
>> 0
>> 509.1 254 83 78 32 56 605 0.61 5 8 17.4 LAD
>>
>> Best Regards,
>>
>> - Bruce
>>
>> Bruce Van Allen
>> PO Box 839
>> Santa Cruz CA 95061
>>
>> Desk: 831-429-1688
>> Cell: 831-332-3649
>> b...@cruzio.com
>> ![](/default.asp?pg=pgDownload&amp;pgType=pgFile&amp;ixBugEvent=2641840&amp;ixAttachment=91884&amp;sFileName=Screen%20Shot%202021-01-13%20at%207.57.27%20PM.png&amp;sTicket=417198_gbj26434
>> "Screen Shot
>> 2021-01-13 at 7.57.27 PM.png")

[Christopher Stone]

On 01/14/2021, at 10:08, Bruce Van Allen <b...@cruzio.com> wrote:

Here’s Rich’s explanation of BBEdit’s behavior.

I’m still not clear why Christopher seems to get different results.

---

Hey Bruce,

I figured it out.

The original data sample from Howard has many tabs in the “blank” lines.

So by Rich's definition they have content, and Process Lines is able to operate on them.

When I replace them with entirely empty lines then Process Lines fails.

My sed text filter works in both cases:

---

#!/usr/bin/env bash

sed -E '/^[[:blank:]]*$/d'

---

And just for practice let's do that with Perl:

---

#!/usr/bin/env perl -sw

while (<>) {

   if (m/\S+/) {print};

}

---

--

Best Regards,

Chris