How can I eliminate a newline character in a variable?

[paris2venice]

So just as I thought I had perfected my Gardiner numbers to Unicode hieroglyphs script (written in both bash and Applescript), I discovered an annoying bug. That is, when I create the final array to send to the clipboard (via Apple's /usr/bin/pbcopy), it somehow now contains a newline ... which when you paste in the result from the script, it now always has a newline. I cannot figure out where this came from but these are my attempts to eliminate it ... none of which succeed at doing so.

The original line:
pbcopy <<< "${glyph_array[*]}"

My first attempt to eliminate the newline:
pbcopy <<< $( tr '\n' '' <<< "${glyph_array[*]}" )

My second attempt:
pbcopy <<< $( tr -cd "[:print:]" <<< "${glyph_array[*]}" )

This command is independent of the Applescript so both the shell and the GUI interface use this line and both continue to contain the newline when later using Command-V to paste. Thanks for any help.

[paris2venice]

Also, I'm wondering if testing for Linux (which I don't have on my Mac), could I use /dev/clip to replicate pbcopy in any Linux environment or would it only work in certain implementations of Linux?

echo "${glyph_array[*]}" > /dev/clip

[paris2venice]

I misinterpreted that. Apparently, there is no such /dev/clip for Linux, nor any other similar built-in feature. While there is xclip, it is 3rd party.

[Jorgen Grahn]

On Sat, 2020-10-31, paris2venice wrote:
...

> I misinterpreted that. Apparently, there is no such /dev/clip for
> Linux, nor any other similar built-in feature. While there is
> xclip, it is 3rd party.

It's Linux; /everything/ is 3rd party. Maybe what you're after is: are
people likely to have it installed already? And if not, do their
distributions carry it for easy installation? The answers are no and
yes, at least if you're on Debian.

/Jorgen

--
// Jorgen Grahn <grahn@ Oo o. . .
\X/ snipabacken.se> O o .

[Chris F.A. Johnson]

If the newline is at the end of the variable:

nofnl=${nl%?}

To remove one newline from anywhere in the string:

nonl=${xx/$'\n'/}

To remove all newlines from the string:

nonl=${xx//$'\n'/}

--
Chris F.A. Johnson <http://cfajohnson.com/>
=========================== Author: ===============================
Shell Scripting Recipes: A Problem-Solution Approach (2005, Apress)
Pro Bash Programming: Scripting the GNU/Linux shell (2009, Apress)

[paris2venice]

So what I discovered is that it is the '<<<' that is forcing the newline. This is given even greater evidence by the suggestion made by Chris which I tried after pinning the problem on my use of '<<<' ... see below where 'maa-kheru' is the name of my script and 'q1 d4 a40' are the Gardiner numbers for the hieroglyphs for Osiris.

$ osiris=$( maa-kheru q1 d4 a40 )
$ echo $osiris
𓊨 𓁹 𓀭
$ pbcopy <<< ${osiris%?}
$ 𓊨 𓁹
-bash: 𓊨: command not found

As you can see, the '%?' strips off the last character ( 𓀭 ) since, in the variable, there is no newline. However, sent to pbcopy via '<<<,' pbcopy gets a newline as shown by the command-v on the following line where an error is generated because a newline forced the error. When I eliminate the '<<<' and just use a pipe, the problem goes away.

printf "$osiris" | pbcopy
<command-v>
𓊨 𓁹 𓀭<blinking insertion point / no newline>

Thanks for the assistance.

[Ed Morton]

On 10/31/2020 5:19 PM, paris2venice wrote:
<snip>

> So what I discovered is that it is the '<<<' that is forcing the newline.

Right, <<< has to produce a valid text line per the POSIX standard and
POSIX lines end in a newline. If it didn't do that then it'd be
undefined what any POSIX text processing tool did with the result.

Ed.

[Chris F.A. Johnson]

Is <<< now included in POSIX?

None of my attempts to get <<< to end in a newline have worked.

[Janis Papanagnou]

On 01.11.2020 19:07, Chris F.A. Johnson wrote:
> On 2020-10-31, Ed Morton wrote:
>> On 10/31/2020 5:19 PM, paris2venice wrote:
>> <snip>
>>> So what I discovered is that it is the '<<<' that is forcing the newline.
>>
>> Right, <<< has to produce a valid text line per the POSIX standard and
>> POSIX lines end in a newline. If it didn't do that then it'd be
>> undefined what any POSIX text processing tool did with the result.
>
> Is <<< now included in POSIX?

I don't think so.

(I interpreted Ed's comment as how lines are generally defined in POSIX.)

> None of my attempts to get <<< to end in a newline have worked.

What were these attempts?

(I get newlines with all the prominent shells; ksh, bash, zsh.)

Janis

[Chris F.A. Johnson]

(Using bash)

The only newline is generated by echo:

$ read xx <<< qwerty
$ echo "$xx"
qwerty
$

The trailing newline is stripped:

$ read xx <<< "$(pwd)"
$ echo "$xx"
/home/chris/Downloads
$

It even strips trailing newlines:

$ cat <<< "$(printf '%s\n\n' qwerty)"
qwerty
$

But not embedded newlines:

$ cat <<< "$(printf '\n%s\n\n' qwe$'\n'rty)"

qwe
rty
$

[Ed Morton]

On 11/1/2020 12:07 PM, Chris F.A. Johnson wrote:
> On 2020-10-31, Ed Morton wrote:
>> On 10/31/2020 5:19 PM, paris2venice wrote:
>> <snip>
>>> So what I discovered is that it is the '<<<' that is forcing the newline.
>>
>> Right, <<< has to produce a valid text line per the POSIX standard and
>> POSIX lines end in a newline. If it didn't do that then it'd be
>> undefined what any POSIX text processing tool did with the result.
>
> Is <<< now included in POSIX?

I doubt it but that's not what I'm saying anyway, I'm saying the `<<<`
has to output a valid POSIX text line for that output to be acceptable
to all the POSIX compliant tools that are required to handle POSIX text
lines and for which lines that do not end in a newline invoke undefined
behavior.

>
> None of my attempts to get <<< to end in a newline have worked.

I can't imagine how you're stopping it from ending in a newline.

Ed.

[Ed Morton]

On 11/1/2020 2:10 PM, Chris F.A. Johnson wrote:
> On 2020-11-01, Janis Papanagnou wrote:
>> On 01.11.2020 19:07, Chris F.A. Johnson wrote:
>>> On 2020-10-31, Ed Morton wrote:
>>>> On 10/31/2020 5:19 PM, paris2venice wrote:
>>>> <snip>
>>>>> So what I discovered is that it is the '<<<' that is forcing the newline.
>>>>
>>>> Right, <<< has to produce a valid text line per the POSIX standard and
>>>> POSIX lines end in a newline. If it didn't do that then it'd be
>>>> undefined what any POSIX text processing tool did with the result.
>>>
>>> Is <<< now included in POSIX?
>>
>> I don't think so.
>>
>> (I interpreted Ed's comment as how lines are generally defined in POSIX.)
>>
>>> None of my attempts to get <<< to end in a newline have worked.
>>
>> What were these attempts?
>>
>> (I get newlines with all the prominent shells; ksh, bash, zsh.)
>
> (Using bash)
>
> The only newline is generated by echo:

No, `read` is consuming the newline generated by `<<<`, just like if you do

$ awk '{print "<" $0 ">"}' <<< qwerty
<qwerty>

awk consumes the newline that was present in the input before printing
the output. Here's how you can see the newline that `<<<` is generating:

$ awk -v RS=X '{print "<" $0 ">"}' <<< qwerty
<qwerty
>

Regards,

Ed.

[Janis Papanagnou]

On 01.11.2020 21:10, Chris F.A. Johnson wrote:
> On 2020-11-01, Janis Papanagnou wrote:
>> On 01.11.2020 19:07, Chris F.A. Johnson wrote:
>>> On 2020-10-31, Ed Morton wrote:
>>>> On 10/31/2020 5:19 PM, paris2venice wrote:
>>>> <snip>
>>>>> So what I discovered is that it is the '<<<' that is forcing the newline.
>>>>
>>>> Right, <<< has to produce a valid text line per the POSIX standard and
>>>> POSIX lines end in a newline. If it didn't do that then it'd be
>>>> undefined what any POSIX text processing tool did with the result.
>>>
>>> Is <<< now included in POSIX?
>>
>> I don't think so.
>>
>> (I interpreted Ed's comment as how lines are generally defined in POSIX.)
>>
>>> None of my attempts to get <<< to end in a newline have worked.
>>
>> What were these attempts?
>>
>> (I get newlines with all the prominent shells; ksh, bash, zsh.)
>
> (Using bash)

I don't think that your examples show what you say to be happening...

> The only newline is generated by echo:
>
> $ read xx <<< qwerty
> $ echo "$xx"
> qwerty
> $

Above 'read' is removing the newlines, also the one added by '<<<'.
(Try read xx <<< $'qwerty\n\n' to see the behaviour.)

Then try non-read contexts...

$ od -c <<< qwerty
0000000 q w e r t y \n
0000007

$ awk 'BEGIN{FS=RS="x"}{print length()}' <<< "foo"
4

>
> The trailing newline is stripped:

In subshells (your examples below) multiple trailing newlines are stripped,
and finally one added by <<< (as I understand it).

> $ read xx <<< "$(pwd)"
> $ echo "$xx"
> /home/chris/Downloads
> $
>
> It even strips trailing newlines:
>
> $ cat <<< "$(printf '%s\n\n' qwerty)"
> qwerty
> $
>
> But not embedded newlines:
>
> $ cat <<< "$(printf '\n%s\n\n' qwe$'\n'rty)"
>
> qwe
> rty
> $

I want to add an Email from David Korn posted 10 years ago in the AST
mailling list, where the basic question was raised and where he gave a
very terse answer...

David Korn
Mi, 19.05.2010 15:37
Subject: Re: RE: [ast-users] wc --bytes <<<"foo" == 4?
--------
> >
> > Is this a bug or a feature that wc --bytes <<<"foo" returns 4? Does
> > the <<<string operator add a new line at the end by default?
>
> Apparently.
>
> $ od -c <<<"foo"
> 0000000 f o o \n
> 0000004

This is intentional.



When the <<< had been introduced in Ksh I racall to have had a discussion
with David Korn about the originally implemented behaviour at that time.
I cannot find those Emails now but as far as memory serves the original
implementation did not show the newline, then it was added to behave -
YMMV - consistent. (Awaiting your posts or emails of complaints, but the
added newline became the de facto standard it seems.)

Janis