how to insert a \ with sed?

[François Patte]

Bonjour,

I want to insert some latex commands (\phantomsection and \ label) after
the string "begin" in a file the name of which is given by a variable:
$file.

Here is what I have done:

sed -i "/begin/ s/$/ \\\\\\phantomsection \\\\\\label{$file}/" "$file"

but, this works only for \phantomsection and, for the command \label,
it returns \abel!

How can I retrieve \label?

Moreover is it necessary to have six backslash to retrieve only one? If
I put single quotes, I need only two backslash, but the variable $file
is not expanded...

Thank you
--
François Patte
Université Paris Descartes

[Janis Papanagnou]

For the latter question, you have the option to use [on shell level]
different alternating quotes, as in 'string1'"$file"'string2' .

Janis

>
> Thank you
>

[Teemu Likonen]

François Patte [2017-10-22 12:46:09+02] wrote:

> Here is what I have done:
>
> sed -i "/begin/ s/$/ \\\\\\phantomsection \\\\\\label{$file}/" "$file"
>
> but, this works only for \phantomsection and, for the command \label,
> it returns \abel!
>
> How can I retrieve \label?

You could build this step by step to see what your shell does.

$ echo "\label"
\label

$ echo "\\"
\

In shell's double quotes the backslash character has special meaning
only before a few characters. In this case only the backslash itself
needs to be escaped.

In sed's "s" command's replacement string you need two backslashes to
output one. Let's try that first:

$ echo "\\\label"
\\label

So:

sed -i -e "/begin/ s/$/ \\\phantomsection \\\label{$file}/" "$file"

--
/// Teemu Likonen - .-.. <https://keybase.io/tlikonen> //
// PGP: 4E10 55DC 84E9 DFF6 13D7 8557 719D 69D3 2453 9450 ///

[Thomas 'PointedEars' Lahn]

François Patte wrote:

> I want to insert some latex commands (\phantomsection and \ label) after

Probably you mean without the space after the second backslash.

> the string "begin" in a file the name of which is given by a variable:
> $file.
>
> Here is what I have done:
>
> sed -i "/begin/ s/$/ \\\\\\phantomsection \\\\\\label{$file}/" "$file"
>
> but, this works only for \phantomsection and, for the command \label,
> it returns \abel!

If you investigate the expansion, you can see why:

| $ printf '%s\n' "/begin/ s/$/ \\\\\\phantomsection \\\\\\label/"
| /begin/ s/$/ \\\phantomsection \\\label/

The second line is what sed(1) will see, where

“\\” is the sed(1) escape sequence for one backslash character (“\”);
“\p” is an unsupported sed(1) escape sequence, so it is parsed as “p”;
“\l” is a GNU sed(1) extension:

| File: sed.info, Node: The "s" Command, […] Up: sed Programs
|
| […] Finally, as a GNU 'sed' extension, you can include a special sequence
| made of a backslash and one of the letters 'L', 'l', 'U', 'u', or 'E'.
| The meaning is as follows:
|
| '\L'
| Turn the replacement to lowercase until a '\U' or '\E' is found,
|
| '\l'
| Turn the next character to lowercase,
| […]

The next character in your case is “{”, so the “\l” is discarded and nothing
else changes.

> How can I retrieve \label?

By using only 4 backslashes if you use a double-quoted string.

| $ echo 'begin' | sed "/begin/ s/$/ \\\\phantomsection \\\\label/"
| begin \phantomsection \label

Because that is what sed(1) will see:

| $ printf '%s\n' "/begin/ s/$/ \\\\phantomsection \\\\label/"
| /begin/ s/$/ \\phantomsection \\label/

> Moreover is it necessary to have six backslash to retrieve only one?

No. (How did you get that idea in the first place?)

| $ echo 'begin' | sed '/begin/ s/$/ \\phantomsection \\label/'
| begin \phantomsection \label

> If I put single quotes, I need only two backslash, but the variable $file
> is not expanded...

You can switch between expansion and non-expansion by concatenation. I have
marked the string literals below (use a fixed-width font):

sed -i '/begin/ s/$/ \\phantomsection \\label{'"$file"'}/' "$file"

^ ^^ ^^ ^ ^ ^
'--------------------------------------''-----''--' '-----'

However, be aware that you need to escape any sed-special character
sequences in what the first "$file" expands to.

--
PointedEars

Twitter: @PointedEars2
Please do not cc me. /Bitte keine Kopien per E-Mail.

[Eric]

Well,

$ echo "/begin/ s/$/ \\\\\\phantomsection \\\\\\label{$file}/"
/begin/ s/$/ \\\phantomsection \\\label{xxxx}/

so that is what sed sees. And yes, the doubling up for the shell is
normal, \\ becomes \ after shell substitution.

A backslash is as special to sed as it is to the shell, so if you want a
real backslash you have to give it two. I do not understand why you put
the third backslash in, but it means that sed sees \p which is just p ,
but \l means lowercase the next character, which is why it disappears
(and you get no hint because the next character is lowercase anyway).

The meaning for \l is a GNU extension!

It means that to get an actual \l you need to escape the backslash but
NOT the l .

Eric
--
ms fnd in a lbry

[Thomas 'PointedEars' Lahn]

Teemu Likonen wrote:

> François Patte [2017-10-22 12:46:09+02] wrote:
>> Here is what I have done:
>>
>> sed -i "/begin/ s/$/ \\\\\\phantomsection \\\\\\label{$file}/" "$file"
>>
>> but, this works only for \phantomsection and, for the command \label,
>> it returns \abel!
>>
>> How can I retrieve \label?
>
> You could build this step by step to see what your shell does.
>
> $ echo "\label"
> \label
>
> $ echo "\\"
> \
>
> In shell's double quotes the backslash character has special meaning
> only before a few characters.

Exactly. And that is why your solution only works by coincidence:

> In this case only the backslash itself needs to be escaped.

That is _not_ the reason why the “l” was removed in the output.

> In sed's "s" command's replacement string you need two backslashes to
> output one. Let's try that first:
>
> $ echo "\\\label"
> \\label

You are missing the point. Do you see how *your shell* expands "\\" to “\”,
but "\l" to “l”?

> So:
>
> sed -i -e "/begin/ s/$/ \\\phantomsection \\\label{$file}/" "$file"

This would work, but only because the OP’s *GNU* sed(1) never sees “\l” as
*the shell* parses "\p" as “p”, and "\l" as “l”:

$ printf '%s\n' "/begin/ s/$/ \\\phantomsection \\\label{$file}/"
/begin/ s/$/ \\phantomsection \\label{}/

It is functionally equivalent to the safer

$ printf '%s\n' "/begin/ s/$/ \\\\phantomsection \\\\label{$file}/"
/begin/ s/$/ \\phantomsection \\label{}/

and

$ printf '%s\n' '/begin/ s/$/ \\phantomsection \\label{'"$file"'}/'
/begin/ s/$/ \\phantomsection \\label{}/

[Thomas 'PointedEars' Lahn]

Thomas 'PointedEars' Lahn wrote:

> Teemu Likonen wrote:
>> In shell's double quotes the backslash character has special meaning
>> only before a few characters.
>
> Exactly. And that is why your solution only works by coincidence:
>
>> In this case only the backslash itself needs to be escaped.
>
> That is _not_ the reason why the “l” was removed in the output.
>
>> In sed's "s" command's replacement string you need two backslashes to
>> output one. Let's try that first:
>>
>> $ echo "\\\label"
>> \\label
>
> You are missing the point. Do you see how *your shell* expands "\\" to
> “\”, but "\l" to “l”?

Correction: The shell expands the unknown escape sequence "\l" to itself
(“\l”), and that is why this works by coincidence.

It is better not to produce unknown escape sequences in the first place.

[Teemu Likonen]

Thomas Lahn [2017-10-22 14:12:47+02] wrote:

> Teemu Likonen wrote:
>> sed -i -e "/begin/ s/$/ \\\phantomsection \\\label{$file}/" "$file"
>
> This would work, but only because the OP’s *GNU* sed(1) never sees
> “\l” as *the shell* parses "\p" as “p”, and "\l" as “l”:

I used three backslashes because in shell's double quote expansion "\\"
is one backslash and "\p" and "\l" are backslash and the letter.

$ echo "\\"
\

$ echo "\p"
\p

$ echo "\l"
\l

I didn't care about sed's \l command because I didn't need to: it wasn't
part of the solution.

[Teemu Likonen]

Thomas Lahn [2017-10-22 14:54:25+02] wrote:

> Correction: The shell expands the unknown escape sequence "\l" to
> itself (“\l”), and that is why this works by coincidence.
>
> It is better not to produce unknown escape sequences in the first
> place.

It's not unknown and not by coincidence. The behavior documented in
bash(1):

QUOTING

[...]

Enclosing characters in double quotes preserves the literal
value of all characters within the quotes, with the exception of
$, `, \, and, when history expansion is enabled, !. [...] The
backslash retains its special meaning only when followed by one
of the following characters: $, `, ", \, or <newline>.

[Thomas 'PointedEars' Lahn]

Teemu Likonen wrote:

> Thomas Lahn [2017-10-22 14:54:25+02] wrote:
>> Correction: The shell expands the unknown escape sequence "\l" to
>> itself (“\l”), and that is why this works by coincidence.
>>
>> It is better not to produce unknown escape sequences in the first
>> place.
>
> It's not unknown and not by coincidence.

Yes, by coincidence. Other shells and GNU sed support more escape
sequences.

> The behavior documented in
> bash(1):

Bash is not the only command-line shell.

> QUOTING
>
> [...]
>
> Enclosing characters in double quotes preserves the literal
> value of all characters within the quotes, with the exception of
> $, `, \, and, when history expansion is enabled, !. [...] The
> backslash retains its special meaning only when followed by one
> of the following characters: $, `, ", \, or <newline>.

Perhaps you would have understood “unspecified” better?

[Thomas 'PointedEars' Lahn]

Teemu Likonen wrote:

> Thomas Lahn [2017-10-22 14:12:47+02] wrote:
>> Teemu Likonen wrote:
>>> sed -i -e "/begin/ s/$/ \\\phantomsection \\\label{$file}/" "$file"
>> This would work, but only because the OP’s *GNU* sed(1) never sees
>> “\l” as *the shell* parses "\p" as “p”, and "\l" as “l”:

See my correction.

> I used three backslashes because in shell's double quote expansion "\\"
> is one backslash and "\p" and "\l" are backslash and the letter.

Not “in shell’s”, but _in the *one* shell that *you* tested with_ (Bash).

> $ echo "\\"
> \
>
> $ echo "\p"
> \p
>
> $ echo "\l"
> \l

And, AISB, you are *lucky* that this works *for you*. There is *no need* to
escape the “p” or “l” in a *double*-quoted string and *hope* that the *used*
shell does *not* expand it. None *at all*:

| $ echo "\\l"
| \l

|
| $ echo '\l'
| \l

> I didn't care about sed's \l command because I didn't need to: it wasn't
> part of the solution.

It was a part of the problem, though. The OP *explicitly* asked why the “l”
would not appear in the output. This can only happen if they use *GNU* sed
(and do not tell it with the “--posix” switch to be compliant with *POSIX*
sed):

,----
| $ echo 'Foo' | sed 's/^\(.\)/\l\1/'
| foo
|
| $ echo 'Foo' | sed --posix 's/^\(.\)/\l\1/'
| lFoo
|
| $ sed --version
| sed (GNU sed) 4.2.2
| Copyright (C) 2012 Free Software Foundation, Inc.
| License GPLv3+: GNU GPL version 3 or later
| <http://gnu.org/licenses/gpl.html>.
| This is free software: you are free to change and redistribute it.
| There is NO WARRANTY, to the extent permitted by law.
|
| Written by Jay Fenlason, Tom Lord, Ken Pizzini,
| and Paolo Bonzini.
| GNU sed home page: <http://www.gnu.org/software/sed/>.
| General help using GNU software: <http://www.gnu.org/gethelp/>.
| E-mail bug reports to: <bug...@gnu.org>.
| Be sure to include the word ``sed'' somewhere in the ``Subject:'' field.
`----