(bash) Which characters are magic inside of double quotes?

[Kenny McCormack]

I can think of only two: $ and `

Am I forgetting anything?

--
The book "1984" used to be a cautionary tale;
Now it is a "how-to" manual.

[Lew Pitcher]

On Sat, 25 Nov 2023 14:34:43 +0000, Kenny McCormack wrote:

> I can think of only two: $ and `
>
> Am I forgetting anything?

From the bash(1) manpage:

"Enclosing characters in double quotes preserves the literal value of
all characters within the quotes, with the exception of $, `, \, and,
when history expansion is enabled, !."

So the list is:
$
`
\
!


--
Lew Pitcher
"In Skills We Trust"

[Kenny McCormack]

In article <ujt24f$2qv42$1...@dont-email.me>,

Thanks. Luckily, I don't have to worry about !. But I think \ is a dark
corner. It sometimes needs to be escaped; sometime not. It is weird. We
had a discussion (started by me) a while back about this and the conclusion
was as stated (that it is sort of a dark corner).

The overall context of this question is embedding scripts (for various
interpreted languages) inside shell scripts. For most languages, such as
"AWK", you just put it all inside single quotes and everything is fine,
Most languages use double quotes internally for string delimiting, and it
all just works. And if you need a single quote, you can usually write \047.

But there are two languages where the string delimiter is single quote
(usually) and for those languages, it ends up being more convenient to
enclose the script (in the shell script) with double quotes. But then you
have to worry about the fact that double quotes are not "strong" quotes.
Hence, this thread.

The two languages I am referring to are "VIM" and "SQL". Both of these
languages are defined in terms of using single quotes as the string
delimiter, but both sortof/kindof/maybe also accept double quotes. So, you
can sometimes get away with using double quotes, but sometimes not.

--
The people who were, are, and always will be, wrong about everything, are still
calling *us* "libtards"...

(John Fugelsang)

[Helmut Waitzmann]

gaz...@shell.xmission.com (Kenny McCormack):

[The question: Which characters are magic inside of double
quotes?]

> The overall context of this question is embedding scripts (for
> various interpreted languages) inside shell scripts. For most
> languages, such as "AWK", you just put it all inside single
> quotes and everything is fine, Most languages use double quotes
> internally for string delimiting, and it all just works. And if
> you need a single quote, you can usually write \047.
>
>
> But there are two languages where the string delimiter is single
> quote (usually) and for those languages, it ends up being more
> convenient to enclose the script (in the shell script) with
> double quotes. But then you have to worry about the fact that
> double quotes are not "strong" quotes. Hence, this thread.
>
>
> The two languages I am referring to are "VIM" and "SQL". Both of
> these languages are defined in terms of using single quotes as
> the string delimiter, but both sortof/kindof/maybe also accept
> double quotes. So, you can sometimes get away with using double
> quotes, but sometimes not.
>
>

But you can always do it like this:


For every script to be embedded inside a shell script, procede
using the following rules:


In the script to be embedded, replace each single quote by a
single quote, a backslash, and 2 single quotes (or insert before
each single quote a single quote, a backslash, and a single quote
or insert after each single quote a backslash and 2 single
quotes).


After this has been done, prepend the whole script to be embedded
with a single quote and append a single quote.


Embed the result.


That's it.  It will work with any script to be embedded into a
shell script regardless the language of the script to be
embedded.


The advantage of this procedure:  It can be done using a
search‐and‐replace command in your favorite text editor, as well
as by a "sed" or "awk" command invocation or even by a shell
script.

[Kenny McCormack]

In article <83bkbhc...@helmutwaitzmann.news.arcor.de>,
Helmut Waitzmann <oe.th...@xoxy.net> wrote:
...

> For every script to be embedded inside a shell script, procede
> using the following rules:
>
>
> In the script to be embedded, replace each single quote by a
> single quote, a backslash, and 2 single quotes (or insert before
> each single quote a single quote, a backslash, and a single quote
> or insert after each single quote a backslash and 2 single
> quotes).

Yes, this works. But it is ugly. In the past, I've used the trick of
replacing ' with '"'"' - but as you say, replacing it with '\'' saves one
character.

But it is just unaesthetically pleasing. In the instant case, I found it
easier to go with " instead of ' and to "hope for the best". It all worked
out fine, as the embedded script didn't have any of the "magic" characters
in it.

The process described above, of replacing the ' with a 4 or 5 character
replacement string, is not too bad for AWK, where ' is rare, but gets ugly
with either of the mentioned languages (VIM and SQL), where the ' is
common.

--
The whole aim of practical politics is to keep the populace alarmed (and hence clamorous
to be led to safety) by menacing it with an endless series of hobgoblins, all of them imaginary.

H. L. Mencken

[Ben Bacarisse]

Helmut Waitzmann <nn.th...@xoxy.net> writes:

> gaz...@shell.xmission.com (Kenny McCormack):
>
> [The question: Which characters are magic inside of double quotes?]

>> The overall context of this question is embedding scripts (for
>> various interpreted languages) inside shell scripts. For most >
>> languages, such as "AWK", you just put it all inside single > quotes
>> and everything is fine, Most languages use double quotes > internally
>> for string delimiting, and it all just works. And if > you need a
>> single quote, you can usually write \047.
>>
>> But there are two languages where the string delimiter is single quote
>> (usually) and for those languages, it ends up being more convenient to
>> enclose the script (in the shell script) with double quotes. But then you
>> have to worry about the fact that double quotes are not "strong"
>> quotes. Hence, this thread. The two languages I am referring to are "VIM"
>> and "SQL". Both of these languages are defined in terms of using single
>> quotes as the string delimiter, but both sortof/kindof/maybe also accept
>> double quotes. So, you can sometimes get away with using double quotes,
>> but sometimes not.
>
> But you can always do it like this: For every script to be embedded inside
> a shell script, procede using the following rules: In the script to be
> embedded, replace each single quote by a single quote, a backslash, and 2

> single quotes [alternatives elided...]

That works well when there is a "build process" for the script that
contains the embedded script because, as you say, it's easy to automate.
But I find it fragile when the embedded script might be edited and hard
to read when the script needs to be inspected. Lots of '\''...'\''s get
in the way.

My preference is for here documents. You can choose a clear and
staggeringly unlikely delimiter, and the text between the start and the
end is taken literally (if you 'quote-the-end-delimiter').

This works best for standard input, but when I need a single string
argument I put "$(cat in front and )" at the end. Not pretty, perhaps,
but it keeps all of the mess away from the actual embedded script:

printf "{%s}\n" "$(cat <<'---END-OF-EMBEDDED-SCRIPT---'
...script here...
---END-OF-EMBEDDED-SCRIPT---
)"

--
Ben.

[Helmut Waitzmann]

Ben Bacarisse <ben.u...@bsb.me.uk>:

> Helmut Waitzmann <nn.th...@xoxy.net> writes:
>
>> But you can always do it like this:  For every script to be
>> embedded inside a shell script, procede using the following
>> rules:  In the script to be embedded, replace each single quote
>> by a single quote, a backslash, and 2 single quotes
>> [alternatives elided...]
>
> That works well when there is a "build process" for the script that
> contains the embedded script because, as you say, it's easy to automate.
> But I find it fragile when the embedded script might be edited and hard
> to read when the script needs to be inspected. Lots of '\''...'\''s get
> in the way.

Yes, that's true.  In this case, I either include the script to
be embedded in its original form, each line of it preceded by


#


in the shell script as well, that is, save it as a shell comment
for a later source of re-editing.


Or, when the embedded script is to be to re-edited, construct a
shell command to be run:  Start that command with the shell
command line fragment


printf '%s' \


then copy and paste the embedded script, which is to be
re-edited, into the following lines, either in a shell script
file or use the features of your favourite editor to run a shell
command line.  Then run that shell script file (resp. the shell
command line).


The standard output will be the original script to be embedded.

> My preference is for here documents. You can choose a clear and
> staggeringly unlikely delimiter, and the text between the start and the
> end is taken literally (if you 'quote-the-end-delimiter').
>
> This works best for standard input, but when I need a single string
> argument I put "$(cat in front and )" at the end. Not pretty, perhaps,
> but it keeps all of the mess away from the actual embedded script:
>
> printf "{%s}\n" "$(cat <<'---END-OF-EMBEDDED-SCRIPT---'
> ...script here...
> ---END-OF-EMBEDDED-SCRIPT---
> )"

When using here documents, I tend to use


cut -d . -f2- <<-EOF
.These are the lines of the script to be embedded,
.each of them prepended with the "cut" delimiter.
.Even indented lines like the following
. indented line
.will have their indentation preserved.
EOF


rather than just "cat", prepending each line of the script to be
embedded with a non-whitespace delimiter (for example a period): 
That allows me to neatly indent the here document using TAB
characters according to the indentation level of its context in
the shell script and also frees me from having to worry about
looking for a suitable here document delimiter: "EOF" (for
example) will always work, as it is different from any line
starting with zero or more white space characters followed by a
period.

[Janis Papanagnou]

On 11.12.2023 12:40, Helmut Waitzmann wrote:
>
> When using here documents, I tend to use
>
> cut -d . -f2- <<-EOF
> .These are the lines of the script to be embedded,
> .each of them prepended with the "cut" delimiter.
> .Even indented lines like the following
> . indented line
> .will have their indentation preserved.
> EOF
>
>
> rather than just "cat", prepending each line of the script to be
> embedded with a non-whitespace delimiter (for example a period): That
> allows me to neatly indent the here document using TAB characters
> according to the indentation level of its context in the shell script
> and also frees me from having to worry about looking for a suitable here
> document delimiter: "EOF" (for example) will always work, as it is
> different from any line starting with zero or more white space
> characters followed by a period.

Ksh users may use '<<#' as in

cat <<# EOF
These are the lines of the script to be embedded.

Even indented lines like the following

tab-indented line
space-indented line

will have their indentation preserved.
EOF

"If # is appended to <<, then leading spaces and tabs will
be stripped off the first line of the document and up to an
equivalent indentation will be stripped from the remaining
lines and from word." [man ksh]

(Note: by "word" the 'EOF' is meant here. It can also be
indented if desired.)

Janis