testing if strconv.Quote() would do change a string, without calling it

[Tim Hockin]

Is there any ready-built function that can tell me whether `strconv.Quote()` would produce a different string than its input, without actually running it?  Or is there a clearly documented set of rules one could use to test each rune in a string?

I am trying to avoid allocations, and MOST of the inputs will be safe (but not all).  Calling strconv.Quote() has a measurable impact, so I'd avoid it if I could...

Does such an animal exist?

Tim

[Robert Engels]

A simple loop calling IsPrint is your best bet. You could then have a custom implementation of Quote that started at a specified index. 

On Oct 13, 2021, at 5:46 PM, 'Tim Hockin' via golang-nuts <golan...@googlegroups.com> wrote:

Is there any ready-built function that can tell me whether `strconv.Quote()` would produce a different string than its input, without actually running it?  Or is there a clearly documented set of rules one could use to test each rune in a string?

I am trying to avoid allocations, and MOST of the inputs will be safe (but not all).  Calling strconv.Quote() has a measurable impact, so I'd avoid it if I could...

Does such an animal exist?

Tim

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/1bb3f806-5930-4866-8249-0bbc0ee383b8n%40googlegroups.com.

[Tim Hockin]

` IsPrint(r) || r == '\\' || r == '"' ` passes tests.  I need to build confidence in that, though :)  Thanks.

[Robert Engels]

I was thinking the other way. If !IsPrint() then strconv.Quote()

On Oct 13, 2021, at 6:24 PM, Tim Hockin <tho...@google.com> wrote:



[Tim Hockin]

If I find a string with a stray backslash in it (which passes IsPrint()) I still need to quote.

I dug into the Quote() impl and this seems like the right path.  Thanks!

[Ian Lance Taylor]

To answer your exact question, strconv.Quote always allocates a new
string, because it always adds quotation marks around the returned
string. That is, the output of strconv.Quote("a") is `"a"`, with
literal quotation characters.

Other than calling strconv.Quote will replace any rune for which
strconv.IsPrint returns false, and it will also replace `\` and `"`.
But I don't think there is a function to check that.

Ian

[Tim Hockin]

Thanks for confirming.  I wrote that function and erased a good bit of the overhead.

bytes.Buffer for the no-escapes path and strconv.Quote otherwise.

[roger peppe]

On Thu, 14 Oct 2021 at 04:58, 'Tim Hockin' via golang-nuts <golan...@googlegroups.com> wrote:

Thanks for confirming.  I wrote that function and erased a good bit of the overhead.

bytes.Buffer for the no-escapes path and strconv.Quote otherwise.

Could you not use strconv.AppendQuote and get the advantage without needing the extra scan?

[Tim Hockin]

I tried:

strconv.Quote()

strconv.AppendQuote() (weird that this is faster than Quote)

fmt.Sprintf("%q")

scanning for !IsPrint() + bytes.Buffer

scanning for !IsPrint() + strings.Builder (sad that this is not faster than Buffer)

scanning for !IsPrint() + string addition

 

```

$ go test -benchtime=5s -bench='XX' ./benchmark/
goos: linux
goarch: amd64
pkg: github.com/go-logr/logr/benchmark
cpu: Intel(R) Xeon(R) W-2135 CPU @ 3.70GHz
BenchmarkXXQuote-6         23233543       258.5 ns/op
BenchmarkXXAppendQuote-6   24370812       207.7 ns/op
BenchmarkXXSprintf-6       18040070       335.2 ns/op
BenchmarkXXScanBuffer-6     47154340       117.4 ns/op
BenchmarkXXScanBuilder-6   42295635       141.9 ns/op
BenchmarkXXScanAdd-6       43635146       137.5 ns/op
PASS
ok   github.com/go-logr/logr/benchmark 35.926s

```

code:

```

//go:noinline
func foo(s string) {
    _ = s
}

func prettyBuffer(s string) string {
    if needsEscape(s) {
        return strconv.Quote(s)
    }
    b := bytes.NewBuffer(make([]byte, 0, 1024))
    b.WriteByte('"')
    b.WriteString(s)
    b.WriteByte('"')
    return b.String()
}

func prettyBuilder(s string) string {
    if needsEscape(s) {
        return strconv.Quote(s)
    }
    b := strings.Builder{}
    b.WriteByte('"')
    b.WriteString(s)
    b.WriteByte('"')
    return b.String()
}
func prettyAdd(s string) string {
    if needsEscape(s) {
        return strconv.Quote(s)
    }
    return `"` + s + `"`
}

// needsEscape determines whether the input string needs to be escaped or not,
// without doing any allocations.
func needsEscape(s string) bool {
    for _, r := range s {
        if !strconv.IsPrint(r) || r == '\\' || r == '"' {
            return true
        }
    }
    return false
}

func BenchmarkXXQuote(b *testing.B) {
    in := "a string with no specials"
    for i := 0; i < b.N; i++ {
        out := strconv.Quote(in)
        foo(out)
    }
}
func BenchmarkXXAppendQuote(b *testing.B) {
    in := "a string with no specials"
    for i := 0; i < b.N; i++ {
        out := strconv.AppendQuote(make([]byte, 0, 1024), in)
        foo(string(out))
    }
}
func BenchmarkXXSprintf(b *testing.B) {
    in := "a string with no specials"
    for i := 0; i < b.N; i++ {
        out := fmt.Sprintf("%q", in)
        foo(out)
    }
}
func BenchmarkXXScanBuffer(b *testing.B) {
    in := "a string with no specials"
    for i := 0; i < b.N; i++ {
        out := prettyBuffer(in)
        foo(out)
    }
}
func BenchmarkXXScanBuilder(b *testing.B) {
    in := "a string with no specials"
    for i := 0; i < b.N; i++ {
        out := prettyBuilder(in)
        foo(out)
    }
}
func BenchmarkXXScanAdd(b *testing.B) {
    in := "a string with no specials"
    for i := 0; i < b.N; i++ {
        out := prettyAdd(in)
        foo(out)
    }
}

```