C++11 Feature Proposal: Raw string literals

[Alex Vakulenko]

What:

Easily specify string literals that contain new lines, tabs, quotes, apostrophes and other special characters without needing to escape them.

http://en.cppreference.com/w/cpp/language/string_literal

Why:

If you ever had to encode a JSON string in a C++ unit test code you know the pain you have to go through by escaping every single (") character.

So, rather than:

const char* json = "{\"name\": \"value\"}";

you can do just:

const char* json = R"({"name":"value"})";

Here is an example of real world usage in Chrome OS:

https://chromium-review.googlesource.com/#/c/218749/4..5/chromeos-dbus-bindings/adaptor_generator_unittest.cc

Where:

Every time one needs to use a string literal with a lot of escaped characters.

[Ryan Sleevi]

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

In general, we tend to favour long-term readability rather than writability.

Concerns with this:

1) A number of tools / syntax highlighters don't support this, and thus get all sorts of confused

2) This suffers from the same issues of an #ifdef soup - that is, without syntax highlighting/coloring (and we still have a lot of tools that lack it - including, for example, the codereview tool being used) - it forces a lot of visual scanning to find when something begins or ends. 

When I first looked at your code, what I saw was a bunch of code. It wasn't until I scanned for a while that I saw the R"literal_string( and the )literal_string" that indicated that, oh yes, this is a string.

Please see Titus Winter's CPPCon 14 talk on the Philosophy of Google's Style Guide 

From my own experience, I'd prefer we don't allow these until we have better tooling, editing, and even then, only in limited circumstances. It definitely feels more reader-hostile (string escapes, while annoying, are at least clear evidence of being in a string)

[Stephen Chenney]

Sure, tooling may be an issue. But in the example linked to from Chrome OS it would be challenging to figure out the desired test result if everything in that chunk of code was escaped. I doubt it would be any easier to rapidly parse than the current situation. That's long term readability to me. Lack of tooling is short term readability.

Stephen.

[Alex Vakulenko]

Well, I don't know about you, but I'd prefer to read through this (even without tooling):

EXPECT_EQ(R("{

  "base":{

    "manufacturer":"",

    "serialNumber":""

  },

  "power":{

    "battery_level":100

  },

  "terminator":{

    "target":""

  }

})", ValueToString(mgr_->GetStateValuesAsJson(nullptr).get()));

than this:

EXPECT_EQ("{

"  \"base\":{\n"

"    \"manufacturer\":\"\",\n"

"    \"serialNumber\":\"\"\n"

"  },\n"

"  \"power\":{\n"

"    \"battery_level\":100\n"

"  },\n"

"  \"terminator\":{\n"

"    \"target\":\"\"\n"

"  }\n"

"}", ValueToString(mgr_->GetStateValuesAsJson(nullptr).get()));

Alex

[Ryan Sleevi]

While I subjectively disagree with that, I will point out that if it was a strong concern, then that test could have done precisely what we do in a variety of our existing tests - which is to have a gold/master/reference file, and compare the output to the contents of that file.

I think having a test with such a large string, escaped or not, is generally indicative of a codesmell itself. In non-Chromium code, where facilities like //base are neither universal nor ubiquitous, I can understand the argument. But here? It doesn't really seem to float with me. 

[Jeffrey Yasskin]

FWIW, I find a separate file hard to read in a test. When I'm writing
the test, sure it's easy to open enough windows to go back and forth,
but if I'm just looking at the test to see what failed, it's a
roadblock to have to find the file (usually some part of the path
prefix is "abstracted" away), and then lose part of my investigation
state in order to open a new window so I can see the contents of the
assertion next to the assertion itself.

[Peter Kasting]

On Thu, Sep 25, 2014 at 11:29 AM, Ryan Sleevi <rsl...@chromium.org> wrote:

In general, we tend to favour long-term readability rather than writability.

I agree, but I agree with Alex that the no-slashes form is significantly more readable.  So to me readability is precisely the argument for doing this.

I think if your code really makes it hard to tell where a string ends with this, it's likely hard to tell where the string ends without it too.  I can't think of a case where I'd have a _harder_ time with raw strings.  Certainly in Alex' examples I find it no harder to spot the end of the string in the new version.  Quotes look like quotes either way.

PK 

[Scott Graham]

On Thu, Sep 25, 2014 at 2:01 PM, Peter Kasting <pkas...@chromium.org> wrote:

On Thu, Sep 25, 2014 at 11:29 AM, Ryan Sleevi <rsl...@chromium.org> wrote:

In general, we tend to favour long-term readability rather than writability.

I agree, but I agree with Alex that the no-slashes form is significantly more readable.  So to me readability is precisely the argument for doing this.

I think if your code really makes it hard to tell where a string ends with this, it's likely hard to tell where the string ends without it too.  I can't think of a case where I'd have a _harder_ time with raw strings.

Sure you can, it's C++! Quick, what's:

R"x%/uR(%b)x%/uR"

?

More seriously, if we want to go for this, perhaps not allowing custom delimiters (or whatever they're called) for a while would break less tools.

 

  Certainly in Alex' examples I find it no harder to spot the end of the string in the new version.  Quotes look like quotes either way.

PK 

--

[Peter Kasting]

On Thu, Sep 25, 2014 at 2:19 PM, Scott Graham <sco...@chromium.org> wrote:

On Thu, Sep 25, 2014 at 2:01 PM, Peter Kasting <pkas...@chromium.org> wrote:

On Thu, Sep 25, 2014 at 11:29 AM, Ryan Sleevi <rsl...@chromium.org> wrote:

In general, we tend to favour long-term readability rather than writability.

I agree, but I agree with Alex that the no-slashes form is significantly more readable.  So to me readability is precisely the argument for doing this.

I think if your code really makes it hard to tell where a string ends with this, it's likely hard to tell where the string ends without it too.  I can't think of a case where I'd have a _harder_ time with raw strings.

Sure you can, it's C++! Quick, what's:

R"x%/uR(%b)x%/uR"

?

More seriously, if we want to go for this, perhaps not allowing custom delimiters (or whatever they're called) for a while would break less tools.

My apologies, I misunderstood the syntax.  I didn't realize the custom delimiter thing was possible.  That does cause problems.

PK 

[Scott Graham]

(I was just being silly, that's obviously "not lgtm".)

But per original post, I'd lean towards being less enthusiastic than "Where: Every time one needs to use a string literal with a lot of escaped characters".

Something like "Allowed in the R"( )" form, but only if substantially more readable than regular strings. Consider using an external data file instead.". I think the inlined json test passes that.

[Peter Kasting]

On Thu, Sep 25, 2014 at 2:39 PM, Scott Graham <sco...@chromium.org> wrote:

(I was just being silly, that's obviously "not lgtm".)

And I was being stupid for opining about something without sufficient understanding of it.  I didn't even realize the () part was in there.  /hangs head in shame

PK

[Bruce Dawson]

Raw string literals should continue to be disallowed for now because of buggy support in VC++ 2013. I have tried using them and they can make code much more readable but they cause VC++ 2013 to get confused about what line number it is on. This then causes warning and error message line numbers to be incorrect which causes mind-bending confusion. It turns out that I depend on the line numbers in warning and error messages quite heavily. It's a shame because VC++'s support for the functionality of raw string literals appears to be fine. I filed a bug two months ago but so far I haven't received any responses. Here's the bug link:

https://connect.microsoft.com/VisualStudio/feedback/details/933079/c-11-multi-line-string-constants-in-macros-cause-error-warning-message-line-numbers-to-be-way-off

[Alex Vakulenko]

By the way, gcc has the same bug, but clang seems fine.

[Nico Weber]

This is now banned for now due to the compiler bugs: http://chromium-cpp.appspot.com/

Alex: Do you happen to have a link to the gcc bug?

[Alex Vakulenko]

Sorry, no. I don't know if it is even logged. I bumped into it while stepping through some code in debugger and noticed that the lines and breakpoints were all off. I narrowed it down to raw literal strings and also the problem went away after recompiling the code with clang.

[Alex Clarke]

I don't think we're using VC++ 2013 anymore.  Is it safe to allow these now?

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev+unsubscribe@chromium.org.

[Jeremy Roman]

On Fri, Aug 19, 2016 at 8:37 AM, 'Alex Clarke' via Chromium-dev <chromi...@chromium.org> wrote:

I don't think we're using VC++ 2013 anymore.  Is it safe to allow these now?

If our bots are happy with it, I have no objections. Unless Dana or Nico objected, I'd lgtm a CL that added a use and updated c++11.html to allow it.

[Jeremy Roman]

On Fri, Aug 19, 2016 at 10:20 AM, Jeremy Roman <jbr...@chromium.org> wrote:

On Fri, Aug 19, 2016 at 8:37 AM, 'Alex Clarke' via Chromium-dev <chromi...@chromium.org> wrote:

I don't think we're using VC++ 2013 anymore.  Is it safe to allow these now?

If our bots are happy with it, I have no objections. Unless Dana or Nico objected, I'd lgtm a CL that added a use and updated c++11.html to allow it.

Errm, by that I mean our build works, _and_ line numbers are correct with MSVC, clang and gcc (assuming I'm correct in thinking we still use gcc on some platform -- one of the CrOS builds, maybe?).

[dan...@chromium.org]

On Fri, Aug 19, 2016 at 7:21 AM, Jeremy Roman <jbr...@chromium.org> wrote:

On Fri, Aug 19, 2016 at 10:20 AM, Jeremy Roman <jbr...@chromium.org> wrote:

On Fri, Aug 19, 2016 at 8:37 AM, 'Alex Clarke' via Chromium-dev <chromi...@chromium.org> wrote:

I don't think we're using VC++ 2013 anymore.  Is it safe to allow these now?

If our bots are happy with it, I have no objections. Unless Dana or Nico objected, I'd lgtm a CL that added a use and updated c++11.html to allow it.

SGTM. Maybe recommend using () as the delimiter in the comments aka "(...)"? The google style guide appears to be silent on this, but having some sorta consistency may be nice.

 

Errm, by that I mean our build works, _and_ line numbers are correct with MSVC, clang and gcc (assuming I'm correct in thinking we still use gcc on some platform -- one of the CrOS builds, maybe?).

I believe cros and android by default use gcc.

[Scott Graham]

VS2015 seems OK as far as line numbers being fine in errors and debugging. (Don't know about our GCCs.)

[Jeremy Roman]

On Fri, Aug 19, 2016 at 2:40 PM, <dan...@chromium.org> wrote:

On Fri, Aug 19, 2016 at 7:21 AM, Jeremy Roman <jbr...@chromium.org> wrote:

On Fri, Aug 19, 2016 at 10:20 AM, Jeremy Roman <jbr...@chromium.org> wrote:

On Fri, Aug 19, 2016 at 8:37 AM, 'Alex Clarke' via Chromium-dev <chromi...@chromium.org> wrote:

I don't think we're using VC++ 2013 anymore.  Is it safe to allow these now?

If our bots are happy with it, I have no objections. Unless Dana or Nico objected, I'd lgtm a CL that added a use and updated c++11.html to allow it.

SGTM. Maybe recommend using () as the delimiter in the comments aka "(...)"? The google style guide appears to be silent on this, but having some sorta consistency may be nice.

To be clear, () is required by the language. You can just add more to that, but I would only expect people to do that if the string )" appears in their string, so I don't think we need to be explicit.

[dan...@chromium.org]

On Fri, Aug 19, 2016 at 12:29 PM, Jeremy Roman <jbr...@chromium.org> wrote:

On Fri, Aug 19, 2016 at 2:40 PM, <dan...@chromium.org> wrote:

On Fri, Aug 19, 2016 at 7:21 AM, Jeremy Roman <jbr...@chromium.org> wrote:

On Fri, Aug 19, 2016 at 10:20 AM, Jeremy Roman <jbr...@chromium.org> wrote:

On Fri, Aug 19, 2016 at 8:37 AM, 'Alex Clarke' via Chromium-dev <chromi...@chromium.org> wrote:

I don't think we're using VC++ 2013 anymore.  Is it safe to allow these now?

If our bots are happy with it, I have no objections. Unless Dana or Nico objected, I'd lgtm a CL that added a use and updated c++11.html to allow it.

SGTM. Maybe recommend using () as the delimiter in the comments aka "(...)"? The google style guide appears to be silent on this, but having some sorta consistency may be nice.

To be clear, () is required by the language. You can just add more to that, but I would only expect people to do that if the string )" appears in their string, so I don't think we need to be explicit.

Oh thanks, I didn't catch that.

[Jean-François Geyelin]

Any update?

[Bruce Dawson]

I'm not aware of any blocking issues. VC++ 2015 support seems fine, based on my brief testing.

[dan...@chromium.org]

On Tue, Aug 23, 2016 at 5:21 AM, Jean-François Geyelin <j...@chromium.org> wrote:

Any update?

Someone just needs to write a CL using the feature and updating the guide. Maybe you can coordinate with the OP (avakulenko@) to get that done.

[Brett Wilson]

I'm enthusiastic about this.

To provide a counter to readability and maintainability concerns, GN has thousands of lines of documentation in-line (I believe keeping the docs with the code is more maintainable than a wiki or something) and the formatting required to make this work is quite cumbersome. I think Raw string literals would make this a lot more readable and maintainable.

Brett

[Nico Weber]

Want to send a CL to move them to allowed and that changes a couple gn strings to raw literals, to doublecheck that all compilers are happy with this now? :-)

[Dirk Pranke]

I think they are still broken in GCC 4.8 (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=57824, maybe?) which I think we still use on CrOS and possibly Android?

I feel like I tested this in some version of 4.9 and they were broken as well, but I might be misremembering and someone should verify both versions.

-- Dirk

[Nico Weber]

On Fri, Sep 2, 2016 at 12:18 PM, Dirk Pranke <dpr...@chromium.org> wrote:

I think they are still broken in GCC 4.8 (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=57824, maybe?) which I think we still use on CrOS and possibly Android?

Android's on 4.9. That bug is about raw string literals as macro arguments; most raw string literals probably aren't used as macro arguments...is that's the only problem, I think we could generally allow them (and mention this known bug).

[Peter Kasting]

On Fri, Sep 2, 2016 at 9:25 AM, Nico Weber <tha...@chromium.org> wrote:

On Fri, Sep 2, 2016 at 12:18 PM, Dirk Pranke <dpr...@chromium.org> wrote:

I think they are still broken in GCC 4.8 (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=57824, maybe?) which I think we still use on CrOS and possibly Android?

Android's on 4.9. That bug is about raw string literals as macro arguments; most raw string literals probably aren't used as macro arguments...is that's the only problem, I think we could generally allow them (and mention this known bug).

As of https://crrev.com/bf8b8103771747ed31d3af4e924de7cd27607deb , this is done.

PK 

[Lutz]

The presubmit check in Chrome OS platform2 complains about tabs in raw strings:

            * Found a tab character in:

                * <myfile>, line 22

Is there a simple way to fix this? I figure the check would have to know when it's processing a raw string, which might end up being very complicated to get right (e.g. R" could be in a comment or a macro).

- Lutz

[Nico Weber]

I don't know much about Chrome OS scripting, but my take is that parsing C++ from python scripts is a losing proposition. In src/, we now have a presubmit check that checks that code is clang-formatted, and that takes care of tabs (and uses the lexer of a real C++ compiler).