f-strings again.

[Carlton Gibson]

Hi All.

f-strings...

There was some discussion a couple of years ago about replacing all string formatting operations with f-strings.

The consensus then was "No, let's not do that": %-formatting and format() are both great. Each has advantages

in readability, depending on context, and there are translation and logging issues with f-strings.

We've since not been allowing (at all) f-strings in the code. A PR comes in with a f-string, we remove it (or ask the author to).

A few months ago this came up again. We said we needed to come back to the mailing, because of the previous discussion,

but the thread never started — but there was an amount of support for at least allowing f-strings. (From Claude, Jon, Nick, Paolo, Simon, ...)

We have a PR again where the "Are f-strings allowed?" conversation has come up — so I'm posting to the list: is it time to update the policy?

So let me quote Simon on #29988:

> I am in favor where it makes sense and has clear readability benefits. I don't have strong rules to provide but I've already seen this feature abused in the wild to compose complex formatting string which makes hard to differentiate placeholders from their delimiters when function calls are involved.

>

> I also share you Claude's concerns about error messages and logging message templates.

So, paraphrasing, in favor of allowing, but it's not just true that f-strings are always more readable, and can't be used for strings that may be translated.

I had the impression in ≈Feb that that kind of view would have broad support. 🤷‍♀️

Beyond that I would like to avoid spending time updating existing uses to use f-strings. Sure, if we're editing, and an f-string is more reasonable, let's use it, but, like replacing quotes, let's not (please) spend time on it beyond that. (I think Jon and Nick may not agree with that from previous comments but...)

As such I've prepared a PR suggesting a change to the Coding Style doc, that should allow us to close #29988:

> * String variable interpolation may use %-formatting, ``format()``, or
>   f-strings as appropriate, with the goal of maximizing code readability.
>   f-strings should not be used for any string that may require translation,
>   including error and logging messages. Don't waste time doing unrelated
>   refactoring of existing code to adjust the formatting method.

Tweaks (as always) welcome on the PR. General discussion of the principle here? Shall we allow f-strings? Three formatting methods seems a lot... but that's what we've got, and folks are getting used to using them.

Thanks for the input all!

Kind Regards,

Carlton

[Mariusz Felisiak]

Hi y'all,

    I will not stand against f-strings, I think we can allow them. My main concerns is readability. f-strings are powerful and it's quite common that they are used with functions calls and complex formatting which makes code hard to understand and maintain, "With great power comes great responsibility" ...

    I'm strongly against any bulk updates to conform to this style. This will just create unnecessary noise in the history, I know that there are tools, please don't start this discussion again :)

    I would also be in favor of keeping only %-formatting and f-strings in Coding style docs. I don't see any reason to use also `format()` in a new code.

Best,
Mariusz

[laym...@gmail.com]

Hi folks,

I personally like to use f-strings wherever it makes sense, so +1.

I agree with Mariusz. I think we should only allow %-formatting and f-strings from now on.

And yes, bulk updates should not be done. Aside from making unnecessary noise, there's always a risk in doing that kind of thing.

Regards,

Sage

[Jacob Rief]

I strongly agree with Mariusz on

 I would also be in favor of keeping only %-formatting and f-strings in Coding style docs. I don't see any reason to use also `format()` in a new code.

so +1 from my side for f-strings.

Actually, in one of my side projects, %-formatting wouldn't even work, so `format()`, or the newer f-strings are an improvement in functionality.

- Jacob

[Adam Johnson]

+1 to allowing f-strings.

Mariusz' position dropping .format() seems reasonable to me too. I can't think of a reason to use them over f-strings, off the top of my head.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/8a0cbf42-1879-4ed1-9d3c-8e6985012155n%40googlegroups.com.

--

Adam

[Carlton Gibson]

I'm going to float this here, just to make sure it's considered...

`format()` allows positional and named kwargs in format strings:

"First, thou shalt count to {0}"  # References first positional argument
"Bring me a {}"                   # Implicitly references the first positional argument
"From {} to {}"                   # Same as "From {0} to {1}"
"My quest is {name}"              # References keyword argument 'name'
"Weight in tons {0.weight}"       # 'weight' attribute of first positional arg
"Units destroyed: {players[0]}"   # First element of keyword argument 'players'.

Are these never useful?

Certainly 99% of cases can be handled as cleanly (or more so, because I guess we fell `format()` is a little verbose) with %-formatting or an f-string.

But if we say format() is not allowed, don't we then guarantee we hit the one case in X where "Actually, this version with format() is much cleaner"?

[אורי‎]

I don't like f-strings (actually I was not aware what they are until I googled them now). I also don't like the %-formatting and I prefer to use {}-format (with or without a number or name) for all my formatting purposes.

In general, f-strings take variables which are not given to them explicitly, which I think is wrong. The {}-format takes only variables which are explicitly given to them.

Also when I use translations, I use the {}-format strings in the translated strings.

אורי

u...@speedy.net

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/f7709597-7f55-4e64-9a2a-f0b62e6e1393o%40googlegroups.com.

[Javier Buzzi]

I'm +1 f-string -- very conformable, easy to work with, and fits 99% of my needs.
I'm  -1 removing .format()  -- there are some limitations to f-string, for example using them for a template-like placeholder/i18n, in these cases .format() comes very handy. Yes, I suppose %-format in these cases, but .format() is a lot more confortable to me... #personalbias.

- buzzi

[Nick Pope]

Hi,

So I'm in favour of using f-strings, but agree that there are places where they are not appropriate.

I am also against bulk updates in this case as mentioned in my previous comment. I don't think we should exclude replacing .format() with f-strings on a case-by-case basis, however, where performance is a concern.

Thanks for your initial documentation, Carlton. Am wondering whether we should be more explicit about the pros and cons of each to help people make the correct decision? Here are my thoughts:

%-formatting:

+ Simple to use printf-style familiar to all.

+ Default (can be changed) style used internally by the logging module, e.g. logging.info('Message with %s.', value)

− Much less flexibility for formatting, see pyformat.info.

``.format()``:

+ Useful for interpolating a stored template string, e.g. from a database, which isn't possible with f-strings.
− Worst performance due to method call.

− Much more verbose, makes code less readable.

f-strings:

+ Better performance than other methods due to dedicated FORMAT_VALUE opcode.

+ Allows for embedding more complex expressions, e.g. f'Hello {user.get_full_name()}'

+ Much more concise, makes code more readable.

− Cannot be used if string must be translated.

− Complex expressions can get out of control. A sensible balance needs to be struck.

Regarding performance, here are some simple numbers:

python -m timeit -s 'x="test"' 'f"{x}"'

20000000 loops, best of 5: 11.8 nsec per loop

$ python -m timeit -s 'x="test"' '"%s" % x'

10000000 loops, best of 5: 39.1 nsec per loop

$ python -m timeit -s 'x="test"' '"{}".format(x)'

5000000 loops, best of 5: 76.1 nsec per loop

I think it is probably also worth updating the documentation added in this commit. It isn't that xgettext doesn't support f-strings... It does:

$ echo "_('Hello %s') % user.username" | xgettext --language=python --omit-header --output=- -

#: standard input:1

#, python-format

msgid "Hello %s"

msgstr ""

$ echo "_('Hello {}').format(user.username)" | xgettext --language=python --omit-header --output=- -

#: standard input:1

msgid "Hello {}"

msgstr ""

$ echo "_('Hello {name}').format(name=user.username)" | xgettext --language=python --omit-header --output=- -

#: standard input:1

#, python-brace-format

msgid "Hello {name}"

msgstr ""

$ echo "_(f'Hello {user.username}')" | xgettext --language=python --omit-header --output=- -

#: standard input:1

#, python-brace-format

msgid "Hello {user.username}"

msgstr ""

$ echo "_(f'Hello {user.get_full_name()}')" | xgettext --language=python --omit-header --output=- -

#: standard input:1

msgid "Hello {user.get_full_name()}"

msgstr ""

It is actually that Python doesn't support modifying the template string prior to interpolating the values which is the requirement for injecting the translated string. PEP 498 makes no explicit mention of this. PEP 501 was initially looking at solving the problem but was deemed a poor fit and was also deferred.

Kind regards,

Nick

[st...@jigsawtech.co.uk]

+1 for f-strings from me I actually find them easier to understand at a glance than %-formatting and .format

[Dave Vernon]

+1, I'd happily help on any PR with this (it would be my first!).

Kind Regards,

Dave

Springbourne Tech Limited

M: +44(0) 79 2107 6483 

W: www.springbourne-tech.com

—
This E-Mail and its contents are confidential, protected by law and legally privileged.  Only access by the addressee is authorised.  Any liability (in negligence, contract or otherwise) arising from any third party taking any action or refraining from taking any action on the basis of any of the information contained in this E-Mail is hereby excluded to the fullest extent of the law.  In the event that you are not the addressee, please notify the sender immediately.  Do not discuss, disclose the contents to any person or store or copy the information in any medium or use it for any purpose whatsoever.

--

You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/ceba9695-9d21-436f-84b5-53381aa3ea15n%40googlegroups.com.

[Tom Carrick]

I'm also +1, I find f-strings easier to read in most cases. You can go overboard -  and I probably do - but I don't think the guard against unreadable f-strings is to ban them outright.

One small negative to % formatting for me is that I sometimes get confused for a second about whether something is interpolated in the code or if they're placeholders for sql params or logging calls. But I don't see a reason to change any of the current ones because of that.

Cheers,

Tom

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CADqmmRwfce5zrF0Yr4Fe_NczXsiSvZ7mePyE_uasuK37ZU-C2A%40mail.gmail.com.

[Markus Holtermann]

I've been one of those who previously were against adding them to Django. Mostly, because I wasn't used to them and didn't see their value. But more importantly, because I was worried about unintended security issues they could cause. Why the latter is still present, I've think the vulnerability vectors are fairly small. Hence:

+1 to allow them in PRs that add new code or adjust existing code
-1 on PRs who's primary focus is to change either % or .format() to use f-strings

/Markus

On Tue, Jul 21, 2020, at 11:24 AM, Tom Carrick wrote:
> I'm also +1, I find f-strings easier to read in most cases. You can go
> overboard - and I probably do - but I don't think the guard against
> unreadable f-strings is to ban them outright.
>
> One small negative to % formatting for me is that I sometimes get
> confused for a second about whether something is interpolated in the
> code or if they're placeholders for sql params or logging calls. But I
> don't see a reason to change any of the current ones because of that.
>
> Cheers,
> Tom
>
> On Tue, 21 Jul 2020 at 10:55, Dave Vernon <da...@springbourne-tech.com> wrote:
> > +1, I'd happily help on any PR with this (it would be my first!).
> >
> > Kind Regards,
> >
> > Dave
> >
> > Springbourne Tech Limited
> > M: +44(0) 79 2107 6483
> > W: www.springbourne-tech.com
> >
> >
> >
> >

> > *—
> > This E-Mail and its contents are confidential, protected by law and legally privileged. Only access by the addressee is authorised. Any liability (in negligence, contract or otherwise) arising from any third party taking any action or refraining from taking any action on the basis of any of the information contained in this E-Mail is hereby excluded to the fullest extent of the law. In the event that you are not the addressee, please notify the sender immediately. Do not discuss, disclose the contents to any person or store or copy the information in any medium or use it for any purpose whatsoever.*

> >
> >
> >
> >
> > On Tue, 21 Jul 2020 at 09:43, Nick Pope <nickpo...@gmail.com> wrote:
> >> Hi,
> >>
> >> So I'm in favour of using f-strings, but agree that there are places where they are not appropriate.
> >>

> >> I am also against bulk updates in this case as mentioned in my previous comment <https://github.com/django/django/pull/12650#issuecomment-607707454>. I don't think we should exclude replacing .format() with f-strings on a case-by-case basis, however, where performance is a concern.

> >>
> >> Thanks for your initial documentation, Carlton. Am wondering whether we should be more explicit about the pros and cons of each to help people make the correct decision? Here are my thoughts:
> >>
> >> %-formatting:
> >>
> >> + Simple to use printf-style familiar to all.

> >> + Default (can be changed <https://docs.python.org/3/howto/logging-cookbook.html#formatting-styles>) style used internally by the logging module, e.g. logging.info('Message with %s.', value)

> >> − Much less flexibility for formatting, see pyformat.info.
> >>
> >> ``.format()``:
> >>
> >> + Useful for interpolating a stored template string, e.g. from a database, which isn't possible with f-strings.
> >> − Worst performance due to method call.
> >> − Much more verbose, makes code less readable.
> >>
> >> f-strings:
> >>

> >> + Better performance than other methods due to dedicated FORMAT_VALUE <https://bugs.python.org/issue25483> opcode.

> >> + Allows for embedding more complex expressions, e.g. f'Hello {user.get_full_name()}'
> >> + Much more concise, makes code more readable.
> >> − Cannot be used if string must be translated.
> >> − Complex expressions can get out of control. A sensible balance needs to be struck.
> >>
> >> Regarding performance, here are some simple numbers:
> >>
> >> python -m timeit -s 'x="test"' 'f"{x}"'
> >> 20000000 loops, best of 5: 11.8 nsec per loop
> >> $ python -m timeit -s 'x="test"' '"%s" % x'
> >> 10000000 loops, best of 5: 39.1 nsec per loop
> >> $ python -m timeit -s 'x="test"' '"{}".format(x)'
> >> 5000000 loops, best of 5: 76.1 nsec per loop
> >>

> >> I think it is probably also worth updating the documentation added in this commit <https://github.com/django/django/commit/c3437f734d03d93f798151f712064394652cabed>. It isn't that xgettext doesn't support f-strings... It does:

> >>
> >> $ echo "_('Hello %s') % user.username" | xgettext --language=python --omit-header --output=- -
> >> #: standard input:1
> >> #, python-format
> >> msgid "Hello %s"
> >> msgstr ""
> >> $ echo "_('Hello {}').format(user.username)" | xgettext --language=python --omit-header --output=- -
> >> #: standard input:1
> >> msgid "Hello {}"
> >> msgstr ""
> >> $ echo "_('Hello {name}').format(name=user.username)" | xgettext --language=python --omit-header --output=- -
> >> #: standard input:1
> >> #, python-brace-format
> >> msgid "Hello {name}"
> >> msgstr ""
> >> $ echo "_(f'Hello {user.username}')" | xgettext --language=python --omit-header --output=- -
> >> #: standard input:1
> >> #, python-brace-format
> >> msgid "Hello {user.username}"
> >> msgstr ""
> >> $ echo "_(f'Hello {user.get_full_name()}')" | xgettext --language=python --omit-header --output=- -
> >> #: standard input:1
> >> msgid "Hello {user.get_full_name()}"
> >> msgstr ""
> >>

> >> It is actually that Python doesn't support modifying the template string prior to interpolating the values which is the requirement for injecting the translated string. PEP 498 <https://www.python.org/dev/peps/pep-0498/> makes no explicit mention of this. PEP 501 <https://www.python.org/dev/peps/pep-0501/> was initially looking at solving the problem but was deemed a poor fit and was also deferred.

> >>
> >> Kind regards,
> >>
> >> Nick
> >> On Tuesday, 21 July 2020 at 08:28:54 UTC+1 Mariusz Felisiak wrote:
> >>> Hi y'all,
> >>>

> >>> I will not stand against f-strings, I think we can allow them. My main concerns is readability. f-strings are powerful and it's quite common that they are used with functions calls and complex formatting which makes code hard to understand and maintain, *"With great power comes great responsibility" ...*

> >>>
> >>> I'm strongly against any bulk updates to conform to this style. This will just create unnecessary noise in the history, I know that there are tools, please don't start this discussion again :)
> >>>
> >>> I would also be in favor of keeping only %-formatting and f-strings in Coding style docs. I don't see any reason to use also `format()` in a new code.
> >>>
> >>> Best,
> >>> Mariusz
>
> >> --
> >> You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
> >> To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.

> >> To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/ceba9695-9d21-436f-84b5-53381aa3ea15n%40googlegroups.com <https://groups.google.com/d/msgid/django-developers/ceba9695-9d21-436f-84b5-53381aa3ea15n%40googlegroups.com?utm_medium=email&utm_source=footer>.

>
> > --
> > You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
> > To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.

> > To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CADqmmRwfce5zrF0Yr4Fe_NczXsiSvZ7mePyE_uasuK37ZU-C2A%40mail.gmail.com <https://groups.google.com/d/msgid/django-developers/CADqmmRwfce5zrF0Yr4Fe_NczXsiSvZ7mePyE_uasuK37ZU-C2A%40mail.gmail.com?utm_medium=email&utm_source=footer>.

>
> --
> You received this message because you are subscribed to the Google
> Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-develop...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/django-developers/CAHoz%3DMYEuzLnJhSp9CoMQL9f7mZwOVeBCiLTi42WST3znLTpUA%40mail.gmail.com <https://groups.google.com/d/msgid/django-developers/CAHoz%3DMYEuzLnJhSp9CoMQL9f7mZwOVeBCiLTi42WST3znLTpUA%40mail.gmail.com?utm_medium=email&utm_source=footer>.

[Shai Berger]

On Tue, 21 Jul 2020 01:28:31 -0700 (PDT)
Carlton Gibson <carlton...@gmail.com> wrote:
>
> Certainly 99% of cases can be handled as cleanly (or more so, because
> I guess we fell `format()` is a little verbose) with %-formatting or
> an f-string.
> But if we say format() is not allowed, don't we then guarantee we hit
> the one case in X where "Actually, this version with format() is much
> cleaner"?
>

FWIW, it seems to me one case where this happens is when you want to
use one argument multiple times, without naming:

'<{0} id="{1}"> {2} </{0}>'.format(tag, id, content)

Arguably, this specific example would look better with named references,
but, IMO, not with % formatting:

'<%(tag)s id="%(id)s"> %(content)s </%(tag)s>' % dict(
tag=tag, id=id, content=content
)

and as noted, f-strings have their limitations.

[Dave Vernon]

Hi Shai,

If you used  '<{0} id="{1}"> {2} </{0}>'.format(tag, id, content) in a templatetag (for example), it could easily be used iteratively which would increase the need for performance and at that point, you could argue that f'<{tag} id="{id}"> {content} </{tag}>' would be a preferable choice, even though you are repeating the 'tag' argument.

More generally:

I understand the value of *not* doing a bulk change in a PR, but I was wondering if it's OK to do a PR based purely on improved performance for a specific element? (I don't have one in mind, the question is purely 'in principle')

Thanks,

Dave

--
You received this message because you are subscribed to the Google Groups "Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/20200721140822.47376bd8.shai%40platonix.com.

[Paolo Melchiorre]

+1 to start using and suggesting f-string in new PRs

I agree to avoid bulk updated only to use f-string in old code.

I think we can write a priority list of string formatting styles to
help new and old contributors in following the same code style in all
PRs.

I will suggest this order:
1) f-string
2) %-formatting
3) format

On Tue, Jul 21, 2020 at 10:21 AM Adam Johnson <m...@adamj.eu> wrote:
> +1 to allowing f-strings.

> Mariusz' position dropping .format() seems reasonable to me too....

> On Tue, 21 Jul 2020 at 09:00, laym...@gmail.com <laym...@gmail.com> wrote:
>> I personally like to use f-strings wherever it makes sense, so +1.
>> I agree with Mariusz. I think we should only allow %-formatting and f-strings from now on.

>> And yes, bulk updates should not be done. ...

>> On Tuesday, 21 July 2020 at 14:28:54 UTC+7 Mariusz Felisiak wrote:

>>> I will not stand against f-strings, I think we can allow them. My main concerns is readability. ...
>>> I'm strongly against any bulk updates to conform to this style....
>>> I would also be in favor of keeping only %-formatting and f-strings in Coding style docs....

--
Paolo Melchiorre

https://www.paulox.net

[Adam Johnson]

 I was wondering if it's OK to do a PR based purely on improved performance for a specific element? (I don't have one in mind, the question is purely 'in principle')

Benchmarked performance improvements are always welcome. I normally use %timeit in ipython for this.

--
You received this message because you are subscribed to the Google Groups "Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAKFO%2Bx6JmXnDaamf8TTE5S3XGXGWbV0MfPkKboS6ZgViN0xJRA%40mail.gmail.com.

--

Adam

[Shai Berger]

Hi Dave and all,

On Tue, 21 Jul 2020 13:56:53 +0100
Dave Vernon <da...@springbourne-tech.com> wrote:

> More generally:
>
> I understand the value of *not* doing a bulk change in a PR, but I was
> wondering if it's OK to do a PR based purely on improved performance
> for a specific element? (I don't have one in mind, the question is
> purely 'in principle')

We are still in discussions here, but if I understand the forming
concensus correctly, it will be ok to replace older-style formatting
with f-strings if you're making changes in the code in question, and
performance-improving changes are, in principal, welcome.

However, whether the performance improvement afforded by such a change
would, on its own, justify the change -- will need to be judged on a
case-by-case basis, and my guess is that justifiable cases will be few
and far between. I believe that in most cases, string formatting is not
a major performance bottleneck.

My non-shot-calling 2-cents' worth,

Shai.

[Dan Davis]

+1 iff flake8 can validate  f-srings as well as PyCharm does f-strings!

I think flake8 would mean that SublimeText and Atom can highlight errors.

Background - Arguments based on readability are so subjective. If I am using Pycharm, f-strings are very readable, and it will check whether a keyword argument is defined, e.g. what about command-line tools?  My guess is that validating format and %- strings is an easier problem because context is localized, but maybe not as solved.

--
You received this message because you are subscribed to the Google Groups "Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/20200721165300.70bd9631.shai%40platonix.com.

[charettes]

I think this reply is the one I'm in the most in agreement with.

Lets avoid a barrage of PR SPAM to change to one style of string formatting to an other.

Defining clear rules about what constitute accepted and disallowed usage of f-strings before allowing their usage will certainly make reviews less likely to turn into bikeshedding discussions.

From my perspective anything beyond variable and property access (e.g. function call, arithmetic) hurts readability and made easier to parse by a prior local variable assignment. That seems like an easy rule to adhere to.

e.g.

Allowed

f'hello {user}'

f'hello {user.name}'

f'hello {self.user.name}'

Disallowed

f'hello {get_user()}'

f'you are {user.age * 365.25} days old'

Allowed with local variable assignment

user = get_user()

f'hello {user}'

user_days_old = user.age * 365.25

'you are {user_days_old} days old'

Simon

[st...@jigsawtech.co.uk]

This is my go to for string formatting comparison. It's from 2018 but was Python 3.8.2 when the op wrote it  https://stackoverflow.com/a/48465349

Table result attached as an image for reference but shad0w_wa1k3r posted his method on that SO thread