Proposal: String.indent and Macro.to_string Inspect.Opts indentation options

[Christopher Keele]

Proposal:

1. Add support for indenting multi-line strings via String.indent:
   
   defmodule String do

   @type indentation ::

   binary

   | Inspect.Algebra.t()

   | [{:spaces, non_neg_integer()}]

   | [{:tabs, non_neg_integer()}]

   
   

   @spec indent(String.t(), indentation) :: String.t()

   def indent(string, indentation)

   end
2. Add support for indenting ast representations in Macro.to_string:
   
   defmodule Macro do

   @type to_string_opt :: {:indent, String.indentation()}

   @spec to_string(Macro.t(), [to_string_opt()]) :: String.t()

   def to_string(string, options \\ [])

   def to_string(string, options)

   end
   
3. Add support for indenting code in Code.format_string!:
   

   defmodule Code do

   @type format_opt ::

   # ... |

   {:indent, String.indentation()}

   @spec format_string!(String.t(), [format_opt()]) :: iodata()

   def format_string!(string, opts \\[])

   end
4. Add support for indenting arbitrary Algebra documents in Inspect.Opts:
   

   defmodule Inspect.Opts do

   @t new_opt ::

   # ... |

   {:indent, String.indentation()}

   end

Motivation:

I often want to indent code I provide as feedback to the user in exception messages in macros. I have to imagine other library authors, metaprogrammers, and LS developers would benefit as well.

I often want to compose multi-line log messages with certain passages indented. I have to imagine other application developers have wanted the same.

Please share if you can think of other ways you would use this functionality, or other stdlib APIs you can think of wanting indentation support for!

Rationale:

I find myself re-implementing this odd job often, not just for my stated use-cases here. The arguments for putting it in stlib are:

1. The String utility is general-purpose enough for many use-cases, and so is potentially useful to many Elixir users.
2. We can provide a better implementation than the naive String.split |> Kernel.<> |> Enum.join.
3. We can provide an implementation that works with Inspect.Algebra documents and inspect.
4. We can provide an option to Macro.to_string and Code.format_string! where displaying multi-line strings is common.

[Zach Daniel]

This is such a good idea. I do this a lot too and didn't think of it, especially in library code. In strong support.

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/elixir-lang-core/f9b60654-29b2-4c06-80ea-6a6fa614f9d3n%40googlegroups.com.

[Dorgan]

This was common enough that it's an option in Sourceror.to_string/2

To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/elixir-lang-core/f9b60654-29b2-4c06-80ea-6a6fa614f9d3n%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/elixir-lang-core/mk4q396z.7bef578b-059a-4f47-9bbe-f8485fa3ecdc%40we.are.superhuman.com.

[José Valim]

Thanks for the proposal.

I am potentially fine with adding an option to some of the code generation helpers we have, but I am not sure we need the function in standard library. A simple indent implementation could be written as:

def indent(string, indent) do

  indent <> String.replace(string, ~r/\r\n|\n/, & &1 <> indent)

end

Or even without a regex:

def indent(string, indent) do

  indent <> String.replace(string, ["\r\n", "\n"], & &1 <> indent)

end

Especially because I don't see a reason to pass anything but a string.

However, you will find that most indent in Elixir codebase is actually a bit different. We don't indent consecutive or trailing newlines, so we would either need to add more options. If we have an official API, now we need to support these different options, while instead you could just change the one liner to use either ~r/(\r\n|\n)+/ or ~r/(\r\n|\n)+(?!$)/ respectively for the desired results.

[Christopher Keele]

I agree it makes sense to accept a `newlines` regex as an option to `String.indent/2`, and have pushed an implementation supporting that to my branch for posterity!

> [The] Elixir codebase is actually a bit different. We don't indent consecutive or trailing newlines

My understanding is that this is an optimization over the direct approach that text editors normally take: indenting the whole document, then stripping trailing whitespace. Since we can't re.search over an algebra document, though, if we support indents in code generation we'd have to either implement the optimization within the algebra formatter when emitting new lines, implement the unoptimized approach with a second pass, or some other option. Neither bring me joy, which is an argument for not implementing, which is an argument for (IMO) just shipping the String.indent variant instead and letting the user provide a regex in post to perform the optimization themselves.

> I am potentially fine with adding an option to some of the code generation helpers we have, but I am not sure we need the function in standard library.

My thought here was that if we add String.indent, then we don't strictly need the option in the code generation helpers, so I started with that in my PR. Looking at the complexity of not just updating `IO.Algebra.format` to insert indentation mid-format, but also considering the complexity that would come from then needing to decide if IO.Algebra should be concerned with trailing whitespace, I'm definitely a fan of cutting scope.

Per the PR discussion, that leaves 2 decisions: 1) should this return the IO.data instead, and 2) should this proposal be approved or not, based on a) not really wanting to support a fairly minor helper vs b) the complexity of maintaining the gnarlier code-formatter implementation vs c) passing on both.

[Christopher Keele]

For reference, the proposal has been updated to the following docs/signatures, with returning IO vs string co-ercion still on the table. 

@typedoc """

A description of how to indent the start of a line in a string.

Spaces, tabs, or arbitrary binaries can all be used in some `amount`,

or a literal `binary` can be applied as-is to indent.

"""

@type indentation ::

{:spaces, amount :: non_neg_integer}

| {:tabs, amount :: non_neg_integer}

| {:binary, {binary, amount :: non_neg_integer}}

| {:binary, binary}

@type indent_opt :: indentation | {:newlines, Regex.t() | list(binary) | binary}

@doc """

Returns a string with indentation applied at the start of every line.

## Options

* `indentation` - An `t:indentation/0` option specifier to apply:

* `spaces: amount`: an `amount` of spaces

* `tabs: amount`: an `amount` of tabs

* `binary: {string, times}`: some `string` multiple `times`

* `binary: string`: an arbitrary `string`

* `:newlines` - Any valid `pattern` to `split/3`. The default

`~r/\r\n|\r|\n/` correctly handles cross-platform newlines. You might use

`~r/(\r\n|\r|\n)+(?!$)/` to skip indenting empty or trailing lines.

## Examples

iex> string = "every\\n\\nwhich\\nway\\n"

iex> String.indent(string)

" every\\n \\n which\\n way\\n "

iex> string = "every\\n\\nwhich\\nway\\n"

iex> String.indent(string, newlines: ~r/(\r\\n|\r|\\n)+(?!$)/)

" every\\n\\n which\\n way\\n"

iex> string = "every\\n\\nwhich\\nway\\n"

iex> String.indent(string, spaces: 4)

" every\\n \\n which\\n way\\n "

iex> string = "every\\n\\nwhich\\nway\\n"

iex> String.indent(string, tabs: 1)

"\tevery\\n\t\\n\twhich\\n\tway\\n\t"

iex> string = "every\\n\\nwhich\\nway\\n"

iex> String.indent(string, binary: {"~", 2})

"~~every\\n~~\\n~~which\\n~~way\\n~~"

iex> string = "every\\n\\nwhich\\nway\\n"

iex> String.indent(string, binary: "+ ")

"+ every\\n+ \\n+ which\\n+ way\\n+ "

"""

@doc since: "1.20.0"

@spec indent(t, list(indent_opt)) :: t

def indent(string, opts \\ [])

[Christopher Keele]

PR feedback on the implementation has led to awaiting for more discussion, so feel free to steal the code there if it doesn't land, but add your examples, use-cases, and +1's here in the meantime!