Issue with macro generated code not working like regular code - tuplify(x)

[Tallak Tveide]

Hi!

I made a small Elixir script that demonstrates a problem I have been having lately (https://gist.github.com/tallakt/d2b883bb8884f1ad9739)

When making the `tuplify` function directly inline in the module, it works well, but when I create the exact same code using a macro, it won't work.

I suspect there is an issue with macro hygiene, but I don't understand why...

    defmodule M do
      defmacro def_tuplify do
        result = quote(unquote: false) do
          def tuplify_from_macro(x) do
            {unquote(Macro.var :x, nil)}
          end
        end
        IO.puts "Returned from macro:\n#{result |> Macro.to_string}"
        result
      end
    end


    defmodule A do
      require M

      def tuplify(x) do
        { unquote(Macro.var :x, nil) }
      end

      M.def_tuplify
    end


    IO.puts "testing results"
    IO.inspect A.tuplify(:ok)
    IO.inspect A.tuplify_from_macro(:ok)

And the output looks something like this:

    $ elixir tuplify.exs
    Returned from macro:
    def(tuplify_from_macro(x)) do
      {unquote(Macro.var(:x, nil))}
    end
    ** (CompileError) tuplify.exs:21: function x/0 undefined
        (stdlib) lists.erl:1336: :lists.foreach/2
        tuplify.exs:14: (file)
        (elixir) lib/code.ex:307: Code.require_file/2

[José Valim]

It is not clear what you are trying to achieve. All unquotes are expanded at *compile time*.  When you pass unquote: false, you are saying "I want any unquote in this quote to be expanded later (and later is still at compile time)". This generates the code:

    def(tuplify_from_macro(x)) do
      {unquote(Macro.var(:x, nil))}
    end

And because this code has an unquote in it, it should be again, expanded at compile time. "Macro.var(:x, nil)" will return the variable x, which will fail when unquoted at compile time because there is no x variable. The x variable will exist only at runtime, when the function is called.

I think you want this:

        quote do
          def tuplify_from_macro(x) do
            {x}
          end
        end

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Director of R&D

--
You received this message because you are subscribed to the Google Groups "elixir-lang-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-ta...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-talk/1eb41980-0e6d-4b6c-85df-a4933d4607d3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Tallak Tveide]

Hi. I simplified the example as much as possible. My intention was to
replace the ':x' dynamically by the macro. So as an example, the macro
could generate:

def test({a, b, c}, 10), do: {b, a, c}
def test({a, b, c}, 20), do: {-a, -c, b}

and so on... That is why i am using Macro.var(...) instead of just
typing the variable name directly.

> You received this message because you are subscribed to a topic in the
> Google Groups "elixir-lang-talk" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/elixir-lang-talk/iHEfwYShO5E/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to

> elixir-lang-ta...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/elixir-lang-talk/CAGnRm4KXPYoJyuz-VwUmUp31d%2BHkSeTgGoC%2BNHCrQ91%2B9ieXfQ%40mail.gmail.com.

[eksperimental]

Hi Tallak, sorry, it is still not clear to me.
in your test/2 example, you have arity 2 and what i can see in your
macro, the generated function has arity 1,
I think if you can put exactly the how you call the macro, and what you
exprect to generate, it will be great.

[Tallak Tveide]

Hi. The stuff I'm actually working on is a bit more involved, so that's why I'm trying to make the code as concise as possible. Sorry if it still doesn't make much sense. I made a second example code that is more similar to my original problem code (but unrelated otherwise): https://gist.github.com/tallakt/728799573609beda7286

Let me try to explain the rationale behind what I'm doing: I am doing about doing 3D rotations on vectors, but optimizing for angles in multiples of 90 degrees, since these are by far most used. For these angles, I can reorder the x-y-z coordinates and swap signs. Other values are handled by a general purpose code. I am not posting that code because it is rather involved.

:)

    defmodule Tuplify.Helpers do
      defmacro def_shift_some(index) do
        result = quote(unquote: false, bind_quoted: [i: index]) do
          resulting_vars =
            [:a, :b, :c, :d]
            |> Stream.cycle
            |> Stream.drop(rem(i, 4))
            |> Enum.take(4)
            |> Enum.map(fn var_name -> Macro.var(var_name, nil) end)

          def shift_some({a, b, c, d}, unquote(i)) do
            {unquote_splicing(resulting_vars)}
          end
        end
        IO.puts "Macro generated code:\n#{result |> Macro.to_string}"
        result
      end
    end


    defmodule Tuplify do
      require Tuplify.Helpers

      # This line has been commented as it results in the error:
      #
      #   ** (CompileError) tuplify.exs:31: function a/0 undefined
      #       (stdlib) lists.erl:1336: :lists.foreach/2
      #       tuplify.exs:21: (file)
      #       (elixir) lib/code.ex:307: Code.require_file/2
      #
      # for x <- 1..20, do: Tuplify.Helpers.def_shift_some(x)

      for x <- 100..110 do
        # Code copied from macro generated code
        # When executed inline in the module definition, it works
        (
          i = x
          (
            resulting_vars = [:a, :b, :c, :d] |> Stream.cycle() |> Stream.drop(rem(i, 4)) |> Enum.take(4) |> Enum.map(fn var_name -> Macro.var(var_name, nil) end)
            def(shift_some({a, b, c, d}, unquote(i))) do
              {unquote_splicing(resulting_vars)}
            end
          )
        )
      end

      # non specialized fallback definition
      def shift_some(tuple, _), do: tuple

    end


    IO.puts "testing results"
    IO.inspect Tuplify.shift_some({1, 2, 3, 4}, 0)   # not specialized
    IO.inspect Tuplify.shift_some({1, 2, 3, 4}, 101) # specialized inline module
    IO.inspect Tuplify.shift_some({1, 2, 3, 4}, 12)  # specialized by macro

[Saša Jurić]

I think your problem is in the hygiene. When you do 

def shift_some({a, b, c, d}, ...)

These vars have different scoping from the ones you generate in resulting_vars. Here's how I fixed it:

defmodule Tuplify.Helpers do

  defmacro def_shift_some(index) do

    result = quote(unquote: false, bind_quoted: [i: index]) do

      vars = Enum.map([:a, :b, :c, :d], &Macro.var(&1, __MODULE__))

      resulting_vars =

        vars

        |> Stream.cycle

        |> Stream.drop(rem(i, 4))

        |> Enum.take(4)

      def shift_some({unquote_splicing(vars)}, unquote(i)) do

        {unquote_splicing(resulting_vars)}

      end

    end

    IO.puts "Macro generated code:\n#{result |> Macro.to_string}"

    result

  end

end

[Tallak Tveide]

Thanks, I would never have thought of that. It worked right away.

I am still a bit puzzled, what is the function of `Macro.var(:x, nil)`?

From the docs:

In order to build a variable, a context is expected. Most of the times, in order to preserve hygiene, the context must be __MODULE__:

    iex> Macro.var(:foo, __MODULE__)
    {:foo, [], __MODULE__}

However, if there is a need to access the user variable, nil can be given:

    
   iex> Macro.var(:foo, nil)
    {:foo, [], nil}

[Saša Jurić]

I think the purpose of Macro.var(:x, nil) is to allow a variable bound in the macro (quoted block) to be accessible outside of the macro. For example, if your macro does something like

defmacro my_macro do

  quote do

    unquote(Macro.var(:foo, nil)) = :bar

  end

end

Then the client code can access var foo:

my_macro

IO.inspect foo