Inspect Redaction / Override

[Jayson Vantuyl]

Hey everybody,

Use Case

I have what I think is actually a pretty common problem.  All over my code base, there are uses of inspect/2.  This is a wonderful thing that helps with debugging.  It is less wonderful when it spews PII all over your logs / error pages and you find yourself having sent somebody's social security number to DataDog or disclosed their home address on a 500 page.  Then your lawyer starts doing that thing where their eye twitches, you need to notify four different regulators on three continents, alert all of your customers with scary messages, are made to attend 3-hours of mandatory re-education wherein you learn to recite the GDPR from memory, and eventually sacrifice three interns to appease the compliance gods.

Temporary Hack

What myself and some co-conspirators have done to address this is overriding the Inspect protocol for the built-in types to redact things by default and then have a whitelist for the bits we want to show.  Given that our top-level logging metadata is a map, we can pretty much just whitelist keys there and call it a day.  This works fabulously, but it violates Erlang's expectations significantly.  While Erlang is probably used to that, it gets quite cross and refuses to generate a release because it doesn't have a good way to know which .beam file to put in the release.

As you might imagine, I'm pretty bummed that I can't use releases and have to ignore tons of very alarming sounding warnings about redefining modules.

Options

Could we consider some solutions to make redaction require less unforgivable sins against code loading?  To start, three directions have been proposed by the various people I've talked to:

1. Instead of implementing Inspect for the built-in types, do that inspection in a handler for Any; thereby allowing overriding of the built-in types easily.
2. Wedge something into a common entry point (maybe in Inspect.Algebra?) allows us to specify a global redaction function.  Perhaps configure this with a global config value?
3.  Implement some sort of overriding layer for just the Inspect protocol.
   

In terms of pros and cons, for #1...

• Pro: Works well for built-ins.
• Pro: Implementing this is very straightforward.
• Pro: This probably doesn't break any existing code, very small blast radius.
  
• Con: Doesn't work at all as soon as anything defines its own inspection protocol.
• Con: Isn't amenable to configuration at runtime (maybe this is not an issue?).
  

As for #2...

• Pro: Can be configured at runtime.
• Pro: I have no idea how to implement this and Inspect.Algebra scares me.
  
• Pro: This probably doesn't break any existing code, very small blast radius.
• Con: Given that the Inspect protocol is pretty much "turn X into string", I'm not sure how much redaction we could really do if we allow the existing protocol to run.

As for #3...

• Pro: This provides a clear way to just replace the protocol for a given type.
• Pro: Implementing this is very straightforward.
• Pro: This probably doesn't break any existing code, very small blast radius.
• Con: It's all fun and games until a library does it, then you need Override2 or 3 or 4...
• Con: Probably gets redundant if there ever is a blessed way to override protocols.
• Con: I already pitched this to a few Elixir celebrities and they thought it was a bit too hacky.

In Closing

So, yeah, in the long term, maybe we'll have a blessed way to override protocols; but, short of that, there's got to be some stopgap, right?  What do people think of adopting something like one of these approaches so that my PII problems evaporate and I can finally build some sweet, sweet releases?  I'll even implement it myself!  I promise!

Thanks,

- J

[Allen Wyma]

I believe that the new changeset "redact" attribute should help with this?

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/d5d9a932-930c-46f7-a3fb-2f1a9ae06112n%40googlegroups.com.

[Jayson Vantuyl]

That would help with Ecto.  Unfortunately we still see a bunch of stuff from the likes of Elixir GRPC and various structs from libraries that don't have that kind of functionality.  We were kind of hoping for a top-down, whitelist approach.  We figure that it's easier to plug the leak from the top (inspect/2 itself) than it is to try to deal with all of the leaks at the bottom (everything that tries to use it).

[Allen Wyma]

Looks like they just use the derive functionality from inspect: "@derive {Inspect, except: @ecto_redact_fields}" so you can also look into using that. If it's for struct outside of your control, that may be more difficult

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/38307a23-2568-4c48-ae8c-76cb70e7541bn%40googlegroups.com.

[José Valim]

After a brief discussion with the Elixir team, we are inclined to go more with option 2. However, in order to make it efficient, we want to use persistent_term, which is only available on 21.3 and Elixir currently supports 21.0+.

We will drop Erlang/OTP 21 support once Erlang/OTP 24 is out but it is unclear if that will be before Elixir v1.12. Alternatively we can require Erlang/OTP 21.3. In any case, I have added an item to track this here: https://github.com/elixir-lang/elixir/issues/8414

Thank you!

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAJuWSyL8xhXbZf0ca_nZ4-dJU_Xej3UJ_8-dQFJp0%2B9XX1znpw%40mail.gmail.com.

[Wiebe-Marten Wijnja]

One possibility to allow this and other features waiting on `persistent_term` already on OTP 21.0+ would be to provide a non-NIF fallback implementation.
After all, `persistent_term is based on userspace-libraries written in Elixir and Erlang that were released before (c.f. the FastGlobal Elixir library on Hex and mochiglobal (part of the mochiweb Erlang library).)

Very nice to see steps being made to support inspect redaction!

~Marten

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4L_CuMDq2JVLz2tYog33eTBLR_YF36B-c_DY5VUaVQNsQ%40mail.gmail.com.

[Jayson Vantuyl]

Marten,

Given they want to use `persistent_term` for performance, I think that the fallback probably wouldn't qualify.

If this was a normal library, we could have a compile-time setting to choose an option and it would just recompile on a change.  Then we could implement it such that it had no runtime impact.  However, since this is the standard library, I believe that such recompilation is off the table.

(Also, sorry I didn't respond sooner.)

-J