Model zoo

27 views
Skip to first unread message

Ian Goodfellow

unread,
Sep 28, 2018, 2:07:45 PM9/28/18
to cleverhans dev
We've recently started toward offering defenses in CleverHans. Previously, we only offered attacks, and you could build adversarial training out of them, but we didn't offer any models, etc.

Now that we're providing reference implementations of defenses, I propose starting a `cleverhans.model_zoo` module. This would in turn have different submodules for different papers, etc. The model zoo would be flagged as explicitly not being as conservative as the rest of the library: we consider attack performance (in terms of accuracy, etc., not runtime) to be part of the API, so we increment the major version number whenever attack performance improves, but we will probably make frequent performance tweaks to models.

What do people think about this? OK to proceed?

Nicolas Papernot

unread,
Sep 28, 2018, 2:22:23 PM9/28/18
to Ian Goodfellow, cleverhans dev
+1 I agree that this would make things easier to maintain than the current ad hoc ways we use to include models in the repo

--
You received this message because you are subscribed to the Google Groups "cleverhans dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cleverhans-de...@googlegroups.com.
To post to this group, send email to cleverh...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cleverhans-dev/CAG5SLK_F8kYtVWr989ObkSaHWbQkmapcn7L68a6NeYyO7ewMjg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Ian Goodfellow

unread,
Sep 28, 2018, 2:38:21 PM9/28/18
to Nicolas Papernot, cleverhans dev

On Fri, Sep 28, 2018 at 11:21 AM, Nicolas Papernot <nic...@papernot.fr> wrote:
+1 I agree that this would make things easier to maintain than the current ad hoc ways we use to include models in the repo

On Fri, Sep 28, 2018 at 11:07 AM 'Ian Goodfellow' via cleverhans dev <cleverhans-dev@googlegroups.com> wrote:
We've recently started toward offering defenses in CleverHans. Previously, we only offered attacks, and you could build adversarial training out of them, but we didn't offer any models, etc.

Now that we're providing reference implementations of defenses, I propose starting a `cleverhans.model_zoo` module. This would in turn have different submodules for different papers, etc. The model zoo would be flagged as explicitly not being as conservative as the rest of the library: we consider attack performance (in terms of accuracy, etc., not runtime) to be part of the API, so we increment the major version number whenever attack performance improves, but we will probably make frequent performance tweaks to models.

What do people think about this? OK to proceed?

--
You received this message because you are subscribed to the Google Groups "cleverhans dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cleverhans-dev+unsubscribe@googlegroups.com.
To post to this group, send email to cleverhans-dev@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages