Inclusion of framework for adversarial example attacks on deep RL - question about memory usage.

63 views
Skip to first unread message

Vahid Behzadan

unread,
Jan 16, 2018, 4:35:38 AM1/16/18
to cleverhans dev
Hi,

I have been working on an interface between Cleverhans and OpenAI Baselines for experimenting with adversarial example attacks against deep RL. The current version (available at https://github.com/behzadanksu/rl-attack ), supports FGSM, Iterative, and CW12 attacks on DQNs. 

First, I would like to ask whether the dev team would be interested in including this code to the main repository as an example of attacks on deep RL? This is a running project, and it would be great if I could continue its development in coordination and collaboration with other CleverHans developers.

Second, my current implementation seem to run into memory issues in extremely extended experiments (e.g., >55M iterations for Breakthrough ). Do the FGSM module and its dependencies automatically clean up after generating the output, or do I have to do it manually? I welcome any advice on how to approach this problem.

Many Thanks,

Vahid


Nicolas Papernot

unread,
Jan 16, 2018, 11:19:57 AM1/16/18
to Vahid Behzadan, cleverhans dev
Hi Vahid,

This sounds like something that would be a good fit for the examples/ directory in the CleverHans repository.

Looking forward to your PR.

Thanks,
-Nicolas

--
You received this message because you are subscribed to the Google Groups "cleverhans dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cleverhans-de...@googlegroups.com.
To post to this group, send email to cleverh...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cleverhans-dev/d7300526-51a0-4ff8-9d5c-6d5e433bd44c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Vahid Behzadan

unread,
Jan 23, 2018, 6:19:23 PM1/23/18
to cleverhans dev
Thanks Nicolas, per our discussions in the PR thread, I am preparing an example-based version to avoid using submodules.

Also, do you have any thoughts regarding my other question? Does the current implementation of FGSM clean up the memory after an example is generated? If not, would a customized clean up disturb its function at all?

Cheers,

Vahid 


On Tuesday, January 16, 2018 at 10:19:57 AM UTC-6, Nicolas Papernot wrote:
Hi Vahid,

This sounds like something that would be a good fit for the examples/ directory in the CleverHans repository.

Looking forward to your PR.

Thanks,
-Nicolas

On Tue, Jan 16, 2018 at 4:35 AM Vahid Behzadan <behz...@gmail.com> wrote:
Hi,

I have been working on an interface between Cleverhans and OpenAI Baselines for experimenting with adversarial example attacks against deep RL. The current version (available at https://github.com/behzadanksu/rl-attack ), supports FGSM, Iterative, and CW12 attacks on DQNs. 

First, I would like to ask whether the dev team would be interested in including this code to the main repository as an example of attacks on deep RL? This is a running project, and it would be great if I could continue its development in coordination and collaboration with other CleverHans developers.

Second, my current implementation seem to run into memory issues in extremely extended experiments (e.g., >55M iterations for Breakthrough ). Do the FGSM module and its dependencies automatically clean up after generating the output, or do I have to do it manually? I welcome any advice on how to approach this problem.

Many Thanks,

Vahid


--
You received this message because you are subscribed to the Google Groups "cleverhans dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cleverhans-dev+unsubscribe@googlegroups.com.
To post to this group, send email to cleverhans-dev@googlegroups.com.

Ian Goodfellow

unread,
Jan 23, 2018, 6:27:53 PM1/23/18
to Vahid Behzadan, cleverhans dev
CleverHans doesn't do anything special in terms of memory management, but TensorFlow should garbage collect every buffer once it's done being used. 

On Tue, Jan 23, 2018 at 3:19 PM, Vahid Behzadan <behz...@gmail.com> wrote:
Thanks Nicolas, per our discussions in the PR thread, I am preparing an example-based version to avoid using submodules.

Also, do you have any thoughts regarding my other question? Does the current implementation of FGSM clean up the memory after an example is generated? If not, would a customized clean up disturb its function at all?

Cheers,

Vahid 


On Tuesday, January 16, 2018 at 10:19:57 AM UTC-6, Nicolas Papernot wrote:
Hi Vahid,

This sounds like something that would be a good fit for the examples/ directory in the CleverHans repository.

Looking forward to your PR.

Thanks,
-Nicolas

On Tue, Jan 16, 2018 at 4:35 AM Vahid Behzadan <behz...@gmail.com> wrote:
Hi,

I have been working on an interface between Cleverhans and OpenAI Baselines for experimenting with adversarial example attacks against deep RL. The current version (available at https://github.com/behzadanksu/rl-attack ), supports FGSM, Iterative, and CW12 attacks on DQNs. 

First, I would like to ask whether the dev team would be interested in including this code to the main repository as an example of attacks on deep RL? This is a running project, and it would be great if I could continue its development in coordination and collaboration with other CleverHans developers.

Second, my current implementation seem to run into memory issues in extremely extended experiments (e.g., >55M iterations for Breakthrough ). Do the FGSM module and its dependencies automatically clean up after generating the output, or do I have to do it manually? I welcome any advice on how to approach this problem.

Many Thanks,

Vahid


--
You received this message because you are subscribed to the Google Groups "cleverhans dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cleverhans-dev+unsubscribe@googlegroups.com.
To post to this group, send email to cleverh...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "cleverhans dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cleverhans-dev+unsubscribe@googlegroups.com.
To post to this group, send email to cleverhans-dev@googlegroups.com.

Vahid Behzadan

unread,
Jan 23, 2018, 8:04:11 PM1/23/18
to cleverhans dev
Thanks Ian. My problem arises at extreme ends of DQN training with many adversarial perturbations -i.e., when more than half of the observations are to be perturbed, TF runs out of memory at around 55M iterations. 

So far, my analysis has led me to think that memory problems happen due to numerous calls to FastGradientMethod() in the same session. My theory is that when one instance of this method is created, it is retained in memory unless it's explicitly deleted. Does this sound reasonable to you? If so, do you have any suggestions on how to go about removing previous FGSM instances without resorting to multi-session implementations?
  

On Tuesday, January 23, 2018 at 5:27:53 PM UTC-6, Ian Goodfellow wrote:
CleverHans doesn't do anything special in terms of memory management, but TensorFlow should garbage collect every buffer once it's done being used. 
On Tue, Jan 23, 2018 at 3:19 PM, Vahid Behzadan <behz...@gmail.com> wrote:
Thanks Nicolas, per our discussions in the PR thread, I am preparing an example-based version to avoid using submodules.

Also, do you have any thoughts regarding my other question? Does the current implementation of FGSM clean up the memory after an example is generated? If not, would a customized clean up disturb its function at all?

Cheers,

Vahid 


On Tuesday, January 16, 2018 at 10:19:57 AM UTC-6, Nicolas Papernot wrote:
Hi Vahid,

This sounds like something that would be a good fit for the examples/ directory in the CleverHans repository.

Looking forward to your PR.

Thanks,
-Nicolas

On Tue, Jan 16, 2018 at 4:35 AM Vahid Behzadan <behz...@gmail.com> wrote:
Hi,

I have been working on an interface between Cleverhans and OpenAI Baselines for experimenting with adversarial example attacks against deep RL. The current version (available at https://github.com/behzadanksu/rl-attack ), supports FGSM, Iterative, and CW12 attacks on DQNs. 

First, I would like to ask whether the dev team would be interested in including this code to the main repository as an example of attacks on deep RL? This is a running project, and it would be great if I could continue its development in coordination and collaboration with other CleverHans developers.

Second, my current implementation seem to run into memory issues in extremely extended experiments (e.g., >55M iterations for Breakthrough ). Do the FGSM module and its dependencies automatically clean up after generating the output, or do I have to do it manually? I welcome any advice on how to approach this problem.

Many Thanks,

Vahid


--
You received this message because you are subscribed to the Google Groups "cleverhans dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cleverhans-dev+unsubscribe@googlegroups.com.

Ian Goodfellow

unread,
Jan 23, 2018, 9:38:16 PM1/23/18
to Vahid Behzadan, cleverhans dev
It sounds like you're adding nodes to the default TensorFlow Graph object on every iteration of DQN and you run out of memory when the graph itself is too big to store. You shouldn't add new tf nodes every time you do an iteration of training. You should add the nodes describing one step of training once and then run those nodes many times. This doesn't really have anything to do with CleverHans though; it would happen for any code that repeatedly constructs new TensorFlow nodes in a long-running loop.

Vahid 
To post to this group, send email to cleverh...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "cleverhans dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cleverhans-dev+unsubscribe@googlegroups.com.
To post to this group, send email to cleverh...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "cleverhans dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cleverhans-dev+unsubscribe@googlegroups.com.
To post to this group, send email to cleverhans-dev@googlegroups.com.

Vahid Behzadan

unread,
Jan 23, 2018, 10:00:20 PM1/23/18
to cleverhans dev
Makes perfect sense Ian, thanks. I'll check to see if this actually is the case and will take it from there.

PS, do let me know if the team needs an extra hand for further work on Cleverhans. I'd be more than happy to contribute to this fantastic project.

Cheers,

Vahid 
Vahid 
To post to this group, send email to cleverhans-dev@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "cleverhans dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cleverhans-dev+unsubscribe@googlegroups.com.
To post to this group, send email to cleverhans-dev@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages