Firstly upgrade to Java8 to get rid of the PermGen :)
Secondly as this is permgen this sound like dynamic classes that are created for groovy may not getting disposed correctly so have some references somewhere - so I would use Java8 and get a heap dump and investigate from there.
Something like the following should help for the java options when you start Jenkins.
-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/disk2/dumps
I've seen memory issues in plugins that are fine in 99% of the installations that use them but that in 1% use them in a way that causes them to bloat memory - so don;t worry if no one else has seen it.
Also - what are your memory options on the JVM - it may be its not leaking at all - but that the headroom is just too low (the defaults from Java are too low for anything other than a simple installation)
/James