Matthew,
The easiest way to uninstall an extension is to revert the commit that you made when installing it. Typically, when installing a gem, after I add to the gem to the gemfile, bundle, and run the install generator, I will do a commit to git. If I ever need to uninstall it, I know exactly what has been changed and can revert the changes with a single command.
If you don't use source control (and you should), what Weston said is the best way.
Regards,
Nate