We can remove all symbols in a particular context by using
Remove["context`*"]. But is it possible to remove "context`" itself
from the system so that it will no longer be listed in Contexts[]?
Yes. The simplest way to do this would be to load the package
CleanSlate before defining the context you want to remove. That
is doing:
<<Utilities`CleanSlate`
followed by
CleanSlate[]
will remove everything defined after CleanSlate was loaded. Or
if you just want to remove one context
CleanSlate["context`"]
will do that.
The file CleanSlate.m can be found at
ToFileName[{$InstallationDirectory,"AddOns","ExtraPackages","Utilities"}]
That file is well commented to describe how CleanSlate works. If
the functions provided aren't what you need, it should form a
good basis for creating your own solution.