I don't know if you can delete it directly, but with GAMADV-X you can find the contactID of it and then delete the unique ID from each user.
gam redirect csv ./bad_contacts.csv all users print contacts emailmatchpattern "exact...@address.here"
That will give you a CSV of that contact within all users' address lists.
You can then push that csv back through gam.
gam csv ./bad_contacts.csv gam delete contacts user ~User delete contactid ~ContactID
But, also maybe this could work.
gam all users delete contacts emailmatchpattern "exact...@address.here"
(No, it doesn't work. I just tested it. There's something wrong with my syntax for a direct delete, if it's even possible.)