To flesh out Ben's answer a bit, you'd want a Persistent reference to obj which you could SetWeak. I guess maybe that's obvious but never hurts to state the obvious. The Persistent reference could be inside Point or a base class of Point (and others) if you have a menagerie of object types to manage.
If you're concerned about the non-deterministic nature of the cleanup that Ben pointed out, you can make Point (and any other such objects) extend String::ExternalStringResource. Heap tear down will call Dispose for every ExternalString in the Isolate. In addition, if obj is collected, and the presumed only reference to p (the interal field) is eliminated, the ExternalString will be collected and Dispose called for p. If all you want Dispose to do is delete p, you don't have to specify a Dispose override.
It might have been nice if V8 had a similar mechanism for plain External objects as it's a bit weird having your objects extend ExternalStringResource but on the grand scheme of weird it's not a big deal.
Note that the determistic aspect of String::ExternalStringResource is only really important if your process creates and destroys multiple isolates as a matter of course so you don't want to wait for process termination to clean everything up.
Have fun!