I suspect you'll need to come at the problem from the other end; of only *exporting* the nodes you want to be referenced later.
There is no way, to my knowledge, to selectively reference a file. The one exception being a reference with another reference inside of it; so potentially if you could split your custom nodes into a reference of their own, and reference that into your file? Effectively a hierarchy of references inside of that one referenced file. Sounds awefully complex haha, but that would be the closest thing.
Not even a post-import/reference callback (which does exists) would work, because nodes can't be deleted from a referenced file.