If you are writing a plugin, then presumably you are using a recent version of Netbox. In Netbox v2.10 onwards, custom fields are not held in their own models. There is a single JSON field "custom_field_data" on each model, containg an object of {"cfname":"cfvalue"}
I am confused by what you're saying. On the one hand you're saying "CustomerID is a custom field", but then you are showing "CustomerID" as a model. I can only assume you mean that "CustomerID" is a custom field which *refers to* the CustomerID model in some way (e.g. it contains the value of CustomerID.name)
Unfortunately, custom fields don't have a way to enforce referential integrity - not to standard netbox models, and certainly not to plugin models. This field would be just plain text or a number. That is, it's easy to have a CustomerID custom field on both Device and IPAddress, but no way to ensure that the data entered is valid. Having said that, it might be possible using the validation feature in the upcoming Netbox 3.x to write a custom validator which looks up the CustomerID and gives an error if there's no corresponding plugin model. I haven't tested this myself.
In your code you also show a class "IPAddress"; does this mean that your plugin also creates a model "IPAddress", separately from Netbox's "IPAddress"? That seems confusing to me, but I don't really know what you're trying to achieve. If you want to model IP addresses separately from Netbox's IP addresses, then I suggest you at least call the model something else. You can of course have custom fields on IPAddress, just as you can on Device. But you cannot modify the core Netbox models in any way.
There's another way to do this in a plugin, avoiding custom fields altogether:
1. make a model "Customer" (which has a "name" field containing the customerID, and also has a database-assigned ID like other Netbox models)
2. make a model which links Customer to Device (i.e. it has customer_id and device_id)
3. make a model which links Customer to IPAddress
2 and 3 give you referential integrity. These are structurally ManyToMany join models, but by adding a unique constraint on the device_id and ipaddress_id (on the join models), you can ensure that each Device and IPAddress can only be linked to one Customer. That is, you can turn it into a ManyToOne as far as the user is concerned, and for database-enforced integrity.