Currently the BaseModelAdmin class has a method called formfield_for_dbfield which has a conditional block that in turn calls different formfield_for_* methods depending on the type of database field it is handling. In order to do any custom logic to these fields we need to override these methods and/or pass in some arguments as kwargs to override attributes of the form field.
In my opinion, a more natural way is to let different objects handle the customization, rather than BaseModelAdmin. I'm proposing AdminField classes with a consistent API that others can subclass from and swap with the defaults if they desire.
I've done so preliminary work that you can see in this PR: