What language are you using? Go?
In Go you can provide a tls.Config with VerifyPeerCertificate set. Other languages will vary or not be readily possible. In Java you'd implement an X509ExtendedTrustManager.
Note that restricting at connection establishment commonly still needs to be combined with restricting on a per-method or per-resource basis, simply because your ACLs can be per-method or per-resource as well.