Clarifying some terminology first
A Reverse Proxy is usually used to provided access to server interface or API in a Secure Zone, from Internet or a less secure network.
Any process inside uses a Forward Proxy, or NAT Gateway, if they need to connect out of the Secure Zone to Internet or other networks.
You would use a Reverse Proxy if clients need to reach a server process in a Secure Zone.
So if you have a WebRTC Peer in a Secure Zone you need to provide either a NAT Gateway or a Forward Proxy to allow it to reach a Peer on another network.
This is the same regardless of whether the route between the 2 Secure Zones needs to traverse Internet or private networks.
If you have 2 Peers in 2 different Secure Zones, then NAT Gateways with access to a STUN server, on the outside either of the Gateways should be sufficient, as long as UDP egress and ingress is available.
The ICE negotiation will establish a connection via the 2 NAT Gateways. That's its reason for being.
If UDP traversal is not available at either end, you will a TURN server to provide TCP to UDP relay. (This also works if both ends can only do TCP, as the TURN server relays TCP to UDP internally)