The Ubiquiti Diaries: A Site-to-Site VPN Story

The CGNAT Spanner in the Works

Carrier-Grade NAT (CGNAT) is used by ISPs to circumvent the lack of IPv4 addresses available for allocation to their customers. As more and more devices come online and connect to the Internet directly, it is not surprising that there is a dearth of IPv4 addresses to hand out. CGNAT is mostly encountered with WAN connections enabled by mobile service providers (think of smartphones connecting to a 4G or 5G network). However, many wired ISPs (such as Airtel in India) have also started rolling out CGNAT. The easiest way to identify if one is behind a CGNAT is to search for one's IP address on Google. Google helpfully displays your public IP address as the first result. If this address is different from the WAN IP reported by your gateway (and that is usually between 100.64.0.0/16 and 100.127.0.0/16), you are likely behind a CGNAT. This is different from a local IP gathered by a gateway placed behind ISP-supplied equipment that acts as a DHCP server itself.

CGNAT is rarely a problem for regular web browsing and content consumption. Problems start cropping up when dealing with services that require opening up ports on the gateway for bidirectional communication - such as VoIP services, multi-player gaming, and VPN setups. Even simple things taken for granted like systems in the network being able to communicate with a remote NTP server (for time synchronization) are not possible. Since the NAT happens at the ISP end, it is not possible for port forwarding to be configured either. Some services manage to find their way around CGNAT using STUN / TURN, but power users end up finding one roadblock after another. On the whole, CGNAT is a pain in the neck for users accustomed to obtaining public-facing WAN IPs.

NTP Synchronization Attempt behind CGNAT, and the 'Solution' (Using a 'Local' NTP Server)

Ideally speaking, configuring the Site-to-Site Manual IPSec VPN on the USG Pro 4 (having a public WAN IP) with a remote server address of 0.0.0.0, and providing the USG Pro 4's WAN IP as the remote server address on the UDM side (behind the CGNAT) should have worked by allowing the CGNAT side to initiate the site-to-site connection with the server. I did try out this suggestion made by an Ubiquiti employee many years ago - however, the initial key exchange / handshake process appeared to be break down with the response from the USG Pro 4 apparently getting dropped by the UDM's ISP.

An online search for enabling site-to-site VPNs with one site behind a CGNAT brought up two sets of results - one suggesting a connection via an intermediate VPS (cloud server hosted by a 3rd party), and the other trying to make the public WAN IP-possessing side to act as an OpenVPN server and the CGNAT side to act as a client and initiate the connection. I was not interested in bringing a VPS into the picture, and my trials with OpenVPN did not yield any promising results.

At this point, I realized that my Android phone was able to open up a Teleport connection with the UDM despite the gateway going behind CGNAT. So, my first attempt was to get a system to connect to the Teleport VPN and route traffic from specific devices through the system's Teleport VPN connection.