A full-mesh SD-WAN design looks elegant when the network is small.
Every branch can reach every other branch directly. Voice and video avoid unnecessary detours. The topology is easy to explain, and the proof of concept usually performs well.
Then the network grows.
At a few hundred sites, the design begins carrying a hidden tax. At a few thousand, that tax can become an operational problem. The issue is not simply bandwidth or appliance size. It is the number of secure tunnels, routing relationships, liveness checks and rekey operations the control plane must continuously maintain.
The lesson for infrastructure leaders is simple: a topology that scales linearly in business terms can scale quadratically in engineering terms.
The Tunnel Math Turns Against You
In a full mesh, every site needs a direct tunnel to every other site.
The number of tunnel pairs grows according to:
n(n-1)/2
At 100 sites, that means 4,950 tunnel pairs.
At 1,000 sites, it becomes 499,500.
At 4,000 sites, the network is approaching 8 million.
Each tunnel carries more than packet forwarding. It requires security associations, liveness monitoring, periodic key rotation and state inside the operating system and VPN process.
That state has to be maintained even when traffic between two branches is almost nonexistent.
The architecture therefore spends increasing amounts of control-plane capacity preserving connectivity that many sites may never use.
The Failure Usually Looks Like Instability, Not Capacity Exhaustion
The most dangerous part of this problem is that it rarely fails cleanly.
An appliance does not simply announce, “Maximum tunnel count reached.”
Instead, operators see strange symptoms.
CPU usage spikes after a WAN flap. Tunnel renegotiations pile up. Healthy peers begin timing out. Routing sessions churn. A device recovers from one failure only to trigger a wave of new failures.
Consider a branch with several thousand peers.
If its WAN connection drops and returns, thousands of VPN sessions may attempt to renegotiate almost simultaneously. Each secure session requires cryptographic work and state management.
The result can resemble a denial-of-service event generated by the network itself.
Central gateways face an even larger version of the same problem. After an outage, thousands of remote sites can reconnect at once, creating a recovery storm precisely when the infrastructure is already under stress.
This is why resilience testing should include recovery behaviour, not just steady-state throughput.
Buying Bigger Hardware Only Delays the Problem
The first reaction is often straightforward: increase appliance capacity.
That helps, but it does not change the growth curve.
If tunnel count grows quadratically, doubling the number of sites creates roughly four times the tunnel relationships. A larger appliance may push the ceiling further out, but the same architectural problem returns at the next growth stage.
Another common response is to abandon the mesh and move everything to hub-and-spoke.
That solves the tunnel explosion, but it introduces a different cost.
Branch-to-branch traffic may now travel through a central hub, adding latency to voice, collaboration or other interactive applications. A design introduced to improve network efficiency ends up recreating the long paths SD-WAN was supposed to eliminate.
The better answer is not simply more hardware or fewer direct paths.
It is selective connectivity.
Measure the Edge Before Redesigning the Topology
Before changing architecture, teams need to understand where control-plane pressure is actually accumulating.
Three measurements matter immediately:
Installed IPsec state. How many active security associations is each branch maintaining, and how close is that number to the platform’s tested limit?
Rekey activity. How many VPN sessions are being renegotiated during normal operation and after connectivity events?
Routing churn. How frequently is the routing table changing, and how much processing does that trigger across the fleet?
The important word is tested.
Datasheets rarely tell teams how a branch appliance behaves when thousands of security associations renegotiate while routing updates are arriving and production traffic remains active.
That threshold needs to be established through representative load testing.
Once teams know where packet loss, CPU saturation or control-plane instability begins, they can identify branches approaching those limits before customers notice the symptoms.
A practical migration rule is to prioritise sites that are already operating close to their proven security-association or renegotiation ceilings.
Replace One Giant Mesh With Smaller Connectivity Groups
The more sustainable design is to partition the overlay.
Instead of allowing every site to learn routes to every other site, divide branches into logical groups. Sites inside a group can retain direct connectivity, while traffic between groups passes through a smaller number of regional or shared gateways.
The difference in tunnel count can be dramatic.
For example, a large network divided into groups of a few hundred sites can reduce the number of tunnels by roughly an order of magnitude compared with a single global mesh, depending on the design.
The key is that the reduction comes from topology, not hardware.
The network stops creating tunnels merely because another branch exists somewhere in the organisation.
It creates them because there is an operational reason for the two locations to communicate directly.
Group by Traffic, Not by Organisational Charts
Geography is a convenient starting point for grouping.
Branches in the same region often communicate more frequently and can benefit from direct paths, while cross-region traffic is comparatively sparse.
But geography should not become a rigid rule.
Flow data should determine the design.
If two sites in different regions exchange latency-sensitive traffic constantly, they may deserve direct connectivity or membership in an additional shared group.
If two neighbouring branches never communicate, a direct tunnel between them may provide little value.
The principle is simple:
Build direct connectivity around traffic affinity, not around the theoretical possibility that any site might someday talk to any other site.
This turns overlay design into a demand-driven architecture.
Route Control Becomes Tunnel Control
The most useful architectural insight is that tunnel count does not always need to be managed directly.
It can be managed through routing.
A branch generally builds tunnels towards the destinations and next hops it learns about. If routing policy prevents the branch from learning every remote site in the network, the VPN layer no longer needs to maintain thousands of unnecessary peer relationships.
Route-target-based segmentation is one established way to create this behaviour.
A branch can import routes belonging to its local group plus routes towards shared gateways. Remote groups remain invisible unless policy explicitly permits communication.
This is much easier to operate than maintaining a giant matrix of per-site tunnel filters.
Group membership becomes a routing-policy decision rather than thousands of individual VPN configuration changes.
That distinction matters at scale.
Accept the Trade-Offs Instead of Pretending They Do Not Exist
Grouping is not free.
The first trade-off is latency.
Traffic crossing group boundaries may need to traverse a gateway, adding another network hop. For ordinary business traffic, that penalty may be insignificant. For latency-sensitive applications, it needs to be measured.
The second trade-off is controller complexity.
Routing systems may need to maintain different policies for different groups, which increases processing and operational responsibility at the control layer.
The third trade-off is blast radius.
A bad policy applied to an entire group can disconnect hundreds of sites at once.
These are manageable risks, but only if they are treated as design inputs.
Redundant control systems, staged policy deployment, canary groups and continuous convergence testing should be part of the operating model.
The goal is not to eliminate every trade-off.
It is to replace uncontrolled quadratic growth with trade-offs the engineering team can actually manage.
Migrations Need Make-Before-Break Discipline
Moving live sites between groups is where good architecture can still fail operationally.
The dangerous approach is simple: remove the site’s old routing policy, then add the new one.
That creates a window where the branch may have neither set of routes.
A safer migration follows a make-before-break sequence.
First, allow the site to learn both the old and new group routes.
Second, confirm that routes from the new group are actually installed.
Third, advertise the site into the new group.
Only after the new path is working should the old membership be removed.
This sequence sounds basic, but it prevents a surprisingly common class of migration outage.
Automation should enforce the sequence and abort if expected routes do not appear.
A Practical Four-Step Action Plan
1. Measure control-plane headroom. Establish tested limits for security associations, renegotiation rates and routing churn on representative branch hardware.
2. Map actual traffic relationships. Identify which branches genuinely need direct east-west paths and which relationships exist only because the topology makes them possible.
3. Build connectivity groups. Use regional or traffic-based grouping to constrain route visibility and tunnel fan-out while retaining direct paths where they create real value.
4. Migrate gradually. Start with low-risk sites, use make-before-break changes and monitor rekey rates, routing convergence, latency and customer impact before expanding the design.
Do not wait for the first large rekey storm to discover that the topology has already exceeded the architecture.
The Real Scaling Lesson Is About Control
SD-WAN scale is often discussed as a hardware problem.
It is more accurately a control-plane problem.
The network becomes unstable when every branch is expected to maintain relationships with every other branch, regardless of whether those relationships create business value.
Smarter topology changes that equation.
Smaller failure domains reduce recovery storms. Smaller routing tables simplify operations. Fewer tunnels reduce cryptographic and kernel state. Traffic that benefits from direct connectivity can keep it, while sparse traffic can use shared paths.
The best scaling strategy is not to make every device powerful enough to support an unlimited mesh.
It is to stop asking the network to build connections it never needed in the first place.
