Second Design Iteration

Design elements

As illustrated above, the second design iteration modifies the initial design iteration with the following additions - Inter-data-center connectivity is provisioned for zones 2 and 3 - Supplementing the existing IDC for zone 1 - Optimizing intra-zone/inter-site flows relative to the previous design iteration - BGP policy configuration to circumvent asymmetric routing due to equal-cost paths

Logical Topology

This design iteration’s logical topology is depicted in the following diagram. The “layers” button on the toolbar above the diagram can be used to expose an “interactive controls” layer to enable/disable example flow illustrations.

diagram

BGP design

The logical topology used in this design results in each zone/site VRF learning two equal-cost (based on AS path-length) routes to networks originated from the other two security zone’s VRFs in the other site. That dynamic is demonstrate3d in the following figure, in which route propagation from the site2/zone2 VRF to the site1/zone3 VRF is illustrated.

diagram

Desired Behavior

Multiple BGP-based strategies are available to eliminate equal-cost paths and resulting asymmetric flows, but before one can be designed, the desired forwarding behavior has to be established. In this case, the question to answer was:

Which firewalls should be traversed by flows with endpoints that reside in separate security-zones and different sites?

Considered options include:

• “Always choose the site-1 (or site-2) FW path”
  • Would result in all inspection for these flows being handled on one site’s firewalls
    • Frustrating the design objective of equal workload distribution across the two sites
    • And creating significant risk of configuration drift between the firewalls at the two sites.
      • The non-preferred site’s firewall only handles these flows during a failure of the primary site’s firewalls.
• “Choose a preferred firewall for each destination zone or each destination site”
  • An attempt to provide some crude level of workload distribution across the firewalls
  • But not really a coherent proposal, because
    • “Destination” isn’t a property of a reflexive flow (A->B and B->A); it’s a property of individual packets in a flow
    • “to-there” and “back-to-here” packets in a given flow are going to have inverted destination zones
  • “Both” was the eventual compromise
    • A single zone’s DCI link was to be preferred for all inter-DC traffic.
    • Ensuring that any inter-zone/intersite flows traversed both firewalls

Implementation details

• Each VRF was configured to assign a higher-than-default local preference to any routes:
  • That matched a list of “the other site’s networks”, AND
  • That included the local site’s zone-1 VRF in the AS-path.
• Inbound route-maps were used with:
  • Match clauses evaluated against IP prefix-lists (of the prefixes expected to be originated at the site)
  • Local-preference set to a value greater than the default value

Gaps / Limitations

This design iteration was sub-optimal in the following regards:

• Security-zone “1” still functions as a de-facto “transit zone” for all inter-site/inter-zone traffic
  • We don’t want traffic between zones 2 and 3 routed across zone 1.
  • We don’t want zone 1’s firewall rule configuration to include traffic that is not to or from zone 1
  • Route-policy configuration overhead increases quickly as the number of sites and zones is increased
• Scalability limitations
  • Configuration complexity
    • Additional contemplated sites and security zones will multiply the IP prefix-list cofniguration complexity and the number of routers they must be replicated across
  • The number of discrete/L3-isolated inter-DC/WAN links increases with each additional site.