Juniper NETWORKS Apstra Drain User Guide

Juniper Apstra Drain Mode Guide
Published
2024-10-16

Contents hide

1 NETWORKS Apstra Drain

2 About This Guide

3 Introduction

4 Activate or Disable Drain Mode

5 IBA Monitoring of Devices in Drain Mode

6 Configuration Examples

7 Documents / Resources

7.1 References

NETWORKS Apstra Drain

Juniper Networks, Inc.
1133 Innovation Way
Sunnyvale, California 94089
USA
408-745-2000
www.juniper.net
Juniper Networks, the Juniper Networks logo, Juniper, and Junos are registered trademarks of Juniper Networks, Inc.
in the United States and other countries. All other trademarks, service marks, registered marks, or registered service marks are the property of their respective owners.
Juniper Networks assumes no responsibility for any inaccuracies in this document. Juniper Networks reserves the right to change, modify, transfer, or otherwise revise this publication without notice.
Juniper Apstra Drain Mode Guide
Copyright © 2024 Juniper Networks, Inc. All rights reserved.
The information in this document is current as of the date on the title page.
YEAR 2000 NOTICE
Juniper Networks hardware and software products are Year 2000 compliant. Junos OS has no known time-related limitations through the year 2038. However, the NTP application is known to have some difficulty in the year 2036.
END USER LICENSE AGREEMENT
The Juniper Networks product that is the subject of this technical documentation consists of (or is intended for use with) Juniper Networks software. Use of such software is subject to the terms and conditions of the End User License Agreement (“EULA”) posted at https://support.juniper.net/support/eula/. By downloading, installing or using such software, you agree to the terms and conditions of that EULA.

About This Guide

This guide provides information about using Drain Mode in Juniper Apstra, with configuration examples.
Drain Mode enables you to gracefully drain traffic from devices without shutting down the BGP neighbor routes.

Introduction

Juniper Apstra supports Drain Mode for managed switches, allowing the operator to gracefully drain traffic from devices without simply shutting down the BGP neighbor relationships. This is implemented through modifications to the BGP process (inbound/outbound route-maps), shutting down connected L2 server ports, and shutting down MLAG peer link ports. By using Drain Mode, operators can minimize the number of dropped/lost traffic during these operations. During maintenance, redundancy is handled by ECMP/MLAG as long as there are suitable redundant systems in place. A visual example of Drain Mode on Spine switches is displayed below:

Activate or Disable Drain Mode

IN THIS SECTION
Activate Drain Mode | 4
Disable Drain Mode | 5
Activate Drain Mode
Activate Drain Mode by switching devices to the Drain state in Juniper Apstra:

Once the device is switched to Drain, the change must be completed with the Commit button. The following image shows an example workflow using the Drain functionality.

Disable Drain Mode
To restore a device to service, switch the Deploy Mode setting back to Deploy, then Commit.

IBA Monitoring of Devices in Drain Mode

IN THIS SECTION
Example | 7
Recommended Usage | 8
A prebuilt IBA (intent-based analytics) probe is available in Juniper Apstra. You can activate it by instantiating a predefined probe named “Drain traffic anomaly”. The required value for Threshold in bps works as follows:

Value is the net sum of traffic on all hosted_interfaces
This does not include traffic on the Ethernet management port which is not part of the probe measurement

These interfaces include all L3 BGP enabled paths
Server facing interfaces are shut during Drain Mode and are not part of this calculation

The threshold describes the amount of traffic you wish to be alerted on (above the value) if devices are in the Drain state
This ensures that you do not perform actual maintenance operations on a device that has not been fully drained.

Example
Spine1 is connected to 4 leaf switches, each connection runs the eBGP routing process. All application (server) based traffic flows are rehashed via ECMP onto other links and the basic BGP neighbor updates are still running. In a lab example with a small topology, this is effectively 1.5KBPS per link. With 4 neighbors, the total traffic we expect to remain on the devices is approximately 6KBPS. If we set the probe Threshold in bps to 10KBPS (10000), the probe generates anomalies if there is more than 10K on all of the 4 interfaces combined.
Recommended Usage
Enable the probe with 100KBPS and leave it running in all Blueprints. When a device enters the Drain state, an anomaly appears as the traffic is removed from the links. This anomaly should only exist for a few seconds. If the anomaly does not clear, the device is not fully in Drain Mode. Once the anomaly clears, you are free to switch the device to the Ready state to take it out of service completely. It is also possible that you will not see the anomaly as it may appear and disappear very quickly.

Configuration Examples

IN THIS SECTION

Drain Spine Devices (L2 and L3 Blueprints) | 10
Drain Leaf Devices (Server-Facing Ports w/ MLAG) | 13

Drain Leaf Devices (L2 Server-Facing Ports no MLAG) | 18
Drain Leaf Devices (L3 Connected Servers) | 23

The following sections provide Drain Mode configuration examples for different OS and device combinations.
Drain Spine Devices (L2 and L3 Blueprints)
IN THIS SECTION

Drain (NX-OS) | 11

Drain (Junos) | 12

The following occurs when draining the Spine:

Outbound routes are removed from the device’s routing table.
Routes to destinations with the device’s ASN (Autonomous System Numbers) in the AS-PATH are removed from all devices in the network.

Packets are forwarded through remaining ECMP (Equal Cost Multi-Path) paths for all destinations.
NOTE: It is highly unlikely that a single in-flight packet will be lost. This is dependent however, on the L3 ECMP to L2 path hashing algorithms in the hardware and NOS.

Drain (NX-OS)

Drain (Junos)

Drain Leaf Devices (Server-Facing Ports w/ MLAG)
IN THIS SECTION

Drain (NX-OS) | 14
Drain (EOS) | 15
Undrain (NS-OS) | 16
Undrain (EOS) | 17

The following occurs when draining Leaf devices with a server-facing port in an MLAG:

A route-map is placed on all BGP neighbors restricting inbound and outbound routes.
Server facing interfaces are shutdown.
MLAG peer interfaces are shutdown.

What happens at L3:

Outbound routes are removed from the device’s routing table.
Routes to destinations with the device’s ASN in the AS-PATH are removed from all devices in the network.
Packets are forwarded through remaining ECMP paths for all destinations.
NOTE: It is highly unlikely that a single in-flight packet will be lost, however, this is dependent on the L3 ECMP to L2 path hashing algorithms in the hardware and NOS.

What happens at L2:

Server interfaces to this device will go DOWN.
Packets from the server that happen to be hashed onto this device via MLAG may be dropped depending on where they are in the forwarding process.

Packets from the server that happen to be hashed onto this device via MLAG may be forwarded over the MLAG peer link depending on where they are in the forwarding process.
Flows will be reestablished on the alternate MLAG interfaces.
New flows will be established on the remaining MLAG interfaces.