Image Image Image Image Image Image Image Image Image Image
Scroll to top

Top

No Comments

Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair)

Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair)

| On 14, Sep 2015




In this post, we will configure our NetScaler virtual appliances for High Availability. 

Make sure to catch up this series' previous posts first!

 

Introduction

In the previous post, we reviewed the architecture of Citrix Netscaler and installed two standalone virtual appliances (VPX). The next step is to configure High Availability with these two VPX. You will learn the best practices to set up HA smoothly, to increase resiliency for your services.

What is High Availability?

Today’s businesses need access to their systems and services all the time. Globalization has made the workplace a 24/7 environment. People work from home, multiple locations, and everyday offices–they cannot afford any downtime caused by maintenance, human error, or hardware failure.

The best practice while building your infrastructure is to make sure that your system is available all the time. One solution to achieve this is through configuring High Availability (HA). This will eliminate a single point of failure, as the service will remain available for your users even if one of the appliances is down. It will also allow you to schedule maintenance for your devices without impacting production.

In the Netscaler world, an HA pair is composed of at least two appliances/nodes (up to 64) permanently exchanging UDP heartbeat messages.

At the same time, there is always only one node in charge of all traffic; this is the primary node. It actively accepts connections, manages servers, and manages all shared IP addresses.

The node not in charge is called the secondary node, and monitors the health of the primary node to make sure that the service is up and running. If there are issues, a failover will occur and the primary role will be transferred to the secondary node.

NetScaler HA Architecture

Below is the generic NetScaler HA Architecture:

Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) LabNetScalerHAArchitecture1 e1441646622249

Lab NetScaler HA Architecture

All IP addresses are said to be floating IP addresses and are shared across the members of the HA pair except the NSIPs which are unique to each appliance.
Only the primary NetScaler(NetScaler 1 in the schema) is in use. The secondary NetScaler is in stand-by mode and is waiting for the primary node to fail. In others words, all the traffic goes through the primary node.

The two nodes are exchanging packets together (HA sync) for health monitoring.

ARP/GARP

Network devices including NetScalers use ARP (Address Resolution Protocol) to find MAC addresses of others devices located on the LAN.

For example, you have a virtual IP address (VIP) owned by a NetScaler. When a request to this VIP arrives to the gateway, it will check its own ARP cache (where are located previously resolved IP addresses) for the corresponding MAC address. If the cache is empty, the gateway will send an ARP broadcast packet on the LAN. The device with the corresponding IP address will reply to the gateway with an unicast packet containing its MAC address. Then the gateway will put this value into its ARP cache.

You can see the NetScaler ARP table with the command below:


Show ARP Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) CitrixGuru Lab Capture 97 1442279116

Show ARP

or this one:


Show ARP Table Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) CitrixGuru Lab Capture 98 1442279272

Show ARP Table

You can clear the NetScaler ARP table:


or remove a specific IP:


In HA environments and especially after a failover, this could cause issues. The previous MAC address in the gateway ARP cache table is the MAC address of the previous NetScaler appliance.

To fix this issue, the new primary appliance will send GARP (Gratuitous ARP) broadcast packets for all NetScaler HA owned IP addresses (VIPs, NSIPs, etc). The gateway will received these packets and update its ARP cache table.

There is few moment when GARP packets are sent:

  • NetScaler startup (for MIP and SNIP)
  • Creation of a new LB server
  • Add new ip
  • Set new vLAN
  • Failover

You can only disable GARP on VIP:

In others words, ARP packets are sent when another device is requesting information and GARP packets are sent on-demand when a change is occurring in the configuration.

Netscaler HA Settings

Node States

  • STAYPRIMARY: This option forces the NetScaler appliance to stay in the primary mode.
  • STAYSECONDARY: This option forces the NetScaler appliance to stay in the secondary mode.
  • ENABLED: This is the default option. This option enables the NetScaler appliance of the high availability pair to fail over based on the high availability events.
  • DISABLED: This option disables the high availability engine.


In the lab, both HA nodes are configured by default (HA Enabled).

Fail-Safe

Fail-safe mode ensures that one node is always primary when both nodes failed the health check. This is to ensure that when a node is only partially available, backup methods are enabled to handle traffic as best as possible. The HA fail-safe mode needs to be configure on each node.


Recommendation: Enable fail-safe on both nodes.

HA behavior

Below is the behavior of the HA resulting of the nodes states.

Fail-Safe disabled

NS01 (Primary)NS02 (Secondary)Nodes statesHA state
Down (failed last)Down (failed first) NS01 - Secondary
NS02 - Secondary
HA is DOWN
Down (failed first) Down (failed last)NS01 - Secondary
NS02 - Secondary
HA is DOWN
UPUPNS01 - Primary
NS02 - Secondary
HA is UP
UP DownNS01 - Primary
NS02 - Secondary
HA is UP
DownUPNS01 - Secondary
NS02 - Primary
HA is UP
DownUP (Stay Secondary)NS01 - Secondary
NS02 - Secondary
HA is DOWN

Fail-Safe enabled

NS01 (Primary)NS02 (Secondary)Nodes statesHA state
Down (failed last)Down (failed first) NS01 - Primary
NS02 - Secondary
HA is UP
Down (failed first) Down (failed last)NS01 - Secondary
NS02 - Primary
HA is UP
UPUPNS01 - Primary
NS02 - Secondary
HA is UP
UP DownNS01 - Primary
NS02 - Secondary
HA is UP
DownUPNS01 - Secondary
NS02 - Primary
HA is UP
DownUP (Stay Secondary)NS01 - Primary
NS02 - Secondary
HA is UP

Default Configuration

  • The primary node is in charge of all traffic.
  • Both nodes have their own NSIPs but share everything else including VIPs, SNIPs and MIPs.
  • Management over the SNIP addresses is recommended.
  • Changes made on the primary node are replicated to the secondary node. (Enabled by default)
  • Heartbeat(Hello interval) is 200ms.  
    • UDP Packets on port 3003.
    • Dead interval is 3secs.
    • Failover occurs when 3 secs of heartbeat packets are missed.
  • Fail-safe is disabled by default and is configured independently on each node.
  • By default, communications are not secure:
    • HA pair synchronization occurs on port 3010 (TCP).
    • Secure HA pair synchronization occurs on port 3008 (TCP).
    • Commands propagation occurs on port 3011 (TCP).
    • Secure Commands propagation occurs on port 3009 (TCP).

Lab NetScaler HA Architecture

Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) LabNetScalerHAArchitecture11

Lab NetScaler HA Architecture

Configure NetScaler High Availability

Requirements

NetScaler model, version and licenses must be the same on all the appliances.

RPC password must be the same.

Management must be done on the primary node only.

Make sure that the two Netscalers are configured with a unique NSIP.


Also make sure that your appliances are not part of any HA configuration:


This command should have no result.

Make sure that all interfaces are up and running.


There is more requirements available here.

Configuration

On NS02, type the following command:


This command will force NS02 to stay the secondary node in the HA pair.

On NS01, type the following command:


Then configure the sync between the nodes.


On NS02, type the following command:


Then you can show the HA configuration with the command below:


NS01 is configured as Primary and NS02 is configured as Secondary.

HA nodes Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) CitrixGuru Lab Capture 48 1441477468

HA nodes

Note that the sync state of the Secondary NetScaler is SUCCESS.

Type the following command on NS02 to enable the HA node:


Verify the configuration:


Verify the configuration with the GUI:

Connect to http://192.168.199 and check the configuration.

Primary node Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) HAPairNetscalerGUI2

Primary node

This node is primary in the HA pair.

Primary node Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) HAPairNetscalerGUI

Primary node

Connect to http://192.168.200 and check the configuration.

NetScaler is warning you that this appliance is the secondary node and nothing will be saved or propagated.

Secondary Node Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) HAPairNetscalerGUI3

Secondary Node

This node is secondary in the HA pair.

Secondary Node Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) HAPairNetscalerGUI4

Secondary Node

Secondary Node Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) HAPairNetscalerGUI5

Secondary Node

Test failover

On NS02, type the following command:


Note that NS01 is the Primary and NS02 is the secondary node.

HA Status Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) CitrixGuru Lab Capture 52 1441480432

HA Status

Now type the command below to force the failover:


Then, show the HA configuration again:


HA Status Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) CitrixGuru Lab Capture 51 1441480416

HA Status

Note that NS01 is now the Secondary node and NS02 is the now the Primary node.

Before the next step, save your NS config on both appliances:


Shutdown NS02:


On NS01, type the following command:


HA Status Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) CitrixGuru Lab Capture 53 1441480454

HA Status

The previous Primary node(NS02) is now unreachable, so the role has been automatically transferred to NS01.

HA Configuration with NS02 Down Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) HAPairNetScalerGUI6

HA Configuration with NS02 Down

Additional configuration

Configure default route

The configuration below will allow the Netscalers to resolve Internet DNS.

Configure SUBNET IP for management

Connect to the primary node, and add a new SNIP.

Add a new SNIP Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) newSNIP

Add a new SNIP

Go to NetScaler -> System -> Network > IPs -> IPV4 and select the new IP and click Edit.

Select the new IP Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) newSNIP2

Select the new IP

And then select Enable Management Access.

Enable management access  Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) newSNIP3

Enable management access

Open your browser and type http://192.168.1.201. As SNIP addresses are shared between HA nodes, this address will always redirect to the primary NetScaler.

Force HA pair sync

You can force the synchronization of the HA pair with the following command:

Secure HA communication

Connect to the Primary node and type the following command:


show rpcnode Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) CitrixGuru Lab Capture 54 1441485225

show rpcnode

The setting is currently configured on OFF and communication is not secure between the two nodes.

You can change the RPC password by typing the following commands on both nodes:


I recommend to change the password before enabling the Secure RPC.


Secured RPC Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) Lab: Part 6 – Configure NetScaler 11 High Availability (HA Pair) CitrixGuru Lab Capture 55 1441485456

Secured RPC

To make sure that the communication is OK between the two nodes, type the following command:


Sync State should be ENABLED on the primary node and SUCCESS on the secondary node.

Reconfigure NTP server

You can reconfigure the ntp server for the HA pair with the command below:


To make sure that NTPd is running:


Force NTP update


In the next post, we will review how to update the firmware of the nodes configured in HA.

Make sure to catch up this series' previous posts first!

 

Nicolas ignoto
Nicolas ignoto
Nicolas ignoto

Submit a Comment

Leave a Reply