Professional Documents
Culture Documents
APV App WebUI
APV App WebUI
6
User Guide
Copyright Statement
Copyright Statement
Copyright©2017 Array Networks, Inc., 1371 McCarthy Blvd, Milpitas, California 95035, USA.
All rights reserved.
This document is protected by copyright and distributed under licenses restricting its use, copying,
distribution, and compilation. No part of this document may be reproduced in any form by any
means without prior written authorization of Array Networks. Documentation is provided “as is”
without warranty of any kind, either express or implied, including any kind of implied or express
warranty of non-infringement or the implied warranties of merchantability or fitness for a
particular purpose.
Array Networks reserves the right to change any products described herein at any time, and
without notice. Array Networks assumes no responsibility or liability arising from the use of
products described herein, except as expressly agreed to in writing by Array Networks. The use
and purchase of this product does not convey a license to any patent copyright, or trademark rights,
or any other intellectual property rights of Array Networks.
Warning: Modifications made to the Array Networks unit, unless expressly approved by
Array Networks, could void the user’s authority to operate the equipment.
Declaration of Conformity
We, Array Networks, Inc., 1371 McCarthy Blvd, Milpitas, CA 95035, 1-866-692-7729; declare
under our sole responsibility that the product(s) Array Networks, Array Appliance complies with
Part 15 of FCC Rules. Operation is subject to the following two conditions: (1) this device may
not cause harmful interference, and (2) this device must accept any interference received,
including interference that may cause undesired operation.
Warning: This is a Class A digital device, pursuant to Part 15 of the FCC rules. These
limits are designed to provide reasonable protection against harmful interference when the
equipment is operated in a commercial environment. This equipment generates, uses, and
can radiate radio frequency energy, and if not installed and used in accordance with the
instruction manual, may cause harmful interference to radio communications. In a
residential area, operation of this equipment is likely to cause harmful interference in
which case the user may be required to take adequate measures. In a domestic
environment this product may cause radio interference in which case the user may be
required to take adequate measures.
Engineered for the modern data center, Array Networks application, desktop and cloud service
delivery solutions support the scalability, price-performance, software agility and leading-edge
feature innovation essential for successfully transforming today's challenges in mobile and cloud
computing into opportunities for mobilizing and accelerating business.
Website:
https://www.arraynetworks.com/
Telephone:
Phone: (408)240-8700
Fax: (408)240-8754
Telephone access to Array Networks is available Monday through Friday, 9 A.M. to 5 P.M. PST.
E-mail:
Address:
Revision History
Date Description
December 11, 2015 First release.
December 25, 2015 Updates for the APV 8.6 patch release in January 2016.
July 26, 2016 Updates for the APV 8.6 release in July 2016.
Table of Contents
Copyright Statement ......................................................................................................................... I
2.1.1 VLAN..................................................................................................................... 12
18.5.3 Associating the Virtual Service with the Setting Script ..................................... 258
18.5.4 Associating the Virtual Service with the Runtime Script................................... 258
1.1 Overview
This section will outline the initial connection, basic setup and configuration of the APV appliance.
The easy to follow setup steps are introduced below.
Console (recommended)
SSH
WebUI
If you choose the console connection, first connect the console cable (supplied) to the System
Console Port on the APV appliance, and then set up your console as follows:
Setting Value
Emulation VT 100
Baud 9600
Number of Bits 8
Parity No
Stop Bits 1
Flow Control No
Open a connection between the console and the APV appliance. Once this connection is opened,
users will see the APV appliance prompt and may begin the configuration process.
Once the IP parameters are configured and the SSH service is activated, the APV appliance is
prepared for custom configuration. You may access the command line interface (CLI) using SSH
connection. Below gives an example.
Note: If you require SSH software for Windows, MacOS or UNIX, it is available on-line
at http://www.openssh.com. Freeware and links to other support sites can be found here.
2. After you establish a connection, the APV appliance will ask you for a privilege password.
Upon the first startup, the user will be prompted for login username and password. The default
username is “array”, and the default password is “admin”.
Note: You must have the IP information setup and basic network connectivity in order to
access the box through SSH.
This section introduces the connection method via APV WebUI (Web User Interface). It provides
graphical user interface for configuring and managing the APV appliance in a visual and friendly
way, which greatly simplifies the configuration operation but achieves the same configuration
effects as the CLI.
Note: WebUI is disabled by default. To configure and manage the APV appliance using
the WebUI, please see section 1.2.2.4 “Setting up the WebUI”.
The APV appliance provides two WebUIs (new WebUI and legacy WebUI), both of which can be
accessed by entering the IP address and port number of the appliance in the address bar of your
browser.
Example for accessing the new WebUI (8888 is the default port number for accessing the new
WebUI):
https://192.168.1.200:8888
Example for accessing the legacy WebUI (8889 is the default port number for accessing the
legacy WebUI):
https://192.168.1.200:8889
And then press “Enter”. The welcome screen should appear, prompting for username and
password. The default username and password is array and admin.
The legacy WebUI supports IE, Firefox, and Chrome browsers, and the new WebUI supports IE,
Firefox, Chrome, and Safari browsers. It is recommended to use IE 6.0 or later to access the
legacy WebUI, and use IE 11 or later to access the new WebUI. Browser resolution should be set
to 1024×786 or higher.
The APV appliance possesses three LEDs in the front panel: one yellow, one green and one blue.
The following is the usage description of each LED in the front panel.
Note: If the yellow LED is lighted, please contact Customer Support. You can view
system logs to get more information about the problem.
The APV appliance provides two LEDs for every Ethernet port in the rear panel:
Link LED: indicates the speed mode of the link, which can be 1 Gbps, 10 Mbps or 100 Mbps.
The following table describes the meaning of each LED on the onboard and add-on NICs of the
APV appliance.
The CLI allows you to configure and control key functions of the APV appliance to better manage
the performance of your servers and the accessibility to the contents therein.
The APV appliance software has been designed with specific enhancements to make interaction
with the Appliance more user friendly, such as Shorthand. Shorthand is the intuitive method by
which the Appliance completes CLI commands based on the first letters entered. Other user
shortcuts are listed below:
Note: The symbol “^” indicates holding down the Control (Ctrl) Key while pressing the
letter that appears after the symbol.
The APV CLI commands will generally adhere to the following style conventions:
Style Convention
Bold The body of a CLI command is in Boldface.
Italic CLI parameters are in Italic.
<> Parameters in angle brackets < > are required.
Parameters in square brackets [ ] are optional.
[]
Subcommand such as “no”, “show” and “clear” commands.
Alternative items are grouped in braces and separated by vertical
{x|y|…}
bars. At least one should be selected.
Optional alternative items are grouped in square brackets and
[x|y|…]
separated by vertical bars. One or none is selected.
For example:
The APV appliance’s Command Line Interface offers three levels of configuration and access to
the ArrayOS. The CLI prompt of each level consists of the host name of the APV appliance
followed by a unique cursor prompt, either “>”, “#” or “(config)#”.
The first level is for basic network troubleshooting and is called the User level. At this level, the
user is only authorized to operate some very basic commands and non-critical functions such as
ping and traceroute. Here is how the User level prompt appears in the CLI.
AN>
The second level of administration is the Enable level. Users at this level have access to a majority
of view only commands such as “show version”. Users in the Enable level may execute
commands from both the User and Enable levels. In order to gain access to this level of appliance
management, the user must employ the command “enable”. Once this command is entered, the
APV appliance prompts the user for the appropriate password. If correct password is entered, the
CLI prompt will change from “AN>” to “AN#”, which means the user is granted access to the
Enable level. The default password for the Enable level is null, i.e. users simply need to press
“Enter”.
AN>enable
Enable password:
AN#
The final access level is the Config level. It is with this level of authority that the user can make
changes to the configuration of the box. No two users can access the Config level at the same time.
Once a user has gained access to this level, he or she can implement commands in all three levels.
To gain access to the full configurable functions of the APV appliance, the user must use the
following command:
AN#config terminal
Once this command is entered, the CLI prompt will change to:
AN(config)#
In the event that Config level is not available because another Config level session has been
opened, the administrator can deploy the following command to gain access to the Config level:
Type “YES” and press “Enter”. You will enter the Config level successfully.
For each level the user can type “?” for available commands. For example, entering
“AN(config)#slb real ?” will prompt users with all the possible parameters or protocols the CLI
will accept with the “slb real” command.
The table below shows the most critical pieces of configurations from the figure above:
IP Address Description
10.10.0.1/24 Gateway IP Address
10.10.0.2/24 Management IP Address
192.168.10.1/24 Port2 Interface IP Address
192.168.10.0/24 NAT
192.168.10.10 Real Server #1
192.168.10.11 Real Server #2
192.168.10.12 Real Server #3
192.168.10.13 Real Server #4
192.168.10.14 Real Server #5
10.10.0.3 Nameserver/NTP server
Operation Command
Configure interface IP ip address {system_ifname|mnet_ifname|vlan_ifname|bond_ifname}
First, the Port1 Interface IP address needs to be assigned followed by the Port2 Interface, both
with the appropriate netmask assignments. Now with our example network addresses and netmask
designations, these commands should be executed as such:
The port1 interface and the port2 interface cannot be on the same IP network. The CLI will issue a
warning message and will not allow you to configure the two interfaces for the same network.
APV supports changing the MAC address of the system interfaces by using the command
“interface mac <interface_name> <mac_address>”.
Note: The administrator will need to provide the method necessary to allow end-users to
direct outbound traffic to a preferred route based on the IP and protocol type.
The final step in this initial introduction of the APV appliance to the network infrastructure
requires you to define the Gateway IP address.
To verify that APV appliance is indeed actively deployed within this network infrastructure, you
may ping both the gateway and backend server by using the “ping” command.
AN(config)#ping 10.10.0.1
PING 10.10.0.1(10.10.0.1): 56 data bytes
64 bytes from 10.10.0.1: icmp_seq=0 ttl=128 time=0.671 ms
64 bytes from 10.10.0.1: icmp_seq=1 ttl=128 time=0.580 ms
64 bytes from 10.10.0.1: icmp_seq=2 ttl=128 time=0.529 ms
64 bytes from 10.10.0.1: icmp_seq=3 ttl=128 time=0.486 ms
64 bytes from 10.10.0.1: icmp_seq=4 ttl=128 time=0.638 ms
AN(config)#ping 192.168.10.1
PING 192.168.10.1(192.168.10.156 data bytes
64 bytes from 192.168.10.1: icmp_seq=0 ttl=128 time=0.661 ms
64 bytes from 192.168.10.1: icmp_seq=1 ttl=128 time=0.581 ms
64 bytes from 192.168.10.1: icmp_seq=2 ttl=128 time=0.552 ms
64 bytes from 192.168.10.1: icmp_seq=3 ttl=128 time=0.484 ms
64 bytes from 192.168.10.1: icmp_seq=4 ttl=128 time=0.632 ms
AN(config)#show ip address
ip address "port1" 10.10.0.2 255.255.255.0
ip address "port2" 192.168.10.1 255.255.255.0
AN(config)#show ip route
Destination Netmask Gateway
default 10.10.0.1
Should changes be required, in most cases, administrators should deploy the “no” version of the
command relating to the configured information to remove any incorrect information before
entering the desired corrections. For example, executing the command “no ip address port1”,
will remove the port1 IP address for you to then reenter the correct information.
If administrators want to take full advantage of the WebUI access to the APV appliance, at least
one unique IP address is required.
In our example, we use the port1 interface IP address as the default WebUI IP address, and the
default port number 8889 to access the legacy WebUI and 8888 to access the new WebUI.
AN(config)#webui on
AN(config)#webui legacy on
Note:
To change the IP address for accessing the WebUI, use the “webui ip” command.
To change the port number for accessing the new WebUI, use the “webui port” command.
To change the port number for accessing the legacy WebUI, use the “webui legacy port”
command.
Now you can access the WebUI of the APV appliance for configuration and management via the
Web browser. For detailed information about how to use the WebUI, please see related
introductions in section 1.1.1.3 “WebUI Connection”.
With Array clustering technology, more than one APV appliance may be used within a single
network server farm. With this in mind, the ArrayOS allows you to assign a “name” to each APV
appliance for monitoring each device’s performance and configuration specifications. Once
you’ve named your APV appliance, the prompt will change from the default “AN” to the newly
assigned name:
AN(config)#hostname SJ-Box1
SJ-Box1(config)#
SJ-Box1(config)#write memory
Now your configuration is saved into the startup file which the APV appliance calls upon at
reboot.
2.1 Overview
This section focuses on introducing the advanced network configurations, including VLAN,
MNET, Port Forwarding, NAT, Dynamic Routing and IP Pool functionalities and configurations
on the APV appliance.
Note: Basic network configurations such as IP address, the default route and advanced
system configurations are usually associated with or used by other configurations. After
these configurations are set up, please do not modify or change them, or it may lead to
unpredicted errors. To modify network configurations, please execute the commands
“write memory/file”, “clear config all” and “config memory/file” in turn after
modifying network configurations.
2.1.1 VLAN
VLAN (Virtual Local Area Network) is used to logically segment a network into smaller networks
by application, or function, without regard to the physical location of the users. Each VLAN is
considered a separate logical network. There are two types of VLAN specifications for Ethernet
network.
Port-based VLAN
Define VLAN based on port number of the switch. Port-based VLAN is easy to configure but
often limited to one single switch.
Tag-based VLAN
Tag-based VLAN allows a group of devices on different physical LAN segments to communicate
with each other as if they were all on the same physical LAN segment. In tag-based VLAN, an
identifying number, called a “VLAN ID” or a “tag”, is written into the Ethernet frame itself, so
that switches and routers can use this information to make switching decisions. A tagged frame is
four bytes longer than an untagged frame and contains two bytes of Tag Protocol Identifier (TPID)
and two bytes of Tag Control Information (TCI).
The APV appliance supports Tag-based VLAN on all interfaces. Tag-based VLAN (also known as
Trunking by some vendors) is where a tag is inserted into the Ethernet frame so that switches and
routers can use this information to make switching decisions.
The APV appliance’s VLAN can work in both the IPv4 and IPv6 network environments.
Administrator can view all the IPv4 and IPv6-based VLAN configurations by executing the
command “show interface”.
For example:
IPv4-based VLAN:
AN(config)#show interface
……
V1(vlan1): flags=8843<UP,BROADCAST,RUNNING,SIMPLEX> mtu 1500
inet 10.8.66.96 netmask 0xffffff00 broadcast 10.8.66.255
ether 00:25:90:39:97:f3
media: autoselect
status: no carrier
vlan : 3 parent interface: port1
webwall status: OFF
packet drop (not permit): 0
tcp 0 udp 0 icmp 0 ah 0 esp 0
packet drop (deny): 0
tcp 0 udp 0 icmp 0 ah 0 esp 0
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
IPv6-based VLAN:
AN(config)#show interface
……
V2(vlan2): flags=8843<UP,BROADCAST,RUNNING,SIMPLEX> mtu 1500
inet6 fe80::230:48ff:fe93:a73e prefixlen 64 scopeid 0xc
ether 00:30:48:93:a7:41
media: autoselect
status: no carrier
vlan : 20 parent interface: port2
webwall status: OFF
packet drop (not permit): 0
tcp 0 udp 0 icmp 0 ah 0 esp 0
packet drop (deny): 0
tcp 0 udp 0 icmp 0 ah 0 esp 0
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
2.1.2 MNET
MNET (Multi-Netting) is used to assign more than one IP address on a physical interface. Here is
an example for MNET:
A new Internet site is under development for a small corporation. The network administrator
knows that the site will grow in the future but today there is no need for a complex network. A
server is installed that will be used as Web server, FTP server, mail server, and the corporation’s
DNS server. Later, when the use of the network services grows, new servers will be used for each
of the functions.
When the time comes to address the current server, the administrator has a choice. A single IP
address can be used on the server. Later when the new servers are needed, new IP addresses can
be assigned to them.
Another way of assigning addresses can be used. The administrator can assign four IP addresses to
the server. Each IP address will match the IP address to be used in the future on the new servers.
The administrator now knows what addresses will be used and can create DNS entries for the new
devices with the correct addresses. This process of providing more than one IP address on an
interface is often called multi-netting.
The APV appliance’s MNET can work in both the IPv4 and IPv6 network environments.
Note: The Port Forwarding feature cannot support FTP, users are recommended to use
SLB feature instead.
For example, say that you are running SSH on a real server on the port2 interface subnet, and you
want to connect to the server from a client in the outside. To get this information from the outside
world to the desired inside IP/port pair, we will have to add a port forward on the APV appliance.
2.1.4 NAT
NAT (Network Address Translation) is the translation of an IP address used within one network to
a different IP address known within another network. One network is designated the inside
network and the other is the outside. NAT is used when you want your real servers on the inside
network to access the outside network. Using NAT, all packets will appear as though they came
from the APV appliance.
When a client on the inside network contacts a machine on the outside network, it sends out IP
packets destined for that machine. These packets contain all the addressing information necessary
to get them to their destination.
When the packets pass through the NAT gateway, they will be modified so that they appear to be
coming from the NAT gateway itself. The NAT gateway will record the changes it makes in its
state table so that it can reverse the changes on return packets and ensure that return packets are
passed through the firewall without being blocked.
The APV appliance supports NAT traversal of PPTP (Point-to-Point Tunneling Protocol).
Static NAT
Mapping an IP address on one-to-one basis. By configuring static NAT, the APV appliance maps
an inside real IP address to a VIP address on the outside. For inbound traffic directed from an
outside VIP, the traffic will be forwarded to a corresponding inside real IP without any change in
the port number or protocol value. Thus, hosts on the inside network will be directly accessible via
the VIP on the port1 interface. The outbound traffic coming from an inside host will use the
corresponding outside VIP as the source IP for the outgoing traffic. The port number and protocol
remain unchanged. TCP, UDP and ICMP are supported for static NAT.
In static NAT, the computer with the IP address (192.168.10.13) will be always translated into
212.0.0.3:
Port-level NAT
Mapping multiple inside real IP addresses to a single VIP address with a different port number
assignment on the port1 interface. By configuring port-level NAT, the group of hosts on the inside
network will be directly accessible via the VIP on the port1 interface.
In port-level NAT, the computers with the IP address in the range from 192.168.10.11 to
192.168.10.13 will be translated into 212.0.0.3 with a different port on the outside network:
If a port-level NAT is configured for an inside real IP address, static NAT should take precedence
over the regular NAT policy. VIPs used by static NAT should not be used by regular NAT. Also,
one static NAT VIP should not map to multiple real IP addresses.
Mapping one or multiple inside real IP addresses to an IP pool which contains multiple IP
addresses. By configuring IP pool-based dynamic NAT, a group of hosts on the inside network
will be translated into via the multiple IP addresses in IP pool.
As shown below, in IP pool-based dynamic NAT, the client IP addresses in the range from
192.168.10.11 to 192.168.10.13 will be translated into 212.0.0.1 to 212.0.0.3 and get connected to
the Internet.
APV supports NAT based on destination IP address. The destination IP based NAT can be
performed only when the destination IP address is in the specified network segment and in the
same network segment as the route gateway. If the gateway is set to the default value 0.0.0.0, the
destination IP/IP pool for NAT and the route gateway should be within the same network segment.
Note: For IPv6 address, only TCP and UDP packets can be NATTed and no gateway can
be configured.
1. Configure IP pool.
The following two commands can be used to check the NAT configurations and statistics.
Dynamic Routing is especially suitable for today’s large, constantly changed networks. It
improves performance by allowing network routers to adjust to changes in the network topology.
And it distributes routing information between routers and chooses the best path for the network,
saving money and improving performance.
2.1.6 IP Pool
An IP pool contains multiple IP addresses from one IP segment. Administrators can use the
pre-defined IP pools for NAT and SLB configurations.
For the NAT module, IP pool can be defined on the outside interface to realize translation of
multiple outgoing IP addresses. This helps increase the concurrent capacity and fully utilize the IP
resources. For the SLB module, IP pool can be defined for real server groups. Under the SLB
reverse mode, when different real server groups are selected, APV can select different IP
addresses in IP pools to connect to real servers. This not only increases the concurrent connection
capacity, but also provides a more flexible configuration method for administrators.
APV appliance supports multiple IP pools, and at most 256 IP addresses can be added in one IP
pool. The maximum number of IP pools allowed on the APV appliance varies with different
system memories. Please see the table below for details.
– IP addresses which are not covered by any interface subnet are illegal.
The table below shows the most critical pieces of configurations from the figure above:
IP Address Description
10.10.0.1/24 Gateway IP Address
10.10.0.2/24 Management IP Address
192.168.10.1/24 Port2 Interface IP Address
192.168.10.0/24 NAT
192.168.10.10 Real Server #1
192.168.10.11 Real Server #2
192.168.10.12 Real Server #3
192.168.10.13 Real Server #4
192.168.10.14 Real Server #5
10.10.0.3 Nameserver/NTP server
Operation Command
Configure VLAN vlan {system_ifname|bond_ifname} <user_interface_name> <vlan_tag>
Configure MNET mnet {system_ifname|bond_ifname} <user_interface_name>
Configure Port fwd tcp <local_ip> <local_port> <remote_ip> <remote_port> [timeout]
Forwarding fwd udp <local_ip> <local_port> <remote_ip> <remote_port>
Operation Command
[timeout]
nat port {pool_name|vip} <source_ip> {netmask|prefix} [timeout]
Configure NAT [gateway] [description]
nat static <vip> <network_ip> [timeout] [gateway] [description]
rip {on|off}
Configure Dynamic rip network <ip_address> <netmask>
Routing ospf {on|off}
ospf network <ip_address> <netmask> <area_id>
ip pool <pool_name> <start_ip> [end_ip]
Configure IP pool slb proxyip global <pool_name>
slb proxyip group <group_name> <pool_name>
In our example, we are going to create two VLANs, “inside-vlan1” and “inside-vlan2”. The
“inside-vlan1” has a tag of 500 and “inside-vlan2” has a tag of 3001. These tags are inserted into
the Ethernet frame.
2. Assign an IP address to each VLAN interface by using the “ip address” command.
For the interface with VLAN configuration, it needs to be connected to a switch or router with Tag
VLAN or Trunking turned on. See your switch vendors’ documentation on how to setup Tag
VLAN.
Configuring MNET on the port2 interfaces is very similar to VLAN configuration. For our
example network, we will run two networks over the port2 interface, 192.168.1.1/24 and
192.168.2.1/24.
Again you need to refer to your vendor’s switch/router documentation on how to setup their
interface for use with MNET.
For our example configuration, we will be adopting the TCP port forwarding protocols as such:
We picked an arbitrary high port to use. You should not use a port below 1024 on the APV
appliance since other services might be listening on those ports, i.e. 443 (for SSL) and 80 (for
HTTP). We can choose a port below 1024 on the real server since that is the service that we want
to connect to. To view or alter these forwarding instructions, employ the show, no or clear
versions of the above commands.
This command will perform NAT on the 192.168.10.0/24 network. In our example, the VIP
10.10.0.2 and the route gateway 10.10.0.1 are within the same network segment. Therefore the
parameter “gateway” in the command “nat port” can be set to the default value 0.0.0.0 or the
route gateway. If the VIP and the route gateway are not in the same network segment, the
parameter “gateway” in the command “nat port” must be set to the route gateway.
We can change the netmask to allow only certain blocks of your inside network to access the
external network. For example, the following command will only allow the IP addresses ranging
192.168.10.0 through192.168.10.128, to access the external network:
If we want to allow the top half of the IP address space range that is left over
(192.168.10.129-192.168.10.254), to access the external network, we will do the following:
If we want to allow one real IP address to access the external network, we will configure static
NAT:
1. RIP Configurations.
AN(config)#rip on
AN(config)#rip version 2
AN(config)#rip network 172.16.31.0 255.255.255.0
AN(config)#rip network 172.16.32.0 255.255.255.0
2. OSPF Configurations.
AN(config)#ospf on
AN(config)#ospf network 172.16.32.0 255.255.255.0 0
AN(config)#ospf network 172.16.31.0 255.255.255.0 0
After these configurations, you can view the dynamically generated routes by using the “show ip
route” command.
AN(config)#show ip route
Destination Netmask Gateway
RIP routes:
Destination Netmask Gateway
172.16.39.0 255.255.255.0 172.16.31.67
OSPF routes:
Destination Netmask Gateway
172.16.41.0 255.255.255.0 172.16.32.2
Now that the very basics of our example network configurations are implemented, it is time to
move forward to configure the APV appliance to operate seamlessly within the network
architecture.
2. Define the IP pool for NAT via the “nat port” command.
2. Define the IP pool as the global proxy IP pool by using the “slb proxyip global” command.
3.1 Overview
This section describes link aggregation functionality of the network. Link Aggregation is also
called trunking, which can greatly improve network performance and stability.
The APV appliance supports at most 8 bond interfaces, and at most 12 system interfaces can be
added to a bond interface. The bond interface will check the status of the system interfaces. If a
system interface becomes down, the traffic processed by this interface will be directed to other
working system interfaces in the bond interface.
To add a system interface into a bond interface, the administrator can further set the interface as
the primary or backup interface in the bond. Multiple primary or backup interfaces can be set in a
bond. When all the primary interfaces in the bond fail, the backup interfaces will take the place of
primary interfaces to work.
Note: To bind a system interface with a bond interface, the system interface should be
configured with no IP address information. If there is IP configuration on the system
interface, the administrator needs to remove the IP configuration first. If otherwise, the
system will refuse to add the system interface into the bond.
In addition, the APV appliance also supports configuring MNET or VLAN on bond interface. The
bond interface configuration must be performed before configuring MNET or VLAN on it.
Note: When the health status of all the bond subinterfaces is marked “down” with the
transmission of ARP and IPv6 NS packets still behaving normally, the bond interface will
still be in an usable state, and all the service traffic will be sent to every subinterface.
Meanwhile, please check if the destination IP of the bond health check is configured
properly.
Operation Command
Bond system
bond interface <bond_name> <interface_name> [1|0]
interfaces
Assign a name for the
bond name <bond_id> <bond_name>
bond interface
Assign IP address to ip address {system_ifname|mnet_ifname|vlan_ifname|bond_ifname}
the bond interface <ip_address> {netmask|prefix}
Assign default route ip route default <gateway_ip>
Configure link bond health <bond_name> <destination_ip> [interval] [timeout]
aggregation health [up_check_times] [down_check_times] [gateway_ip]
check
In our example, we bind the “port1” interface and the “port4” interface with the bond interface
bond1, and further set the “port1” interface as the primary, and “port4” as the backup in the bond
interface.
We can set the bond name for the configured bond interface by using the “bond name” command.
To verify that APV appliance is indeed actively deployed within this network infrastructure, you
may ping the gateway IP by using the “ping” command.
If these configurations are entered correctly, you will receive the following return messages.
AN(config)#ping 10.10.0.1
PING 10.10.0.1(10.10.0.1): 56 data bytes
64 bytes from 10.10.0.1: icmp_seq=0 ttl=128 time=0.671 ms
64 bytes from 10.10.0.1: icmp_seq=1 ttl=128 time=0.580 ms
64 bytes from 10.10.0.1: icmp_seq=2 ttl=128 time=0.529 ms
64 bytes from 10.10.0.1: icmp_seq=3 ttl=128 time=0.486 ms
64 bytes from 10.10.0.1: icmp_seq=4 ttl=128 time=0.638 ms
Chapter 4 Clustering
4.1 Overview
The Array Clustering function allows you to maintain high availability within a local site. With
other options you can also distribute load across multiple boxes within a cluster.
Active-Standby mode – In Active-Standby mode, all VIPs on one APV appliance in the cluster
will be the master, and all VIPs on the other APV appliances in the cluster are standby. In this
mode, clustering supports fast failover.
Active-Active mode – In Active-Active mode, each APV appliance in the cluster has a different
master VIP or cluster ID.
Discreet Backup mode is designed to prevent a double-master state. In this mode, the system
determines whether a state transition is needed for the devices based on their state information
detected by a heartbeat cable. This mode makes the state transition more reliable, and any VRRP
packet loss will not result in double-master state.
1. After turning on clustering, the device enters into Init state. Then, in order to check the health
of the heartbeat cable, the Init device switches to FFO state.
2. The device collected the health information of the heartbeat cable. If the heartbeat cable is
well connected, it will switch to Backup state.
Note: Even though the heartbeat cable is disconnected, the device will still switch to
Backup state, and clustering will work well. However, the discreet mode is invalid.
3. If the backup receives a higher priority VRRP packet, it will switch to Discreet Backup state.
4. In the following events, the discreet backup will switch to Backup state:
– The device in Discreet Backup state receives a lower priority VRRP packet (after the
successful state transition, the backup will go on to switch to Master state.).
– The device in Discreet Backup state will check the heartbeat cable health. If the
heartbeat cable is disconnected, it will log out to Backup state.
– The backup receives a lower priority VRRP packet (in Preemption mode).
– In three continuous broadcast intervals (the default interval is 5 seconds, three intervals
are 15 seconds), the backup does not receive the VRRP packet from the master.
6. If the master receives a higher priority VRRP packet, it will switch to Backup state.
7. If the heartbeat cable detected the master’s NIC is down, the discreet backup will switch to
Master state directly.
Note: All cluster state transitions can be traced by the command “show cluster virtual
transition”.
To configure the discreet backup mode, the following two commands MUST be configured first to
turn on the discreet backup mode.
If the interface for Clustering is configured with both the IPv4 and IPv6 addresses or with only the
IPv4 address, then the IPv4-based VRRP packets will be used for communication between the
APV appliances. If only the IPv6 address is configured on the interface for Clustering, then the
IPv6-based VRRP packets will be used.
Note: The VRRP packets are incompatible with each other among different ArrayOS
versions. So please use the same ArrayOS version for the APV appliances in a cluster.
For information about SLB, please refer to Chapter 7 Server Load Balancing (SLB).
Configuration Guidelines
In Active-Standby mode, one node in the cluster will be the master of the VIP, and thus active.
The other node in the cluster will be in standby mode. Upon failure of the active node, the standby
node will take over the VIP and become master. If preemption has been enabled on the initial
master node, it will reassume mastership when it returns to a working state. Otherwise, the VIP
will stay with the new master node until the node fails.
Refer to the following figure for the typical layout of Active-Standby architecture, in which:
APV1 is the current master, and handles SLB traffic for VIP.
APV2 is the backup, and listens for advertisements from the master. It will resume master
status if APV1 stops sending advertisements (i.e. APV1 fails).
Operation Command
Configure SLB Refer to SLB Configuration section.
Configure a virtual
cluster virtual ifname <interface_name> <cluster_id>
interface
Configure virtual
cluster virtual auth <interface_name> <cluster_id> {0|1} [password]
cluster authentication
Configure preemption cluster virtual preempt <interface_name> <cluster_id> <mode>
Configure virtual IP cluster virtual vip <interface_name> <cluster_id> <vip>
cluster virtual priority <interface_name> <cluster_id> <priority>
Configure priority
[synconfig_peer_name]
Enable the virtual
cluster virtual {on|off} [cluster_id|0] [interface_name]
cluster
Operation Command
Enable fast failover cluster virtual ffo {on|off}
feature cluster virtual ffo interface carrier loss timeout <interface_timeout>
It is recommended that you run clustering with an authentication string to avoid unauthorized
participation in your cluster.
Now we configure APV1 to preempt the VIP when the initial master returns online. For APV2, it
will not preempt the VIP from the master node, but will take over if the master ceases operations.
Cluster priority determines which node becomes the master. The node with highest priority
becomes the master. Since we want APV1 to always be master of the VIP, we will set its priority
to 255. For APV2, we will leave its priority at 100. In a two-node cluster, this is permissible.
Though, when you include more nodes in your cluster, you will need to set a unique priority for
each VIP to properly communicate and fail-over. To do this, use the following command:
Note: The state is the backup on APV2. This is expected since it is of lower priority than
the master.
APV1(config)#cluster virtual on
APV2(config)#cluster virtual on
Configuration Guidelines
In Active-Active mode, node 1 will be the master for VIP1, and the backup for VIP2. Node 2 will
act as the master for VIP2, and serve as the backup for VIP1. This increases the performance of
your site while maintaining high availability.
The next illustration shows a typical deployment. To achieve active-active status, we need to have
two virtual cluster IDs (VCID), each containing at least one VIP.
In the above figure, APV1 is the master for VIP1 and the backup for VIP2 and APV2 is the master
for VIP2 and the backup for VIP1.
VCID 1 will have VIP1 (192.168.2.100) and VCID 2 will have VIP2 (192.168.2.101).
Operation Command
Configure SLB Refer to the SLB Configuration section.
Configure a virtual
cluster virtual ifname <interface_name> <cluster_id>
interface
Configure virtual
cluster cluster virtual auth <interface_name> <cluster_id> {0|1} [password]
authentication
Configure
cluster virtual preempt <interface_name> <cluster_id> <mode>
preemption
Configure virtual IP cluster virtual vip <interface_name> <cluster_id> <vip>
cluster virtual priority <interface_name> <cluster_id> <priority>
Configure priority
[synconfig_peer_name]
Enable the virtual
cluster virtual {on|off} [cluster_id|0] [interface_name]
cluster
We will setup node 1 as the master of VIP1 and the backup of VIP2. Node 2 will be the master of
VIP2 and the backup for VIP1.
It is recommended that you run clustering with an authentication string to avoid unauthorized
participation in your cluster.
Cluster priority determines which node becomes the master. The node with highest priority
becomes the master.
APV1(config)#cluster virtual on
APV2(config)#cluster virtual on
Note: NATing is highly recommended if the machines in your inside network need to
communicate to other networks via the APV appliance.
There are two methods of setting up the inside interface. The first is to use one VIP that will
belong to one of the appliances in the Virtual Cluster. If you want to or need to share the load
between the nodes you will have to setup an Active-Active configuration for the inside interfaces.
We will cover how to setup both scenarios in this section.
Configuration Guidelines
In Active-Standby mode, one box will serve as the gateway for the inside network. Upon
unexpected failure of the master node, the standby node in the cluster will take over. For our
purpose, we are going to pick an unused IP address on the inside network (192.168.1.3) and use it
as the gateway for our inside network.
Operation Command
Configure a virtual
cluster virtual ifname <interface_name> <cluster_id>
interface
Configure virtual IP cluster virtual vip <interface_name> <cluster_id> <vip>
cluster virtual priority <interface_name> <cluster_id> <priority>
Configure priority
[synconfig_peer_name]
Enable the virtual
cluster virtual {on|off} [cluster_id|0] [interface_name]
cluster
Cluster priority determines which node becomes the master. The node with highest priority
becomes the master.
APV1(config)#cluster virtual on
APV2(config)#cluster virtual on
Configuration Guidelines
In Active-Active configuration, we will create two VIPs to serve as gateways. Half of your
servers’ default routes will point to the first VIP and the other half will point to the second VIP,
thus equally dividing the load between the APV appliances.
Operation Command
Configure a virtual
cluster virtual ifname <interface_name> <cluster_id>
interface
Configure virtual IP cluster virtual vip <interface_name> <cluster_id> <vip>
cluster virtual priority <interface_name> <cluster_id> <priority>
Configure priority
[synconfig_peer_name]
Enable the virtual
cluster virtual {on|off} [cluster_id|0] [interface_name]
cluster
Cluster priority determines which node becomes the master. The node with highest priority
becomes the master.
APV1(config)#cluster virtual on
APV2(config)#cluster virtual on
Configuration Guidelines
In Active-Standby mode, one node in the cluster will be the master of the VIP, and thus active.
The other node in the cluster will be in standby mode. Upon failure of the active node, the standby
node will take over the VIP and become master. If preemption has been enabled on the initial
master node, it will reassume mastership when it returns to a working state. Otherwise, the VIP
will stay with the new master node until the node fails.
Refer to the following figure for the typical layout of Active-Standby architecture, in which:
APV1 is the current master, and handles SLB traffic for VIP.
APV2 is the backup, and listens for advertisements from the master. It will resume master
status if APV1 stops sending advertisements (i.e. APV1 fails).
Operation Command
Configure SLB Refer to the SLB Configuration section.
Configure a virtual
cluster virtual ifname <interface_name> <cluster_id>
interface
Configure virtual
cluster virtual auth <interface_name> <cluster_id> {0|1} [password]
cluster authentication
Configure preemption cluster virtual preempt <interface_name> <cluster_id> <mode>
Configure virtual IP cluster virtual vip <interface_name> <cluster_id> <vip>
Configure priority cluster virtual priority <interface_name> <cluster_id> <priority>
Operation Command
[synconfig_peer_name]
Enable the virtual
cluster virtual {on|off} [cluster_id|0] [interface_name]
cluster
Enable fast failover cluster virtual ffo {on|off}
feature cluster virtual ffo interface carrier loss timeout <interface_timeout>
It is recommended that you run clustering with an authentication string to avoid unauthorized
participation in your cluster.
Now we configure APV1 to preempt the VIP when the initial master returns online. For APV2, it
will not preempt the VIP from the master node, but will take over if the master ceases operations.
Cluster priority determines which node becomes the master. The node with highest priority
becomes the master. Since we want APV1 to always be master of the VIP, we will set its priority
to 255. For APV2, we will leave its priority at 100. In a two-node cluster, this is permissible.
Though, when you include more nodes in your cluster, you will need to set a unique priority for
each VIP to properly communicate and fail-over. To do this, use the following command:
Note: The state is the backup on APV2. This is expected since it is of lower priority than
the master.
APV1(config)#cluster virtual on
APV2(config)#cluster virtual on
Configuration Guidelines
In Active-Active mode, node 1 will be the master for VIP1, and the backup for VIP2. Node 2 will
act as the master for VIP2, and serve as the backup for VIP1. This increases the performance of
your site while maintaining high availability.
The next illustration shows a typical deployment. To achieve active-active status, we need to have
two virtual cluster IDs (VCID), each containing at least one VIP.
In the above figure, APV1 is the master for VIP1 and the backup for VIP2 and APV2 is the master
for VIP2 and the backup for VIP1.
VCID 1 will have VIP1 (192.168.2.100) and VCID 2 will have VIP2 (192.168.2.101).
Operation Command
Configure SLB Refer to the SLB Configuration section.
Configure a virtual
cluster virtual ifname <interface_name> <cluster_id>
interface
Configure virtual
cluster cluster virtual auth <interface_name> <cluster_id> {0|1} [password]
authentication
Configure
cluster virtual preempt <interface_name> <cluster_id> <mode>
preemption
Configure virtual IP cluster virtual vip <interface_name> <cluster_id> <vip>
cluster virtual priority <interface_name> <cluster_id> <priority>
Configure priority
[synconfig_peer_name]
Enable the virtual
cluster virtual {on|off} [cluster_id|0] [interface_name]
cluster
We will setup node 1 as the master of VIP1 and the backup of VIP2. Node 2 will be the master of
VIP2 and the backup for VIP1.
It is recommended that you run clustering with an authentication string to avoid unauthorized
participation in your cluster.
Cluster priority determines which node becomes the master. The node with highest priority
becomes the master.
APV1(config)#cluster virtual on
APV2(config)#cluster virtual on
Note: NATing is highly recommended if the machines in your inside network need to
communicate to other networks via the APV appliance.
There are two methods of setting up the inside interface. The first is to use one VIP that will
belong to one of the appliances in the Virtual Cluster. If you want to or need to share the load
between the nodes you will have to setup an Active-Active configuration for the inside interfaces.
We will cover how to setup both scenarios in this section.
Configuration Guidelines
In Active-Standby mode, one box will serve as the gateway for the inside network. Upon
unexpected failure of the master node, the standby node in the cluster will take over. For our
purpose, we are going to pick an unused IP address on the inside network (192.168.1.3) and use it
as the gateway for our inside network.
Operation Command
Configure a virtual
cluster virtual ifname <interface_name> <cluster_id>
interface
Configure virtual IP cluster virtual vip <interface_name> <cluster_id> <vip>
cluster virtual priority <interface_name> <cluster_id> <priority>
Configure priority
[synconfig_peer_name]
Enable the virtual
cluster virtual {on|off} [cluster_id|0] [interface_name]
cluster
Cluster priority determines which node becomes the master. The node with highest priority
becomes the master.
APV1(config)#cluster virtual on
APV2(config)#cluster virtual on
Configuration Guidelines
In Active-Active configuration, we will create two VIPs to serve as gateways. Half of your
servers’ default routes will point to the first VIP and the other half will point to the second VIP,
thus equally dividing the load between the APV appliances.
Operation Command
Configure a virtual
cluster virtual ifname <interface_name> <cluster_id>
interface
Configure virtual IP cluster virtual vip <interface_name> <cluster_id> <vip>
cluster virtual priority <interface_name> <cluster_id> <priority>
Configure priority
[synconfig_peer_name]
Enable the virtual
cluster virtual {on|off} [cluster_id|0] [interface_name]
cluster
Cluster priority determines which node becomes the master. The node with highest priority
becomes the master.
APV1(config)#cluster virtual on
APV2(config)#cluster virtual on
5.1 Overview
As the network applications develop, customers have higher and higher requirements for the
reliability of the network and network appliances. During network planning and design, to
improve the reliability of the network, some critical network appliances must have redundancy
protection mechanisms. The Clustering function mentioned in the “Clusterin” chapter uses the
VRRP technology to solve the single-point failure. This chapter will introduce the High
Availability (HA) function that newly provided by Array APV appliances. The HA function not
only solves the single-point failure, but also provides more policies to ensure the network
reliability.
The HA function allows two or more APV appliances to continuously exchange the running status
with each other, and keep their configurations synchronized. When an appliance becomes down,
other available appliances will take over the application services on the faulty appliance, which
ensures the high availability of application services.
Besides, the HA function provides the Stateful Session Failover (SSF) function. With the SSF
function, when a service failover occurs, connections on the service will be switched to the new
appliance. This avoids the interruption of connections and therefore improves user experience.
The HA function can be deployed flexibly. Besides the Active/Active and Active/Standby
deployment scenarios, the HA function can be deployed among multiple appliances to achieve
mutual-backup.
5.2 HA Basics
To ensure the consistency and flexibility of service failover, the Array HA technology groups the
floating IP addresses and switches floating IP addresses by group. The floating IP addresses can
be switched only after they are added to a floating IP group. At the same moment, the status of all
floating IP addresses in the same floating IP group are the same. The status is also called the status
of the floating IP group.
The status of a floating IP group is determined by the group priority, failover mode, and results of
the health checks related to the group. After the floating IP group is configured correctly, the HA
module will the check the running environment of the group based on the configured health check
conditions. Based the health check results, the group status can be one of the following two types:
Active/Standby: The results of all health checks related to the group are “Up”, indicating the
group is ready to provide services. In this case, the group status is “Active” or “Standby”. If
the group status is “Active”, this unit will obtain all the floating IP addresses of the group and
provide services. If the group status is “Standby”, this unit will provides backup for services
and will take over services in case of service failover.
Init: Initial group status. If the result of any health checks related to the group is “Down”, the
group status is “Init”, which indicates that this unit is not qualified for provides services of
the group. Even if service failover occurs on the group, this unit cannot take over services.
Note: When the group status is “Init”, check the group configurations or the health check
results to make the group status change to “Active” or “Standby” so that the unit will
provide services or backup for services.
On one unit, multiple floating IP groups can be configured. The status of every groups are
independent from each other. If all groups on a unit need to be switched over together, the
“Unit_Failover” mode (see the section “Failover Rules”) is required.
The floating IP address are configured by using the “ha group fip” or “ha group fiprange”
command. For details, please see the ArrayOS APV CLI Handbook.
If the non-preempt mode is enabled, the group status on the local unit will not change until a
failover occurs.
In the preempt mode, if the group priority on the local unit is higher than those of all peer
units, the group status on the local unit will be forcibly switched to “Active”. If the group
status on a peer unit was “Active” before this, its group status will be forcibly switched to
“Standby”.
because the MAC addresses of the appliances that provide application services before and after
group status switch remain unchanged.
Note:
By default, the floating MAC function is disabled. Before this function is enabled,
the HA function must be first disabled by executing the command “ha off”.
The parameter “interface_name” in the command “ha floatmac mac” determines
that the floating MAC function takes effect on the floating IP group with which the
interface is associated. The floating MAC addresses configured for different floating
IP groups cannot be the same.
Active/Active deployment scenario: The HA domain comprises two units; on each unit, there
are “Active” floating IP groups and “Standby” floating IP groups, the status of which are
“Active” on the peer unit. The HA domain comprises two units. On each unit, there are
“Active” floating IP groups, and the status of the “Active” group on the peer unit is
“Standby”.
Active/Standby deployment scenario: The HA domain comprises two units; the status of all
floating IP groups are “Active” on one unit and are “Standby” on the other unit.
When the HA function is deployed among multiple appliances to achieve mutual-backup, the
HA domain comprises multiple units to provide services or backup for services. Among these
scenarios, the “N+1” deployment scenario is commonest one. In the “N+1” deployment
scenario, the HA domain comprises N+1 units. Among these units, the group status on N
units are all “Active” and all the group status on the remaining one are “Standby”.
Primary Link
Secondary Link
The FFO link is established by directly connecting two HA units’ FFO ports through a dedicated
FFO cable. Therefore, it can only be used for the Active/Active and Active/Standby deployment
scenarios with two units. By default, the FFO link is disabled. The main functions of the FFO link
are as follows:
Heartbeat packet transmission: When the HA function is running, the local unit can use the
FFO link to send the heartbeat packets to detect the peer unit’s status.
Bootup Synconfig: When the FFO link and Bootup Synconfig are both enabled, the local unit
can synchronize the HA unit and link configurations from the peer unit.
Fast Failover: When the local unit is down, the peer unit can perform fast VIP failover
(within 3 milliseconds) through the FFO link.
The primary and secondary links are also called network links, because both of them connect the
two units through ordinary network cables. Only one primary link can be established between any
two units, while at most 31 secondary links are allowed between any two units. By defaults, the
network links are enabled.
After adding multiple units for an HA domain, the system will establish primary link connections
between each two units automatically. The main functions of the primary link are as follows:
Heartbeat packet transmission: The local unit can send the heartbeat packets to its peer units
through the primary link to detect the peer units’ status.
Bootup Synconfig: With both Bootup Synconfig and the HA function enabled, the local unit
can synchronize the configurations saved by the “write memory” command from the peer
units.
Runtime Synconfig: With Runtime Synconfig enabled, when the whitelist configurations of
the local unit are modified, the local unit can synchronize the modifications to the peer units
via the primary link. When the blacklist configurations of the local unit are modified, the
local unit will not synchronize the modifications to the peer units.
The secondary link is optional and just used for heartbeat packets transmission. The administrator
has to manually set up the same secondary link configurations on the local unit and the peer units.
Please be noted that to establish a secondary link between two units, you need to configure a
secondary link with the same ID on the two units respectively.
For example, the IP address of two HA units “u1” and “u2” are 192.168.1.1 and 192.168.1.2
respectively. To establish a secondary link between the two units, the following two commands
must be executed on both units:
After the above configurations are finished on both units, a secondary link with ID “1” is
established between “u1” and “u2”.
The table below shows the differences and similarities among the three types of HA
communication links.
Failover rules are defined by associating failover conditions with failover actions. Failover
conditions indicate the monitoring status on system hardware or software, such as network
interface status, CPU utilization and so on. Failover actions are the operations to be performed by
the system when the associated failover conditions occur. HA provides three failover actions:
Group_Failover: Switch over the status of the floating IP group. For this action, the system
will select a new unit based on the health condition and group priority, and change the status
of the floating IP group enabled on that unit to be “Active” to take over the services.
Unit_Failover: Switch over the status of all the floating IP groups enabled on a unit.
Reboot: Switch over the status of all the floating IP groups enabled on a unit, and then restart
the unit.
Note:
Only when the network connections of all interfaces in a bond interface become
down, will the “Group_Failover” action be taken for the floating IP group to which
the IP addresses of the bond interface belong.
For the units who provide failover support for the same floating IP group in the HA
domain, the failover rules of the floating IP group configured on the units should be
exactly the same, including the name of the health check condition in failover rules.
For the units who provide failover support for “Unit_Failover” in the HA domain, the
failover rules of the “Unit_Failover” configured on the units should be exactly the
same, including the name of the health check condition in failover rules.
2. System health check: HA supports the reuse of the system health check as failover conditions.
Administrator can define the following health check conditions for HA failover.
Hardware:
Software:
Network condition:
In some complex application environments, more complicated failover rules are required. For
example, theoretically, in the environment with a bond interface deployed, only when the network
connections of all interfaces in the bond interface become down will failover action be taken.
However, in practical application, it is required to take failover action as long as the network
connection of one interface becomes down. To meet this kind of complicated applications, HA
further introduces the concept of health check condition group (vcondition). A vcondition
comprises multiple health check sub-conditions. A sub-condition can be a real health check
condition or another vcondition, which further comprises sub-conditions. The logical relationship
among multiple sub-conditions can be either “AND” or “OR”. To apply vcondition to the above
application, administrators can first define health check conditions for each of the network
interfaces in a bond interface, and combine these conditions into a vcondition by setting the
logical relationship to “OR”. Then, associate the vcondition with some failover actions.
3. Self-defined health check script: The system supports the import of self-defined health check
scripts from an SCP or TFTP server. Self-defined health check scripts take effect only after
signature verification. To use this function, please contact Array Networks Customer
Support.
Make sure the same Challenge Code is configured on both local unit and peer unit using the
“synconfig challenge” command.
Configure synchronization peers on both peer unit and local unit using the “synconfig peer
<peer_name> <peer_ip>” command. In addition, the value of the “peer_name” parameter
should be identical with the value of the “unit_name” parameter specified in the “ha unit
<unit_name> <ip_address> [port]” command.
The local unit can log into the HA domain in two methods:
FFO Login: The unit logs into the HA domain through the FFO link.
Network Login: The unit logs into the HA domain through the network link.
To use the FFO Login method, administrators need to perform the following operations on the
local unit:
1. Execute the “ha link ffo on” command to enable the FFO link.
2. Execute the “ha synconfig bootup on” command to enable the Bootup Synconfig function of
HA.
Note: After the FFO link and the Bootup Synconfig function are enabled, the local unit
will start to synchronize configurations about HA units and links from the peer unit.
3. Execute the “ha link network on” command to enable the network links.
To use the Network Login method, administrators need to perform the following operations on the
local unit:
1. Execute the “ha unit” command to add the local unit and the peer unit.
2. Execute the “ha synconfig bootup on” command to enable the Bootup Synconfig function of
HA.
3. Execute the “ha link network on” command to enable the network links.
Note: In Bootup Synconfig, only the configurations saved by executing the command
“write memory” can be synchronized.
Note: To synchronize the configuration changes from the local unit to the peer units,
please make sure that Runtime Synconfig is enabled on both the local unit and the peer
units
The SSF function supports TCP, UDP, FTP and IP types of SLB applications as well as NAT
applications. It can be enabled or disabled per virtual service.
Note:
To ensure the SSF function works well, please make sure that the HA-related
configurations on all the units in one HA domain are the same. It is recommended to
use Runtime Synconfig while the SSF function is enabled.
The SSF function uses a stable network link between two HA units to transmit SSF
session information. If the network link used for SSF goes down, session information
cannot be exchanged between two units. If a group failover occurs subsequently, the
existing connections might be reset.
5.7 HA Logging
The HA function provide logging function. By default, the HA logging function is disabled. If the
HA function is enabled, the logging function will be enabled too. If the HA function is disabled,
the logging function will be disabled too.
The HA logging function allows administrators to set the level of the HA logs that the system
generates. Eight levels HA logs are supported: emerg, alert, crit, err, warning, notice, info, and
debug. Once the level of HA logs is specified, the log messages lower than this level will be
ignored, that is will not be recorded in the system. The default log level is info.
HA internal information is recorded in HA log, and the changes of floating IP groups and units is
recorded in the system log.
Scenario 1: Active/Standby
Scenario 2: Active/Active
Scenario 3: N+1
The following sections describe the configuration examples for all the three scenarios.
The Active/Standby deployment scenario can be used to achieve the following configuration
objectives:
The HA domain contains two HA units, each of which is enabled with the same floating IP
group.
The Floating IP group contains the VIP addresses of two application services.
Unit 1 provides application services, while unit 2 provides backup for such services.
The following figure shows the network topology for the preceding configuration objectives.
APV1:
2. Execute the following commands to configure HA units, synchronization peers and links:
AN(config)#ha group id 1
AN(config)#ha group fip 1 192.168.10.2 port1
AN(config)#ha group fip 1 192.168.10.3 port1
AN(config)#ha group fip 1 192.168.100.2 port3
AN(config)#ha group fip 1 192.168.100.3 port3
AN(config)#ha group priority unit1 1 10
AN(config)#ha group priority unit2 1 5
AN(config)#ha group preempt on 1
AN(config)#ha group enable 1
4. (Optional) Execute the following commands to configure health check conditions, taking the
health check for gateway and CPU utilization as examples.
7. (Optional) Execute the following commands to set the configuration synchronization mode:
AN(config)#ha log on
9. Execute the following commands to enable the HA function and save the HA configurations
to the memory:
AN(config)#ha on
AN(config)#write memory
APV2:
In the Active/Standby scenario, it is recommended to use the FFO link and primary link to
synchronize configuration information from the peer unit.
2. Execute the following commands to enable the FFO function and the Bootup Synconfig
mode:
AN(config)#ha on
Once the HA function is enabled, unit2 (APV2) will join the HA domain. After unit2 joins the HA
domain, it first synchronizes configuration information about HA units, FFO link and network
links through the FFO link and then synchronizes other configurations through the primary link
from unit1 (APV1).
The Active/Active deployment scenario can be used to achieve the following configuration
objectives:
The HA domain contains two HA units and provides two floating IP groups.
Each floating IP group contains the VIP address of one application service.
Unit1 provides the application service for group1, while unit2 provides the application
service for group2. Unit1 and unit2 provide backup for each other.
The following figure shows the network topology for the preceding configuration objectives.
APV1:
2. Execute the following commands to configure HA units, synchronization peers and links:
AN(config)#ha group id 1
AN(config)#ha group fip 1 192.168.10.2 port1
AN(config)#ha group fip 1 192.168.100.2 port3
AN(config)#ha group priority unit1 1 10
AN(config)#ha group priority unit2 1 5
AN(config)#ha group preempt on 1
AN(config)#ha group enable 1
AN(config)#ha group id 2
AN(config)#ha group fip 2 192.168.10.3 port1
AN(config)#ha group fip 2 192.168.100.3 port3
AN(config)#ha group priority unit1 2 5
AN(config)#ha group priority unit2 2 10
AN(config)#ha group preempt on 2
AN(config)#ha group enable 2
4. (Optional) Execute the following command to configure health check conditions, taking the
health check for gateway and CPU utilization as examples.
7. (Optional) Execute the following commands to set the configuration synchronization mode:
AN(config)#ha log on
9. Execute the following commands to enable the HA function and save the HA configurations
to the memory:
AN(config)#ha on
AN(config)#write memory
APV2:
In the Active/Active scenario, it is recommended to use the FFO link and primary link to
synchronize configuration information from the peer unit.
2. Execute the following commands to enable the FFO function and the Bootup Synconfig
mode:
AN(config)#ha on
Once the HA function is enabled, unit2 (APV2) will join the HA domain. After unit2 joins the HA
domain, it first synchronizes configuration information about HA units, FFO link and network
links through the FFO link and then synchronizes other configurations through the primary link
from unit1 (APV1).
The “3+1” deployment scenario can be used to achieve the following configuration objectives:
The HA domain contains four HA units and provides three floating IP groups.
Unit1 to unit3 provide the virtual services of group1 to group3 respectively, while unit4
provides backup for unit1 to unit3.
The following figure shows the network topology for the preceding configuration objectives.
APV1:
2. Execute the following commands to configure HA units, synchronization peers and links:
AN(config)#ha group id 1
AN(config)#ha group fip 1 192.168.10.2 port1
AN(config)#ha group fip 1 192.168.100.2 port3
AN(config)#ha group priority unit1 1 200
AN(config)#ha group priority unit2 1 100
AN(config)#ha group priority unit3 1 50
AN(config)#ha group priority unit4 1 150
AN(config)#ha group preempt on 1
AN(config)#ha group enable 1
AN(config)#ha group id 2
AN(config)#ha group fip 2 192.168.10.3 port1
AN(config)#ha group fip 2 192.168.100.3 port3
AN(config)#ha group priority unit1 2 50
AN(config)#ha group priority unit2 2 200
AN(config)#ha group priority unit3 2 100
AN(config)#ha group priority unit4 2 150
AN(config)#ha group preempt on 2
AN(config)#ha group enable 2
AN(config)#ha group id 3
AN(config)#ha group fip 3 192.168.10.4 port1
AN(config)#ha group fip 3 192.168.100.4 port3
AN(config)#ha group priority unit1 3 100
AN(config)#ha group priority unit2 3 50
4. (Optional) Execute the following command to configure health check conditions, taking the
health check for gateway and CPU utilization as examples.
6. (Optional) Execute the following commands to set the configuration synchronization mode:
AN(config)#ha log on
8. Execute the following commands to enable the HA function and save the HA-related
configurations to the memory:
AN(config)#ha on
AN(config)#write memory
APV2:
1. Execute the following commands to configure HA units, synchronization peers and links:
AN(config)#ha on
Once the HA function is enabled, unit2 (APV2) will join the HA domain and start to synchronize
configuration information from unit1 (APV1).
APV3:
1. Execute the following commands to configure HA units, synchronization peers and links:
AN(config)#ha on
Once the HA function is enabled, unit3 (APV3) will join the HA domain and start to synchronize
configuration information from unit1 (APV1).
APV4:
1. Execute the following commands to configure HA units, synchronization peers and links:
AN(config)#ha on
Once the HA function is enabled, unit4 (APV4) will join the HA domain and start to synchronize
configuration information from unit1 (APV1).
6.1 Overview
A single system image (SSI) domain is a group of APV appliances that appears to be one single
system. Logically, the multiple appliances in SSI deployment are regarded as one appliance.
The SSI feature allows the administrators to provide one virtual service by deploying multiple
parallel APV appliances, to meet higher performance demand.
Note: The SSI feature and HA feature cannot be deployed at the same time.
Note: The SSI feature only supports SLB reverse proxy mode
As shown in SSI Work Flow, the data flow in SSI deployment is:
2. The switch1 forwards the data packets to an APV appliance (because the APV appliances in
the SSI domain have the same virtual service IP address and floating MAC configuration, the
data packets will be sent to certain APV appliance according to certain algorithm of the
switch, hash based on the source IP and port, for example).
3. The APV appliance forwards the data packets to the real servers through switch2.
5. The switch2 sends the data packets to the APV appliance (the data packets can be returned to
the APV appliance originally performed the processing, because the APV appliances have
different IP pools for proxy IP).
As shown in the above picture, the following configurations need particular attention:
Add the switch1 ports connecting with the APV inbound ports in a bond.
Add the switch2 (intranet switch) ports connecting with the APV1 outbound ports in a bond.
Add the switch2 ports connecting with the APV2 outbound ports in a bond.
Note: The SSI feature only supports SLB reverse proxy mode. The inbound ports of the
SSI units can only serve as service port instead of management port, that is they cannot
work for SSH link establishment, WebUI link establishment or system upgrade. If separate
management for each unit is required, please assign unique management port to each unit.
The failover support for the switch between the client and the SSI domain provided by the switch
failover function allows the virtual service can be switched by the SSI units to the standby switch,
when the current switch undergoes electricity or connection interruption, thus guaranteeing the
service continuance.
The current switch and the standby unit have the same port configuration, including bond
configuration, LACP protocol enabling and flow distribution setting.
Add the APV ports connecting with the switches into two floating IP groups, that is, add
port1 and port2 of APV1 and APV2 into group1, and add port3 and port4 of APV1 and
APV2 into group2.
Switch configuration
Bond and interface IP configuration, and heartbeat interface configuration to include the
APV appliances into one broadcast domain
SLB configuration
SSI configuration
Note: The SSI function requires that configuration on each APV appliance except proxy
IP be identical.
Switch configuration
– Connect port1 and port2 of APV1 to switch1, and port3 and port4 to switch2.
– Connect port1 and port2 of APV2 to switch1, and port3 and port4 to switch2.
– Add the switch1 ports connecting with port1 and port2 of APV1 and APV2 into a bond.
– Configure flow distribution (for example, hash based on the source IP and port) on the
bond of switch1.
– Add the switch2 ports connecting with por3 and port4 of APV1 into a bond.
– Add the switch2 ports connecting with por3 and port4 of APV2 into a bond.
#APV1
AN(config)#bond interface bond1 port1
AN(config)#bond interface bond1 port2
AN(config)#bond interface bond2 port3
AN(config)#bond interface bond2 port4
AN(config)#ip address bond1 100.8.1.1 255.255.255.0
AN(config)#ip address bond2 192.168.1.1 255.255.255.0
#heartbeat configuration for SSI domain
AN(config)#ip address port5 172.16.1.1 255.255.255.0
#APV2
AN(config)#bond interface bond1 port1
AN(config)#bond interface bond1 port2
AN(config)#bond interface bond2 port3
AN(config)#bond interface bond2 port4
AN(config)#ip address bond1 100.8.1.1 255.255.255.0
AN(config)#ip address bond2 192.168.1.2 255.255.255.0
#heartbeat configuration for SSI domain
AN(config)#ip address port5 172.16.1.2 255.255.255.0
#APV1
AN(config)#slb real tcp "r1" 192.168.1.100 80 65535 tcp 3 3
AN(config)#slb real tcp "r2" 192.168.1.200 80 65535 tcp 3 3
AN(config)#slb group method "slb_g1" rr
AN(config)#slb group member "slb_g1" "r1" 1 0
AN(config)#slb group member "slb_g1" "r2" 1 0
AN(config)#slb virtual tcp "v1" 100.8.1.20 80 arp 0
AN(config)#slb policy default "v1" "slb_g1"
AN(config)#ip pool p1 192.168.1.3 192.168.1.3
#APV2
AN(config)#slb real tcp "r1" 192.168.1.100 80 65535 tcp 3 3
AN(config)#slb real tcp "r2" 192.168.1.200 80 65535 tcp 3 3
AN(config)#slb group method "slb_g1" rr
AN(config)#slb group member "slb_g1" "r1" 1 0
AN(config)#slb group member "slb_g1" "r2" 1 0
AN(config)#slb virtual tcp "v1" 100.8.1.20 80 arp 0
AN(config)#slb policy default "v1" "slb_g1"
AN(config)#ip pool p1 192.168.1.4 192.168.1.4
AN(config)#slb proxyip group slb_g1 p1
AN(config)#ha group id 1
AN(config)#ha group port 1 bond1 port1
AN(config)#ha group port 1 bond1 port2
#configuring the priority of a specified floating IP group on all units in the SSI domain
AN(config)#ha group priority ssi 1 100
AN(config)#ha group enable 1
AN(config)#ha floatmac on
AN(config)#ha floatmac mac bond1 40:aa:bb:cc:dd:ee
AN(config)#monitor network gateway APV_1 192.168.1.30 GATEWAY_1 1000 3 3
AN(config)#monitor network gateway APV_2 192.168.1.30 GATEWAY_1 1000 3 3
AN(config)#ha decision rule "GATEWAY_1" Group_Failover 1
AN(config)#ha on
Switch configuration
Bond and interface IP configuration, and heartbeat interface configuration to include the
APV appliances into one broadcast domain
SLB configuration
SSI configuration
Switch configuration
– Connect port1 and port2 of APV1 to the current switch, and port3 and port4 to the
standby switch.
– Connect port1 and port2 of APV2 to the current switch, and port3 and port4 to the
standby switch.
– Add the current switch ports connecting with port1 and port2 of APV1 and APV2 into a
bond.
– Add the standby switch ports connecting with port3 and port4 of APV1 and APV2 into a
bond.
– Enable the LACP protocol on the bond of the current and standby switches.
– Configure flow distribution (for example, hash based on the source IP and port) on the
bond of the current and standby switches.
– Add the intranet switch ports connecting with port5 and port6 of APV1 into a bond.
– Add the intranet switch ports connecting with port5 and port6 of APV2 into a bond.
#APV1
AN(config)#bond interface bond1 port1
AN(config)#bond interface bond1 port2
AN(config)#bond interface bond1 port3
AN(config)#bond interface bond1 port4
AN(config)#bond interface bond2 port5
AN(config)#bond interface bond2 port6
AN(config)#ip address bond1 100.8.1.1 255.255.255.0
AN(config)#ip address bond2 192.168.1.1 255.255.255.0
#heartbeat configuration for SSI domain
AN(config)#ip address port7 172.16.1.1 255.255.255.0
#APV2
AN(config)#bond interface bond1 port1
AN(config)#bond interface bond1 port2
AN(config)#bond interface bond1 port3
AN(config)#bond interface bond1 port4
AN(config)#bond interface bond2 port5
AN(config)#bond interface bond2 port6
AN(config)#ip address bond1 100.8.1.1 255.255.255.0
#APV1
AN(config)#slb real tcp "r1" 192.168.1.100 80 65535 tcp 3 3
AN(config)#slb real tcp "r2" 192.168.1.200 80 65535 tcp 3 3
AN(config)#slb group method "slb_g1" rr
AN(config)#slb group member "slb_g1" "r1" 1 0
AN(config)#slb group member "slb_g1" "r2" 1 0
AN(config)#slb virtual tcp "v1" 100.8.1.20 80 arp 0
AN(config)#slb policy default "v1" "slb_g1"
AN(config)#ip pool p1 192.168.1.3 192.168.1.3
AN(config)#slb proxyip group slb_g1 p1
#APV2
AN(config)#slb real tcp "r1" 192.168.1.100 80 65535 tcp 3 3
AN(config)#slb real tcp "r2" 192.168.1.200 80 65535 tcp 3 3
AN(config)#slb group method "slb_g1" rr
AN(config)#slb group member "slb_g1" "r1" 1 0
AN(config)#slb group member "slb_g1" "r2" 1 0
AN(config)#slb virtual tcp "v1" 100.8.1.20 80 arp 0
AN(config)#slb policy default "v1" "slb_g1"
AN(config)#ip pool p1 192.168.1.4 192.168.1.4
AN(config)#slb proxyip group slb_g1 p1
AN(config)#ha group id 1
AN(config)#ha group port 1 bond1 port1
AN(config)#ha group port 1 bond1 port2
AN(config)#ha group priority ssi 1 200
AN(config)#ha group enable 1
AN(config)#ha group id 2
AN(config)#ha group port 2 bond1 port3
AN(config)#ha group port 2 bond1 port4
AN(config)#ha group priority ssi 2 100
AN(config)#ha group enable 2
AN(config)#ha floatmac on
AN(config)#ha floatmac mac bond1 40:aa:bb:cc:dd:ee
AN(config)#monitor network gateway APV_1 192.168.1.30 GATEWAY_1 1000 3 3
AN(config)#monitor network gateway APV_2 192.168.1.30 GATEWAY_1 1000 3 3
AN(config)#ha decision rule "GATEWAY_1" Group_Failover 1
AN(config)#ha on
7.1 Overview
SLB (Server Load Balancing) allows you to distribute load and traffic to specific groups of servers
or to a specific server. The APV appliance supports server load balancing in Layers 2-7 of the OSI
network model. Layer 2 SLB is based on network interfaces. Layer 3 SLB works on IP addresses.
Layer 4 SLB is mostly concerned with port based load balancing. Layer 7 is used when you want
to perform load balancing based on URLs, HTTP headers or Cookies. The basic steps for setting
up SLB are:
The real server, the VIP and the virtual service are the fundamental components of SLB
deployment.
The real server is an application server hosting varied applications or services. It processes
the requests from the client side.
The VIP in general is a public IP address that can be accessed from the external clients. As an
entrance, it receives and forwards external requests, and sends the processed results from the
real servers back to the client side.
For the Layer 4 and Layer 7 SLB, the virtual service is commonly represented with a
VIP/port pair and can be accessed by the external clients to get their target network resources.
For example, if a client wants to access some Web resources by a predefined VIP or a Web
site name (with DNS), all the requests from this client will go through the VIP and be sent
out to different real servers by the APV appliance hosting the VIP and real servers. With the
virtual service, the internal network architecture and backend real servers are hidden from the
external clients by only exposing the VIP address.
The remainder of this chapter will cover these steps and cover some examples of Layer 2, Layer 3,
Layer 4 and Layer 7 load balancing strategies.
This following figure is a logical overview of load balancing using the APV appliance.
Additional Load Balancing methods include: Persistent URL (pu), Persistent Hostname (ph), Hash
Cookie (hc), Hash Header (hh), SSL SID (sslsid), Hash IP (hi), Consistent Hash IP (chi),
Consistent Hash RADIUS User Name (radchu), Consistent Hash RADIUS Session ID (radchs),
and persistence (Individual Session Persistence).
For more information on these additional SLB methods, please consult the Array APV CLI
Handbook.
Different types of policies have different priorities. Currently, multiple APV appliance SLB
policies can be configured for one SLB virtual service and the APV appliance will route requests
based on the first matched policy with the highest priority. The following is the order of priority
that APV appliance SLB will follow:
Priority Policy
a Redirect
b Static
c DNSSEC
QoS Clientport
QoS Network
Persistent URL
Rewrite Cookie
Insert Cookie
Persistent Cookie
QoS Cookie
d1
QoS Hostname
QoS URL
QoS Body
QoS Dnsdomain
Regex
Header
Hash URL
RADIUS Username
d2 RADIUS Session ID
Filetype
h Default
i Backup
For Layer 4 SLB, only three SLB policies (QoS Network, QoS Clientport, and QoS Dnsdomain)
can be used besides the Static, Default and Backup policies.
For backward compatibility, the default policy precedence is the same as before. To configure
specific precedence and associate it with specific virtual service, the system administrator can use
the following CLI commands:
SLB Policy Nesting is the mechanism for different policies to be nested to perform server load
balancing.
In traditional SLB policy mechanisms, one policy is defined to bind a virtual service with a group.
For example, in the command “slb policy qos url policy1 vs1 group1 ‘news’ 1”, the virtual
service vs1 and group1 are associated by policy1. The requests that match the policy1 will be
forwarded to group1.
In SLB Policy Nesting mechanism, a new object “vlink” is defined for policy nesting. A virtual
service or a vlink can be associated with a group or another vlink via policies. The requests will be
distributed to a group or a vlink according to the policies. For a request sent to a vlink, the system
will distribute the request to a group or another vlink associated to the vlink according to the
policies. Therefore, the requests will be distributed to a group associated to a vlink only when they
match two or more nested policies.
Some policies cannot be nested, such as static, redirect, filetype and external policies.
The icookie, rcookie, persistent cookie and persistent URL policies can only associate a
virtual service or a vlink with a group, but cannot be associated to a virtual service or a vlink
with another vlink.
To simplify the configuration, the policy nesting layer should be less than or equal to 3. If
more than 3 layers are configured, the system will return a 503 error.
The persistence method maintains a mapping table to keep track of each session ID and its
associated real server. Furthermore, the persistence method dynamically processes the timeout
information for each session ID to ensure independence of each session.
ip: This session ID type is the client IP address that the persistence method extracts from the
client request.
ip+port: This session ID type is the client IP address and port number that the persistence
method extracts from the client request.
string: This session ID type is a specified string that the persistence method either extracts
from the client request (or real server response) itself or obtains through extraction by an
existing SLB policy. You may also specify an offset and ID length for more granular control
of this session ID type.
When used independently, the persistence method can directly extract the following session ID
types from the client request or real server response:
Client IP address or IP address and port number from the client request
A specified string from the HTTP URL query, HTTP cookie, HTTP header, or HTTP body
field in the client request (configured by the “slb group persistence request” command).
A specified string from the HTTP cookie, HTTP header, or HTTP body field in the server
response. In this case, you also need to configure the APV appliance to obtain the matching
specified string from the HTTP URL query, HTTP cookie, HTTP header, or HTTP body
field in the client request (configured by the “slb group persistence response” command).
Note: When multiple session IDs of the string type are configured, the APV appliance will
obtain the session ID in priority-descending order from the HTTP URL query, HTTP
cookie, HTTP header, and HTTP body.
In collaboration mode, the persistence method can indirectly obtain the session ID through the
following types of SLB persistence policies:
Header
Persistent cookie
Persistent url
Qos body
Note: For details about the Session ID matching mechanism with the SLB persistence
method and SLB persistence policy, please contact Array Customer Support for related
documents.
For more information about the collaboration of the persistence method (configured by the “slb
group method <group_name> persistence” command) with existing SLB persistence policies,
refer to example “Persistence Method Collaborating with an SLB Persistence Policy”.
The persistence method can dynamically processes the timeout information of each session ID
using one of the following modes:
idle: If the APV appliance receives no new client requests within the specified “idle” time
period, the APV appliance will terminate the client’s session and clear its associated session
ID.
duration: If the specified “duration” of time has passed since a client’s session was created,
the APV appliance will terminate the client’s session and clear its associated session ID
regardless of the arrival of any new requests.
Note: To dynamically manage the timeout information of session IDs, you need to specify
the management mode to ensure that the APV appliance clears outdated session IDs and
records and new ones.
In addition, the persistence method can statically reserve a session ID (configured by the “slb
group persistence session static” command). Requests matching a statically reserved session ID
will be consistently sent to the same real server.
Basic health checks determine the application, service or server availability (Up/Down) status
using a specified protocol format. APV supports basic health check types that include ARP, ICMP
(ping), TCP, TCPS, DNS, HTTP and HTTPS. The default health check is assigned with the
individual real server (for example, representing any backend physical or virtual server in the
server farm) configuration. APV will perform basic health checks on the IP/port pair of the real
server to decide the real service availability. This is also called main health check.
Many times application or service infrastructure components are spread across multiple server
types that need to be run simultaneously. For example, an application might require Web servers,
application servers, and database servers to be run simultaneously. In this case determining the
health of one server is not sufficient, and health checks must be performed on the entire suit of
servers for determining the health of entire application infrastructure. In addition to the basic
health check, multiple diverse health checks can be configured for one real server. The AND or
OR relationship for multiple additional health checks can be set. APV supports configuring
additional health check name, which helps administrators easily configure and identify the
additional health check. See the “health relation” command.
There are many applications and non-standard protocols (in addition to standard TCP/IP based
protocols) that follow specific request/response sequence for communication purposes. To support
custom application availability check, customers can make use of scripted health check. APV
offers a generic application aware health check that runs over a TCP or UDP connection. It
determines an application’s health by exchanging messages with the application in the specified
format. Multiple application messages, request/response sequences can be scripted for emulating
normal application communication between APV and applications.
Two health check types are available, “script-tcp” and “script-udp” for building generic script
health checks. Script health checks support the following advanced application health checks: FTP,
SMTP, LDAP, RADIUS, POP3, DNS, and TELNET applications. Additional application health
check can be built by importing application request and response into the APV appliance.
The group health check is used to determine the health status of each real service of the associated
group. The group health check includes ICMP, TCP, TCPS, HTTP and HTTPS health check. In
application, administrators can configure the group health check to obtain the status of each real
service without configuring health check for each real service, which simplifies the configurations
of the health check.
Besides, the group health check supports the Server Load Balancing in the open cloud
environment. For more details about how to configure the group health check in the open cloud
environment, contact Array Customer Support for related documents.
Note: The relationship of the basic health check and the group health check is “AND”.
Health Check allows the APV appliance to perform diagnostic observations on the performance of
Web servers and Web server farms associated with each APV appliance. These observations on
the performance health of the real server will allow the appliance to know how the servers are
performing and which backend servers can best handle the incoming client requests.
It is a limited health check method that simply sends an ICMP echo (ping) to the server. If the
server responds with an ICMP reply then the server is marked as “up”. The server is marked as
“down” otherwise. This does NOT check for the running service or the quality of the service.
TCP health check simply opens a TCP connection to a specific port of the real server. If that
connection fails, the server will be marked as “down”. The server will be marked as “up” if the
TCP connection succeeds. This health check does not indicate if the service is actually functioning.
For checking if a particular service in question is functioning correctly, a specific health check
must be used (for example, HTTP health check for Web applications).
For Layer 2 SLB, the TCP health check method needs to configure a health check reflector on
another APV appliance. The reflector will open a port and listen upon the port, and responds to
health check requests. In this way, the health check process can go through the real server and
check the entire link status. If the reflector is able to respond to the health check request, the real
server will be marked as “UP”, else marked as “DOWN”.
TCPS health check provides an SSL health check for SLB real servers. If the SSL handshake fails,
the server will be marked as “down”. If the SSL handshake succeeds, the server will be marked as
“up”. This health check function will check for the availability of the real service by opening an
SSL connection to a specific port of the real server or defaults to 443.
The basic built-in HTTP health check opens a TCP connection and sends an HTTP request with
one of the HTTP methods (such as GET, POST, etc.) pre-defined in the health request table. The
APV appliance expects the health response as defined in the response table. The default index
chosen to reference request/response table is 0. If the response is not satisfied with the conditions
configured in the response table, the server will be marked as “down”, otherwise it is marked as
“up”.
HTTPS Health Check provides an SSL health check for real servers. If the SSL handshake
succeeds, the APV will send a pre-defined HTTP request with proper method format to real
servers. If the response from the real server is the same as the expected response, the real server
will be marked as “up”; otherwise, it is marked as “down”. When using HTTPS Health Check,
users should pre-define HTTP requests/methods and corresponding expected responses for
matching purposes. When using client certificates, the imported client certificate must be encoded
by DER rules during client authentication.
Script-TCP and script-UDP are provided for the generic script health check. They run over TCP
and UDP connection respectively. Only when the “hc_type” is set to these two types, the health
check list can work while doing health check.
Script-TCPS Health Check provides an SSL health check for HTTPS real servers. It works the
same way as script-TCP Health Check once the SSL handshake succeeds.
DNS health check is one of the built-in application health checks for DNS service that uses
scripted health check. DNS health check opens a UDP connection and sends a DNS request to a
destination DNS server, and the APV expects a special DNS response. The DNS request and
response are not configurable because they are unchangeable. If the required conditions are
satisfied, then server will be marked as “UP”, otherwise, the server will be marked as “DOWN”.
Radius-Auth health check and Radius-Acct health check are provided for checking the availability
of the RADIUS servers.
APV supports health check on commonly used LDAP servers like Windows AD, OpenLDAP and
SunOne Directory, to better meet the customers’ needs for health check on LDAP binding and
search operations.
The LDAP additional health check is only supported for TCP real servers.
RTSP health check opens a TCP connection and sends an RTSP “OPTIONS” (Get available
methods on the streaming server) request to a RTSP real server. If the real server responds with
any of the RFC-defined RTSP status codes, then the server will be marked as “up”, otherwise, it
will be marked as “down”.
SIP health check opens a UDP or TCP connection and sends a SIP “OPTIONS” request to a real
server. This request is used to ask the SIP server for the list of SIP methods it supports. The
response may contain a set of capabilities (i.e. audio/video codecs) of the responding SIP server. If
the real server responds with RFC-defined methods, the server will be marked as “up”, otherwise,
it will be marked as “down”.
Database health check checks the health status of real database servers by using a series of health
checks between the APV appliance and the real database servers. The APV appliance first
establishes a TCP connection with the real database server, and then it logs into the real database
server and send a database query to the real database server. If the real database server responds
with an expected response, it will be marked as “up”; otherwise, it will be marked as “down”.
Currently, the system supports two types of database health checks: MySQL and MsSQL. MySQL
database health check includes MySQL-DB and MySQL-DBS, and MySQL-DBS is an SSL-based
MySQL database health check. MsSQL database health check includes MsSQL-DB and
MsSQL-DBS, and MsSQL-DBS is an SSL-based MsSQL database health check.
When the method of health check is set as script_tcp or script_udp, HC checker and HC checker
list can work while the APV appliance does health check. An HC checker is defined as one
transaction of health check. It consists of sending one message and receiving one response. A list
of HC checkers can compose an HC checker list, which is identified by the HC checker list name.
Below are the commands used to define the HC checker and HC checker list:
By default ArrayOS defines an HTTP health table of HTTP requests and HTTP responses to be
used by the HTTP health check. The default index inside the health table for HTTP requests and
responses is “0, 0”. The “show health request” command shows the table of requests defined.
Likewise “show health response” shows the responses.
AN(config)#health on
AN(config)#show health request
Row Request
0 HEAD / HTTP/1.0
1 HEAD / HTTP/1.0
2 HEAD / HTTP/1.0
3 HEAD / HTTP/1.0
4 HEAD / HTTP/1.0
5 HEAD / HTTP/1.0
6 HEAD / HTTP/1.0
7 HEAD / HTTP/1.0
8 HEAD / HTTP/1.0
9 HEAD / HTTP/1.0
10 HEAD /HTTP/1.0
11 HEAD / HTTP/1.0
12 HEAD /HTTP/1.0
13 HEAD /HTTP/1.0
14 HEAD / HTTP/1.0
15 HEAD /HTTP/1.0
AN(config)#show health response
Row Response:
0 200 OK
1 200 OK
2 200 OK
3 200 OK
4 200 OK
5 200 OK
6 200 OK
7 200 OK
8 200 OK
9 200 OK
10 200 OK
11 200 OK
12 200 OK
13 200 OK
14 200 OK
15 200 OK
Index 0,0 in the health request table results with the following request being sent to the real server:
HEAD / HTTP/1.0
200 OK
You may change the response header to accommodate your network. Refer to your Web server’s
documentation on what valid HTTP responses you may use.
By default, we use a HEAD request as the health check. A HEAD request will only ask for the
HEADER information for the object being requested. If we were to use the GET request, you will
get the entire page back as a response. If you have a large site or content that is fairly large you
should choose the HTTP response that will cause the least amount of overhead.
You can define your own HTTP requests and the responses to be used by the HTTP health check.
For example, you may simply change the request to get a CGI script that returns an HTTP status
200 OK when the database server is up and a 404 NOT FOUND when the database server is
“down”.
Below are the commands to use to perform this task as well as an example from our network
model.
The health check provided by the ArrayOS starts with simple TCP health checks and allows you
to perform advanced health checks by utilizing different health requests and responses.
HTTP health check supports keyword matching for the specific real server response Web page.
The HTTP health check daemon will check the real server’s health by searching a keyword in the
server’s response content. If the keyword is found, this health check is successful. Otherwise, this
health check fails.
3. Associate the configured HTTP health check with the real service.
AN(config)#health server rs 1 1
Note: Keyword HTTP health check can only support ASCII string search in the Web page,
but not support double-byte keyword (for example, simplified Chinese, traditional
Chinese).
The TCP-based SLB virtual service health check supports external devices to detect the
availability of TCP-based virtual services defined in APV appliances.
How it works
If all real services corresponding to TCP-based SLB virtual service go down, the status of the
virtual service will be marked as “DOWN” and the TCP connection attempting from outside will
NOT be responded to so that the other devices will know the unavailability of the detected virtual
service.
Note: If WebWall is turned on for the interface (“port2” for most cases) that SLB real
services are using, you will need to add an access list rule to allow traffic between the
APV appliance and the real backend servers. If you also have health check turned on for
SLB real services, you will need to add corresponding access list rules to allow the health
check replies. For example, if the health check type is ICMP, it needs to add the
corresponding access list rules to allow ICMP echo reply messages. Otherwise, the SLB
real services will fail.
If all real servers configured in an APV appliance are marked as DOWN by Health Check, other
APV appliances will take over the traffic. As long as at least one real server configured in an APV
appliance is marked as UP by Health Check, the APV appliance will take over the traffic again if
its mode is preemptive.
Transmission
For transparent transmission, the APV appliance will direct the connections toward specified real
servers directly when forwarding requests from clients. That is, the server is aware of the client’s
IP address and therefore knows which client accesses the server. In transparent mode, the source
IP stays unchanged, while the source port might change. If administrators want to keep the source
port unchanged under transparent mode, the difference value between virtual service port number
and real service port number must be a multiple of APV CPU core numbers.
2. The APV appliance switches the destination address of the request into real server IP.
3. The APV appliance locates the optimal server available based on the policy and health check
conditions, and then forwards the request to the server.
4. After receiving the request, the server sends the response to the APV appliance.
5. The APV appliance switches the source address into virtual IP;
Structure/router design requires responses from source servers to pass through the APV
appliance.
In the one-armed structure, transparent transmission through APV appliance is not applicable
when the client is at the same network segment with the real server.
Because requests have different IPs, the system performance cannot be enhanced through the
connection pool.
Reverse proxy transmission is used to forward requests to internal servers. The APV appliance can
distribute the requests evenly among internal servers to ensure server load balancing. As the name
of reverse proxy mode indicates, the reverse proxy mode enables clients to access internal servers,
whereas the normal proxy transmission enables clients to access external servers. The following
figure shows how reverse proxy transmission works:
2. The APV appliance locates the optimal server based on the policy and health check
conditions, and then forwards the request to the server.
The IPs of the client that have accessed the server cannot be recorded.
Solution
The APV appliance can insert a field “X-Forwarded-For” into the header of the HTTP
request of the client to trace the client IP.
manner. In addition to “reverse” and “transparent” modes, a new system mode “triangle” is added
for this new feature.
For Triangle Transmisison, when selecting a proper real server from a group, administrators can
use Round Robin (rr), Persistent IP (pi), Hash IP (hi), Consistent Hash IP (chi), Least connections
(lc) and SNMP (snmp) group methods. In triangle mode, only TCP, UDP and IP virtual services
are supported.
1. Client sends a request to a virtual IP on the APV appliance via the router.
3. The real service returns a response to the router directly. Since the default route IP on the real
service is set to be port2 interface IP of the router, the response will be sent to the router
directly.
In this packet flow, the request will pass through the APV appliance, but the response will be sent
from the real server to the client directly without hitting the APV appliance.
Note: Triangle transmission SLB health check is based on the system IP addresses of the
real servers, not the loopback IP addresses. This means when health check is up, the real
service might not be available.
applications), this method might cause uneven load balancing. To solve this problem, packet based
UDP SLB can be used to support single UDP packet based application and more evenly spread
load to different real servers.
APV SIP Load Balancing intelligently distributes and balances SIP traffic among multiple SIP
servers and provides application persistence based on the unique SIP caller ID to ensure
application and transaction integrity. Additionally, to ensure reliability and availability of SIP
services, SIP load balancing is able to perform advanced health checks on SIP devices, routing
SIP clients away from unstable or unreliable devices. SIP load balancing is critically important for
successful, scalable VoIP telephony environments.
SIP Load Balancing performs stateful inspection of SIP messages to scan and hash calls based on
a SIP Call-ID header destined for a SIP server. Stateful inspection means that a packet is inspected
not only for its source and destination information found in the header, but also packet contents
found at Layer 7 (the Application Layer). Once the APV appliance has identified the Call-ID
which identifies a specific SIP session, it sends future messages from the same Call-ID to the
same SIP server.
A SIP proxy server is implemented to NAT the session packets originated from inside real servers.
By SIP NAT, real servers can reside in a private network and do not have to own global IP
addresses.
Note: A SIP proxy server is the server controlling the management of connections and IP
addresses in a SIP-enabled network..
To synchronize SIP registration information, SIP SLB supports the broadcasting of SIP
registration requests to all the SIP register servers in the same SLB group.
RTSP (Real Time Streaming Protocol) is an application layer protocol which is used for real-time
media traffic. Normally, an RTSP session includes two channels: control channel for control
signals and data channel for media stream. The control channel is a TCP connection while the data
channel can be either TCP or UDP connection.
The use of streaming audio and video is growing among enterprises for applications such as
eLearning and corporate communications. RTSP streaming delivers higher performance, and is
more secure and easier to manage than HTTP streaming implementations. APV appliance enables
companies to optimize streaming media resources by intelligently and transparently switching
requests to RTSP media servers or caches.
RTSP Load Balancing only supports filetype, default, backup and static policy. In filetype policy,
the real service is selected based on the file extension names. For example, client request for
“rtsp://mp3.xyz.com/test.mp3” will hit the RTSP SLB group corresponding to “mp3” filetype.
Particularly, RTSP Load Balancing supports two working modes: “REDIRECT” and “Dynamic
NAT”.
Redirect mode
The media stream will not pass through APV appliance. After APV appliance retrieves
Request-URL from the client request, it will select a real service according to the file type,
and then send a “REDIRECT” response to inform the client to access the selected real
service.
Media stream will pass through APV appliance. After APV appliance selects a real service
according to the policy, it will retrieve the media transport information from the negotiation
between the client and the real service. And then implement dynamic NAT for media
streaming packets according to their requirement. All connections (TCP and UDP) of one
RTSP session will be closed when the control connection tears down.
Different from higher layer SLB (Layer 4 and Layer 7 SLB), the traffic to be balanced in Layer 2
does not need to access a particular IP address or “IP address + Port” pair defined in the interface
(normally the port1 interface) on APV appliance for balancing purpose. As long as the incoming
traffic can be routed to APV appliance’s interface with defined Layer 2 virtual services (for
example, virtual services are defined on APV appliance’s port1 interface and APV appliance port1
interface’s system IP address is set as the gateway of client machines), it will be balanced among a
pool of Layer 2 real services according to configured load balance algorithms.
Layer 2 SLB feature has its own definitions for virtual service, real service, group method, health
check and policy:
Virtual service is defined by IP address. Internally, the associated input interface will be
found.
Real service can be defined by either IP or MAC address. Internally, the associated output
interfaces will be discovered.
Layer 2 SLB Health Check: Layer 2 SLB real services support all the health check methods
available in Layer 4/Layer 7 SLB health check. Only the additional health check can be
configured to the Layer 2 SLB real services.
The supported groups methods are rr (Round Robin) and hi (Hash IP).
Cross-port applications: some session protocols bind multiple connections together, one is the
initial connection, others could be generated from dynamic ports.
Cross-protocol applications: most streaming protocols use UDP connection for data
transferring and TCP connection to transfer control information between the same client and
server.
Port range virtual service has lower priority than Layer 4 and Layer 7 virtual services. That means,
if an input request hits both a Layer 4/Layer 7 virtual service and a port range virtual service, the
Layer 4/Layer 7 virtual service will be matched first.
Port range real service and static port real service can not be configured in one group. Port range
real service and port range group can only be associated with port range virtual service. However,
port range virtual service can associate with static port real service. Port range SLB does not work
for “FTP” type real services and virtual services.
The health check type of a port range real service can only be “none” or “ICMP” because its port
is 0. If other health checks are needed, additional health checks can be used:
Terminal Services Session Directory Service is a database that keeps track of sessions on terminal
servers in a load-balanced farm. The database maintains a list of the user names that are associated
with the session IDs that are connected to the servers in a load-balanced terminal server farm. It
can either reside on a server that is separate from the terminal servers in the farm, or be hosted on
a member of the terminal server farm.
When a client sends an authentication request to APV appliance, APV appliance will forward the
request to one of the terminal servers in the farm, for example TS1. TS1 prompts the client with a
login screen. After the client enters the user name and password, TS1 validates the user name and
password, and queries Session Directory database with the user name. If a session with the same
user name already exists on one of the terminal servers in the farm, for example TS2, Session
Directory will send the information (including TS2’s IP address and port number) to TS1, and TS1
will send a routing token with the TS2’s IP address and port to the client and drop the connection
with the client. After that, the client sends APV appliance another authentication request with the
routing token. APV appliance will reconnect the client to TS2 according to the IP address and port
in the routing token.
7.2.14 DirectFWD
DirectFWD is a new Layer 4 SLB function by utilizing a multi-thread and non-lock architecture
based on a multi-core system. This new architecture has maximized the advantage of the
multi-core system. Compared with the traditional Layer 4 SLB, DirectFWD provides remarkably
better Layer 4 SLB performance. This function is controlled by the command “slb direcfwd
{on|off}”.
Since only limited functions are implemented in the new architecture now, the DirectFWD
function still has some limitations as follows:
It is only used for TCP SLB, and temporarily only supports static, default and backup
policies. All the Layer 4 SLB methods, except Shortest Response Time, are supported.
(Please refer to the policy-method supporting matrix to check the details about the Layer 4
SLB methods).
It cannot be used in different MTU environments. In other words, all the interfaces must have
the same MTU; otherwise, the connection may fail to handle the big packets.
Note:
The more the interfaces used for DirectFWD, the better the performance.
DirctFWD is not supported in Triangle Transmission SLB.
DirectFWD Syncache
DirectFWD syncache effectively and efficiently protects the real servers from SYN flood DOS
attacks. When DirectFWD syncache function is on, all the SYN packets from a client will not be
forwarded to a real server directly. The APV appliance will hold the useful information in those
SYN packets firstly, and then send a SYN-ACK packet to the client. After that, the APV appliance
will receive an ACK packet from the client and setup a TCP connection with the real sever.
Note: When the DirectFWD syncache function is on, clients cannot negotiate MTU with
real servers.
Activation
By default, when a real service is disabled or deleted, the APV appliance SLB shall not send
session requests to the real service that has been disabled. However, for the real services using
cookie-based group method and load balancing polices, such as pc (Persistent Cookie), ic (Insert
Cookie), rc (Rewrite Cookie), and the session ID-based and generally-applied persistence method,
SLB will still send the existing session requests that match the cookie to the disabled real service
to ensure service persistence. While the new session requests will be sent to other working real
services. This function is called “Graceful Shutdown”, as shown below.
After disabling the real service named “service”, users can check the status of the real service by
using the command “show statistics slb real”.
After a while, users can run the command “show statistics slb real” again to check the status of
the real service.
As shown in the above output information, the status of “service” now is displayed as
“INACTIVE(suspend)”, which means it is shut down completely.
After a real service is enabled, it might receive a large number of requests immediately before
getting fully prepared to work. The sudden increase of traffic on the real service might lead to
failure of the service. To solve this problem, the APV appliance supports warm-up activation of
real services. Administrators can set recovery time and warm-up time for real services.
Right after a real service is enabled, no connection requests will be sent to the real service for a
period of time. This period is called “recovery time”. When the recovery time ends, the real
service enters the “warm-up time”. In this period of time, firstly only a small amount of
connection requests are forwarded to the real service for processing. The number of the requests
will be increased gradually. When it reaches the maximum number allowed for the real service, it
means the real service works normally.
Administrators can use the command “show statistics slb real” to check the status of a real
service. As shown in the following output information, the status of the real service in “recovery
time” is displayed as “UP(softup)”. No connection requests will be sent to a real service in
“softup” status.
Balancing
This section covers more than one SLB configuration example. At first, we will give an example
with basic SLB configuration. Then more configuration examples are given based on the different
policies. Herein we use HTTP protocol as an example. The configurations of other protocols are
similar.
Before you start to configure SLB, you need to get familiar with the following SLB network
architecture.
Operation Command
slb real tcp <real_service> <ip> <port> [max_connection] [hc_type]
[hc_up] [hc_down]
slb real ftp <real_service> <ip> [port] [max_connection] [hc_type]
Configure real
[hc_up] [hc_down]
services
slb real http <real_service> <ip> [port] [max_connection] [hc_type]
[hc_up] [hc_down]
slb real udp <real_service> <ip> <port> [max_connection] [hc_up]
Operation Command
[hc_down] [timeout] [hc_type]
slb real https <real_service> <ip> [port] [max_connection] [hc_type]
[hc_up] [hc_down]
slb real tcps <real_service> <ip> <port> [max_connection] [hc_type]
[hc_up] [hc_down]
slb real dns <real_service> <ip> [port] [max_connection] [hc_type]
[hc_up] [hc_down] [timeout]
slb real health <add_hc_name> <real_service> <ip> <port> [hc_type]
[hc_up] [hc_down]
slb group method <group_name> [rr|pu|sr]
slb group method <group_name> hc [first_choice_method]
[threshold_granularity]
slb group method <group_name> ic [cookie_name] [path_attribute]
[first_choice_method] [threshold_granularity]
slb group method <group_name> rc [cookie_name] [offset]
Define group
[first_choice_method] [threshold_granularity]
methods
slb group method <group_name> lc [threshold_granularity] [yes|no]
slb group method <group_name> hh <header_name>
[first_choice_method] [threshold_granularity] [prefix] [delimiter]
slb group method <group_name> ec <cookie_name>
[first_choice_method] [threshold_granularity]
slb group method <group_name> lb [threshold_granularity]
slb group member <group_name> <real_service>
[weight|cookie_value|url_tag_value] [priority]
Add the real servers
slb health <health_name> [type] [interval] [timeout] [hc_up] [hc_down]
into the group
[http_method] [url_path] [expected_codes]
slb group health <group_name> <health_name>
slb virtual {http|https|dns|siptcp|sipudp} <virtual_service> <vip>
[vport] [arp_support] [max_connection]
slb virtual {tcp|tcps|udp} <virtual_service> <vip> <vport>
[arp_support] [max_connection]
Define virtual
slb virtual rtsp <virtual_service> <vip> [vport] [mode] [arp_support]
services
[max_connection]
slb virtual {ftp|ftps} <virtual_service> <vip> [vport] [max_connection]
slb virtual ip <virtual_service> <vip> [max_connection]
slb virtual l2ip <virtual_service> <vip> [gateway_ip]
slb policy default {virtual_service|vlink_name} {group_name|vlink_name}
slb policy static <virtual_service> <real_service>
Bind the group (or a slb policy qos url <policy_name> {virtual_service|vlink_name}
real service) to the {group_name|vlink_name} <url_string> <precedence>
virtual service slb policy qos cookie <policy_name> {virtual_service|vlink_name}
{group_name|vlink_name} <policy_string> <precedence>
slb policy persistent cookie <policy_name> {virtual_service|vlink_name}
Operation Command
<group_name> <cookie_name> <precedence>
slb policy qos hostname <policy_name> {virtual_service|vlink_name}
{group_name|vlink_name} <host_name> <precedence>
slb policy redirect <policy_name> <virtual_service> <group_name>
<redirected_from_host>
The first step in setting up your network architecture with the APV appliance performing SLB
tasks is to create and configure your real services. To define real services for our model, use the
following command:
When you define an HTTP real service with just the real service name and the IP address, it will
use the following default values:
Port: 80
For service2 we will define what kind of health check to use, HTTP in this case.
Now all five real services are defined for the APV appliance. To verify the configuration, you may
use the “show slb real http” command. You may either specify a service by supplying the real
name, or leave this field blank and view all of your configured real services.
For setting additional health check for the first real service:
So the real server “service1http” will be healthy only when the main health check (192.168.10.10,
80, tcp), and the two additional health check servers (192.168.10.10 80 http and 192.168.10.10
8080 http), are all reported as healthy.
Sometimes it becomes necessary to disable a real service for maintenance or for temporary shifts
in resource allocation. To disable a real service, use the “slb real disable” command:
To re-enable the real server just use the “slb enable” command.
By default, when a real service is disabled or deleted, the APV appliance SLB shall not send
session requests to the real services that have been disabled. However, for cookie-based group
method and load balancing polices: PC (Persistent Cookie), IC (Insert Cookie), RC (Rewrite
Cookie), SLB supports the “graceful shutdown” of real services.
Graceful Shutdown
If a real service is disabled when it is already in a cookie-based group, SLB will still send the
existing session requests which match the cookie to the disabled real service. The new requests
will be sent to other working real services.
For example, you can configure the graceful shutdown function as follows:
After the above configurations, the existing session requests will still be sent to r1.
2. Define groups.
Now that the real services have been defined, it is time to assign them to groups. A group is first
defined by using the “slb group method” command. Below is an example from our model.
Once the group method is defined, the method changes from rr, sr and lc methods switching to rr,
sr, lc, pi, hi, and chi methods by employing the “slb group method” command can work without
removing the configured method. For other method changes, the old group method has to be
removed firstly, and then a new one can be added.
We have just defined a group with the name of “rrgroup” and a metric of Round Robin.
Associate the group health check condition with the specified group.
Once we have defined a group we simply add the real service by using the “slb group member”
command:
A virtual service is an access point that will service requests for the content which a group is
designed for. For Layer 4 and Layer 7 SLB, a virtual service is just a (VIP, port) pair. A VIP
(virtual IP) is mostly a public IP address which can be accessed from external clients. For example,
if group1 is a set of image servers, then we could define a VIP of 10.10.0.10 that is tied to group1.
Any requests made to this Virtual IP will be passed to either the Cache or SLB subsystem
depending on your cache and SLB settings. In essence you are hiding your internal architecture by
only exposing one IP and not many. Sometimes, the word “VIP” in this chapter is used for “virtual
service”.
Note:
interface on the appliance (except the 0.0.0.0 and noarp cases). For the VIP address
that does not match the subnet of any interface, the APV appliance will not allow it
to be configured.
If a VIP address is not tied to a policy the client will get a 503 Service Unavailable
response from the APV appliance. 503 will also be returned when all real servers are
down in a group.
We now have the IP address 10.10.0.10, listening for requests on port 80. Next we must define a
policy so incoming requests will be directed to the proper group.
5. Define policies.
The final step is to define a default policy to bind a virtual service to a Layer 4 “group”.
This command sets the default policy for a request on virtual1http to be serviced from rrgroup. We
now have Layer 4 load balancing using the round robin metric, live on the APV appliance. You
should be able to test the configuration by typing the HTTP URL: “http://10.10.0.10/” in a Web
browser.
QoS URL
Using QoS URL, we can setup policies that will look into the URL string and make a decision
based upon the information housed within that string. For our next example, we will setup two
groups. Group 1 will service all requests with ‘.jpg’ in the URL while Group 2 will service all
requests with ‘english’ in the URL.
Members:
S1.sj.example.com
S2.sj.example.com
Members:
S3.sj.example.com
S4.sj.example.com
We do not have to redefine the real servers and virtual service so we will leave them as they were
originally defined in the basic SLB configuration example above.
2. Add the real services into the groups (the real services are defined in the last example).
3. Set the policies and the URL associated with each group (the virtual service is defined in the
last example).
If we receive a request that has neither .jpg or ‘english’ in the URL, then the APV appliance will
returne a 503 Service Unavailable since it does not know how to handle the request. This is fine if
it is guaranteed every URL will always contain one of the strings. If not, it is best to define the
default policy. In our case we are just going to direct any request that does not contain one of the
strings to go to the “english” servers belonging to group 2.
QoS Cookie
QoS cookie allows you to add policies to the Layer 7 rules to load balance requests to a group
based on a cookie name and value pair. The following figure shows the QoS cookie in action. We
are going to define our two groups, define our cookies, and then setup the policies to make
cookie-based load balancing work within our model network.
Since we are using QoS cookie, which works on top of the existing groups, we define the groups
as would with Layer 4 rules.
Members:
S1.sj.example.com
S2.sj.example.com
Members:
S3.sj.example.com
S4.sj.example.com
Members: S5.sj.example.com
If there is a Cookie sent with the request that has the name “Service” and value “Gold”, then
go to Group 1.
If there is a Cookie sent with the request that has a name “Service” and value “Silver”, then
go to Group 2.
Group3 will be our cookie setters. If a client has never been to a site then the request will not
contain a cookie since it is the server that sets the cookie for the first time. These types of requests
will not match Group1 or Group2 and will be served through Group3, the default. After the client
has been to the Web site through Group3 and the cookie has been set by the server, the next time
the client accesses the Web site the browser will send the cookies in the HTTP request headers.
These cookies will ensure we pick the appropriate service from Group1 or Group2.
Note: If we do not have the default policy, the rule will “drop through” and the APV
appliance will return a 503 service unavailable
Now that we have defined what needs to happen, let’s configure the appliance. We do not have to
redefine the real services so we will leave them as they have been originally defined in the basic
SLB configuration example.
3. Set the default policy for group “cookie_set_group”, where cookies will be set.
4. Set the QoS Cookie policy for group “gold_group” and “silver_group”.
Persistent Cookie
Using persistent cookies, you can set cookie names and values and tie them to specific real servers.
This is different from QoS cookie in which the requests go directly to a server. And it so happens
that the configuration of persistent cookie is completely different.
Group 1
Group 2
Definition of the real services has been already completed. Now let’s proceed to setting up the
cookies, groups and then the policies.
Note: Persistent Cookie must be used with either Hash Cookie or Persistent Cookie group
balancing methods.
3. Set the default policy for “group2”, where cookies will be set.
QoS Hostname
QoS hostname based load balancing makes decisions on the host name within the URL. This is
also known as Virtual Hosting. This setup is similar to QoS cookie, but we are going to use
hostname policies instead of qos cookie policies.
In the following figure we have three groups. In our example, c-one.example.com refers to
“customer one” while c-two.example.com refers to “customer two”. Any other host name that is
used to access the VIP will go to the default policy.
Members:
S1.sj.example.com
S2.sj.example.com
Members:
S3.sj.example.com
S4.sj.example.com
Members: S5.sj.example.com
Note: QoS hostname policy can be used with any balancing method except Persistent
Cookie and Persistent URL.
By default, we want “www_group” to service any requests that do not have “c-one.example.com”
or “c-two.example.com” as their host names.
Persistent URL
Persistent URL works by looking for a string which has the format: tag=value. This is typically
used within a URL when the browser is submitting a form to the server. This way you can set
persistence to the backend server by the value of a parameter being passed to the CGI script. For
example a URL of the form:
http://www.example.com/find_user.pl?username=bob
This will match our Layer 7 policy and direct the request to server1. The following configuration
example shows you how to set up a persistent URL policy.
Note: Persistent URL policy must be used with a persistent URL (pu) and hash query (hq)
group.
Insert Cookie
Insert Cookie (ic) is a method that allows users to insert a cookie into the server response in order
to maintain the persistency between the clients and servers.
1. Create an SLB group and configure Insert Cookie algorithm for it.
Note: Insert Cookie policy must be used with an Insert Cookie (ic) group.
4. Set up the SLB policies to associate the group with virtual services.
By default you do not need to define a cookie name. The system will generate a random cookie
name and save it in the configuration file. You can view the cookie name by running the “show
slb group method” command, as follows:
If you want to set the name of the cookie, just add the name by using the command “slb group
method <group_name> ic [cookie_name] [add_path] [rr|sr|lc] [threshold]”. For example:
Now the cookie “CookieExampleName” will be inserted into the responses from the real services.
The value will be used by the APV appliance to determine which real service to keep persistent to.
For example:
CookieExampleName=server1http
Rewrite Cookie
Rewrite cookie allows us to rewrite a section of a cookie value to make sure that the cookie gets
sent back to the same server. We will not strip out the modifications that the ArrayOS has made.
So the backend server will see the modified cookie.
Note: For insert cookie, we will not strip out the modifications that the ArrayOS has made
too. So the backend server will see the inserted cookie now.
Note: Rewrite Cookie policy must be used with Rewrite Cookie group method and Embed
Cookie group method.
Then we use the rcookie policy to bind the virtual service to the group:
Now the cookie, CookieExampleName, will be rewritten with the service name. For example the
name will look like the following if the response comes from service1http:
CookieExaampleName=valueabcdefghijk
CookieExampleName= service1http!?ijk
Note: The length of cookie value has to be not less than the length of real service name
+2. Otherwise the rewrite operation will not be performed.
Redirect
A Redirect policy is applied to the URL host in incoming HTTP requests. Redirect policy allows
users to redirect the client’s HTTP request from one host to another host. By doing this, all the
HTTP requests are balanced to different real services and the persistency is kept by the new host
name at the same time.
In our example, the requests for the homepage “http://www.abc.com/index.htm” from the client
will be redirected to be “http://www1.abc.com/index.htm”, “http://www2.abc.com/index.htm” or
“http://www3.abc.com/index.htm” depending on the configured group method (rr).
1. Define the real services that HTTP requests will be redirected to.
2. Create a group.
4. Use the “slb policy redirect” command to associate an existing virtual service with the
group.
Currently, the Redirect policy supports rr (Round Robin), lc (Least Connection), sr (Shortest
Response Time) and prox (Proximity) load balancing methods.
In this section, the example is a one-arm case. The default Gateway of two servers is APV
appliance (i.e. 172.16.30.170). The server subnet (VLAN 30) and client subnet (VLAN 10) are
connected by router 172.16.30.1.
Operation Command
slb real siptcp <real_service> <ip> [port] [max_connection] [hc_type]
Configure real [hc_up] [hc_down]
services slb real sipudp <real_service> <ip> [port] [max_connection] [hc_type]
[hc_up] [hc_down] [timeout]
slb group method <group_name> {sipcid|sipuid} [first_choice_method]
[threshold_granularity]
slb group method <group_name> {rr|pu|sr}
slb group method <group_name> lc [threshold_granularity] [yes|no]
Define group slb group method <group_name> pi [hash_bits] [first_choice_method]
methods [threshold_granularity]
slb group method <group_name> hi [hash_bits]
slb group method <group_name> chi [hash_bits]
slb group method <group_name> snmp [snmp_mode] [community]
[oid_count] [oid1] [oid1_weight] [oid2] [oid2_weight] [check_interval]
Add the real
slb group member <group_name> <real_service>
servers into the
[weight|cookie_value|url_tag_value] [priority]
group
slb virtual {http|https|dns|siptcp|sipudp} <virtual_service> <vip> [vport]
[arp_support] [max_connection]
slb virtual {tcp|tcps|udp} <virtual_service> <vip> <vport> [arp_support]
Define virtual
[max_connection]
services
slb virtual rtsp <virtual_service> <vip> [vport] [mode] [arp_support]
[max_connection]
slb virtual {ftp|ftps} <virtual_service> <vip> [vport] [max_connection]
Bind the group to slb policy default {virtual_service|vlink_name} {group_name|vlink_name}
the virtual slb policy static <virtual_service> <real_service>
service slb policy backup {virtual_service|vlink_name} {group_name|vlink_name}
2. Create a group for SIP load balancing by using the “slb group method” command.
Then you can define the SIPUDP virtual services by using the “slb virtual siptcp” or “slb virtual
sipudp” command.
If the backend servers do not share database, turn on the multi-register function.
AN(config)#sip multireg on
To handle network traffic originated from real servers, you need to set the SIP NAT rules for the
defined SIP real services.
Configuration Guidelines
In our example, the client sends a request “rtsp://10.5.1.80/test.mp3” to virtual service “vs_rtsp1”
(10.5.1.80). APV appliance chooses a real service according to some policy and method. In
redirect mode, APV appliance responds the client with the chosen real server’s URL
“rtsp://audio2.array.com:554/test.mp3”. The APV appliance and the client get disconnected, and
the client begins to communicate with the real server “audio2.array.com:554” In this mode, all the
real servers should have public IP addresses which can be accessible from Internet clients.
Operation Command
Configure real slb real rtsp <real_service> <ip> [port] [max_connection] [hc_type]
services [hc_up] [hc_down] [timeout]
slb group method <group_name> {rr|pu|sr}
slb group method <group_name> lc [threshold_granularity] [yes|no]
slb group method <group_name> pi [hash_bits] [first_choice_method]
Define group [threshold_granularity]
methods slb group method <group_name> hi [hash_bits]
slb group method <group_name> chi [hash_bits]
slb group method <group_name> snmp [snmp_mode] [community]
[oid_count] [oid1] [oid1_weight] [oid2] [oid2_weight] [check_interval]
Add the real servers slb group member <group_name> <real_service>
into the group [weight|cookie_value|url_tag_value] [priority]
slb virtual {http|https|dns|siptcp|sipudp} <virtual_service> <vip>
[vport] [arp_support] [max_connection]
slb virtual {tcp|tcps|udp} <virtual_service> <vip> <vport>
Define virtual
[arp_support] [max_connection]
services
slb virtual rtsp <virtual_service> <vip> [vport] [mode] [arp_support]
[max_connection]
slb virtual {ftp|ftps} <virtual_service> <vip> [vport] [max_connection]
slb policy default {virtual_service|vlink_name} {group_name|vlink_name}
slb policy static <virtual_service> <real_service>
Bind the group to slb policy backup {virtual_service|vlink_name}
the virtual service {group_name|vlink_name}
slb policy filetype <policy_name> <vs_name> <group_name >
<filetype>
1. Define RTSP real services by using the command “slb real rtsp”.
When the virtual service mode is “Redirect”, the real service name should be “real IP[:port]” or
“domainname[:port]”.
We can use rr (Round Robin), pi (Persistent IP), hi (Hash IP), chi (Consistent Hash IP), snmp
method to choose RTSP real service in one group.
The default mode of the RTSP virtual service is “Redirect”, and the port is 554.
If you add default policy, you will choose that group when you can not find available real services
by filetype policy.
Configuration Guidelines
In NAT mode, all the RTSP control messages will be balanced to multiple backend media servers
across the APV appliance. Packets originated from backend media servers (normally the media
data) will be NATTed to outside clients. Different from redirect mode, the real servers do not have
to use public IP addresses. The internal private IP addresses will be translated into global IP
address on APV appliance.
Operation Command
Configure real slb real rtsp <real_name> <ip> [port] [max_conn]
services [rtsp-tcp|tcp|icmp|script-tcp|script-udp|dns] [hc_up] [hc_down] [timeout]
slb group method <group_name> {rr|pu|sr}
slb group method <group_name> lc [threshold] [yes|no]
slb group method <group_name> pi [hash_bits] [rr|sr|lc|lb] [threshold]
Define group
slb group method <group_name> hi [hash_bits]
methods
slb group method <group_name> chi [hash_bits]
slb group method <group_name> snmp [weight|cpu] [community]
[oidcount] [oid1] [oidweight1] [oid2] [oidweight2] [check_interval]
Add the real servers
slb group member <group_name> <real_name> [weight]
into the group
slb virtual {http|https|dns|siptcp|sipudp} <virtual_name> <vip> [vport]
[arp|noarp] [max_conn]
slb virtual {tcp|tcps|udp} <virtual_name> <vip> <vport> [arp|noarp]
Define virtual
[max_conn]
services
slb virtual rtsp <virtual_name> <vip> [vport] [mode] [arp|noarp]
[max_conn]
slb virtual {ftp|ftps} <virtual_name> <vip> [vport] [max_conn]
slb policy default {virtual_name|vlink_name} {group_name|vlink_name}
Bind the group to slb policy static <virtual_name> <real_name>
the virtual service slb policy backup {virtual_name|vlink_name} {group_name|vlink_name}
slb policy filetype <policy_name> <vs_name> <group> <filetype>
1. Define RTSP real services by using the command “slb real rtsp”.
We can use rr (Round Robin), pi (Persistent IP), hi (Hash IP), chi (Consistent Hash IP), and snmp
method to choose RTSP real service in one group.
Configuration Guidelines
Before we show how to set this up, we should describe the relative concepts in our system.
Let’s begin to setup the environment for firewall load balance. We will describe several different
cases.
To make Layer 2 SLB work, the clients, servers and firewalls should have the default gateway or
some static route gateway configured as one of the APV appliance’s IP addresses so that the
traffic can be forwarded to the APV appliance.
For example, the following routes can be added for the clients, servers and firewalls respectively:
Client route:
Server route:
Firewall1 routes:
Firewall2 routes:
Note: We assume all the systems are Unix-alike. For Windows, different versions of route
commands may need to be applied.
Note: One real service can only be included by one real service group. Layer 3 real
service and Layer 2 real service can not be on the same interface.
Operation Command
Configure real slb real l2ip <real_service> <real_ip>
services slb real l2mac <real_service> <real_mac> <output_interface>
Define group
slb group method <group_name> {hi|rr|chi} [hash_bits]
methods
Add the real servers slb group member <group_name> <real_service>
Operation Command
into the group [weight|cookie_value|url_tag_value] [priority]
Define virtual
slb virtual l2ip <virtual_service> <vip> [gateway_ip]
services
Bind the group to
slb policy default {virtual_service|vlink_name} {group_name|vlink_name}
the virtual service
Define the
slb real health <add_hc_name> <real_service> <ip> <port> [hc_type]
additional health
[hc_up] [hc_down]
check
Configure reflector health ipreflect <reflector_name> <ip_address> <port> [protocol]
Then we begin to configure the left box APV1 according to the above figure.
On the other hand, we can use MAC address to define a real service as well.
Note: To get the MAC address, please use relative IP address command for your specific
system. For example, if you use Linux, then you can use the command “ifconfig -u” to get
the MAC address of NIC.
3. Define the group for the real service and add its members to the group.
Note: Layer 2 SLB supports three methods for the group: rr, hi and chi.
When the “slb group method” command is used to define a Layer 2 SLB group, a new parameter
is introduced as the last argument: “route mode”. This parameter is used to route a data packet
coming from a Layer 2 real service. Possible values for route mode are: direct and route. “direct”
the data packet from a Layer 2 real service will be sent out from the related Layer 2 virtual
service’s interface directly without bothering any route settings. On the contrary, if the route mode
is valued “route”, route settings will be used to send the data packet.
Here, we have finished the configuration on APV1 for the Layer 2 IP/MAC based SLB. Now we
will begin to configure APV2:
Configuration Guidelines
Same as two-arm scenario, new routes need to be configured in the client, server and firewalls for
packet forwarding:
Client route:
Server routes:
Firewall1 route:
Firewall2 route:
The general settings for single-arm network are the same as the settings for two-arm network.
The commands used to configure Layer 3 SLB are summarized in the following table:
Operation Command
Configure real slb real ip <real_service> <ip> [max_connection] [hc_type] [hc_up]
services [hc_down] [udp_timeout]
lb group method <group_name> {rr|sr}
slb group method <group_name> lc [threshold_granularity] [yes|no]
Define group
slb group method <group_name> pi [hash_bits] [first_choice_method]
methods
[threshold_granularity]
slb group method <group_name> hi [hash_bits]
Operation Command
slb group method <group_name> chi [hash_bits]
slb group method <group_name> snmp [snmp_mode] [community]
[oid_count] [oid1] [oid1_weight] [oid2] [oid2_weight] [check_interval]
Add the real servers slb group member <group_name> <real_service>
into the group [weight|cookie_value|url_tag_value] [priority]
Define virtual
slb virtual ip <virtual_service> <vip> [max_connection]
services
slb policy default {virtual_service|vlink_name} {group_name|vlink_name}
Bind the group (or
slb policy static <virtual_service> <real_service>
the real service) to
slb policy backup {virtual_service|vlink_name}
the virtual service
{group_name|vlink_name}
2. Define a group for Layer 3 load balancing by using the “slb group method” command.
You may associate the group or the real service to the virtual service of Layer 3 load balancing by
using the command “slb policy default” or associate the real service to the virtual service by the
command “slb policy static”.
The commands used to configure Port Range SLB are summarized in the following table:
Operation Command
slb real tcp <real_service> <ip> <port> [max_connection] [hc_type]
[hc_up] [hc_down]
slb real http <real_service> <ip> [port] [max_connection] [hc_type]
[hc_up] [hc_down]
slb real udp <real_service> <ip> <port> [max_connection] [hc_up]
Configure real [hc_down] [timeout] [hc_type]
services slb real https <real_service> <ip> [port] [max_connection] [hc_type]
[hc_up] [hc_down]
slb real tcps <real_service> <ip> <port> [max_connection] [hc_type]
[hc_up] [hc_down]
slb real dns <real_service> <ip> [port] [max_connection] [hc_type]
[hc_up] [hc_down] [timeout]
Define group
slb group method <group_name> [method]
methods
Add the real
slb group member <group_name> <real_service>
servers into the
[weight|cookie_value|url_tag_value] [priority]
group
slb virtual {http|https|dns|siptcp|sipudp} <virtual_service> <vip> [vport]
[arp_support] [max_connection]
slb virtual {tcp|tcps|udp} <virtual_service> <vip> <vport> [arp_support]
Define virtual
[max_connection]
services
slb virtual rtsp <virtual_service> <vip> [vport] [mode] [arp_support]
[max_connection]
slb virtual {ftp|ftps} <virtual_service> <vip> [vport] [max_connection]
Define the port
slb virtual portrange <virtual_service> <start_port> <end_port>
range of a virtual
[protocol] [port_type]
service
Bind the group
slb policy default {virtual_service|vlink_name} {group_name|vlink_name}
(or the real
slb policy static <virtual_service> <real_service>
service) to the
slb policy backup {virtual_service|vlink_name} {group_name|vlink_name}
virtual service
Herein we use HTTP protocol as an example. The configurations of other protocols are similar.
When the real services are port range services, the health check can only be “icmp” or “none”.
2. Define a group.
At most three port ranges can be defined for an SLB virtual service.
In the above example, all data packets with destination IP address to be “10.3.14.50” and port
falling into the range 80-90 or 8000-9000 will be handled by the port range virtual service
“vhttp0”.
Note: Port range real services and static real services can not be added into one group.
Associate the group to the port-range virtual service with the command “slb policy default” and
associate the real service to the port-range virtual service with the command “slb policy static”.
The commands used to configure Terminal Server Load Balancing are summarized in the
following table:
Operation Command
slb real rdp <real_service> <ip> [port]
Create RDP real services
[max_connection] [hc_type] [hc_up] [hc_down]
slb group method <group_name> rdprt
Create RDP groups
[first_choice_method]
slb group member <group_name> <real_service>
Add the real services into the group
[weight|cookie_value|url_tag_value] [priority]
slb virtual rdp <virtual_service> <vip> [vport]
Create RDP virtual services
[arp_support] [max_connection]
Associate the real server group with slb policy default {virtual_service|vlink_name}
virtual services {group_name|vlink_name}
Note: For the RDP real services, only the “icmp” and “tcp” types of health check can be
used.
The commands used to configure Policy Nesting are summarized in the following table:
Operation Command
slb real http <real_service> <ip> [port] [max_connection] [hc_type]
Configure real [hc_up] [hc_down]
services slb real https <real_service> <ip> [port] [max_connection] [hc_type]
[hc_up] [hc_down]
Define group
slb group method <group_name> [method]
methods
Add the real servers slb group member <group_name> <real_service>
into the group [weight|cookie_value|url_tag_value] [priority]
slb virtual {http|https|dns|siptcp|sipudp} <virtual_service> <vip>
[vport] [arp_support] [max_connection]
slb virtual {tcp|tcps|udp} <virtual_service> <vip> <vport>
Define virtual
[arp_support] [max_connection]
services
slb virtual rtsp <virtual_service> <vip> [vport] [mode] [arp_support]
[max_connection]
slb virtual {ftp|ftps} <virtual_service> <vip> [vport] [max_connection]
Create the vlink slb vlink <vlink_name>
slb policy default {virtual_service|vlink_name} {group_name|vlink_name}
slb policy static <virtual_service> <real_service>
slb policy icookie <policy_name> {virtual_service|vlink_name}
<group_name> <precedence>
Bind the group to
slb policy rcookie <policy_name> {virtual_service|vlink_name}
the virtual service or
<group_name> <precedence>
the vlink
slb policy persistent cookie <policy_name> {virtual_service|vlink_name}
<group_name> <cookie_name> <precedence>
slb policy persistent url <policy_name> {virtual_service|vlink_name}
<group_name> <url_tag> <precedence>
To help you better understand the example, the following graphic shows the logical relationship
among VIPs, vlinks and groups.
In this example:
If the source IP address of a request is within the sub network, it will match policy1 and go to
vlink1. Otherwise, the request will be distributed to group1.
If the request goes to vlink1 and there is “news” in the URL, it will match policy2 and go to
vlink2. Otherwise, the request will be forwarded to group2.
If the request goes to vlink2 and there is “sports” in the URL, it will match policy3 and go to
group3. Otherwise, the request will be sent to group 4.
Configuration purpose:
To implement session persistence by independently using the persistence method with the client
IP address as the session ID. Once configured, the APV appliance will:
Send the first access request from a client to the real server selected by the first choice
method (rr in this example).
Obtain and record the client IP address as the unique session ID.
Send subsequent requests from the client to the same real server. If the client remains idle
(for example, no new client requests are detected) for 5 minutes, the APV appliance will
terminate the client’s session and clear the session ID.
Prerequisites:
Layer 4 or 7 real services r1 and r2 are already defined. In this example, real services are of
the HTTP type.
A Layer 4 or 7 virtual service v1 is already defined. In this example, the virtual service is of
the HTTP type.
1. Execute the following command to configure the service group and persistence method:
For example:
2. Execute the following command to add real services to the service group:
For example:
3. Execute the following command to bind the service group and virtual service with the default
policy:
For example:
For example:
Configuration purpose:
To implement session persistence by independently using the persistence method with a specified
string (obtained from the HTTP response cookie) as the session ID. Once configured, the APV
appliance will:
Send the first access request from a client to the real server selected by the first choice
method (rr in this example).
Obtain the “mycookie” value from the real server response and record it as the session ID.
When receiving subsequent requests from the client, check whether the value of “telnum” in
the request URL Query matches with the session ID. If yes, send the request to the same real
server. If the client remains idle (for example, no new client requests are detected) for 5
minutes, the APV appliance will terminate the client’s session and clear the session ID.
Prerequisites:
Layer 4 or 7 real services r1 and r2 are already defined. In this example, the virtual service is
of the HTTP type.
A Layer 4 or 7 virtual service v1 is already defined. In this example, the virtual service is of
the HTTP type.
1. Execute the following command to configure the service group and persistence method:
For example:
2. Execute the following command to add real services to the service group:
For example:
3. Execute the following commands to obtain the value of “mycookie” in the response from the
real server as the session ID:
For example:
4. Execute the following command to bind the service group and virtual service with the default
policy:
For example:
For example:
Note: Before configuring the APV to implement session persistence based on the HTTP
cookie, you need to specify the cookie in the real service configurations to ensure that the
HTTP response can carry the corresponding cookie.
Persistence Policy
Configuration purpose:
Implement session persistence by applying both the persistence method and a Layer 7 SLB
persistence policy. In this case, the session ID is obtained through the policy.
The following table describes the configuration for the method and policy:
Item Configurations
The header policy is configured to obtain the content of
“x-up-calling-line-id” in the HTTP request header as the target
Policy
string.
The default policy is configured.
The first choice method is round robin (rr).
The session ID is of the string type.
Whether to obtain the session ID from the client request or
Persistence method
server response is not specified.
The offset and ID length for obtaining the session ID are
specified.
If the request from the client matches with the header policy, the APV appliance will obtain
the session ID based on the offset and ID length from the content of “x-up-calling-line-id”.
If the request from the client matches with the default policy, the APV appliance will use the
first choice method to direct the request to a real server and not implement session
persistence.
Prerequisites:
Layer 4 or 7 real services r1 and r2 are already defined. In this example, the real services are
of the HTTP type.
A Layer 4 or 7 virtual service v1 is already defined. In this example, the virtual service is of
the HTTP type.
1. Execute the following command to configure the service group and persistence method:
For example:
2. Execute the following command to add real services to the service group:
For example:
3. Execute the following command to bind the service group and virtual service with the header
and default policies:
For example:
4. Execute the following command to configure the offset and length for the session ID:
For example:
For example:
Priority
Virtual Real
SLB Type (1 is the Health check Scenarios
Service Service
highest)
service
Non-zero
port RS:
In addition to Layer 7
Layer 7
Layer 7 VS Layer 7 RS SLB, cross-port and
Port range health check
3 + Port Layer 7 RS dynamic port
(for Layer 7) Zero port
range (0 port) application traffic
RS:
balance is supported
ICMP
Additional
Non-zero
port RS:
In addition to Layer 4
Layer 4
Layer 4 VS Layer 4 RS SLB, cross-port and
Port range health check
3 + Port Layer 4 RS dynamic port
(for Layer 4) Zero port
range (0 port) application traffic
RS:
balance is supported
ICMP
Additional
In addition to port
range SLB,
cross-protocol
None
application traffic
Layer 3 4 IP IP ICMP
balance is supported.
Additional
Currently, only TCP
and UDP protocol are
supported
1. The backend real
services do not have
usable IP addresses so
that the traffic cannot
be balanced according
to IP addresses;
ARP 2. The backend real
IP + port
Layer 2 1 IP, MAC Additional services are not the
ranges
(only ICMP) destination of the
input traffic (for
example, virus
scanners check every
packet before
forwarding it to the
real destination).
8.1 Overview
This chapter will cover the concepts and strategies of using the Reverse Proxy Cache to better
enhance the overall speed and performance of your Web servers. Using the cache function will
improve website performance and throughput, and will also reduce server load by caching heavily
requested data in the temporary memory of the APV appliance.
1. The client sends a request to the APV appliance, requesting for a file on the Web server. The
APV appliance will forward the request to the cache module for processing.
2. If the requested content has been cached on the APV appliance and the cache does not expire
(cache hit), the APV appliance will send the file copy to the client directly without
forwarding the request to the Web server. If the requested content does not exist in cache
(cache miss), the request will be forwarded to the Web server for processing.
3. The Web server responds to the APV appliance with the requested content.
4. The APV appliance responds to the client with the requested content, and caches the content.
Future requests for the same content will be responded directly from the APV appliance
cache module.
The default behavior of the cache is to send the cached object to the client while the cache is being
filled with new objects.
The maximum size of the cache objects depends on different system memories of the APV
appliances. See the table below:
The Reverse Proxy Cache function is mainly featured with the following advantages:
Improved performance
– The cache function is an independent module in the ArrayOS. Turning it off will not
impact the functionality of other modules in the system.
– The cache module strictly controls its memory consumption under 25% of the total
system memory.
– The ability to cache both the compressed and uncompressed contents allows the APV to
send compressed contents to appropriate clients without having to involve the
compression engine. This greatly enhances compression throughput.
Outstanding stability
– If any error occurs to the cache module, administrators can turn off the module
immediately, which will not affect the running of other system functions.
– All cache tuning parameters now use the “cache filter” mechanism, and the global
control parameters are reduced. This new approach gives administrators more flexibility
and control and minimizes confusion during configuration.
Intelligent monitoring
– The APV process monitor also monitors the cache (in addition to the reverse proxy). If it
detects any issues (or if the cache process crashes), it will restart the cache after
appropriate cleanup.
Flexible configuration
– The statistics from “show statistics cache” are more detailed and are designed to allow
administrators to get data that would help them understand how the APV is making
caching decisions. This should help the customer tune the APV or their website to
optimize performance.
– The “cache filter” mechanism reduces global control parameters, which increases the
precision and flexibility of command control by administrators.
By default, if there are no HTTP headers that restrict caching for an object, the reverse-proxy
cache will cache the content. The following are the more popular cache-control headers that will
control if the content will be cached and if so, for how long.
Cache-Control: public
The public keyword indicates that any available and configured cache may store the content.
Cache-Control: private
This directive is intended for a single user and will not be cached by the reverse-proxy cache.
Cache-Control: no-cache
This directive lets the reverse-proxy cache know that it can cache the content and can only
use the cached content if the appliance first re-validates the content with the origin server.
This header tells the cache when the content will expire and when to re-fetch it from an
origin server when the request for that object is made. In this example it tells the cache to
make the content expire on Tuesday, 30 Oct 2010 14:00:00 GMT.
Any content with Cache-Control directives which allow caching of the content will always be
cached. If the content does not contain a Cache-Control directive, then it will always be cached
and will not be re-validated until it is manually flushed from the cache. If the content does not
contain an “Expires” header, after it expires, the APV appliance will re-validate the content with
the origin server and update the content when it is requested next time.
Content that has Cache-Control headers which restrict caching of the content will not be cached.
Responses to requests with cookies are not served from cache, unless configured by the user to do
so.
The following gives several application examples of cache filter. For detailed configuration
examples, refer to section 0 Three types of cache expiration time are involved during the cache
process:
Cache expiration time defined by the cache-related field (the Expires and Cache-Control
headers) in the HTTP responses
Cache expiration time specified by the “ttl” parameter in the “cache filter rule” command
1. The cache expiration time set by the “ttl” parameter in the “cache filter rule” command will
be preferentially used. If the “ttl” parameter is not specified, the cache expiration time
specified by the “cache settings expire” command applies.
2. For the cache content that does not match any cache filter rule, the expiration time defined in
the cache-related headers of the HTTP response applies.
3. If the cache expiration time is unavailable in the cache-related headers of the HTTP response,
the cache expiration time set by the “cache settings expire” command applies.
Reverse Proxy Cache Configuration and the Array APV CLI Handbook.
Cache all “*.jpg” files from the host “www.xyz.com”, and the TTL of cache contents is
200,000 seconds.
Only cache the image files in common formats, such as JPG, GIF or BMP.
For the cache objects from the same host, some can be cached by following the cache filter
rules, and others can be cached by following the definitions in the cache control header, such
as the TTL of the cache.
Ignore the specific type of query strings in the request URL when looking up for an object in
cache.
Cache expiration time defined by the cache-related field (the Expires and Cache-Control
headers) in the HTTP responses
Cache expiration time specified by the “ttl” parameter in the “cache filter rule” command
4. The cache expiration time set by the “ttl” parameter in the “cache filter rule” command will
be preferentially used. If the “ttl” parameter is not specified, the cache expiration time
specified by the “cache settings expire” command applies.
5. For the cache content that does not match any cache filter rule, the expiration time defined in
the cache-related headers of the HTTP response applies.
6. If the cache expiration time is unavailable in the cache-related headers of the HTTP response,
the cache expiration time set by the “cache settings expire” command applies.
Operation Command
Enable cache cache {on|off} [virtual_service]
View cache status show cache status
Configure global
cache settings expire <expire_time>
cache expire time
Configure the
maximum size for a cache settings objectsize <size>
cache object
View cache basic
show cache settings
settings
View cache statistics show statistics cache [virtual_service]
Clear cache statistics clear statistics cache [virtual_service]
Operation Command
View contents of
show cache content <host_name> <url_regex>
cache objects
Remove all cache
cache objects by clear cache content
force
Enable cache filter cache filter {on|off}
Configure cache
cache filter rule <host_name> <url> <cache_behavior>
filter rule
View cache filter
show cache filter status
configuration
View all cache filters
about the specified show cache filter hostname <host_name>
host name
View all cache filter
show cache filter all
rules
View the cache filter
rules matching the
cache filter match <host_name> <url_regex>
specified host name
and URL
Remove specified
no cache filter rule <host_name> <url>
cache filter rules
Clear cache filter
rules matched with clear cache filter hostname <host_name>
the specified host
Clear all cache filter
clear cache filter all
rules
View cache filter
show statistics cachefilter <host_name> <url_regex>
statistics
Clear cache filter
clear statistics cachefilter [host_name]
statistics
To use cache, we need to first enable the Cache function for the specified virtual service.
In this example, we enable the Cache function for the virtual service “virtual_MOSS”.
AN(config)#cache on virtual_MOSS
The current status of cache can be viewed by using the “show cache status” command.
We start to define basic cache rules for APV appliance to follow. The settings that can be
configured include:
The current Cache settings can be viewed by using the “show cache settings” command.
The above cache settings are the default values, which are the optimal values. If your application
has some special requirements, you can make the above cache settings as your needs determine.
To set the global cache expiration time, we can use the “cache settings expire” command. The
default value is 82800 seconds (23 hours). The global default expiration time will be used as the
expiration time for an object in cache only if it is impossible to calculate the expiration time using
the Expiration Model specified in Section 13.2 of RFC 2616.
To set the maximum size of an object in cache, the “cache settings objectsize” command should
be used. The command takes the size in kilobytes. The default value is 5120 KB. If the size of an
object being sent to the client is greater than the configured maximum object size, the object will
not be cached even if it is otherwise cacheable.
Now we use using the “show cache settings” command to view current cache settings:
First, enable the cache filter function by using the command “cache filter {on|off}
<virtual_service>”. By default, the cache filter function is disabled.
AN(config)#cache filter on
Then, define cache filter rules by using the command “cache filter rule <host_name> <url>
<cache_behavior>”. Cache filter rules conveniently controls whether to cache an object and how
long to cache it.
In our example, cache all “.jpg” objects from the host “www.xyz.com” and set the TTL to be
200,000 seconds:
To view all cache filter rules we have configured.We can execute the command “show cache
filter all”.
Once you’ve configured your cache functions, the ArrayOS will allow you to view the running
status of the appliance as it pertains to the caching requirements you’ve configured.
(Notice: the real server's time should be in sync with this machine.
Otherwise, the time difference could expire the cachable objects
resulting in low cache hit ratio.)
Advanced Statistics:
Number of cache objects: 0
Number of cache frames: 0
Successful cache probes: 0
9.1 Overview
The HTTP Content Rewrite feature allows end users to visit the HTTP contents on the Web
servers behind the APV appliance. This feature aims to reduce network latency and improve user
experience.
This chapter will cover the theories and configurations of the HTTP Content Rewrite feature.
The solution to the above problems is to rewrite contents of the HTML pages before it is sent to
the client. To be more precise, the contents of every HTML page will be processed and all the
HTML links will be found and rewritten so that the end user can access the pages via a browser. In
addition, the rewritten pages can be cached by the APV appliance so that APV does not need to
rewrite the same pages over and over again.
Upon receiving the responses from the real server, the APV appliance will communicate with the
uproxy module to read the response data that needs to be rewritten, and send these data to the
rewrite module for rewriting. The IP addresses and domain names in the files that need rewriting
according to configured rules will be rewritten. Finally the APV appliance responds the rewritten
data to the end user, and further caches the rewritten data.
1. The end user sends an HTTP request to the real server via the APV appliance.
2. The response data coming from the real server reaches the APV appliance.
4. The rewrite module reads the data that needs rewriting, and rewrites the data.
5. The APV appliance sends the rewritten data to the end user, and caches the data.
The HTTP Content Rewrite help external end users from visiting the internal real server
hidden behind the APV appliance. It rewrites the Web page file into the valid format to end
users. The APV appliance also supports to rewrite the multi-byte character files such as
Chinese and Japanese.
Comparing with the URL rewrite method, the HTTP Content Rewrite feature causes less
communication with the real server to reduce the effect to the performance.
In addition, the rewritten pages can be cached by the APV appliance so that APV does not
need to rewrite the same pages over and over again. When the client requests the real server
for the same Web page, the APV appliance will respond the client with the cached data to
decrease the response time.
Easy to maintain
The HTTP Content Rewrite feature helps enterprise decrease the cost of the human resource.
The administrator needs less monitoring because the HTTP Content Rewrite feature rewrites
the Web page file by the rewrite driver and the rewrite module automatically.
The administrator can enable/disable the HTTP content rewrite globally or per virtual service.
Note:
By default, the HTTP content rewrite function is disabled globally, while enabled per
virtual service.
Only with the global HTTP content rewrite enabled, will the per virtual service
HTTP content rewrite enabling and configurations take effect.
The HTTP content rewrite function allows administrators to define global rewrite rules to rewrite
the IP addresses, domain names or other strings in the Web page files into new strings as
pre-defined.
ProxyHTMLURLMap
Only rewrite the URLs inside the HTML tags. The other strings will not be rewritten. This kind of
rewrite rules can only be applied to rewriting of HTML and XHTML files.
For example:
<p><a href="10.3.129.1">10.3.129.1</a></p>
<p><a href="http://10.3.129.1/">http://10.3.129.1</a></p>
<p><a href="https://10.3.129.1/">https://10.3.129.1</a></p>
<p><a href="172.16.85.74">10.3.129.1</a></p>
<p><a href="http://172.16.85.74/">http://10.3.129.1</a></p>
<p><a href="https://172.16.85.74/">https://10.3.129.1</a></p>
The IP address “10.3.129.1” in the URLs inside the HTML tags has been rewritten into
“172.16.85.74”, while others remain unchanged.
Substitute
For example:
<p><a href="10.3.129.1">10.3.129.1</a></p>
<p><a href="http://10.3.129.1/">http://10.3.129.1</a></p>
<p><a href="https://10.3.129.1/">https://10.3.129.1</a></p>
<p><a href="172.16.85.74">172.16.85.74</a></p>
<p><a href="http://172.16.85.74/">http://172.16.85.74</a></p>
<p><a href="https://172.16.85.74/">https://172.16.85.74</a></p>
Note:
The configuration strings of the parameter “rule” must be framed in double quotes.
The configuration strings of “ProxyHTMLURLMap” and “Substitute” are strictly
case-sensitive.
When both “ProxyHTMLURLMap” and “Substitute” rules are configured, the
“ProxyHTMLURLMap” rules will be applied first, and then the “Substitute” rules. If
the “ProxyHTMLURLMap” and “Substitute” rules have been configured to map to
the same regex, the “Substitute” rules will overwrite the “ProxyHTMLURLMap”
rules.
To change the rewrite rules, the currently running rewrite operations will fail, and
APV will reset the related connections. Therefore, it is suggested not change the
rewrite rules while the APV appliance is processing network traffic.
If the HTTP content rewrite function is enabled and content rewrite rules are
configured, the length of each line in responses cannot be greater than 1MB;
otherwise, the APV appliance will send a RST packet to the client to abort the TCP
connection.
The HTTP Content Rewrite function allows administrators to specify the type of the files to be
rewritten. The following are the supported file types:
text/html
text/plain
text/richtext
text/xml
application/xml
application/xhtml+xml
text/css
text/javascript
application/javascript
Define the URL regex for per virtual service HTTP content rewrite
The administrator can define the URL regex to permit or deny rewriting of the files that match the
URL regex per virtual service.
To specify the URL regex, the administrator should first define a URL list, and then add URL
regexes into the URL list. The URL regex can be defined as the extension name, the file content or
a part of the file name. Then the administrator need to associate the URL list with a virtual service
through a permit/deny rule. After all these are done, the files that match any URL regex in the
URL list will be rewritten according to the associated permit/deny rules.
Multiple URL regexes can be added into a URL list, in “OR” relationship. That is to say, the
permit/deny rule will work as long as any of the extension name, file name or file contents
matches the URL regex.
The HTTP Content Rewrite function also supports rewriting the Web page files that contain
specific HTTP response status code. The “200” HTTP response status code is supported by default.
That is to say, the APV appliance will only rewrite the Web page files with the “200” HTTP
response status code by default.
As shown above, the real server is hidden behind the APV appliance, and the end user can only
access the IP address “172.16.85.74” of the virtual service “vs1”. If the end user wants to visit the
Web resource “a.doc” on the real server “10.3.129.1”, the source URL address
“http://10.3.129.1/a.doc” should be rewritten into “http://172.16.85.74/a.doc”.
Operation Command
Enable HTTP Content http rewrite body {on|off} [virtual_service]
Operation Command
Rewrite
Configure HTTP Content
http rewrite body rule <rule> [flags]
Rewrite rules
Configure the file type to be
http rewrite body mimetype <mime_type>
rewritten
Add a URL regex into a URL
http rewrite body url list <url_list> <url>
list
Configure to permit rewriting
the string that matches the http rewrite body url permit <virtual_service> <url_list>
URL regex in a URL list
Configure to deny rewriting
the string that matches the http rewrite body url deny <virtual_service> <url_list >
URL regex in a URL list
Configure to rewrite the files
that contain specific HTTP http rewrite body statuscode <status_code>
response status code
4. Configure an HTTP Content Rewrite rule to rewrite the IP address “10.3.129.1” into
“172.16.85.74”.
10.1 Overview
The DNS SLB mechanism used by APV appliance supports DNS cache feature. Upon receiving
any type “A” or “AAAA” DNS responses, which are mapping of host names to IP addresses, APV
will save them in SLB DNS cache. Then, when the APV appliance receives any DNS requests for
the cached “A” or “AAAA” records from clients, the appliance will directly send back the “A” or
“AAAA” responses to the clients. If there is no records in cache that hit the requests, the APV
appliance will communicate with the remote DNS server(s) directly, and then save the server
responses in cache for responding to the coming requests.
Operation Command
slb real dns <real_service> <ip> [port] [max_connection] [hc_type]
[hc_up] [hc_down] [timeout]
Define related SLB
slb virtual dns <virtual_service> <vip> [vport] [arp_support]
component
[max_connection]
slb policy static <virtual_service> <real_service>
Enable DNS cache dns cache {on|off}
Configure the DNS
dns cache expire <min_seconds> <max_seconds>
cache expiration time
Establish hosts for the
dns cache host <host_name> <ip>
DNS cache
Since DNS cache is interdependent with SLB configuration strategies, please refer to Chapter 7
Server Load Balancing (SLB). Below is a configuration example for DNS cache deployment. First,
the SLB component needs to be established.
The commands above set up an SLB configuration where the real service is named and bound to a
real IP address/port pair. This real service is then, in turn, bound to the configured virtual service
via the static policy. These commands are covered in depth in the CLI Handbook.
To enable DNS cache, the “dns cache {on|off}” command should be used. The DNS cache is
disabled by default.
AN(config)#dns cache on
11.1 Overview
The APV appliance supports in-line compression of HTTP objects. By employing this licensed
feature, administrators may maximize throughput to the desired site while end-users will
experience quicker download speeds. This chapter describes the configuration of HTTP
Compression capabilities which are part of ArrayOS platform. Configuration of HTTP
Compression functionality can be divided into two main parts. The first part is the basic
configuration and the second part is dedicated to advanced configuration.
By default, the following MIME types can be compressed by the APV appliance for all the
browsers:
Text (text/plain)
HTML (text/HTML)
XML (text/XML)
The following MIME types are able to be compressed by the APV appliance for certain browsers
via “http compression policy useragent” command:
DOC (application/MSWord)
Now, if Administrators do not want to compress some textual contents from Web servers to
browsers, they are able to configure a url-exclude HTTP compression rule for the client request
via the the “http compression policy urlexclude <virtual_service> <url>” command. If the
URL of the client request matches the rule, the textual content from the Web servers to the
browsers will not be compressed even if HTTP compression is on.
Not all files are suitable for compression. For obvious reasons, files that are already compressed,
such as JPEGs, GIFs, PNGs, movies, and bundled contents (for example, Zip, Gzip, and bzip2
files) are not going to be compressed appreciably further with a simple HTTP compression filter.
Therefore, you are not going to get much benefit from compressing these files or a site that relies
heavily on them.
Note: For the data files in very small size, the size of the compressed data may be larger
than the original data size.
Operation Command
Enable HTTP data
http compression {on|off} [virtual_service]
compression
View HTTP data
show http compression settings
compression status
Set advanced HTTP http compression policy useragent <user_agent_string> <mime_type>
compression http compression advanced useragent on
Set the url-exclude
http compression policy urlexclude <virtual_service> <url>
compression rule
AN(config)#http compression on
This command enables the HTTP Compression functionality with default settings. By default, the
APV appliance compresses the following MIME types for all the browsers:
Text (text/plain)
HTML (text/HTML)
XML (text/XML)
If you want to enable the compression of Java Script for Microsoft IE 5.5, you can enable it by
specifying the following parameters:
There are other types of Web contents that are compressible such as:
DOC (application/MSWord)
Not all browsers are able to process the compressed forms of these MIME types. Support for the
above MIME types requires detection of appropriate user agents that can deal with the compressed
forms of these types, and then apply the compression functionality to only those user agents. To
process variations in the handling of these MIME types by browsers, the APV appliance provides
the administrator with the capability to turn ON compression based on specific user agent and
MIME types.
Note: TEXT, XML and HTML of HTTP compression are default values, so they do not
need to be configured by the command “http compression policy useragent”.
Array Networks provides a tested list of browsers that can handle the compressed form of
additional MIME types. The APV appliance provides administrators with a way to enable the
compression of additional MIME types for a best-known-working-set of browsers by using the
following command:
It activates the compression of Java Script and CSS types for IE 6, IE 7, IE 8 and Mozilla 5.0
browsers.
If the URL of a client request to the virtual service “v1” matches the string “/abc”, the textual
contents in the response will not be compressed even if HTTP compression is turned on.
If the URL of a client request to the virtual service “v1” starts from the string “/def”, the textual
contents in the response will not be compressed even if HTTP compression is turned on.
If the URL of a client request to the virtual service “v1” ends with the string “ghi.txt”, the textual
contents in the response will not be compressed even if HTTP compression is turned on.
If the URL of a client request to the virtual service “v1” matches the string “abc*def”, the textual
contents in the response will not be compressed even if HTTP compression is turned on.
12.1 Overview
Now that the basic SLB and Caching are setup on the APV appliance, we can set up the SSL
(Secure Sockets Layer) acceleration functionality to provide secure transactions with your clients.
The SSL Accelerator works by decrypting the secure traffic and passing the unencrypted traffic to
the original server. In an alternative mode the SSL accelerator can be used to decrypt the secure
traffic, apply traffic management (SLB, caching, etc.) processing on decrypted traffic and then
encrypt it back before passing it to SSL enabled origin server.
12.2.1 Cryptography
SSL protects confidential information through the use of cryptography. Sensitive data is encrypted
across public networks to achieve a level of confidentiality.
There are two types of data encryption: secret key cryptography and public key cryptography.
Also known as symmetric cryptography. It uses the same key for encryption and decryption. An
example of symmetric cryptography is a decoder ring. Alice has a ring and Bob has the same ring.
Alice can encode messages to Bob using her ring as the cipher. Bob can then decode the sent
message using his ring. In cryptography, the "decoder ring" is considered a preshared key. The
key is agreed upon by both sides and can remain static. Both sides must know each other already
and have agreed upon what key to use for the encryption and decryption of messages.
It uses one key for encryption of data, and then a separate key for decryption. It is more favorable
than secret key cryptography because even if the encryption key is learned in one direction, the
third party still needs to know the other key in order to decrypt the message in the other direction.
In SSL handshake process, SSL virtual host and real host support the negotiation of RSA and ECC
signature algorithms and the validation of RSA-signed and ECDSA-signed digital certificates.In
Ephemeral Elliptic Curve Diffie-Hellman (ECDHE) key exchange section, the server will send the
ephemeral public key via the Server Key Exchange message which carries a digital signature. The
client will validate the digital signature to confirm the reliability and integrity of the public key.
In client certificate authentication section, the client will send a digital signature to the server via
the Certificate Verify message. This digital signature is to enable the server to confirm the identity
of client. The server will use the public key in the client certificate to validate the digital signature.
The following table lists the signature algorithms supported by SSL virtual hosts and real hosts.
Name Meaning
Indicates the certificate version number. Certificate format varies by
Version
versions.
Indicates the certificate serial number, which is unique among all
Serial Number certificates issued by an authority. If a certificate is revoked, its serial
number will be added to the Certificate Revocation List (CRL).
Indicates the authority that issues the certificate. Generally, a certificate
Issuer
is issued by a Certificate Authority (CA).
Valid from Indicates the valid start date of the certificate.
Signature algorithm Indicates the signature algorithm used to generate the signature.
Indicates the thumbprint and thumbprint algorithm. Thumbprint is a
Thumbprint,
hashed value of the entire certificate calculated by the thumbprint
Thumprint algorithm
algorithm and then encrypted by the private key of the CA.
To obtain a certificate from CA, individuals or enterprises need to send Certificate Signing
Request (CSR) to CA. The APV appliance supports generation of RSA and ECC CSRs for a
virtual host.
To apply for an RSA certificate, execute the “ssl csr” command to generate an RSA CSR.
To apply for an ECC certificate, execute the “ssl ecc csr” command to generate an ECC
CSR.
CA will send back the desired certificate via Email. After receiving the certificate, administrators
need to import and activate it using the “ssl import certficate” and “ssl activate certificate”
commands, whether it is an RSA or ECC certificate. The number of certificates that can be
imported and activated for a virtual host and a real host is different.
If it is associated with one or more domain names, you can import three RSA certificates and
three ECC certificates for each domain of the virtual host. Each domain of it can have one
active RSA certificate and one active ECC certificate.
If it is not associated with any domain name, you can import a maximum of three RSA
certificates and three ECC certificates for it. It can have one active RSA certificate and one
active ECC certificate.
For a real host, you can import three RSA certificates and three ECC certificates. It can have one
active RSA certificate and one ECC certificate.
Note: The APV appliance running FIPS HSM does not support ECC. In FIPS HSM
environment, each virtual host can have one active RSA certificate for each of its
associated domain. Each real host can have only one RSA certificate.
The following is an example for importing and activate an RSA certificate and an ECC certificate
with index 2 for real host “rhost”.
bg6YLHnpI+SdpoNwjsrd25K6AvCmBMANtqFighIwjcbNJqINGWWxZUIquHVmyD+L/53x2Ni
h4qJ5zjMfWzPbR7KbwF0cDJs1/YfMEwg==
-----END CERTIFICATE-----
...
PEM format.
Certificate import successful !!!
Warning: RSA certificate chain is incomplete for rhost. Please add interca or rootca certificate.
RSA Certificate #2 is activated successfully!
Warning: ECC certificate chain is incomplete for rhost. Please add interca or rootca certificate.
ECC Certificate #2 is activated successfully!
In client certificate authentication, a client sends its certificate to a server for authentication upon
receiving a Client Certificiate Request message from the server. The APV appliance supports
using the Array certificate parser (Array Networks patent) to verify the client certificate in a fast
way.
When the APV appliance functions as an SSL proxy client (SSL real host), if client
certificate authentication is enabled for the SSL real host, it will send its certificate to the real
service. In this scenario, if the SSL real host has both RSA and ECC certificates activated, it
will select the certificate to be sent according to the following principles:
– If RSA certificate type is specified in the Client Certificate Request message, the RSA
certificate will be sent to the real service.
– If only ECC certificate type is specified in the Client Certificate Request message, and
the negotiated cipher suite is of the “ECDHE…” type, the ECC certificate wil be sent to
the real service.
– If only ECC certificate type is specified in the Client Certificate Request message, but
the negotiated cipher suite is not an “ECDHE…” one, the SSL real host will reset the
connection.
When the APV appliance functions as an SSL proxy server (SSL virtual host), if client
certificate authentication is enabled for the SSL virtual host, it will request SSL client to
provide a certificate for authentication.
Configuration Example
Prerequisites:
A private key and the corresponding certificate have been imported for the specified SSL
virtual host, and the certificate has been activated.
CLI
1. Configure SLB.
Add the real service “rhost1” and “rhost2”. Associate the real service “rhost1” and “rhost2”with
group “vhg1” and “vhg2” respectively. Add the virtual service “vshost1”. To select the hit real
service based on the domain, configure QoS hostname policy for the group “vhg1” and “vhg2”.
2. Configure an SSL virtual host and associate with the virtual service “vshost1”.
Associate the domain “www.a.com” and “www.b.com” with the SSL virtual host.
For how to get the private key and certificate and other ways to import the private key and
certificate, please refer to section 12.3.2 Configuration Example via CLI.
WwU2hyREMjGi+6k5nq5gecnTSJSTk8q3ori+1jooOfgvRV72x/e3hGRFIysKb15J
1xeyUHFjtNJIHlEVjNg2XURXIRDO+qatI0cnvW1sYbuQodat0jGzpP/oOxGVcRsg
kTuP8xObpUou0RevvZVbfYHWsdWZnY5qhLjje1UGKUu8HwaMp3Q7OJ7pper7TOAX
CkiVn1zHgeFkB4p2lGwCHxbdNJg7ee5U2HFiSoC/8P8G64kIkEfxuMFwzkaVlia+
Anf40WlBcvixZaNZqHkZdcfJGlqKnqnpcJWiyLd8oPr3npYu6+c1PyHiyEua67nE
zK5G4KVFacW0POPLHfj/Q1yMqYcWySyecJvtq0jxYLujGH6qckLhMMltdptCfFvZ
6gfncRanNWEU+2v+nc509XKv9zJeR58WsAh81AN7aChuEBj69cjL4HwOtk3nttVm
pr5+cluF6mpbk2rJuZ14l+Hwc63XRAaHXV0bIQfo+rQBIWHkItvN4S2a0G23IC9N
32Y1qTqQgoj9qxSXvw1oKmV+UcVjtt78GyT3m7pYPtNc/wdehUPrR4Lc8hSiV3m2
9ZGIoNpdhVgCGw/MM65tr3dRZfYcG8cPJ4pk/3WKq9OfznlugAxo4KWebea51CaT
4YBMO5nETHT9WesX9viiKtbhPaAIKQOd4Rwcs5FK0ETHSfBBpW0tsXw562s6QnNS
mnWuoksvqqXqCbnuNSz611z3dOhiLwHdqCqLEDNEhv231ielOk1qoUo+ad0eiM0E
TrP6Zyw+j6i9K71sSe28Zm4gVQMnSHxGyp0K+zK3cYs=
-----END RSA PRIVATE KEY-----
...
PEM format.
Enter passphrase for the private key:
sbg6YLHnpI+SdpoNwjsrd25K6AvCmBMANtqFighIwjcbNJqINGWWxZUIquHVmyD+L/53x2
Nih4qJ5zjMfWzPbR7KbwF0cDJs1/YfMEwg==
-----END CERTIFICATE-----
...
PEM format.
Certificate import successful !!!
WwU2hyREMjGi+6k5nq5gecnTSJSTk8q3ori+1jooOfgvRV72x/e3hGRFIysKb15J
1xeyUHFjtNJIHlEVjNg2XURXIRDO+qatI0cnvW1sYbuQodat0jGzpP/oOxGVcRsg
kTuP8xObpUou0RevvZVbfYHWsdWZnY5qhLjje1UGKUu8HwaMp3Q7OJ7pper7TOAX
CkiVn1zHgeFkB4p2lGwCHxbdNJg7ee5U2HFiSoC/8P8G64kIkEfxuMFwzkaVlia+
Anf40WlBcvixZaNZqHkZdcfJGlqKnqnpcJWiyLd8oPr3npYu6+c1PyHiyEua67nE
zK5G4KVFacW0POPLHfj/Q1yMqYcWySyecJvtq0jxYLujGH6qckLhMMltdptCfFvZ
6gfncRanNWEU+2v+nc509XKv9zJeR58WsAh81AN7aChuEBj69cjL4HwOtk3nttVm
pr5+cluF6mpbk2rJuZ14l+Hwc63XRAaHXV0bIQfo+rQBIWHkItvN4S2a0G23IC9N
32Y1qTqQgoj9qxSXvw1oKmV+UcVjtt78GyT3m7pYPtNc/wdehUPrR4Lc8hSiV3m2
9ZGIoNpdhVgCGw/MM65tr3dRZfYcG8cPJ4pk/3WKq9OfznlugAxo4KWebea51CaT
4YBMO5nETHT9WesX9viiKtbhPaAIKQOd4Rwcs5FK0ETHSfBBpW0tsXw562s6QnNS
mnWuoksvqqXqCbnuNSz611z3dOhiLwHdqCqLEDNEhv231ielOk1qoUo+ad0eiM0E
TrP6Zyw+j6i9K71sSe28Zm4gVQMnSHxGyp0K+zK3cYs=
-----END RSA PRIVATE KEY-----
...
PEM format.
Enter passphrase for the private key:
-----BEGIN CERTIFICATE-----
MIICIzCCAcqgAwIBAgIFAI88UzkwCgYIKoZIzj0EAwIwgaoxCzAJBgNVBAYTAlVTMQswC
QYDVQQIDAJDQTERMA8GA1UEBwwITWlscGl0YXMxGzAZBgNVBAoMEkFycmF5TmV
0d29ya3MgSW5jLjEUMBIGA1UECwwLQVBWIFByb2R1Y3QxHjAcBgNVBAMMFXd3dy5h
cnJheW5ldHdvcmtzLm5ldDEoMCYGCSqGSIb3DQEJARYZc3VwcG9ydEBhcnJheW5ldHdvcm
tzLm5ldDAeFw0xNjA3MTEwOTMxMjdaFw0yNDA5MjcwOTMxMjdaMIGHMQswCQYDVQ
QGEwJDTjELMAkGA1UECAwCQ04xCzAJBgNVBAcMAkNOMQswCQYDVQQKDAJDTjE
LMAkGA1UECwwCQ04xCzAJBgNVBAsMAkNOMQswCQYDVQQLDAJDTjEOMAwGA1U
EAwwFdmhvc3QxGjAYBgkqhkiG9w0BCQEWC2FiY0AxNjIuY29tMFkwEwYHKoZIzj0CAQY
IKoZIzj0DAQcDQgAECXCDImdSY1/eq400+rReCE5qLfL9VeIHygJR8lAOzFTG58stV9kBjKR
BTBL5p5tdZqFX1DbxZ0bT7+mCuBvS7jAKBggqhkjOPQQDAgNHADBEAiAfDVKFeeeyEq9
HvOmXEGEueaYDCMoVg1zm2T396BOBVQIgKZUTOqn+Kb0Nh64b9mS0Fr8mtTqps5Fl7Q/
v2YO4MqQ=
-----END CERTIFICATE-----
...
PEM format.
Certificate import successful !!!
For an SSL virtual host, it must meet the following configurations to support running HTTP/2;
otherwise, HTTP/1.1 will be used:
1. The SLB virtual service associated with the virtual host is enabled with HTTP/2.
For an SSL real host, it must meet the following configurations to support running HTTP/2;
otherwise, HTTP/1.1 will be used:
1. The SLB real service associated with the real host supports and is enabled with HTTP/2.
2. The virtual host supports running HTTP/2 (see the three conditions for HTTP/2 support by
virtual host stated previously).
Note: In the deployment scenario where HTTP/2 runs over TLS, SSL renegotiation must
be disabled. Use the “ssl globals renegotiation off” command to disable it globally, and
the “ssl settings reneg <virtual_host_name>” command to disable it on individual virtual
host.
Terminology Descryption
An SSL host associated with an SLB virtual. An SSL virtual host acts as an
Virtual Host SSL server and is used to communicate by using SSL between browser and
APV appliance.
An SSL host associated with an SLB real. An SSL real host acts as an SSL
Real Host client and is used to communicate by using SSL between APV appliance
and backend origin server.
Origin Server A backend server that will accept clear-text or encrypted requests.
Clear-text Any traffic that is not encrypted.
Virtual Host Port The port that SSL virtual host will listen on. Typically port 443 is used.
A private key that is stored on the APV appliance for PKI (Public Key
Key (private) Infrastructure) authentication purposes. APV appliance supports keys up-to
4096 bits in size.
This is used for authentication purpose and to help set up secure
Certificate
communications between the appliance and the browser.
Certificate A certificate authority is an entity that will create a certificate from a CSR
Authority (CA) (Certificate Signing Request).
Current Web Browsers have a list of known CA’s public keys that are used
Trusted Certificate to verify certificates authenticity. If the browser cannot identify the CA it
Authority will inform the user as such. In a similar manner the APV appliance also
maintains a list of Trusted Certificate Authorities to verify certificates.
For our example environment, we have a domain name of “www.example.com”. For our SSL
purposes we will be using “www.example.com” as our SSL virtual host. This SSL virtual host is
associated with an SLB virtual host using IP 10.10.0.10 and port 443.
There are two methods for setting up SSL acceleration. The first method applies if you have never
set up SSL, and you will need to walk through the whole process of setting up the SSL virtual host
and generation of a CSR to send to the CA of your choice. The CA will send you a signed
certificate that you will then import. The second method applies if you already have a key and
certificate, and you can skip the CSR step and import your key and certificate. Let’s go ahead and
setup SSL as though we have never set it up.
Operation Command
slb virtual https <virtual_service> <vip> [vport] [arp_support]
Create an SSL
[max_connection]
virtual host
ssl host {real|virtual} <host_name> [slb_service]
ssl csr <virtual_host_name> [key_length] [certificate_index]
[signature_algorithm_index] [domain_name]
ssl ecc csr <virtual_host_name> [curve_name] [certificate_index]
[signature_algorithm_index] [domain_name]
Import certificate
ssl import certificate <host_name> [certificate_index] [domain_name]
and key for SSL
[tftp_ip] [file_name]
virtual host
ssl import key <host_name> [certificate_index] [domain_name] [tftp_ip]
[file_name]
ssl activate certificate <host_name> [certificate_index] [domain_name]
[certificate_type]
slb real https <real_service> <ip> [port] [max_connection] [hc_type]
[hc_up] [hc_down]
Create an SSL real
slb real tcps <real_service> <ip> <port> [max_connection] [hc_type]
host
[hc_up] [hc_down]
ssl host {real|virtual} <host_name> [slb_service]
ssl stop <host_name>
ssl settings ciphersuite <host_name> <cipher_string>
ssl settings protocol <host_name> <version>
Advanced ssl settings reuse < host_name>
configuration for ssl settings clientauth <host_name>
an SSL virtual host ssl settings certfilter <virtual_host_name> <condition_1> [condition_2]
ssl import rootca [host_name] [domain_name] [tftp_ip] [file_name]
ssl settings crl offline <virtual_host_name> <cdp_name> <cdp_url>
[domain_name] [time_interval] [delay_time]
Operation Command
ssl settings crl online <virtual_host_name>
ssl settings ocsp <virtual_host_name> <ocsp_server_url>
ssl settings minimum <virtual_host_name> <cipher_strength>
<redirect_url>
ssl start <host_name>
ssl import error <error_code> <url> [virtual_host_name]
ssl load error <error_code> [virtual_host_name]
ssl settings ciphersuite <host_name> <cipher_string>
ssl settings protocol <host_name> <version>
Advanced
ssl settings reuse <host_name>
configuration for
ssl settings clientauth <host_name>
an SSL real host
ssl import rootca [host_name] [domain_name] [tftp_ip] [file_name]
ssl settings servername <real_host_name> <common_name>
Firstly, execute the “slb virtual…” command to create an SLB TCPS or HTTPS virtual service.
Secondly, run the “ssl host” command to create an SSL virtual host and associate with the virtual
service.
In this example, “vitrual1https” is the created SLB virtual service, and “www.example.com” is the
created SSL virtual host.
Importing Certificate and Key for the Newly Created SSL Virtual Host
The following configuration illustrates an example for applying for, importing and activating an
RSA key and certificate for virtual host “www.example.com”.
1. Run the “ssl csr” command to generate an RSA CSR for “www.example.com”. (To apply for
an ECC certificate, run the “ssl ecc csr” command.)
Warning: RSA certificate chain is incomplete for www.example.com. Please add interca or rootca
certificate.
Besides the RSA CSR, this command also generates an RSA key pair and a testing certificate for
virtual host “www.example.com”. The testing certificate is used for testing purpose only. To use it
for testing or demonstration, you can directly start the SSL virtual host.
Now, you can connect to the website securely via a web browser.
In the output of the “ssl csr” command, copy the line starting from “-----BEGIN CERTIFICATE
REQUEST----” to “----END CERTIFICATE REQUEST-----” and send it to CA. CA will reply
with a digital certificate in the Email. The following is an example certificate.
-----BEGIN CERTIFICATE-----
MIICnjCANgcANgEUMA0GCSqGSIb3DQEBBAUAMIG5MQswCQYDVQQGEwJVUzETMB
EGA1UECBMKQ2FsaWZvcm5pYTERMA8GA1UEBxMIU2FuIEpvc2UxHDAaBgNVBAoTE0
NsaWNrQXJyYXkgTmV0d29ya3MxFDASBgNVBAsTC0RldmVsb3BtZW50MSMwIQYDVQ
QDExpkZXZlbG9wbWVudC5jbGlja2FycmF5LmNvbTEpMCcGCSqGSIb3DQEJARYaZGV2Z
WxvcG1lbnRAY2xpY2thcnJheS5jb20wHhcNMDIwMjEzMTgwMTI5WhcNMDMwMjA4MTgw
MTI5WjB0MQswCQYDVQQGEwJVUzEMMAoGA1UECBMDRE9EMQwwCgYDVQQHEw
NET08xCzAJBgNVBAoTAkRPMQswCQYDVQQLEwJETzETMBEGA1UEAxMKMTAuMTIu
MC4xNDEaMBgGCSqGSIb3DQEJARYLbWhAZGtkay5jb20wgZ8wDQYJKoZIhvcNAQEBBQ
ADgY0AMIGJAoGBAMx4r+ae4kTZggtyU047OsKUyqCt+V1MHgTPTpVxdtxYhSTSOZwYIX
gRqBEdJvs2/ua1XZRzLOCTa58VI/8I3derAPqz79WpBRsxD25rCT1rzmalfkTea3V8jHJYP6Yin
DTWKFKztxeUclkzukzPUZO6M0fI5ToXNuLEe+IwvOkfAgMBAAEwDQYJKoZIhvcNAQEEB
QADgYEAodV5O0LKUr/O0BbxOnwmyP/DkLj4bpe9XxQO6B4psDey/+xBHs6tgGKuy8spbcJ4
pQc+5KLydK1ZYcTkbxJq41K4RHM11OClXVjm3xRhqKQnjzNboExIvkZsKIBbfLkBrM1eBnE
aiYWXmsYGfxPkwdhKlQCLQgN+G3IKu2cRQLU=
-----END CERTIFICATE-----
Note: It is imperative that you do not delete the SSL virtual host before importing the
certificate received from CA. Otherwise, you will have to send another CSR to your CA to
re-apply for a certificate. Fortunately, most CAs provide a 30-day trial period for getting
another certificate in case that something goes wrong. If the trial period elapses, you might
have to pay for another certificate.
3. Run the “ssl import certificate” command to import the received certificate for the SSL
virtual host.
pQc+5KLydK1ZYcTkbxJq41K4RHM11OClXVjm3xRhqKQnjzNboExIvkZsKIBbfLkBrM1eBnE
aiYWXmsYGfxPkwdhKlQCLQgN+G3IKu2cRQLU=
-----END CERTIFICATE-----
…
You can also import the certificate from a remote TFTP server, which needs the TFTP server’s IP
address and certificate file name (“example.crt” in this case).
Note: Available key and certificate files can be directly imported into an SSL virtual host
using the “ssl import key” and “ssl import certificate” commands (without applying for
a certificate using CSR). These commands support copy and paste of the key or certificate
content into the command prompt or import of the files from a remote TFTP server.
Whichever method is used, please note following principles:
The key file must be imported first and then the certificate file.
Keys and certificates in non-PEM format can only be imported using the TFTP
method.
Note: The system will check the certificate chain when the certificate is activated. A
warning message, stating that the certificate chain is incomplete, will be printed if its root
CA certificate or intermediate CA certificate cannot be found in the virtual host’s global
trusted CA file or intermediate CA file. These certificates can be imported by using the
“ssl import rootca” and “ssl import interca” commands. These certificates can be
imported by using the “ssl import rootca” and “ssl import interca” commands.
Till now, SSL acceleration configuration for “www.example.com” is completed. A client can
access the virtual host via HTTPS.
Import Certificate and Key from IIS and NS iPlanet Web Servers
IIS
If you are using the Microsoft IIS server, the APV appliance will allow you to import the
certificate from IIS versions 4 and 5 through TFTP mechanism. IIS stores the SSL key and
certificate in the same file. This file is in .PFX format. You need to put this file onto a TFTP
server in its TFTP root directory and rename it as <host_name>.crt. This file then can be imported
into APV appliance through the “ssl import certificate” command. This command takes TFTP
server IP as an extra argument.
This command will download a file that is named <host_name>.crt. In our case it is
“www.example.com.crt” from the TFTP server (10.10.0.3).
After importing the certificate successfully, you will get a response from the CLI prompt. Then
you can activate the certificate via the command “ssl activate certificate”. For example:
Once the certificate and key import is successful through TFTP server, you need to start the SSL
service with the “ssl start” command.
Netscape/iPlanet
If you are using the Netscape or iPlanet servers, the APV appliance will also allow you to import
the certificate and key. The iPlanet server stores the key/cert pair in the directory
/<serverroot>/alias/ where <serverroot> is the directory where the server is installed. In that
directory there will be two files of the form <serverid-hostname>-key3.db and
<serverid-hostname>-cert7.db. You will need to copy the first file to your TFTP server's root
directory and name it the same as your virtual host with a .key extension. The cert will be the
same, but with a .crt extension. These have to be exact, or the SSL subsystem will not load them
correctly.
This command imports the key from 10.10.0.3 with the file name “www.example.com.key”.
Note: You must first import the certificate and then import the key when importing an
SSL cert/key pair from iPlanet.
This command imports the certificate from 10.10.0.3 with the filename “www.example.com.crt”.
Once the key is imported, the ArrayOS will ask you for a password. This password is the one you
use for the database password on the iPlanet server.
After importing the certificate successfully, you will get a response from the CLI prompt. Then
you can activate the specific certificate via the command “ssl activate certificate”. For example:
Now the APV appliance is configured to take full advantage of the SSL functionality within the
ArrayOS.
Note: In this section we have created an SLB virtual service and configured SSL for it.
This SLB virtual service is now ready to be used, and need to be linked with one or more
SLB real services so that the SLB module can complete the SSL requests coming to this
SLB virtual. To get information on “How to associate an SLB virtual with an SLB real”,
please refer to the SLB configuration section.
The APV appliance allows you to use the SSL subsystem to talk to SSL enabled real servers. This
allows an encrypted transaction between the ArrayOS and the backend servers.
Configuration of SSL real host is very simple and can be explained as follows:
The first step in this procedure is to define the SLB real service. To do this first we will employ an
SLB related command. This command will create the SLB real service. The second step is to use
the “ssl host” command to define the SSL real host.
For the definition and meaning of each parameter supplied in this command, please refer to the
SLB CLI section of CLI Handbook.
In the above example, please note that “real1https” is our newly created SLB real service, which
represents a backend server running on IP 192.168.1.20 and port 443 and is capable of handling
SSL requests. As a final step, we can start the SSL subsystem:
Now the APV appliance is configured to take full advantage of the SSL functionality while
communicating with the backend server.
Note: In this section we have created an SLB real service and configured SSL for it. This
SLB real service is now ready to be used, and need to be linked with an SLB virtual
service so that the SLB module can direct the traffic to this SSL enabled SLB real service.
To get information on “How to associate an SLB real with an SLB virtual” please refer to
the SLB configuration section.
SSL virtual host must be disabled before you change its configurations.
The following table lists the RSA and ECC (ECDHE-ECDSA…) cipher suites allowed for an SSL
virtual host with different SSL protocol versions. “Y” indicates that the cipher suite is supported.
“N” indicates that the cipher suite is not supported.
To enable multiple ciphers for a single SSL virtual host, you will need to specify the ciphers in the
form of a colon (:) separated list.
The APV appliance supports the protocols Secure Sockets Layer version 3 (SSLv3), Transport
Layer Security Protocol version 1.0 (TLSv1), and Transport Layer Security Protocol version 1.2
(TLSv1.2). You can use one, two or all of these protocols for the SSL virtual host settings.
This allows you to enable SSL session reuse for an SSL virtual host. This feature is enabled by
default.
The APV appliance supports the SSL based client authentication. You can enable client
authentication for an SSL virtual host. If enabled, the APV appliance will require each client to
present an SSL certificate for authorization, before the client can access the SSL virtual host.
Note: If you enable SSL client authentication for an SSL virtual host, you must provide a
trusted CA certificate. This will be used by the APV appliance to verify client certificates.
This command will prompt you to cut and paste the trusted authority certificate in PEM format.
You may configure multiple trusted authorities for one SSL virtual host.
Furthermore, the SSL virtual host will check the client certificate based on the configured
certificate filters (by using the command “ssl settings certfilter”). If the client certificate fails the
certificate verification, the SSL host will reject the client’s access. At most three pieces of
“certfilter” configuration (by using the “ssl settings certfilter” command) can be configured for
an SSL virtual host. The logical relationship among the three pieces of “certfilter” configuration is
“OR”. If the client certificate does not match any piece of “certfilter” configuration, the SSL
virtual host will reject the client’s access.
The filters can be configured with any of the supported RDNs on the APV appliances.
For example:
In this example, client certificates can pass the certificate verification only when the following
conditions are both met:
In the “subject” field, “C” is “US”, “O” is “Array”, “OU” is “QA” and “emailAddress” is
“[email protected]”.
Two kinds of client authentication modes are supported: mandatory and non-mandatory. Client
authentication mode defaults to mandatory. In non-mandatory client authentication mode, when
the server sends a certificate request to the client, if the client has no matched certificate or cancels
the authentication by clicking the Cancel button, the server will permit the client to access limited
network resources instead of dropping the SSL connection. However, all the networks resources
which can be published to non-authenticated clients need to be defined by using the “http acl url”
command.
APV supports the CRL (Certificate Revocation List) functionality. You can configure the APV
appliance to fetch the CRL file periodically from a CRL Distribution Point (CDP) by using HTTP
or FTP.
For our example, let’s consider a case when you have put your CRL file (Array.crl) on an HTTP
Web server (www.crldp.com) and you want to fetch it every one minute.
This will cause the APV appliance to fetch the CRL file at the regular interval of one minute from
the “www.crldp.com” site by utilizing HTTP.
You can also specify an FTP URL to download the CRL file.
You may also specify an LDAP URL to download the CRL file.
7. Configure OCSP for SSL virtual host to check the certificate validation online.
The APV appliance supports the OCSP (Online Certificate Status Protocol) protocol. You may
configure the APV appliance to validate the certificate on an OCSP server online.
For our example, configure an OCSP server (ocsp.crldp.com:8888) and to validate the certificate
online, you may configure the APV appliance as follows:
Note: The OCSP has top priority. When configured, the OCSP will validate the
certification by only checking the OCSP server.
The APV appliance provides you with a facility to redirect the weak clients (clients who are not
using strong ciphers) to another URL. You can specify the minimum strength of the cipher as
acceptance criteria. Any client that uses a cipher weaker than this will be redirected to the
configured URL.
For example, consider a scenario where you want to redirect all clients that does not support
cipher suites with at least 168 bits key length to a different site “www.example2.com”.
You will need to activate the SSL virtual host to take advantage of all the configuration steps
taken to this point.
SSL real host must be disabled before you change its configurations.
The following table lists the RSA and ECC (ECDHE-ECDSA…) cipher suites allowed for an SSL
real host with different SSL protocol versions. “Y” indicates that the cipher suite is supported. “N”
indicates that the cipher suite is not supported.
SSL protocols allowed to be set include SSLv3, TLSv1 or TLSv12. You can set one, two or all of
them.
This allows you to enable SSL session reuses between the APV appliance and backend servers.
This feature is enabled by default.
The APV appliance can use SSL client authentication while communicating with the backend
server. If this setting is enabled, the APV appliance will submit the client certificate to the
backend sever for authentication during SSL handshake.
Note: If you want to enable client authentication for an SSL real host, you will need to
import a certificate and key pair for the SSL real host. The SSL real host will present this
certificate to the backend server for authentication. This may be accomplished by using
the “ssl import certificate” and “ssl import key” commands for an SSL real host. These
two commands work exactly the same for an SSL virtual host and an SSL real host. For
detailed instruction on using these commands, please refer to the SSL virtual host
configuration described earlier.
If you want to verify the certificate of the real backend server, you will need to turn on global
settings for verifying the server certificate. In addition, make certain the common name of the
server certificate matches a specific name by running the command “ssl settings servername”.
For example, if the certificate common name of the real server associated with the real host
“www.myreal.com” is “Myreal Inc.”, you can use the following command:
Since the SSL subsystem acts like a client to the real server, it has several root CA certificates just
like a common Web browser. If you are using a self-signed certificate, or a certificate issued by
your own local CA on your origin servers, then you need to use the “ssl import rootca” command
to import the self-signed certificate that is on the real server or the local CA certificate.
The certificate must be in PEM format and is imported the same way you import a PEM certificate.
The APV appliance will prompt you to cut and paste the text to the terminal and enter “...” to
accept the certificate.
You will need to activate the SSL real host to take advantage of all the configuration steps taken to
this point.
13.1 Overview
This chapter introduces how to setup the QoS (Quality of Service) function on the APV appliance.
We setup the QoS functionality to provide administrators with the control over network bandwidth
and allow them to manage the network from the business perspective, rather than the technical
perspective.
QoS provides network administrators with the capacity of TCP, UDP and ICMP flow management
by using queuing mechanism and packet filtering policies. By using queuing mechanism and filter
rules, QoS supports both bandwidth management and priority control.
Each queue is bound with a particular network interface and controls either incoming or outgoing
network traffic of that interface. QoS queues are organized in tree-like structures. On the top of a
tree, a root queue is defined for either incoming or outgoing traffic of a network interface. Under
the root queue, there can be multiple sub-queues. Sub-queues can also have their sub-queues. For
each interface, at most two queue trees can be configured: one for the incoming traffic, and the
other for the outgoing data.
Each queue is configured with bandwidth limit and priority for packet processing.
In filter rule, the network traffic is specified by five parameters: source IP subnet, source port,
destination IP subnet, destination port and protocol. By this association, administrators can deploy
either application-oriented or link-oriented QoS control. Normally, application-oriented filter rules
have TCP or UDP ports defined while link-oriented filter rules focus on source or destination IP
addresses.
This priority mechanism works well especially when the network become crowded. If the traffic
reaches a peak, packet loss will arise when the number of packets waiting for processing exceeds
the maximum queuing buffers. Under such circumstance, the packets belonging to the queues with
the highest priority will be processed in the first place, while other packets with lower priorities
may be dropped. In this way, the mission-critical applications will be assigned with the highest
priority, therefore the functionality of the most important transactions is guaranteed.
Operation Command
Configure QoS
qos interface <interface_name> [direction] [bandwidth]
interface
qos queue root <queue_name> <interface_name> [direction]
[bandwidth] [priority] [borrow] [default]
Define QoS queue
qos queue sub <queue_name> <parent_queue> [bandwidth] [priority]
[borrow] [default]
Define QoS filter qos filter <filter_name> <queue_name> < src_addr> <smask> <sport>
rules <dst_addr> <dmask> <dport> <proto> [priority]
Default queue is for all the other packets which cannot hit any defined queues.
5. Enable QoS.
14.1 Overview
This chapter details the configuration of the following Inbound and Outbound Link Load
Balancing implementations:
The APV appliance identifies links based on the logical port and peer MAC address. The statistics
of LLB links are also collected based on the logical port and peer MAC address.
For example, let’s say you have Internet connectivity provided by two ISPs: ISP1 and ISP2. ISP1
assigns address range 100.1.1.0/24 so that you may use them on your network devices. ISP2
assigns address range 200.1.1.0/24 so that you may use them on your network devices. Outbound
LLB allows you to load balance outbound connections traffic through ISP1 and ISP2. Connections
forwarded through ISP1 are NATTed to an address from the range assigned by ISP1. Connections
forwarded through ISP2 are NATTed to an address from the range assigned by ISP2. Thus,
inbound responses for those connections will return through the ISP that they were originally sent
through. If Internet connectivity through one of the ISP links is lost or interrupted, the outbound
traffic will no longer be sent through that ISP. All traffic will be distributed to the functional ISP.
APV outbound LLB methods can work well on the data traffic based on TCP and UDP protocols,
as well as the packets based on IP, IPsec or GRE protocols.
To illustrate, let’s use the same example ISPs as mentioned previously. All external clients trying
to connect to the addresses assigned by ISP1 will be routed through ISP1’s backbone. All external
clients trying to connect to addresses assigned by ISP2 will be routed through ISP2’s backbone.
Inbound LLB allows you to advertise a device or Virtual IP (VIP) using two IP addresses: one
from ISP1 and the other from ISP2. A DNS server on the APV appliance will respond to queries
for configured domain names. The responses will contain an IP address from ISP1 or ISP2, both
representing the same device or VIP. If Internet connectivity through one of the ISP links is lost,
the DNS server will not respond with the address from the failed ISP. Clients will receive only the
address from the functional ISP.
Broadcasting ARP requests at regular intervals can check the availability of the link between APV
interface and the upstream ISP router. Pinging a user-defined upstream IP address not only can
verify if the link between APV interface and the upstream ISP router is available, but also verify
the link between upstream ISP router and user-defined upstream IP address. Multiple upstream IP
addresses can be defined for reliable checking. If any of check point is pingable, the related link is
usable. This ensures that the WAN link is up before forwarding traffic across that link.
ICMP- and TCP-based remote site accessibility checks are supported by this function.
This function works only for outbound traffic with the “rr” or “wrr” as outbound LLB method.
rr (Round Robin)
dd (Dynamic Detecting)
hi (Hash IP)
rr (Round Robin)
proximity
Round Robin distributes each new session to gateways in an alternating (round robin) way. This
is the default load balancing method.
Weighted Round Robin is similar to Round Robin except that a bias (or weight) may be assigned
to each gateway so that some gateways may receive more sessions than others. This allows more
traffic to be directed through an ISP with higher bandwidth capacity.
Shortest Response Time: The link with the shortest response time will get the next request.
Calculation of shortest response time of a link is based on the initiation process of each TCP
connection (both inbound and outbound connections). For the most accurate result, there should
be enough TCP traffic instead of a few long existing TCP connections or only UDP traffic.
Note:
If neither SLB traffic nor NAT traffic goes through the system, the LLB SR method
cannot work properly.
The “sr” method cannot be used to load balance IP fragments, non-TCP/UDP
packets, and reassembled UDP packets.
Dynamic Detecting performs proximity calculations through all available ISP links to the
destinations. By using parallel probe arithmetic, a request from the client will be sent to a
destination by different ISP links at the same time. When the first response returns, the optimal
ISP with the shortest response time will be selected for this request and other ISP connections will
be failed. For future outbound traffic to the same destination, APV appliance will choose the best
ISP connection, according to the results derived from these proximity calculations.
Hash IP distributes the outbound traffic among links in the way that the link with higher weight is
routed with higher probability, by performing Hash operation on the source IP. When the chosen
link is down, the system will carry another Hash operation on the links available. When HI is
deployed as the LLB method, the IPflow function can be disabled.
Proximity: The IP address of the nearest DNS server will be sent to the client as the response.
When a DNS request arrives, APV will first search in the Eroute table reversely to find a
proximity route matching the source address of the DNS request, and then give response to the
client with the corresponding DNS server’s IP address (A record) according to the Eroute
gateway.
etc.).Policy based routing, unlike regular routing, allows the inclusion of the source IP, source port
and destination port as well as the protocol into the route selection. For example, using routing
policy can ensure that all the traffic generated by AOL instant messenger always uses the same
link. If instant messenger client uses different destination IP addresses in its requests and these
requests are sent through the different routes, this might confuse the server and cause login failure.
Configuring routing policy will prevent this problem. The CLI command for that would be:
AN(config)#ip eroute aol_route 1500 0.0.0.0 0.0.0.0 0 0.0.0.0 0.0.0.0 5190 tcp gateway_ip 1
IP region
Eroute supports IP region. Administrators are allowed to import pre-defined IP region table via
HTTP, FTP or Local File method, and then execute the command “ipregion route” to apply the
imported IP region table. This will generate a large number of Eroute configurations, without
making complex configurations. Administrators are also allowed to export the IP region table via
FTP URL or Local File method.
APV appliance will check the contents of the file instead of the file type when an IP region file is
imported. To ensure that the IP region file can be imported successfully, please pre-define the file
contents strictly with the following items included in each entry:
Note:
When performing link selection for the outbound traffic, the system considers not only the routing
policies configured for links but also the load status of each link. That is, when the current link has
reached the configured bandwidth threshold, the APV appliance will search for available links
from matched routes according to the descending sequence of priorities. The APV appliance first
searches for available links from routes with the same priority as the current link. If all available
links reach their bandwidth thresholds, the APV appliance will search for available links from
routes with lower priorities. If the gateways of all matched routes are down or reach the
configured bandwidth thresholds, the APV appliance will still choose the current link to transmit
traffic.
In addition, the APV appliance allows administrators to configure a priority for the LLB link
bandwidth. If the priority of a matched route is higher than the LLB link bandwidth priority, the
traffic will be directly forwarded through this route.
With the LLB bandwidth management function, you do not need to configure Eroutes with the
same priorities for multiple links. This improves the efficiency and flexibility of link bandwidth
configuration and management.
Note:
If the traffic hits an RTS or IPflow route, the traffic will be directly forwarded
through the relevant LLB link no matter whether the LLB link reaches the bandwidth
threshold.
If an Eroute has been configured with the source IP address, source mask, source port
number, destination IP address, destination mask, and destination port number and
these IP addresses and masks are set to 0.0.0.0 and port numbers are set to 0, the
APV appliance will not search for available links from the matched routes whose
priorities are lower than 1000.
When receiving a DNS query from an intranet client, the APV appliance will select a DNS
server from all the DNS servers of ISP links according to the load balancing method, and
change the destination IP address of the DNS query to the IP address of the DNS server.
The APV appliance forwards the DNS query to the DNS server of the ISP link.
rr: Distributes new DNS queries to the DNS servers of the ISP links in a round robin way.
wrr: Assigns every DNS server a weight. The DNS server with the higher weight will receive
more DNS queries.
Note: If the same “weight” is specified for every DNS server or no “weight” is specified,
the rr method will be used to select the DNS server; otherwise, the wrr method will be
used.
Configuration Example:
Besides, the DNS proxy function supports selecting the DNS server using the domain
customization policy and DNS network policy for the outbound DNS query. The domain
customization policy has a higher priority than the DNS network policy, and the DNS network
policy has a higher priority than the load balancing method. Please refer to section 14.2.10.1 DNS
Network Policy and section 14.2.10.2 Domain Customization Policy for details.
The DNS network policy is used to select a DNS server for the DNS query whose source IP
address belongs to a specific network segment. When the source IP address of a DNS query
belongs to the network segment specified by a DNS network policy, the system will select the
DNS server specified by this policy for this DNS query. If a DNS query matches multiple DNS
network policies, the system will select the DNS network policy that has the longest subnet mask.
Configuration example:
After the above configuration is executed, the system will select the DNS server named
“dns_server2” for the DNS query from the 10.0.1.3 source IP address.
Static domain policy: The system always selects the specified DNS server regularly for the
DNS query of the specific domain name.
Bypass domain policy: The system does not use the DNS proxy function for the DNS query
of the specified domain name.
Persistent domain policy: The system persistently selects the specified DNS server for the
DNS queries of the specified domain based on the source IP address. When the DNS query
matches the persistent policy of the domain name for the first time, the system selects the
DNS server based on the DNS network policy (if it is configured and matched) or the load
balancing method for the DNS query, and records the corresponding relation between the
source IP address and the selected DNS server. When subsequent DNS queries from the same
IP address hit this policy, the system will select the specified same DNS server persistently.
Configuration Example:
After the above configuration is executed, the system will always select the DNS server named
“edu_dns” for the DNS queries matching the “*.edu.cn” domain.
After the above configuration is executed, the system will not use the DNS proxy function for the
DNS queries matching the “www.abc.com” domain.
After the above configuration is executed, the system will select the DNS server based on the
persistent policy for the DNS queries matching the “www.abc.com” domain.
The DNS proxy function also supports link bandwidth check. When the traffic of an LLB link
reaches the peak, the DNS servers of this LLB link will no longer be selected until this LLB link
has spare bandwidth. The system will select a DNS server from other available DNS servers
configured for the DNS proxy function.
Configuration Example:
The DNS proxy function supports the link health check. The link health check allows the APV
appliance to send the DNS query to the destination host to detect the health status of the specified
link.
Configuration Example:
Configuration Objective:
The DNS queries from the intranet will be distributed “evenly” to link ISP1 and link ISP2.
Configure the interface IP addresses for the LLB links ISP1 and ISP2.
Configure the interface IP address for the interface connecting to the intranet.
AN(config)#llb dnsproxy on
can be configured in one IP region table. For outbound LLB, only route-based LLB supports IPv6
configurations, while NAT-based LLB does not.
If the single APV appliance stopped working, the network connectivity would be interrupted.
Therefore, we recommend the implementation example with two APV appliances in section
14.3.2 Outbound LLB Configuration (Two APV Appliances).
Operation Command
Configure interface ip address {system_ifname|mnet_ifname|vlan_ifname|bond_ifname}
IP address <ip_address> {netmask|prefix} [overlap]
Configure LLB health llb link route <link_name> <route_ip> [weight] [hc_srcip]
check [bandwidth_threshold]
llb link health {on|off}
llb link health checker icmp <link_name> <host> [hc_interval]
[timeout] [hc_up] [hc_down]
The Port1 interface IP will have an address from ISP1’s address range. In order to assign an
additional IP address on the Port1 interface, you must define and configure a multi-netted virtual
interface (MNET). You will create an MNET named “outside_isp2” and assign it an IP address
from ISP2’s address range.
2. Configure link.
ISP link health checks are performed to ensure that the link between the local router (APV
appliance) and the remote ISP router is up.
Multiple health checks can be configured for the same link. Here, 100.1.1.2, 100.1.1.3 and
100.1.1.4 are another three remote routers of ISP1.
Only when all the health checks for ISP1 have failed, will the link ISP1 be deemed as down.
If a link is unstable, administrators can manually disable the link via the command “llb link
disable <link_name>”. For example, if the link ISP1 is found unstable, executing the command
“llb link disable ISP1” will disable the link, and no outbound traffic will go through this link
anymore. To enable a link, use the command “llb link enable <link_name>”.
LLB remote site accessibility checks are performed to ensure that the remote site is accessible to
the APV appliance.
The configured LLB remote site accessibility check can be associated with a remote site.
Enter the following command to enable LLB remote site accessibility check.
Hash IP (hi)
To make different traffic go through different links, configure the Eroutes for two LLB links.
AN(config)#ip eroute "er1" 1600 192.168.1.0 255.255.255.0 0 0.0.0.0 0.0.0.0 0 any 100.10.1.1 1
AN(config)#ip eroute "er2" 1400 192.168.1.0 255.255.255.0 0 0.0.0.0 0.0.0.0 0 any 200.20.1.1 1
To make traffic that does not match the preceding Eroute configurations go through ISP1,
configure the following Eroute:
AN(config)#ip eroute "er3" 1001 0.0.0.0 0.0.0.0 0 0.0.0.0 0.0.0.0 0 any 100.10.1.1 1
You can set a priority for the link bandwidth threshold to determine whether the configured link
bandwidth threshold takes effect for the relevant LLB link.
Because the priority of Eroute “er1” is higher than the bandwidth priority, the gateway specified
by the Eroute is not affected by the bandwidth threshold of ISP1. By comparison, the gateway
specified by Eroute “er2” is affected by the bandwidth threshold of ISP2.
For an ISP that is selected for a session based on specific LLB method, the NAT rules for the ISP
VIP must be pre-configured. These rules will be applied to the outbound traffic.
Execute the following command to ensure that packets from the same connection will be directed
to the same link by using the same NAT rule. By default, the IPflow function is disabled.
AN(config)#ip ipflow on
RTS (Return to Sender) should be turned on by executing the following command to ensure that a
response packet (for example, ICMP response) will be directed to the link from which its
corresponding request packet (for example, ICMP request) is sent. By default, the RTS function is
disabled.
AN(config)#ip rts on
Operation Command
Follow these steps to configure Outbound Link Load Balancing with clustered APV appliances.
Due to the additional configuration required for a secondary APV appliance and to eliminate
redundancy, this example assumes an understanding of the basic configuration that was illustrated
in the previous section. Also, optional configuration settings will be left at their default values, and
as a result, will not be illustrated in this example.
You will need to define IP addresses on both APV appliances. The same MNET names may be
used on both APV appliances.
Outbound traffic (from behind the APV appliances) must be forwarded to an IP address on the
APV appliances. To provide a fault tolerant gateway for inside devices, a virtual cluster VIP must
be configured.
(APV1) Configure the first APV appliance as the master virtual router for all interfaces so it will
process outbound traffic. Assign it a higher priority than the secondary APV appliance.
(APV2) Configure the secondary APV appliance as a backup virtual router for all interfaces so it
will not process outbound traffic unless the primary APV appliance fails. Assign it a lower priority
than the primary APV appliance.
3. Configure link.
Health check can be configured to monitor the status of network, and turn on health check.
Remote site accessibility check can be configured to monitor the accessibility of remote sites.
To make different traffic go through different links, configure the Eroutes for two LLB links.
AN(APV1)#ip eroute "er1" 1600 192.168.1.0 255.255.255.0 0 0.0.0.0 0.0.0.0 0 any 100.10.1.1 1
AN(APV1)#ip eroute "er2" 1400 192.168.1.0 255.255.255.0 0 0.0.0.0 0.0.0.0 0 any 200.20.1.1 1
AN(APV2)#ip eroute "er1" 1600 192.168.1.0 255.255.255.0 0 0.0.0.0 0.0.0.0 0 any 100.10.1.1 1
AN(APV2)#ip eroute "er2" 1400 192.168.1.0 255.255.255.0 0 0.0.0.0 0.0.0.0 0 any 200.20.1.1 1
To make traffic that does not match the preceding Eroute configurations go through ISP1,
configure the following Eroute:
AN(APV1)#ip eroute "er3" 1001 0.0.0.0 0.0.0.0 0 0.0.0.0 0.0.0.0 0 any 100.10.1.1 1
AN(APV2)#ip eroute "er3" 1001 0.0.0.0 0.0.0.0 0 0.0.0.0 0.0.0.0 0 any 100.10.1.1 1
You can set a priority for the link bandwidth threshold to determine whether the configured link
bandwidth threshold takes effect for the relevant LLB link.
Because the priority of Eroute “er1” is higher than the bandwidth priority, the gateway specified
by the Eroute is not affected by the bandwidth threshold of ISP1. By comparison, the gateway
specified by Eroute “er2” is affected by the bandwidth threshold of ISP2.
(APV1) Cluster VIPs for NAT on each ISP. Assign a higher priority than the secondary APV
appliance.
(APV2) Cluster VIPs for NAT on each ISP. Assign them a lower priority than the primary APV
appliance.
Execute the following command to ensure that packets from the same connection will be directed
to the same link by using the same NAT rule. By default, the IPflow function is disabled.
AN(config)#ip ipflow on
RTS (Return to Sender) should be turned on by executing the following command to ensure that a
response packet (for example, ICMP response) will be directed to the link from which its
corresponding request packet (for example, ICMP request) is sent. By default, the RTS function is
disabled.
AN(config)#ip rts on
Operation Command
Configure
ip address {system_ifname|mnet_ifname|vlan_ifname|bond_ifname}
interface IP
<ip_address> {netmask|prefix} [overlap]
address
Follow these steps to configure Inbound Link Load Balancing with a single APV appliance.
The Port1 interface IP will have an address from ISP1’s address range. In order to assign an
additional IP address on the Port1 interface, you must define and configure a multi-netted virtual
interface (MNET). You will create an MNET named “outside_isp2” and assign it an IP address
from ISP2’s address range.
2. Configure link.
Health check can also be configured to monitor the status of network. ISP link health checks are
performed to ensure that the link between the local router (APV appliance) and the remote ISP
router is up.
Note: To use the “proximity” method for inbound load balancing, please first make
configurations about “ip eroute”.
Execute the following command to ensure that packets from the same connection will be directed
to the same link by using the same NAT rule. By default, the IPflow function is disabled.
AN(config)#ip ipflow on
RTS should be turned on by executing the following command to ensure that a response packet
(for example, ICMP response) will be directed to the link from which its corresponding request
packet (for example, ICMP request) is sent. By default, the RTS function is disabled.
AN(config)#ip rts on
15.1 Overview
Global Server Load Balancing (GSLB) distributes network traffic among servers providing the
same service and deployed in different locations by returning different DNS resolution results.
The APV appliance adopts the Smart DNS (SDNS) technology to implement GSLB. SDNS makes
smart decisions at DNS resolution to achieve the following goals:
High access efficiency: Based on the geographical location and network proximity of the
user’s local DNS, SDNS returns the IP address of the server that is “logically” nearest to the
user or has the shortest response time, which improves the user’s service access efficiency.
Disaster tolerance: When detecting that a server is faulty, SDNS will pick up the IP addresses
of other available servers to respond to DNS queries. This ensures that users can still access
the service normally when a server is faulty and therefore enhances the service continuity and
availability. When the faulty server restores, SDNS can also switch the traffic back to it.
Load balancing: SDNS can distribute traffic among multiple available servers.
The following sections will describe the function principles and configuration examples of SDNS.
1. The client sends the DNS query to the local DNS to resolve the domain name
“www.example.com”.
2. After a few rounds of lookups, the local DNS finds that SDNS is the authoritative DNS for
the domain name and forwards the DNS query to SDNS.
3. SDNS makes smart decision to pick up healthy service IPs and return them to the local DNS.
1. Based on the domain name in the DNS query, SDNS finds the hit SDNS policy.
2. Based on the SDNS policy, SDNS finds the hit SDNS service pool.
3. Based on the method of the hit SDNS service pool, SDNS picks up service IPs.
The SDNS host name is the basic unit of the domain name resolution provided by SDNS. In
SDNS, the domain names that can be resolved are defined as SDNS host names. The system
supports a maximum of 16,000 SDNS host names.
For example:
In SDNS, one SDNS host name can be resolved into multiple IP addresses. SDNS allows the
administrator to add these resolved IP addresses in to a pool, which is called a “service pool”. The
IP address included in the service pool is called a “service IP”.
The service IP can be an IPv4 or IPv6 address and the service IPs in a service pool can be either
all IPv4 addresses or IPv6 addresses. Note that a service pool cannot contain both IPv4 and IPv6
addresses.
For example:
The SDNS service pool method determines which service IPs will be picked up from the service
pool to return to the local DNS. Service IPs picked up from the same service pool vary with
service pool methods. SDNS supports the following service pool methods:
IP Overflow (ipo)
Hash IP (hi)
drop
The following table describes the meanings of SDNS service pool methods.
Method Meaning
If the service pool contains three service IPs, the maximum_rr_count is
rr 1, and the rr method is used, the service IPs will be returned in the DNS
responses in the sequence of “1, 2, 3, 1, 2, 3…”.
The wrr method is similar to the rr method. The difference is that every
service IP in the service pool is assigned a weight. The second service
wrr
IP will not be returned in the DNS responses until the number of times
that the first service IP is returned reaches the weight value.
If the ipo method is used, the service IP with the highest priority will
ipo
always be returned in the DNS responses.
If the hi method is used, SDNS will compute the hash value based on
hi the source IP address. For DNS queries with the same source IP
address, SDNS returns the same service IP.
If the snmp method is used, SDNS will collect the data of specified
OIDs from every service IP through the SNMP protocol, compute the
metric of every service IP using the specified SNMP expression, and
snmp
sort up service IPs in the specified sorting mode. The service IP in the
in the first order will be returned in the DNS response. For details, refer
to section 15.2.6.2 SNMP Data Collection.
If the drop method is used, SDNS will directly discard the DNS query
drop
and return no DNS response.
Besides, SDNS supports configuring two levels of service pool methods: primary and secondary.
When failing to use the primary method to pick up any available service IP from the service pool,
SDNS will use the secondary method. Therefore, the administrator must configure the primary
method and can choose to configure the secondary method.
For example:
SDNS provides the Auto Failover function for the SDNS service pool using the ipo method. When
the service IP(s) with the highest priority become(s) unavailable, the Auto Failover function will
automatically switch the currently selected service IP(s) to the service IP(s) with the second
highest priority in the same SDNS service pool. By default, this function is enabled.
For example:
15.2.2.2.3 Preemption
SDNS provides the Preemption function for the SDNS service pool using the ipo method. When
the service IP(s) with the highest priority become(s) available again, the Preemption function will
automatically switch the currently selected service IP(s) in the same SDNS service pool to the
service IP(s) with the highest priority. By default, this function is enabled.
For example:
SDNS provides the Manual Switchover function for the SDNS service pool using the ipo method.
This function allows the administrator to manually switch the currently selected service IP(s) to
the service IP(s) with the specified priority in the same SDNS service pool.
For example:
Note:
If both the Auto Failover and Preemption functions are enabled, the manual
switchover operation will not work.
The manual switchover operation cannot be saved into memory. Therefore, the
manual switchover operation will stop working after system reboot.
When all the service IPs with the specified priority become down, DNS resolution
will fail.
The SDNS policy determines the SDNS service pool for the host name. SDNS supports two types
of policies:
Region policy: determines the SDNS service pool based on the SDNS host name and the SDNS
region into which the DNS query is divided.
Default policy: determines the SDNS service pool based on the SDNS host name. SDNS will use
the default policy only when failing to pick up any available service IP from the service pool by
using the hit region policy.
SDNS allows configuring multiple region policies and one default policy for the same host name.
When the domain name in the DNS query matches multiple region policies, SDNS will first pick
up available service IP(s) from the service pool associated with the region policy with the highest
priority. If failing to pick up any available service IP from this service pool, SDNS will pick up
available service IP(s) from the service pool associated with the default policy. If still failing to
pick up any available service IP, SDNS will return the empty DNS response to the local DNS. For
details on SDNS region policy, please refer to section 15.2.5 SDNS Region Policy.
For example:
When receiving the DNS CNAME query for the domain name “www.example.com”, SDNS
returns the CNAME “www.web.example.com” to the local DNS.
When receiving the DNS A/AAAA query for the domain name “www.example.com”, SDNS
first finds the CNAME record of “www.example.com”, then performs resolution on the
CNAME “www.web.example.com”, and finally returns the service IP “web1” or “web2” to
the local DNS.
One SDNS service pool has only one fallback pool but can be the fallback pools of multiple other
service pools. Multiple SDNS service pools can form a tree structure of fallback, as shown in the
following figure.
For example:
The service pools “Beijing” and “Tokyo” can fall back to the service pool “Asia” when they
are unavailable.
The service pool “New_York” can fall back to the service pool “World” when the service
pool “New_York” is unavailable.
The service pool “Asia” can fall back to the service pool “World” when the service pool
“Asia” is unavailable.
Note:
SDNS disallows multiple SDNS service pools to form a loop structure of fallback.
An SDNS service pool can fall back to an SDNS CNAME pool, but an SDNS
CNAME pool cannot fall back to an SDNS service pool.
localities, such as Unites States, Japan and China. The administrator can also divide SDNS regions
based on actual requirements, such as based on the Network Service Provider (ISP).
For an SDNS host name, the administrator can configure only one default policy and multiple
region policies. These region policies associate DNS queries of different SDNS regions with
different SDNS service pools.
The region policy determines the SDNS service pool based on the SDNS host name and the SDNS
region into which the DNS query is divided. In multiple region policies configured for the same
SDNS host name, the SDNS service pools vary with the SDNS regions. SDNS uses the SDNS
proximity rules to determine the SDNS regions into which DNS queries coming from a specified
subnet are divided. When receiving DNS queries, SDNS uses the longest prefix matching rule to
find the hit proximity rules and therefore determines the SDNS regions into which the DNS
queries are divided. When a DNS query matched multiple region policies, SDNS determines the
hit region policy based on the priority. The DNS query is processed by the SDNS service pool
associated with the region policy with the highest priority.
SDNS allows the administrator to configure static proximity rules to associate specified subnets
with SDNS regions, and to associate the ipregion table with the SDNS region to generate
proximity rules in batch. The proximity rules generated using the ipregion table are also called
ipregion proximity rules.
In addition, SDNS supports using the Dynamic Proximity System (DPS) to generate dynamic
proximity rules. For details on DPS, please refer to section 15.2.7 SDNS Dynamic Proximity
System (DPS).
For example:
The DNS queries for the host name “www.xyz.com” coming from the subnet 211.100.0.0/16
will hit the region policy “policy_beijing”.
The DNS queries for the host name “www.xyz.com” coming from the subnet 210.10.0.0/16
will hit the region policy “policy_tokyo”.
The DNS queries for the host name “www.xyz.com” coming from the subnet 216.100.0.0/16
will hit the region policy “policy_newyork”.
SDNS monitor supports the ICMP-type, TCP-type HTTP-type and HTTPS-type health check
templates and the SNMP-type data collection template. A maximum of 200 health check and data
collection templates are supported.
When a template is applied to an SDNS service, the system will generate a monitor instance for
the SDNS service; while the template is applied to an SDNS service pool, the system will generate
a monitor instance for every SDNS service in the service pool. The monitor instance generated
based on the health check template is called the health check instance, and the monitor instance
generated based on the SNMP data collection template is called the SNMP data collection instance.
A maximum of 5000 monitor instances are supported and a maximum of 1024 SNMP data
collection instances are supported.
SDNS provides the health check function to check the health status of the service IPs in the SDNS
service pool. When a service IP becomes unavailable, SDNS will pick up other available service
IPs to respond to DNS queries. This ensures that service IPs returned to the local DNS is healthy
and available and therefore enhances the service continuity and availability.
The ICMP-type health check determines the availability of a service IP by sending the ICMP Echo
Request (ping) to the service IP. If receiving the ICMP Echo Response from the service IP, SDNS
regards this service IP as available (Up); otherwise, SDNS regards this this service IP as
unavailable (Down).
The TCP-type health check determines the availability of a service IP by establishing the TCP
connection with the service IP. If the TCP connection is established successfully, SDNS regards
this service IP as available (Up); otherwise, SDNS regards this this service IP as unavailable
(Down).
The HTTP-type health check determines the availability of a service IP by sending the HTTP
request to the service IP. If the service IP returns the expected response, SDNS determines the
health result based on the result flag. If the result flag is “up”, SDNS regards this service IP as
available (Up); if the result flag is “down”, SDNS regards this service IP as unavailable (Down).
The HTTPS-type health check determines the availability of a service IP by implementing SSL
handshake with the service IP, and then sending an HTTP request to this service IP afterwards. If
the SSL handshake is successful and the service IP returns the expected response, SDNS
determines the health check result based on the result flag. If the result flag is “up”, SDNS regards
this service IP as available (Up); if the result flag is “down”, SDNS regards this service IP as
unavailable (Down).
To use the SDNS health check function, the administrator needs to first define health check
templates and then apply health check templates to service IPs or service pools to generate health
check instances.
Request line: specifies the predefined HTTP request content, which includes HTTP method,
URL path and HTTP version. This parameter is available only for the HTTP-type and
HTTPS-type health check templates. The default value is “HEAD / HTTP/1.0\r\n\r\n”.
Expected response: specifies the expected HTTP response(s). This parameter is available
only for the HTTP-type and HTTPS-type health check templates. The default value is “200
OK”.
Result flag: specifies the determined health check result when the expected HTTP response
is received. This parameter is available only for the HTTP-type and HTTPS-type health
check templates. The default value is “up”.
Result flag: specifies the determined health check result when the expected HTTP response
is received. This parameter is available only for the HTTP-type health check template. The
default value is “up”.
Health check interval: specifies the interval of sending health check packets.
Health check timeout: specifies the timeout value of the health check packet. The value of
the health check timeout should not be greater than that of the health check interval.
Maximum retry times: specifies the maximum times of health checks to determine the
health status of the destination IP address. For example, if it is set to 3, the health status of the
destination IP address will be regarded as Up when the health check result is Up for three
consecutive times; the health status of the destination IP address will be regarded as Down
when the health check result is Down for three consecutive times.
Destination IP address: specifies the IP address of the health check object. Usually this
parameter is left empty. When the health check template is applied to the service IP or
service pool, the destination IP address will be replaced by the corresponding service IP in
the generated health check instances.
Destination port: specifies the port of the health check object. This parameter is available
only for the TCP-type, HTTP-type and HTTPS-type health check templates. Usually, this
parameter is set to 0. When the health check template is applied to the service IP or service
pool, the destination IP port will be replaced by the health check port of the corresponding
service IP in the generated health check instances. Note that the destination port of the
TCP-type, HTTP-type and HTTPS-type health check template and the health check port of
the associated service IP cannot be both 0s.
Source IP address: specifies the source IP address of the health check packets.
Gateway IP address: specifies the IP address of the gateway forwarding health check
packets in a multi-link network environment.
After the health check template is applied to the specified service IP, SDNS will generate the
health check instance where the destination IP address is replaced by the service IP, and send the
health check packet to the service IP. If the health check template is applied to the specified
service pool, SDNS will generate the health check instance for each member (service IP) of the
service pool. The administrator can choose to apply the health check templates to the service IP or
service pool based on actual requirements.
If multiple health check templates are applied to a service IP, the service IP has multiple health
check instances; if multiple health check templates are applied to a service pool, these health
check templates will be applied to each service IP in the service pool. Therefore, each service IP
in the service pool has multiple health check instances. SDNS allows the administrator to specify
the relationship among health check instances of a service IP as an individual service IP or as a
member of a service pool.
“AND”: When the results of all health check instances of the service IP are Up, the status of
the service IP is set as Up.
“OR”: When the result of any health check instance for the service IP is Up, the status of the
service IP is set as Up.
For example:
SDNS provides the SNMP data collection function to collect the data of specified SNMP OIDs
from SDNS services. The snmp method of the SDNS service pool needs to depend on the SNMP
data collection function. SDNS computes the metric of every SDNS service based on the SNMP
expression associated with the service pool. The SNMP expression determines the data of what
SNMP OIDs will be used to compute the metric. To collect the data of such SNMP OIDs from
every SDNS service, the administrator needs to configure SNMP data collection.
For the snmp method to work normally, the administrator needs to:
2. Define the SNMP expression used to compute the metric of the SDNS service.
3. Define the SDNS service pool using the snmp method and the associated SNMP expression.
5. Apply the SNMP-type data collection templates to the SDNS service pool to generate SNMP
data collection instances.
Every SNMP expression uses the data of one SNMP OID at minimum and three SNMP OIDs at
maximum. The format of the SNMP expression is as follows:
Data of OID1 × Weight of OID1 + Data of OID2 × Weight of OID2 + Data of OID3 ×
Weight of OID3
Before defining the SNMP expression, the administrator needs to define the SNMP OIDs to be
used. A maximum of eight SNMP OIDs can be defined.
After the system receives the data of the SNMP OIDs, SDNS will compute the metric of every
SDNS service based on the SNMP expression, and sort up SDNS services according to the sorting
mode configured for the SDNS service pool. The SDNS service pool supports two sorting modes:
Ascending: SDNS service IPs are sorted up in the ascending order of the metrics.
Descending: SDNS service IPs are sorted up in the descending order of the metrics.
To collect the data of SNMP OIDs from the SDNS services of an SDNS service pool, the
administrator needs to create SNMP data collection templates for these SNMP OIDs and apply
these templates to the SDNS service pool. In this way, the system generates SNMP data collection
instances for every SDNS service in the SDNS service pool.
OID name: specifies name of the SNMP OID whose data needs to be collected.
Community string: specifies the community string of the SNMP server. The default value is
“public”.
Destination IP address: specifies the IP address of the data collection object. Usually this
parameter is left empty. When the data collection template is applied to the service pool, the
destination IP address will be replaced by the corresponding service IP in the generated data
collection instances.
Destination port: specifies the SNMP port of the data collection object. The default value is
161.
Gateway IP address: specifies the IP address of the gateway forwarding SNMP requests in a
multi-link network environment.
Note: The SNMP-type data collection template can be applied only to the SDNS service
pool.
Configuration example:
DPS Server
The DPS master (the APV appliance) mainly provides the following function:
Collects the IP addresses of local DNSs and sends the address list to DPS detectors for
detection.
DPS Detector
Sends detection packets to local DNSs to obtain three types of proximity information: Round
Trip Time (RTT), Packet Loss Rate (PLR) and hops.
Note: The DPS detector can be deployed on the APV appliance or on other servers
running the Linux or FreeBSD operating system.
When receiving the proximity detection results from the specified SDNS region’s DPS detector,
the DPS server will calculate the metrics of the paths between the DPS detector (region) and all
local DNSs. The DPS server will then generate a dynamic proximity rule between the local DNS
and SDNS region whose path has the smallest metric.
When the mix DPS method is used, the administrator needs to specify the weights for RTT, PLR
and hops.
After a while, the DPS server collects a list of local DNSs and their IP addresses.
The DPS server sends the IP addresses of the collected local DNSs to the DPS detector of
each region (CDN site) for detection.
The DPS detector starts to detect the proximity between its region and all local DNSs.
When the DPS server queries the proximity detection results, the DPS detector reports
proximity detection results to the DPS server.
The DPS server generates the dynamic proximity rule based on received proximity detection
results.
For example:
When receiving DNS queries of the A, AAAA and CNAME types, the APV appliance dispatches
them to SDNS for processing. When receiving DNS queries of other types, the APV appliance
dispatches them to Full-DNS for processing. If the domain names in the DNS queries of the A,
AAAA and CNAME types are not configured on SDNS, the APV appliance will return empty
DNS responses. When Full-DNS fails to process DNS queries of other types, the APV appliance
will also return empty DNS responses.
The APV appliances will forward the DNS queries of the A, AAAA and CNAME types to
Full-DNS for processing if the domain names in the DNS queries have not been configured
on SDNS.
The APV appliances will forward the DNS queries to other external DNSs to perform
recursive query if Full-DNS cannot process them and finally returns the resolution results to
the local DNS.
By default, the Recursive Query function is disabled. To enable the Recursive Query function,
execute the following command:
AN(config)#sdns recursion on
Assume that only resource records of the A, AAAA, or CNAME type are available on SDNS for
the domain name “image.example.com”. To ensure that the DNS queries of the other record types
can be correctly processed, you are advised to configure a TXT record in the following format for
the domain name “example.com” in the Full-DNS zone file:
Note: The “text string” can be any description about this domain name.
AAAA DNS resolution: SDNS can process the DNS query of the AAAA type.
SDNS service pool: The service IPs in a service pool can be either all IPv4 addresses or IPv6
addresses. Note that a service pool cannot contain both IPv4 and IPv6 addresses.
SDNS DPS: SDNS supports IPv6 DPS detectors and supporting detecting IPv6 local DNSs.
SDNS health check: SDNS supports performing health checks on IPv4 and IPv6 service IPs.
For the DNS query sent from the local DNSs near to the “Beijing” site, SDNS will return the
IP address of either server on the “Beijing” site. When both servers on the “Beijing” site
become unavailable, SDNS will return the address of either server on the “Tokyo” site. When
both servers on the “Tokyo” site also become unavailable, SDNS will return the address of
either server on the “New_York” site.
For the DNS query sent from the local DNSs near to the “Tokyo” site, SDNS will return the
IP address of either server on the “Tokyo” site. When both servers on the “Tokyo” site
become unavailable, SDNS will return the address of either server on the “Beijing” site.
When both servers on the “Beijing” site also become unavailable, SDNS will return the
address of either server on the “New_York” site.
For the DNS query sent from the local DNSs near to the “New_York” site, SDNS will return
the IP address of either server on the “New_York” site. When both servers on the
“New_York” site become unavailable, SDNS will return the address of any server on the
“Beijing” or “Tokyo” site.
Define three CDN sites as regions “Beijing”, “Tokyo”, and “New_York” respectively.
Define region policies to associate DNS queries coming from the regions “Beijing”, “Tokyo”,
and “New_York” with the service pools “pool_beijing”, “pool_tokyo” and “pool_newyork”
respectively. The service pool “pool_beijing” includes the service IPs “beijing1” and
“beijing2”; the service pool “pool_newyork” includes the service IPs “tokyo1” and “tokyo2”;
the service pool “pool_newyork” includes the service IPs “newyork1” and “newyork2”.
Configure fallback pools for the service pools “pool_beijing”, “pool_tokyo” and
“pool_newyork”. The service pools “pool_beijing” and “pool_tokyo” can fall back to the
service pool “pool_asia” that includes all service IPs of the service pools “pool_beijing” and
“pool_tokyo”. The service pools “pool_newyork” and “pool_asia” can fall back to the service
pool “pool_world” that includes all service IPs of the service pools “pool_beijing”,
“pool_tokyo”, and “pool_newyork”.
For example:
For example:
For example:
For example:
For example:
For example:
Enable SDNS
For example:
AN(config)#sdns on
16.1 WebWall
16.1.1 Overview
The WebWall functionality of the APV appliance allows you to create permit/deny rules to filter
packets passing through your network infrastructure. The WebWall supports the filtering of TCP,
UDP and ICMP packets that are using the IPv4 or IPv6 address. To use access lists you will define
these “permit” and “deny” rules and apply them to access groups. Once the access lists are
configured, you may apply or bind the group to an interface within the network.
The steps for basic WebWall configurations are explained in this section, along with some
advanced features and general knowledge of how WebWall works. For ArrayOS, the WebWall
feature can independently control each interface.
WebWall permits TCP and UDP health check traffic, but cannot permit ICMP health check traffic
automatically.
WebWall contains several security mechanisms to protect Web servers from attack, including:
ACL Filtering provides tight control over who may and may not enter the network by utilizing
APV’s ultra-fast rules engine. WebWall access control list filtering mechanism ensures virtually
no performance loss with up to 1,000 ACL rules, while never consuming more than one percent of
ArrayOS capability.
In addition to ACL filtering, the WebWall provides stateful packet inspection and protects against
Syn-Flood, fragmentation, DoS and single packet attacks.
The WebWall is a default-deny firewall. Default-Deny refers to the notion that if you do not have
any permit rules in your access control lists, no packets will be allowed to pass through the
appliance. During the initial installation of the box it might be helpful to leave the WebWall in the
off or disengaged state until your total configuration is complete.
Note: By default the WebWall is turned off. The WebWall function will remain disabled
until it is activated via the “webwall on” command.
Configure WebWall on the APV appliance to make sure it processes management traffic and
service traffic based on the following rules:
Permit the IP address 10.10.10.30 to configure and manage the APV appliance through port
22 (for SSH access).
Permit the IP address 10.10.10.30 to configure and manage the APV appliance through port
8888 (for New WebUI access).
Permit all IP addresses except 10.10.10.30 to access the virtual IP address 10.10.0.10.
Permit all clients on the internal network to ping the IP address of port2 interface
(192.168.10.1).
This configuration example is based on the network topology in the following figure.
The following table lists the commands for configuring the WebWall. For detailed information of
each command, please see the ArrayOS APV 10.0 CLI Handbook.
Operation Command
Configure access
accessgroup <accesslist_id> <interface>
group
accesslist permit icmp echoreply <source_ip>
{source_mask|source_prefix} <destination_ip>
{destination_mask|destination_prefix} <accesslist_id>
accesslist permit icmp echorequest <source_ip>
{source_mask|source_prefix} <destination_ip>
{destination_mask|destination_prefix} <accesslist_id>
accesslist permit tcp <source_ip> {source_mask|source_prefix}
<source_port> <destination_ip> {destination_mask|destination_prefix}
<destination_port> <accesslist_id>
accesslist permit udp <source_ip> {source_mask|source_prefix}
Configure ACL <source_port> <destination_ip> {destination_mask|destination_prefix}
rules <destination_port> <accesslist_id>
accesslist permit esp <source_ip> {source_mask|source_prefix}
<destination_ip> {destination_mask|destination_prefix} <accesslist_id>
accesslist permit ah <source_ip> {source_mask|source_prefix}
<destination_ip> {destination_mask|destination_prefix} <accesslist_id>
accesslist deny icmp echoreply <source_ip> {source_mask|source_prefix}
<destination_ip> {destination_mask|destination_prefix} <accesslist_id>
accesslist deny icmp echorequest <source_ip>
{source_mask|source_prefix} <destination_ip>
{destination_mask|destination_prefix} <accesslist_id>
accesslist deny tcp <source_ip> {source_mask|source_prefix}
Operation Command
<source_port> <destination_ip> {destination_mask|destination_prefix}
<destination_port> <accesslist_id>
accesslist deny udp <source_ip> {source_mask|source_prefix}
<source_port> <destination_ip> {destination_mask|destination_prefix}
<destination_port> <accesslist_id>
accesslist deny esp <source_ip> {source_mask|source_prefix}
<destination_ip> {destination_mask|destination_prefix} <accesslist_id>
accesslist deny ah <source_ip> {source_mask|source_prefix}
<destination_ip> {destination_mask|destination_prefix} <accesslist_id>
Enable/Disable webwall <interface> on [mode]
WebWall webwall <interface> off
show interface
View WebWall
show accesslist
configurations
show accessgroup
1. Configure access group 50, which is used to associate ACL rules of miscellaneous types.
AN(config)#accessgroup 50 port2
2. Configure access group 100, which is used to associate ACL rules for management IP access.
3. Configure access group 150, which is used to associate ACL rules for virtual IP access.
1. Permit the IP address 10.10.10.30 to configure and manage the APV appliance by using SSH
at port 22
2. Permit the IP address 10.10.10.30 to configure and manage the APV appliance by using the
new WebUI at port 8888.
3. Permit all IP addresses except 10.10.10.30 to access the virtual IP address 10.10.0.10 through
port 80.
4. Permit all IP address on the internal network to ping 192.168.10.1, IP address of the port2
interface.
AN(config)#webwall port2 on
AN(config)#webwall port1 on
If you have configured a DNS server and need to enable WebWall on the interface through
which DNS traffic will pass, you need to add corresponding access rules to permit traffic on
port 53.
If you need to enable WebWall on the interface where the “synconfig to” or “synconfig
from” command is applied, you need to manually add access rules to permit traffic on port
65,519; otherwise, configuration synchronization will fail. In an HA scenario, you can either
configure the “ha synconfig bootup on” command on the local and peer units or manually
add access rules to permit the synchronization data.
After WebWall is enabled, if you need to adjust configurations, please note that the SSH or
WebUI session in use might be terminated due to misconfigurations.
After finishing configuration of access rules and groups, you can use the “show accesslist” and
“show accessgroup” commands to check the configurations.
AN(config)#show accesslist
accesslist permit tcp 10.10.10.30 255.255.255.255 0 192.168.10.1 255.255.255.255 22 100
accesslist permit tcp 10.10.10.30 255.255.255.255 0 192.168.10.1 255.255.255.255 8888 100
accesslist permit tcp 0.0.0.0 0.0.0.0 0 10.10. 0.10 255.255.255.255 80 150
accesslist deny tcp 10.10.10.30 255.255.255.255 0 10.10. 0.10 255.255.255.255 0 150
accesslist permit icmp echorequest 192.168.10.0 255.255.255.255 192.168.10.1 255.255.255.255
50
accesslist permit icmp echoreply 192.168.10.1 255.255.255.255 192.168.10.0 255.255.255.0 50
AN(config)#show accessgroup
accessgroup 50 port2
accessgroup 100 port2
accessgroup 150 port1
If you run into problems with access rules, keep your configurations simple. With multiple access
groups, you can apply them once at a time to check which access rule is causing problems. You
can turn the WebWall off to determine if the WebWall itself is indeed causing the problem.
To view the interfaces on which WebWall is enabled, run the “show interface” command. For
example:
AN(config)#show webwall
webwall "port2" on 0
webwall "port3" on 0
To view the packet drop statistics on the webwall, run the “show interface” command. See italics
in the following example.
AN(config)#show interface
port2(port2): flags=2008842<BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
ether 00:25:90:39:97:f1
media: autoselect
status: no carrier
webwall status: ON
Hardware is i82574l
Input queue: 0/4096 (size/max)
total: 0 packets, good: 0 packets, 0 bytes
broadcasts: 0, multicasts: 0
0 64 bytes, 0 65-127 bytes,0 128-255 bytes
0 255-511 bytes,0 512-1023 bytes,0 1024-1522 bytes
0 input errors
0 runts, 0 giants, 0 Jabbers, 0 CRCs
0 Flow Control, 0 Fragments, 0 Receive errors
0 Driver dropped, 0 Frame, 0 Lengths, 0 No Buffers
0 overruns, Carrier extension errors: 0
Output queue: 0/4096 (size/max)
total: 0 packets, good: 0 packets, 0 bytes
broadcasts: 0, multicasts: 0
0 64 bytes, 0 65-127 bytes,0 128-255 bytes
0 255-511 bytes,0 512-1023 bytes,0 1024-1522 bytes
0 output errors
0 Collisions, 0 Late collisions, 0 Deferred
0 Single Collisions, 0 Multiple Collisions, 0 Excessive collisions
0 lost carrier, 0 WDT reset
packet drop (not permit): 0(Number of packets dropped by default denial rules)
tcp 0 udp 0 icmp 0 ah 0 esp 0
packet drop (deny): 0(Number of packets dropped by configured denial rules)
tcp 0 udp 0 icmp 0 ah 0 esp 0
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
16.2.1 Overview
While providing the SLB service, the appliance supports the configuration of advanced ACL to
improve the security of internal resources. This function helps:
Protect against attackers that maliciously launch large volume of access traffic.
ACL rules impose limitations on the connections per second (CPS) and the number of concurrent
connections (CC) initiated by clients in a specific subnet. ACL rules apply to all virtual services
that run over TCP.
Note:
The ACL rule supports two control modes and two control types.
Control modes
“total” mode indicates that all the clients on a subnet will be restricted by the ACL rule as a
whole.
“per-ip” mode indicates that every client on a subnet will be restricted by the ACL rule
individually.
Control types
The administrator needs to combine the control modes and control types as needed for configuring
ACL rules for a subnet. There are four combinations of ACL rules for the same subnet. See the
following table.
The total CPS of all the clients on the The total CC of all the clients on
total
subnet cannot exceed the specified the subnet cannot exceed the
One or more rules of the preceding combinations are allowed for a subnet on the APV appliance.
If multiple rules have been configured for a subnet, these rules will work at the same time.
Subnet Nesting
The ACL rule supports subnet nesting. If the IP address of a client hits multiple subnets, the client
is subject to the restriction of all the ACL rules of the smallest subnet hit, and of the “total” rules
of all the upper subnets of the smallest subnet. The system supports 3-layer subnet nesting at most.
For example:
Then, clients on the subnet of 10.8.1.0/24 will be subject to the restriction of:
The administrator can configure ACL whitelists to make the clients on the whitelists free from
restriction of ACL rules. When both ACL rules and whitelists are applied to the same virtual
service, the whitelists have higher processing priorities than the ACL rules.
If the IP address of a client belongs to the subnet defined in an ACL whitelist, the client will not
be subject to the restriction of any ACL rule. However, the CPS and CC of the client will be
counted in the total CPS and CC of the subnet it belongs to respectively.
For example:
Whitelist: 10.8.1.10/32
Then, the maximum total CC of the subnet 10.8.1.0/24 (excluding 10.8.1.10) is 9000.
ACL rules and ACL whitelists can be applied not only to an individual virtual service, but also to
virtual services globally. The advanced ACL function supports all TCP-based virtual services.
When accessing a specified virtual service, a client is subject to the restriction of not only the ACL
rules applied to this virtual service individually but also the ACL rules applied globally.
For example:
Then, when the client whose IP address is 10.8.1.10 accesses the virtual service “vs1”, the
allowed maximum CC is 900.
Services
For Layer 7 HTTP and HTTPS virtual services with the insert cookie policy used, the APV
appliance will insert cookies in the responses sent to the clients. The APV appliance can identify
these clients with the inserted cookies. When a client accesses the virtual service again and carries
the cookie inserted by the APV appliance in the requests, the client will not be subject to the
restriction of any ACL rule. However, the CPS and CC of the client will be counted in the total
CPS and CC of the subnet it belongs to respectively.
Configuration Steps
To configure ACL rules, you need to first add ACL rules and then apply these rules to a specified
virtual service or apply them globally.
For example:
For example:
To configure ACL whitelists, you need to first add ACL whitelists and then apply these whitelists
to a specified virtual service or apply them globally. ACL whitelist configuration is optional.
For example:
For example:
After the preceding configurations are completed, the APV appliance will:
Restrict the maximum total CPS of all clients on the subnet 61.130.10.0/24 to 1,000,000.
Restrict the maximum CPS of every client on the subnet 61.130.10.0/24 to 10,000.
Not restrict the maximum CPS of the client whose IP address is 61.130.10.10.
Restrict the maximum total CC of all clients on the subnet 61.130.10.0/24 to 50,000.
17.1 Overview
As the IPv4 addresses exhaust, how to transit from the IPv4 network to the IPv6 network becomes
a challenge for many enterprises and organizations.
The APV appliance provides comprehensive support for IPv6 to help enterprises and
organizations with the IPv4-to-IPv6 transition without any business interruption. With the
IPv4/IPv6 dual stack support on APV, the IPv4 resources can be delivered to the IPv6 users, and
vice versa. As a result, the IPv4-based and IPv6-based networks can be easily interconnected and
intercommunicated. What’s more, the APV appliance in the IPv6 network can achieve the same
level of secure and efficient application delivery as it does in the IPv4 network.
This chapter will introduce functions and configurations about IPv6 SLB, DNS64/NAT64,
DNS46/NAT46, IPv6 NAT and NDP.
17.2.1 Overview
APV provides comprehensive IPv6 support for the SLB module. For the service layer, SLB
services from Layer 2 to 7 all support IPv6. For the service type, all of the service types except
RDP, SIPUDP and SIPTCP support IPv6. For the SLB method and policy, all of the SLB methods
except SNMP and all of the SLB policies support IPv6. Additionally, for the three SLB
deployment modes, IPv6 is supported as follows:
IPv6 to IPv6 (supported by the reverse proxy mode, transparent mode and triangle mode)
Among which, the reverse proxy mode can work in the IPv4/IPv6 co-existing network
environment, as well as in the IPv4 only or IPv6 only network environment. The transparent mode
and triangle mode can only work in the IPv4 only or IPv6 only network environment.
The following figure shows the topology of the “IPv6 to IPv4” deployment method.
Likewise, the “IPv4 to IPv6” deployment method can be adopted by configuring an IPv4 address
for the virtual service and an IPv6 address for the real server.
For the “IPv6 to IPv6” deployment method, both the virtual service and real server should be
configured with IPv6 addresses.
For example:
For example:
3. Execute the following command to configure the real server and SLB group:
For example:
17.3.1 Overview
The DNS64 function converts the DNS AAAA queries sent from IPv6 clients to DNS A queries
and then converts the DNS A responses to DNS AAAA responses. This ensures that IPv6 clients
can access IPv4 servers. The APV appliance returns the translated IPv6 addresses to IPv6 clients.
When IPv6 clients use these IP addresses to access IPv6 servers, the NAT64 (Network Address
Translation IPv6 to IPv4) function converts the IPv6 packets sent from these clients to IPv4
packets. When the APV appliance receives IPv4 packets from IPv4 servers, the NAT64 function
converts IPv4 packets to IPv6 packets. This ensures that IPv6 clients can communicate with IPv4
servers normally. The DNS64 and NAT64 functions can be deployed on two APV appliances
separately, or deployed on one APV appliance.
1. An IPv6 client (2012:1081::a03:30b) sends a DNS AAAA query to the APV appliance
(2012:1001::3ff:40b) to resolve the domain name “www.example.com”.
2. The APV appliance sends the DNS AAAA query to the DNS AAAA authoritative server for
the domain name.
3. If the DNS AAAA authoritative server has no AAAA record for the domain name, it will
return an empty DNS AAAA response to the APV appliance. The APV appliance will ignore
this response.
4. The APV appliance waits for 100 ms after sending the DNS AAAA query. If the APV
appliance does not receive any valid DNS AAAA response, it will send a DNS A query to the
DNS A authoritative server for the domain name.
5. The APV appliance receives the DNS A response (for example, A: www.example.com -
192.168.2.10) from the DNS A authoritative server.
6. The APV appliance converts the DNS A response to a DNS AAAA response (for example,
AAAA: www.example.com - 64:ff9b::192.168.2.10) by adding the configured IPv6 prefix.
Then, the APV appliance returns the converted DNS AAAA response to the IPv6 client.
7. The IPv6 client uses the converted IPv6 address to access “www.example.com”.
8. The APV appliance converts the IPv6 packet (src: 2012:1081::a03:30b; dst:
64:ff9b::192.168.2.10) sent from the client to an IPv4 packet (src: 192.168.2.1; dst:
192.168.2.10), and sends the IPv4 packet to the target IPv4 server.
9. The IPv4 sever returns an IPv4 packet (src: 192.168.2.10; dst: 192.168.2.1) to the APV
appliance.
10. The APV appliance converts the IPv4 packet to an IPv6 packet (src: 64:ff9b::192.168.2.10;
dst: 2012:1081::a03:30b) and returns the IPv6 packet to the IPv6 client.
To make the DNS64 function work properly, you need to configure the “default” and “backup”
policies for this virtual service. The APV appliance forwards DNS AAAA queries based on the
“default” policy and forwards DNS A queries based on the “backup” policy. Therefore, the real
servers associated with the “default” policy should be DNS servers that can answer AAAA
records, and those associated with the “backup” policy should be DNS servers that can answer A
records.
For example:
For example:
Note: The DNS virtual service on which the DNS64 function is enabled can be associated
with groups by using the “default” or “backup” policy only.
For example:
For example:
Note:
If the DNS64 and NAT64 functions need to work together on one APV appliance,
make sure that the values of the parameters “dns64_prefix” and “nat64_prefix” are
the same.
After an IPv4 address pool is configured for the NAT64 function and the function is
enabled, please do not delete the IPv4 address pool. Otherwise, the NAT64 function
will be disabled and its related configuration will be deleted.
17.4.1 Overview
The DNS46 function converts the DNS A queries sent from IPv4 clients to DNS AAAA queries
and then converts the DNS AAAA responses to DNS A responses. It also creates mapping records
between the IPv6 addresses and IPv4 addresses. The APV appliance returns the translated IPv4
addresses to IPv4 clients. When IPv4 clients use these IPv4 addresses to access IPv6 servers, the
NAT46 function converts IPv4 packets sent from clients to IPv6 packets based the created
mapping records. When the APV appliance receives IPv6 packets from IPv6 servers, the NAT46
function converts IPv6 packets to IPv4 packets. This ensures that IPv4 clients can communicate
with IPv6 servers normally. Because the DNS46 and NAT46 functions both use the mapping
records, they must be deployed on one APV appliance.
1. An IPv4 client (61.130.10.10) sends a DNS A query to the APV appliance (210.108.10.1) to
resolve the domain name “www.example.com”.
2. The APV appliance sends the DNS A query to the DNS A authoritative server for the domain
name.
3. If the DNS A authoritative server has no A record for the domain name, it will return an
empty DNS A response to the APV appliance. The APV appliance will ignore this response.
4. The APV appliance waits for 100 ms after sending the DNS A query. If the APV appliance
does not receive any valid DNS A response, it will send a DNS AAAA query to the DNS
AAAA authoritative server for the domain name.
5. The APV appliance receives the DNS AAAA response (for example, AAAA:
www.example.com - 2012:1081::a03:30b) from the DNS AAAA authoritative server.
6. The APV appliance converts the DNS AAAA response to a DNS A response (for example, A:
www.example.com - 210.108.10.10) based on the configured address pool (that is, the IPv4
mapping subnet configured by using the command “ipv6 dnsnat46 ipmap”). Then, the APV
appliance returns the converted DNS A response to the IPv4 client. Meanwhile, the system
creates a mapping record between 2012:1081::a03:30b and 210.108.10.10.
7. The IPv4 client uses the converted IPv4 address to access “www.example.com”.
8. The APV appliance converts the IPv4 packet (src: 61.130.10.10; dst: 210.108.10.10) sent
from the client to an IPv6 packet (src: 2012:1081::a03:30a; dst:2012:1081::a03:30b) based
on the created address mapping record, and sends the IPv6 packet to the target IPv6 server.
9. The IPv6 server returns an IPv6 packet (src: 2012:1081::a03:30b; dst: 2012:1081::a03:30a) to
the APV appliance.
10. The APV appliance converts the IPv6 packet to an IPv4 packet (src: 210.108.10.10; dst:
61.130.10.10) and returns the IPv6 packet to the IPv4 client.
To make the DNS46 and NAT46 functions work properly, you need to configure the “default” and
“backup” policies for this virtual service. The APV appliance forwards DNS A queries based on
the “default” policy and forwards DNS AAAA queries based on the “backup” policy. Therefore,
the real servers associated with the “default” policy should be DNS servers that can answer A
records, and those associated with the “backup” policy should be DNS servers that can answer
AAAA records.
For example:
2. Execute the following command to enable both the DNS46 and NAT46 functions for a
specified DNS virtual service:
For example:
3. Execute the following command to configure an IPv4 subnet used to create the address
mapping table:
For example:
4. Execute the following command to specify the IPv6 address pool used by the NAT46
function:
For example:
Note: After an IPv6 address pool is configured for the NAT46 function and the function is
enabled, please do not delete the IPv6 address pool. Otherwise, the NAT46function will be
disabled and its related configuration will be deleted.
5. Execute the following command to set the idle timeout period for NAT46 TCP connections:
For example:
17.5.1 Overview
The APV appliance not only supports NAT64 and NAT46, but also supports “IPv6 to IPv6” NAT,
which is able to translate the IPv6 addresses on the internal network to the IPv6 addresses on the
Internet. In addition, NAT pool configurations also support IPv6.
Note:
“IPv6 to IPv6” NAT supports TCP and UDP packets, but does not support ICMP and
FTP packets.
When configuring the “IPv6 to IPv6” NAT, the parameter “gateway” cannot be
configured in the “nat port {pool_name|vip} <source_ip> {netmask|prefix}
[timeout] [gateway] [description]” command.
For example:
17.6 NDP
17.6.1 Overview
NDP (Neighbor Discovery Protocol), a key protocol of the IPv6 stack, can be used for obtaining
the link address information of other neighbor nodes connected with the local nodes.
Similar to the ARP (Address Resolution Protocol) of the IPv4 stack, NDP can perform address
transformation between the network layer and the link layer. The difference is that NDP uses
ICMPv6 (Internet Control Message Protocol version 6) and multicast to manage the information
exchanged among the neighbored nodes (within the same link), and keeps the address mapping
between the network layer and the link layer in the same subnet.
For example:
Chapter 18 ePolicy
18.1 Overview
ePolicy is a script-based function for extending the capabilities of the APV appliance. Using the
scripts written in Tools Command Language (TCL), you can customize new features in addition to
the existing functions on the APV appliance. For example, the APV appliance can be customized
to support more application protocols, precisely control IP application traffic in both incoming and
outgoing directions, or control the access of the specified client to real services.
Event
Command
18.2.1 Event
ePolicy uses an event-driven and message-response mechanism. The APV appliance defines an
event for every action occurring in each Client-APV-Server connection. When such an event
occurs, the APV appliance will process traffic according to preconfigured ePolicy commands.
18.2.2 Command
ePolicy uses commands to instruct the APV appliance to process traffic after an event occurs, such
as rewriting packet contents, selecting real servers, selecting groups, or querying whether a group
has valid real servers.
For detailed information of events, commands, and command invocation rules, contact Array
Customer Support for related documents.
Setting script: specifies the traffic type of a virtual service. The following table lists the
setting scripts that are currently supported:
Runtime script: specifies the action of the APV appliance for an event. The content of a
runtime script should be written according to the actual requirement based on events,
commands, and command invocation rules. For the examples of the runtime scripts, contact
Customer Support for related documents.
Balance loads. When being applied to Server Load Balancing (SLB), ePolicy can work as
SLB policies and collaborate with SLB methods to realize load balancing among real
services.
Analyze the packet contents of the HTTP, Simple Object Access Protocol (SOAP),
eXtensible Markup Language (XML), and Diameter protocols.
Receive, send, analyze, and discard Generic TCP and TCPS packets.
Persistent IP (pi)
Hash IP (hi)
SNMP (snmp)
1. Prepare setting and runtime scripts according to events, commands, and command invocation
rules.
The following takes balancing loads among servers according to HTTP request packet method to
describe how to configure ePolicy.
Name Content
when HTTP_REQUEST_HEADER {
if { [http::method] == "GET" } {
slb::select_server realserver_1
Runtime Script http_slb.tcl } else {
slb::select_server realserver_2
}
}
In which, “http_slb.tcl”, “realserver_1” and “realserver_2” are the names of SLB real servers.
For detailed information of events, commands, and command invocation rules, contact Customer
Support for related documents.
For example:
For example:
Script
Execute the following command to associate the virtual service with the setting script:
For example:
Script
Execute the following command to associate the virtual service with the runtime script:
For example:
Direct HTTP request packets whose method is GET to real server realserver_1.
Direct HTTP request packets with other methods to real server realserver_2.
Chapter 19 Logging
19.1 Overview
The Logging mechanism used by the APV appliance is Syslog compliant. System error and HTTP
access information during proxy application are logged by using the logging subsystem. Syslog is
a standard program for Unix and there are also Syslog implementations for Windows. On the Unix
platform, syslog is started by the syslogd daemon. The syslogd daemon takes charge of receiving
and storing log messages from local machine or remote machine, which listens at UDP 514 port.
APV appliance supports three remote log servers.
19.2.1 Syslog
Syslog is a protocol that is used for the transmission of event notification message across
networks.
Syslog logging has eight valid levels of log message severity: emerg, alert, crit, err, warning,
notice, info and debug. And the supported facilities are LOCAL0 to LOCAL7. Users can view the
internal log buffer, select the transport protocol, and configure syslog source and destination ports
and the alerts on log message string match.
HTTP Access Logging supports four standard formats: Combined, WELF (WebTrends Enhanced
Log), Common and Squid. And users can define their own logging format by using the “log http
custom” command.
Note: The APV appliance will record an HTTP access log only after the HTTP
communication between the client and the Web server is completed successfully.
Log filtering in ArrayOS allows administrators to collect only the logs that they are interested in
instead of having to capture all the logs. For example, the administrator of “www.site1.com” may
want to only collect the HTTP access logs for “www.site1.com”. Knowing if the logs contain a
keyword “site1.com”, the administrator can create a filter for a log definition that captures only
the logs which match the keyword. The administrator will now have a log file which contains only
the desired logs.
If multiple log filters are set on a syslog host, the logs matching one of the filter strings will go to
the syslog host.
After being enabled, the local syslog host starts to receive the system logs sent by the system on
the IP address 127.0.0.1 and the UDP port 515. The local syslog host can save a maximum of
50,000 system logs for every log level. These system logs stored on the local syslog host can be
viewed and exported via the WebUI.
To enable the local syslog host, execute the “log localhost on” command in CLI. Alternatively, on
the WebUI, select Admin Tools > System Logs > Log Settings, set the Local Syslog Host slider
to Enable in the Log Settings area and click the Save Changes button, as shown in the following
figure.
To view the system logs stored on the local syslog host, click the Local Syslog Host tab, as shown
in the following figure.
Operation Command
Enable the logging log {on|off}
AN(config)#log on
AN(config)#log rfc5424 on
The remote host IP address must be specified in dotted IP format. The remote port is optional and
the default value is 514. The transport protocol for the syslog messages can be either UDP or TCP
and the default is UDP. In our example, the host of 10.2.37.1 is listening for log message at UDP
514 port.
No more than 3 log filters can be set on one syslog host. Log filter canot be set on the syslog host
whose ID is 0 (it is configured by the command “log host”). After this command is executed, only
the logs matching this filter string go to the syslog host.
Once a log level is set, messages with level below the configured level will be ignored. The
default level is info.
HTTP access information can be logged in one of the standard formats Squid, WELF, Common
and Combined, or it can be logged in a custom format specified by the user.
You can run the command “log test” to generate an emerg-level log.
AN(config)#log test
You can run the following command “show log buff {forward|backward} [match_str]” to view
logs in the log buffer. The parameters “backward” and “forward” are used to display the logs that
are latest and first generated respectively.
You can run the command “clear log buff” to clear logs from the log buffer.
20.1.1 Overview
This chapter will focus on various configuration maintenance elements, such as downloading new
ArrayOS software, rebooting your APV appliance, reverting your configuration to a previously
saved status or returning the APV appliance to its factory default settings among other closing
strategies.
The final series of configuration options concern the running operation of your APV appliance and
its relationship with the rest of the network architecture. Through the various subfolders (within
the WebUI) that are revealed once you click on the “Admin Tools” folder you will discover a
series of sub-folders allowing you to set administrative passwords, perform configuration
synchronization, set SNMP traps and define reboot strategies among other operations. Otherwise
all of these features may be configured via the CLI.
Operation Command
admin aaa {on|off}
Configuring External admin aaa method [radius|tac_x]
Authentication admin aaa server <server_id> <host_name|ip_address> <port>
<secret>
System shutdown and system shutdown [halt|poweroff]
reboot system reboot [interactive|noninteractive]
clear config file
clear config secondary
clear config primary
clear config all
clear config factorydefault
Configuration file clear config timeout
maintenance write memory
write file <file_name>
write net tftp <ip_tftp> <file_name>
write net scp {remote_server_ip|name} <user_name>
<config_file_name>
config memory
Operation Command
config net tftp <tftp_server_ip> <config_file_name>
config file <file_name>
Software upgrade system update <url> [immediate|deferred]
synconfig peer <peer_name> <peer_ip>
Configuration synconfig challenge <code>
Synchronization synconfig to <name>
synconfig from <name>
synconfig peer <peer_name> <peer_ip>
SDNS
synconfig challenge <code>
Synchronization
synconfig sdns to <peer_name>
graph name <new_name>
graph rename <old_name> <new_name>
Monitoring graph settings displaymode {nostack|stack} <graph_name>
graph item <graph_name> <module_name> <type> [service] <scale>
<color> [order] [legend_string]
ntp {on|off}
ntp server <ip> [version]
NTP
show ntp
clear ntp
xmlrpc {on|off} [https|http]
xmlrpc port <port>
XML RPC
show xmlrpc
clear xmlrpc
ssh remote “user@hostname”
Remote access
telnet “host port”
If you have an external authentication server (RADIUS/TACACS+), you may use these servers to
authenticate the SSH/WebUI logon request. The external authentication will be performed when
the “admin aaa” command is set to ON and the logon user name does not exist in the ArrayOS
system.
AN(config)#admin aaa on
AN(config)#admin aaa method RADIUS
AN(config)#admin aaa server es01 “10.1.1.1” 1812 radiussecret
AN(config)#admin aaa server es02 radius_host 1812 radiussecret
Simply enough, employing the “quit” command will allow you to exit the CLI. In the event you
want to terminate all APV appliance interactions with your network, you will need to use the
“system shutdown” command.
AN(config)#system shutdown
The APV appliance will prompt you with an alert to verify the shutting down process. By entering
“YES”, case sensitive, the APV appliance will commence the shutting down operation. After a
brief, 60-second period, users may turn off the appliance.
In some cases when dealing with configuration changes you might need to reboot the box.
AN(config)#system reboot
When working with configurations there may come a time that you want to experiment with a new
configuration strategy, but not overwrite your known working configuration. The ArrayOS
possesses several options for working with configurations files.
In general, you work with the running configuration and write it to disk by using the “write
memory” command. You can also save the configuration to a file by using the “config file”
command, on the APV appliance. Finally, you may export and import the configuration by using
TFTP.
Now the APV appliance has been returned to its factory default settings.
When working with the “write memory” command, keep in mind that this is the configuration
file that will be loaded when the APV reboots. If you have made changes and want to clear the
configuration currently running, use the “clear config” command.
At any point when you want to import a previously saved configuration, you will need to clear the
current, running configuration as previously discussed in this chapter. Once this is completed, you
can import the new configuration. The APV appliance affords you the opportunity to save
configurations to three separate places; the “memory” file which is where the APV appliance calls
up configuration settings upon reboot, the “file” where the APV appliance can store several
different configurations, and to the “net” which refers to saving a file to a remote location on the
network. To save configuration files:
To recall a previously saved configuration and merge it into the running parameters of the
appliance:
AN(config)#config memory
AN(config)#config file new_lb
AN(config)#config net tftp 10.10.0.3 default_config
When loading the configuration file while the box is running, it is important to remember that the
configuration is merged with the running configuration. So you need to choose to clear the
appropriate configuration from the APV appliance before you load a configuration file. For
example, if you have 5 real servers defined and execute the “config net tftp 10.10.0.3
default_config” command and if that configuration file has 5 real servers using the same real
names you will get an error since you cannot have duplicate real server names.
To see the current version of ArrayOS software that is running, we use the “show version”
command.
AN(config)#show version
Host name : AN
System CPU : Intel(R) Core(TM)2 Quad CPU
System RAM : 3842964 kbytes.
System boot time : Wed Aug 07 15:35:45 GMT (+0000) 2015
Current time : Wed Aug 07 15:44:56 GMT (+0000) 2015
System up time : 9 mins
Platform Bld Date : Mon Aug 5 23:20:10 CST 2015
SSL HW : HW ( 1X16C ) Initialized
Compression HW : No HW Available
Power supply : 2U, AC, 2-cords, Redundancy
Network Interface : 4 x Gigabit Ethernet copper
Model : Array APV 5600
Serial Number : 0437A3345200010003011044316464
Licensed Features : WebWall Clustering L4SLB L7SLB Caching
SSL tProxy AppGateway SwCompression LLB GSLB
QoS MultiLang DynRoute FFO REDUNDANT IPv6
License Key : 1e87c3c2-347f78a0-839c1404-39aae1d5-6c47b97d-00000001-0
1d5d9ab-20130409-99999999
First, contact Customer Support to gain access to the software and documentation repository.
Contact your customer support representative or send email to: [email protected].
Once you have received a password and verified with a customer support engineer that ArrayOS
needs upgrade, you can download the software image using the Array Networks website. You
should download the image to either a local Web server or anonymous FTP server.
It is recommended that you use the serial console to upgrade the ArrayOS. Once you have a
console connection you can upgrade the appliance by using the “system update” command.
Currently the upgrade procedure supports two upgrade methods: HTTP or FTP. The commands
are identical except from the URL.
For example, use the command to upgrade the appliance from 192.168.10.10:
Note: If you are to use a DNS name like: system-update http://s5.sj.example.com, make
sure that you have correctly setup the resolving on the APV appliance, using the “ip
nameserver” command to define your DNS server for the “s5” host or use the “ip host”
command to locally define the IP address of the “s5” host. Otherwise you will get an error
when you try to download the software image.
The ArrayOS will then shutdown all load balancing features and download the software image,
verify that the software is produced at Array Networks and then install it. If there is any problem
with the software image, the CLI will abort the upgrade and display a prompt on the screen.
Otherwise you should get a prompt on the console stating that the upgrade was successful and the
APV appliance will reboot. Upon reboot, you should use the “show version” command to verify
that the upgrade is successful.
Caution:
If executing this command via an SSH connection and if the connection is lost during update
procedure, the APV appliance will not be able to complete the update process.
Do not disconnect the connections to the APV appliance during the system updating process.
Software Licenses
Some software features of the APV appliance may be under software license key control. If you
need these software features, please contact customer support ([email protected]) to
obtain a new license key.
The Configuration Synchronization feature of the APV appliance allows administrators to transfer
configuration information among APV appliances within the same network. Configuration
Synchronization is a set of commands that allow you to manage and configure boxes within a
network. You may transfer configuration information from one APV appliance in a network to
other APV appliances within the same network. By using configuration synchronization, you can
quickly setup an Active-Standby configuration. The rest of the section will cover how to use this
feature.
Assume that IP addresses of APV1 and APV2 are 192.168.1.1 and 192.168.1.2. To synchronize
the configuration from APV1 to APV2, the following configurations need to be made:
AN1(config)#synconfig to machine2
Note: If WebWall is enabled for the interface that is used by the “synconfig” command for
configuration synchronization with peers, you need to add the corresponding accesslist
rules to allow the traffic to come in through port 65519 on both Array appliances (both
local and remote peers).
Administrators can synchronize SDNS configurations and BIND9 zone files except SDNS
member configurations from a local APV appliance to remote peers.
In the following example, SDNS configurations and BIND9 zone files except SDNS member
configurations on APV1 are synchronized to remote APV2.
20.1.2.2.7 Monitoring
The APV appliance allows the administrator to view a wide range of pertinent network data
through a series of pre-designed and custom (administrator defined) graphs.
AN(config)#graph name aa
AN(config)#graph rename aa bb
AN(config)#graph settings displaymode stack bb
AN(config)#graph item bb "System" "CPU Utilization" "1" "red" "2"
The Network Time Protocol (NTP) time synchronizer enables the APV appliance to synchronize
the system time with the specified NTP server.
After the NTP time synchronizer is enabled, the APV appliance will automatically synchronize the
system time with the specified NTP server at the interval of about 15 minutes.
Attention:
Do not change the system time of the APV appliance after enabling the NTP time
synchronizer.
Array APV appliance should be used as the NTP client rather than the NTP server.
If multiple NTP servers are configured, the APV appliance will calculate the round-trip delays
according to the time information in the response packet from each NTP server, and synchronize
its system time with the NTP server with the minimum delay.
AN1(config)#ntp on
Users also can use the command “show ntp” to view the current NTP configuration.
AN1(config)#show ntp
ntp server 207.46.197.32 4
ntp on
time since restart: 1481
time since reset: 1481
packets received: 21
packets processed: 0
current version: 0
previous version: 0
bad version: 0
access denied: 0
bad length or format: 0
bad authentication: 0
rate exceeded: 0
The following explains the items in the output information:
Time since restart: The time in hours since the system was last rebooted.
Time since reset: The time since the statistics were reset and the system statistics monitoring
file was updated. This is designed for busy servers, such as those operated by NIST, USNO, and
intended as early warning detector of clogging attacks.
Packets processed: The number of packets received in response to previous packets sent.
Current version: The number of packets matching the current NTP version.
Previous version: The number of packets matching the previous NTP version.
Access denied: The number of packets denied access for any reason.
Bad length or format: The number of packets with invalid length, format or port number.
XML RPC allows clients to run some CLI commands remotely in ArrayOS. This enables system
programmers to automate remote configuration which is difficult with WebUI.
XML RPC is a Remote Procedure Calling protocol that works over the Internet, which uses HTTP
as a transport mechanism and XML as an encoding.
As shown in the figure below, Client sends an HTTP POST Request to Array APV. XML RPC
message is the body of the HTTP Request, in which the commands to run and the commands’
parameters are specified. Then, Array APV decodes the XML PRC message and executes the
called commands. At last it returns the results formatted in XML to Client.
To realize the communication between the Client and the APV appliance, a Perl script, called
array_xmlrpc.pl, MUST be first executed on Client. The command executed the script is:
In this command, <address> specifies the APV IP address. <port> specifies the port on which the
HTTP server is listening. <data_file> specifies the full path and filename of XML RPC message.
XML RPC message is formatted in XML and contains a <methodCall> tag in which
<methodName> and <params> tags are embedded.
The following is an HTTP POST Request whose body is an XML RPC message:
<value>
<string>****</string>
</value>
</member>
<member>
<name>protocol</name>
<value>
<string>http</string>
</value>
</member>
<member>
<name>name</name>
<value>
<string>array</string>
</value>
</member>
<member>
<name>ip</name>
<value>
<string>10.1.1.1</string>
</value>
</member>
<member>
<name>port</name>
<value>
<int>80</int>
</value>
</member>
<member>
<name>maxconns</name>
<value>
<int>1000</int>
</value>
</member>
<member>
<name>hctype</name>
<value>
<string>tcp</string>
</value>
</member>
<member>
<name>hcup</name>
<value>
<int>1</int>
</value>
</member>
<member>
<name>hcdown</name>
<value>
<int>1</int>
</value>
</member>
</struct>
</value>
</param>
</params>
</methodCall>
In this example, the first three lines (as below) constitute the HTTP Request Header, and the
remaining part HTTP Request body.
In the first three lines of XML RPC message (as below), “slb_real” is the XML RPC method of
the called command “slb real <protocol> <name> <ip> [port] [maxconns] [hc_type] [hc_up]
[hc_down]”. XML PRC method is embedded in a <methodName> tag (Please refer to Appendix
III, in which all XML RPC methods supported by APV are listed.).
The following part specifies the Enable mode and its password, which indicates the user will log
in the Enable mode. “enable_password” is the keyword. The actual password value is embedded
in a <string> tag. Enable password is included in every XML RPC message.
<member>
<name>enable_passwd</name>
<value>
<string>****</string>
</value>
</member>
This portion (as below) specifies the “protocol” parameter of the called “slb_real” method.
“protocol” is the keyword, whose value is embedded in a <string> tag.
<member>
<name>protocol</name>
<value>
<string>http</string>
</value>
</member>
In this example, the parameters of the “slb_real” method include protocol, name, ip, port,
maxconns, hctype, hcup and hcdown。Protocol, name and ip are required, while port, maxconns,
hctype, hcup and hcdown are optional.
Note: In an HTTP Request, more than one XML RPC method can be called.
If the calling is successful, APV will return an HTTP Response formatted in as follows:
If the called command is a “show” command, its output will be displayed in the place of “xmlrpc
command successful”. If there is any error, the error is displayed.
To configure the XML PRC function on APV, you need to configure two commands:
AN1(config)#xml on https
The Remote Management feature of the APV appliance allows administrators to access remote
devices via Telnet & SSH.
To use the Telnet feature on the APV appliance, users can execute the command “telnet “host
port”” as follows:
Password:
[ SRA accepts you ].................succeed
To use the SSH feature on the APV appliance, users can execute the command “ssh remote
“user@hostname”” as follows:
Welcome to Ylmf_OS!
* Information: http://www.ylmf.com/
The APV appliance monitors a variety of useful statistics that provide a good indication of
performance, user and network activity. The APV appliance provides a graphical interface that
can be used to easily monitor various statistics and get a comprehensive picture of the status of the
APV appliance. This graphical interface is called the Flight Deck.
The Array Flight Deck is an additional pop up browser window that, once set, can display a wide
range of real time network operational data. Across the top of the browser window, you will
discover readouts concerning the server health, request rate, cache hits and system usage. Moving
to the left side of the window, you will find reading for the TCP, HTTP and SSL connections. The
three connection figures sum up to total used “TCP pcb” displayed in the output of the “show
memory” command. Sometimes, a pair of TCP connections is created for the same client request,
for example, an SLB client request normally will generate two connections, one is from the client
to APV appliance, and the other is from the APV appliance to the server.
The central portion of the Array Flight Deck is occupied by two configurable graphs. Simply use
the pull-down menu to choose the desired data you wish to track in the real time graphical output.
You can access the Flight Deck from the APV appliance WebUI by clicking the “Flight Deck”
node at the bottom of the WebUI Home configuration tree.
There exists two drop down menus above each graph. The first menu, called “Graph Type”
contains a list of the statistics that can be displayed in the graph. Note that the list is identical for
each graph. The second menu, called “Interval”, is used to control the granularity of the time units
shown on the horizontal axis of the graph, and how often the APV appliance will update the graph.
The default menu option is 5 seconds, which is also the smallest value that can be chosen. When
the value is 5 seconds, the APV appliance will update the graph display every 5 seconds, and the
time will be shown on the horizontal axis in multiples of 5.
For some statistics, it makes sense to use a smaller interval. For example, it might be useful to see
how the number of packets processed by the APV appliance varies in 30 sec. intervals. On the
other hand, you may want to view some statistics over a wider interval. For example, you may
want to look at how the number of concurrent sessions varies from hour to hour, to get a feel for
when most of your end users are logging in.
It is important to note that in order to view any of the statistics in the graphs, you must enable
SNMP. This can be done via the WebUI from the “Graph SNMP Monitoring” page under the
“Admin Tools” node. Some of the statistics also require additional configuration, which will be
described below.
Note: For the sake of security, it is strongly recommended to modify the default SNMP
community string to avoid possible system information interception.
TCP
Active Opens
The number of times CLICKTCP connections have made to a direct transition to the
SYN-SENT state from the CLOSED state
Passive Opens
The number of times CLICKTCP connections have made a direct transition to the
SYN-RCVD state from the LISTEN state
Open Failures
The number of times CLICKTCP connections have made a direct transition to the CLOSED
state from either the SYN-SENT state or the SYN-RCVD state, plus the number of times
TCP connections have made a direct transition to the LISTEN state from the SYN-RCVD
state
Established Conns
The number of CLICKTCP connections for which the current state is either ESTABLISHED
or CLOSE- WAIT
Resets Received
The number of times CLICKTCP connections have made a direct transition to the CLOSED
state from either the ESTABLISHED state or the CLOSE-WAIT state
Resets Sent
Retransmits
The total number of segments retransmitted - that is, the number of CLICKTCP segments
transmitted containing one or more previously transmitted octets
Packet Errors
The total number of segments received in error (for example, bad CLICKTCP checksums)
IP
Packets Received
Packets Sent
Bytes Received
Bytes Sent
Header Errors
Unknown Protocol
No Route Out
UDP
Packets Received
Packets Sent
Invalid Ports
Packet Errors
ICMP
Messages In
Errors In
Unreachable In
Echoes In
Echo Replies In
Messages Out
Errors Out
Unreachable Out
Echoes Out
CPU
% CPU Utilization
When this option is selected, the graph will represent what the percentage of the APV
appliance CPU that is utilized over time. For example, if the plot line on the graph is at the
“75” mark on the graph, then the APV appliance’s CPU is 75% utilized at that time, or is
only 25% idle.
Proxy-Cache
% HTTP/HTTPS requests
Compression
% Bytes Received
% Bytes Sent
20.2.1 Overview
The APV appliance allows creating system administrators and specifying the access control level
(Enable or Config) for administrators. If more flexible control over the administrator privilege is
needed, the role-based privilege management can be used to control the CLI commands an
administrator can execute.
One administrator can be assigned one or more roles, and the logic among the multiple roles is
“OR”. If any role assigned to an administrator permits the execution of a command, then the
administrator can execute this command; if all roles assigned to an administrator deny (or none of
the roles permits) the execution of a command, the administrator cannot execute this command. If
an administrator is not assigned any role, the administrator is allowed to execute all the commands
of his access control level.
Role is a set of privilege rules. A privilege rule consists two parts: rule string and operation
privilege.
Rule String
Rule string defines one or a group of command configurations. Three forms of rule string are
supported:
To configure eligible rule strings, please pay attention to the following notes:
The whole rule string needs to be enclosed in double quotes. If double quotes are further
needed by some parameters in the rule string, please use single quotes instead.
If the entered string has multiple consecutive spaces, the spaces will be regarded as one
space.
The system will check the command entered by the administrator against the rule strings from the
first letter. If the command is identical with or comprises a rule string, the command is regarded as
matching the rule string and will follow the rule.
Operation Privilege
There are two kinds of operation privileges: “permit” and “deny”. The privilege rules are
consequently divided into “permit” rules and “deny” rules, which are respectively used to control
that one command or a group of commands can or cannot be executed by certain roles. If a role is
not configured with any “permit” rule, the role is not permitted to execute any command.
When a command matches a “permit” rule and a “deny” rule at the same time, the system will
assume that the “deny” rule has a higher priority.
For example:
role1:
The system will deny the execution of commands starting with “no slb group method” by
role1, but the execution of other commands starting with “no slb group” is permitted.
role2:
Because the “deny” rule has a higher priority, the system will deny the execution of the
commands starting with “no slb group” by role2, although the “permit” rule allows the
execution of the commands starting with “no slb group method”.
To disallow a role from executing configuration, display and deletion operations about a feature
(for example LLB), please configure the following privilege rule for this role:
Note:
The system provides two pre-defined roles: “SLB” and “NETWORK”. You can
execute the “show role predefined” command to see the privilege rules of the two
roles.
Monitoring WebUI requires the privilege for executing the “show” CLI commands.
Please make sure the administrators who need to monitor a specific feature on
WebUI are assigned the role with the correspondent “show” privileges.
Execute the following command to add an administrator account and specify the access control
level:
For example:
For example:
2. Execute the following commands to configure the privilege rules for the role:
For example:
For example:
Appendix II Abbreviations
Acronym Full Spelling
AAA Authentication, Authorization & Accounting
ACL Access Control List
ADC Application Delivery Controller
API Application Programming Interface
ARP Address Resolution Protocol
ASCII American Standard Code for Information Interchange
ASN.1 Abstract Syntax Notation One
ATCP Array TCP
BGP Border Gateway Protocol
CA Certificate Authority
CDN Content Distribution Network
CDP CRL Distribution Point
CGI Common Gateway Interface
CLI Command Line Interface
CNAME Canonical Name
CPS Connections Per Second
CPU Central Processing Unit
CRC Cyclic Redundancy Check
CRL Certificate Revocation List
CRS Core Rule Set
CSR Certificate Signing Request
DMZ DeMilitarized Zone
DNS Domain Name Service
DoS Denial Of Service
DPS Dynamic Proximity System
FFO Fast Failover
FIFO First-In First-Out
FTP File Transfer Protocol
FTPS FTP over SSL
GMT Greenwich Mean Time
GRE Generic Routing Encapsulation
GSLB Global Server Load Balance (also known as SDNS)
HA High Availability
HC Health Check
HTML HyperText Markup Language
HTTP HyperText Transfer Protocol
HTTPS HyperText Transfer Protocol over Secure Sockets Layer
ICMP Internet Control Message Protocol
ICMPv6 Internet Control Message Protocol version 6