LVS

From Nix-Pro
Jump to: navigation, search

Main idea of LVS

LVS hides real servers behind a virtual IP and load balances the incoming request across all cluster nodes based on a scheduling algorithm. It implements transport-layer load balancing inside the Linux kernel, also called Layer-4 switching.

LVS load balancing types

Network Address Translation (NAT)

Incoming requests arrive at the virtual IP and are forwarded to the real servers by changing the destination IP address. The real servers send the response to the load balancer which in turn changes the destination IP address and forwards the response back to the client. As all traffic goes through the load balancer, it usually becomes a bottleneck for the cluster.

IP Tunneling

LVS sends requests to real servers through an IP tunnel (redirecting to a different IP address) and the real servers reply directly to the client using their own routing tables. Cluster members can be in different subnets.

Direct routing

Packets from end users are forwarded directly to the real server. The IP packet is not modified as the real servers are configured to accept traffic for the shared cluster virtual IP address by using a virtual non-ARP alias interface. The response from the real server is send directly to the client. The real servers and load-balancer (LVS) have to be in the same physical network segment. (layer 2)

Load balancing algorithms

  • round robin
  • weighted round robin
  • least-connec-tion
  • weighted least-connection
  • locality-based least-connection
  • locality-based least-connection with replication
  • destination-hashing
  • source-hashing

As the load-balancer is the only entry point for all incoming requests, it would present a single point of failure for the cluster. A backup load-balancer is needed as well as a monitoring program that can fail over the service along with the connection statuses.

Components

Linux-director - load-balancing server.

Real server - a node that actually processes requests from a client forwarded to it by a Linux-director.

Linux Director Daemon (ldirectord) is used to monitor and administer real servers in the LVS cluster and heartbeat 2 is used as the fail-over monitor for the load balancers (ldirectord).

VIP' - shared virtual IP, that hides a pool of the real servers. It also could be called a clustered IP.

ipvsadm - средство командной строки для настройки и управления LVS таблицами в ядре Linux. The Linux Virtual Server can be used to build scalable network services based on a cluster of two or more nodes.

Ldirectord monitors the health of the real servers by periodically requesting a known URL and checking that the response contains an expected string. If a service fails on a server, then the server is taken out of the pool of real-servers and will be reinserted once it comes back on line.

Load balancing behaviour

In Brief the Linux-director of the cluster redirects service requests to a collection of real servers that will actually perform the services. Файл:lvs2.png
Linux-directors and real servers will have 1 real interface with their IP address and 1 virtual alias interface that will be configured with the shared Virtual IP (VIP) 192.168.0.200.
1. A client will send a request for a web page from 192.168.0.200.
2. Ldirectord will check the IP and port number and If they are matched for a virtual service, a real server is chosen from the cluster by a scheduling algorithm, and the connection is added into the hash table which records connections. 3. The load balancer forwards the packet (VIP is unchanged) to the chosen real server. 4. When the real server receives the forwarded packet, it finds that the packet is for the address on its loopback alias interface, it processes the request and returns the result directly to the client
High availability behaviour
1.Node level monitoring

If one of the nodes (ldirector1/ldirector2) running cluster resources stops sending out heartbeat signals, declare it dead, reboot the node and fail over all resources to a different node.
2.Service level monitoring:
If the VIP or ldirectord service fails, try to restart the service, if it fails, reboot the node and fail over all resources to a different node.
3.Service "stickiness"
If a dead or stand-by node becomes active again, keep the resources where they run now and don't fail-back.

Configuritaion

I this example, Linux Director Daemon (ldirectord) is used to monitor and administer real servers in the LVS cluster and heartbeat 2 is used as the fail-over monitor for the load balancers (ldirectord).

Добавление IPVS правил

Добавление правил осуществляется командой ipvsadm. Ниже список наиболее часто используемых(нужных) параметров:

-A, --add-service - Add a virtual service. A service address is uniquely defined by a triplet: IP address, port number, and protocol. Alternatively, a virtual service may be defined by a firewall-mark.

-t, --tcp-service service-address - Use TCP service. The service-address is of the form host[:port]. Host may be one of a plain IP address or a hostname. Port may be either a plain port number or the service name of port. The Port may be omitted, in which case zero will be used. A Port of zero is only valid if the service is persistent as the -p|--persis-tent option, in which case it is a wild-card port, that is con-nections will be accepted to any port.

-u, --udp-service service-address - Use UDP service. See the -t|--tcp-service for the description of the service-address.

EXAMPLE 1 - Simple Virtual Service

      The following commands configure a Linux Director to distribute  incom-
      ing  requests addressed to port 80 on 207.175.44.110 equally to port 80
      on five real servers. The forwarding method used  in  this  example  is
      NAT,  with  each  of  the  real  servers being masqueraded by the Linux
      Director.
      ipvsadm -A -t 207.175.44.110:80 -s rr
      ipvsadm -a -t 207.175.44.110:80 -r 192.168.10.1:80 -m
      ipvsadm -a -t 207.175.44.110:80 -r 192.168.10.2:80 -m
      ipvsadm -a -t 207.175.44.110:80 -r 192.168.10.3:80 -m
      ipvsadm -a -t 207.175.44.110:80 -r 192.168.10.4:80 -m
      ipvsadm -a -t 207.175.44.110:80 -r 192.168.10.5:80 -m


      As masquerading is used as the forwarding mechanism  in  this  example,
      the  default  route of the real servers must be set to the linux direc-
      tor, which will need to be configured to forward and  masquerade  pack-
      ets. This can be achieved using the following commands:
      echo "1" > /proc/sys/net/ipv4/ip_forward

TODO

http://habrahabr.ru/blogs/sysadm/104621/
http://www.novell.com/communities/node/4846/load-balancing-howto-lvs-ldirectord-heartbeat-2#Goals
http://linuxcommand.org/man_pages/ipvsadm8.html