managing haproxy connections

Posted on 9th January 2012 in haproxy

I have a question about my haproxy config and Willy Tarreau, the creator of haproxy, got me some answers. His response is in bold:


#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
    log         127.0.0.1 syslog emerg
    maxconn     4000
    quiet
    user        haproxy
    group       haproxy
    daemon
#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will 
# use if not designated in their block
#---------------------------------------------------------------------
defaults
    mode        http
    log         global
    option      abortonclose
    option      dontlognull
    option      httpclose
    option      httplog
    option      forwardfor
    option      redispatch
    timeout connect 10000 # default 10 second time out if a backend is not found
    timeout client 300000 # 5 min timeout for client
    timeout server 300000 # 5 min timeout for server
    stats       enable

listen  http_proxy  localhost:81

    balance     roundrobin
    option      httpchk GET /empty.php
    server      server1 app-1:80 maxconn 15 check inter 10000

I am a bit confused about how the maxconn properties work. There is the global one and the maxconn on the server, in the listen block.

And there is also another one in the listen block which defaults to something like 2000.

My thinking is this: the global one manages the total number of connections that haproxy, as a service, will que or process at one time.

Correct. It’s the per-process max number of concurrent connections.

If the number gets above that, it either kills the connection, or pools in some linux socket?

The later, it simply stops accepting new connections and they remain in the socket queue in the kernel. The number of queuable sockets is determined by the min of (net.core.somaxconn, net.ipv4.tcp_max_syn_backlog, and the listen block’s maxconn).

I have no idea what happens if the number gets higher than 4000.

The excess connections wait for another one to complete before being accepted. However, as long as the kernel’s queue is not saturated, the client does not even notice this, as the connection is accepted at the TCP level but is not processed. So the client only notices some delay to process the request. But in practice, the listen block’s maxconn is much more important, since by default it’s smaller than the global one. The listen’s maxconn limits the number of connections per listener. In general it’s wise to configure it for the number of connections you want for the service, and to configure the global maxconn to the max number of connections you let the haproxy process handle. When you have only one service, both can be set to the same value. But when you have many services, you can easily understand it makes a huge difference, as you don’t want a single service to take all the connections and prevent the other ones from working.

Then you have the server maxconn property set at 15. First off, I set that at 15 because my php-fpm, this is forwarding to on a separate server, only has so many child processes it can use, so I make sure I am pooling the requests here, instead of in php-fpm. Which I think is faster.

Yes, not only it should be faster, but it allows haproxy to find another available server whenever possible, and also it allows it to kill the request in the queue if the client hits “stop” before the connection is forwarded to the server.

But back on the subject, my theory about this number is each server in this block will only be sent 15 connections at a time. And then the connections will wait for an open server. If I had cookies on, the connections would wait for the CORRECT open server. But I don’t.

That’s exactly the principle. There is a per-proxy queue and a per-server queue. Connections with a persistence cookie go to the server queue and other connections go to the proxy queue. However since in your case no cookie is configured, all connections go to the proxy queue. You can look at the diagram doc/queuing.fig in haproxy sources if you want, it explains how/where decisions are taken.

So questions are:

  1. What happens if the global connections get above 4000? Do they die? Or pool in Linux somehow?

They’re queued in linux. Once you overwhelm the kernel’s queue, then they’re dropped in the kernel.

  1. Are the global connection related to the server connections, other than the fact you can’t have a total number of server connections greater than global?

No, global and server connection settings are independant.

  1. When figuring out the global connections, shouldn’t it be the amount of connections added up in the server section, plus a certain percentage for pooling? And obviously you have other restrains on the connections, but really it is how many you want to send to the proxies?

You got it right. If your server’s response time is short, there is nothing wrong with queueing thousands of connections to serve only a few at a time, because it substantially reduces the request processing time. Practically, establishing a connection nowadays takes about 5 microseconds on a gigabit LAN. So it makes a lot of sense to let haproxy distribute the connections as fast as possible from its queue to a server with a very small maxconn. I remember one gaming site queuing more than 30000 concurrent connections and running with a queue of 30 per server ! It was an apache server, and apache is much faster with small numbers of connections than with large numbers. But for this you really need a fast server, because you don’t want to have all your clients queued waiting for a connection slot because the server is waiting for a database for instance. Also something which works very well is to dedicate servers. If your site has many statics, you can direct the static requests to a pool of servers (or caches) so that you don’t queue static requests on them and that the static requests don’t eat expensive connection slots.

Hoping this helps,

Willy

 

It sure did. Thanks Willy!

comments: 0 »

cloud computing infrastructure. simple.

Posted on 4th October 2011 in Cloud Computing, Technology

Cloud computing is an esoteric word to your average layman. You might have heard of it from the Apple employee when you bought your last iPhone, or maybe you heard the term “cloud” from a co-worker when he was showing off his new tablet.

First thing to know.

Cloud computing IS the wave of the future. Ask IBM, Apple or even Microsoft. Everyone will be using cloud computing in the next ten years in one form or another, whether they know it or not.

Okay well what is it?

Cloud computing is the delivery of computing as a service rather than a product, whereby shared resources, software, and information are provided to computers and other devices as a utility (like the electricity grid) over a network (typically the Internet). (http://en.wikipedia.org/wiki/Cloud_computing)

Thanks Wikipedia. I guess you could have just googled it if you wanted to know. But now you do. You are basically using the computing power and storage of a bunch a different machines “somewhere” in the world. You don’t really know where, or care. For example, you punch 2+2 into your laptop’s calculator, instead of your laptop doing the “thinking” it sends the math off to the “cloud” where some server somewhere computes the answer and sends it back. That’s all there is to it. You are using “virtual” computers somewhere in the world.

Why is it good?

Take Wikipedia’s definition and imagine you had to build your own power generator for your house. You will need to first buy one, which is very expensive, then pay the fuel costs, repair costs and hire someone to run it. On top of which, you will have to spend the time finding one you like, hiring the person to take care of it, purchasing the fuel, etc. What a hassle. The problem your local electric company solved long ago has now been solved by cloud computing.

Before cloud computing you were forced to put your companies IT infrastructure into a datacenter. A datacenter keeps the machines at the correct temperature, powers them, provides internet, and typically a low level of support for your systems. But everything else is up to you. Buying the machines, installing the OS, the software, you get the idea. It is work. Plain and simple. More work. More work = more money.

If you were running in a cloud, you could have a new server booted up and running in a hour. No hardware to buy, no additional people to hire and train, nothing. I estimate that one person can manage over 2000 servers in a cloud (obviously depending on what the servers were running).

Okay. Well why is is bad?

It’s like a first car you would have to share with your sister:

1. You have to share it. Computers do not like sharing any more than we do. They want their own memory, their own disk and their own processors. This problem is being solved but it also leads to problem #2.

2. You don’t always know who else is going to ride in the car. For financial, healthcare and other sensitive materials, cloud computing is seen as too loose. Joe Blow might be using the same physical memory that your local bank is. Difficult to sell for most high-security applications.

3.  It will occasionally break down. There are solutions for this, but the point is, there are solutions for it, which indicates this is a problem. It is getting better, but frankly it does happen.

That is it. Nothing crazy about it. Some machines somewhere divided up into pieces to make them look like a bunch of little machines.  This is cloud computing as an infrastructure, not as a service or platform, two additional cloud computing concepts.

comments: 1 »