Parsing and Analyzing Apache Log file using linux commands

This content has been moved to a new URL.

Click here to read the actual article

Load Balancing using Apache mod_jk

After installing apache, install mod_jk module.

1. $ apt-get install libapache2-mod-jk

2. create a file jk.conf(if not present) in mods-available directory and write these lines
JkWorkersFile   /etc/apache2/workers.properties
JkLogFile       /var/log/apache2/mod_jk.log
JkShmFile       /var/log/apache2/mod_jk.shm
JkLogLevel      error

After creating file, may be you need to do symbol linking to mods-enabled directory. Or just disabling and enabling the mod_jk will do.

$ a2dismod jk
$ a2enmod jk

3. create a file /etc/apache2/workers.properties
and write lines for load balancing, creating a load balancer and workers and assembling the workers in load balancer. In below configuration server1 will take 60% load and server2 will take 40%.
#
worker.list=loadbalancer

worker.server1.port=8009 (-- The product server port where you want to forward the request)
worker.server1.host=server1 IP Address (-- The proxy server IP where you want to forward the request)
worker.server1.type=ajp13 (-- Protocol setting, in case of mod_jk module, it will be always ajp13)
worker.server1.lbfactor=60 (-- This parameter to set, how much load this server1 will be given)

worker.server2.port=8009
worker.server2.host=server2 IP Address
worker.server2.type=ajp13
worker.server2.lbfactor=40

worker.loadbalancer.type=lb
worker.loadbalancer.sticky_session=true
worker.loadbalancer.balance_workers=server1,server2

4. In virtual host configuration of Apache
In below example, all the request starting with /web/static/css(js)(images), will be JkUnMounted so , they will be served from apache document root directory(which is /var/www ), and rest all requests will be JkMounted, so they will be forwarded to load balancer and load balancer will forward the reuqest either to server1 or server2. This is done using AJP protocol, so make sure that you have configured AJP protocol in your application server like tomcat. TO configure the tomcat, you can check this url Configuration Apache and Tomcat to user Mod_jk connector for proxy passing


  ServerAlias www.example.com
  DocumentRoot /var/www
  ServerName example.com

  DocumentRoot /var/www/

  JkMount /* loadbalancer
  JkUnMount /web/static/css/* loadbalancer
  JkUnMount /web/static/js/* loadbalancer
  JkUnMount /web/static/images/* loadbalancer

  ErrorLog /var/log/apache/example-com-error_log
  CustomLog /var/log/apache/example-com-access_log combined

Apache performance tuning and security tuning

MaxKeepAliveRequests

It's actually the maximum number of requests to serve on a TCP connection. If you set it up to 100, clients with keepalive support will be forced to reconnect after downloading 100 items. Default in Apache is 100, you can increase it if you have enough memory on the system. If you are serving a page which contain high number of images then keeping is high is better because then it utilize the alive connections to serve the image requests.

KeepAliveTimeout

KeepAliveTimeout determines how long to wait for the next request. Set this to a low value, perhaps between two to five seconds. If it is set too high, child processed are tied up waiting for the client when they could be used for serving new clients.

MaxRequestsPerChild

The MaxRequestsPerChild directive sets the limit on the number of requests that an individual child server process will handle. After MaxRequestsPerChild requests, the child process will die. It's set to 0 by default, the child process will never expire. It is appropriate to set this to a value of few thousands. This can help prevent memory leakage, since the process dies after serving a certain number of requests. Don't set this too low, since creating new processes does have overhead.

Proper user of MPM (Multi-Processing Module)

This I have already explain at this URLConfiguring Apache/Tomcat for serving Maximum number of requests

Security tweaks

1. ServerTokens
This directive configures what you return as the Server HTTP response
Header. The default is 'Full' which sends information about the OS-Type and compiled in modules.
# Set to one of:  Full | OS | Minimal | Minor | Major | Prod
where Full conveys the most information, and Prod the least, you can also set it to "ProductOnly" which is best

ServerTokens ProductOnly

2. ServerSignature
Optionally add a line containing the server version and virtual host
# Set to one of:  On | Off | EMail
You can Set to "EMail" to also include a mailto: link to the ServerAdmin, better to set it to Off

ServerSignature Off

3. TraceEnable 
This Allow TRACE method to enable/disabled
# Set to one of:  On | Off | extended
Set to "extended" to also reflect the request body, best it to make it Off

TraceEnable Off

Configuring Apache/Tomcat for serving Maximum number of requests

In Apache if you get to receive more number of request then it's time to configure your web server.

Apache has got MPM (Multi-Processing-Module) which has got many implementations, but mostly people are using 2 models, either prefork model or worker model. Prefork model is based on creating process as per request comes and the all the existing processes (as a pool) are like shared socket of master socket, So the delegation of request to any respective process is been handled by kernel. Worker model has got threads to handle the request. Prefork is stable in terms of No multi-threading so no clash or ambiguity of any shared library of Apache, though any unknown library can behave erroneous in case of worker model. But at the same time prefork model consumes more memory than worker model.

Using PHP, people suggest to use worker model, in case you are using AJP and then Tomcat, people suggest to use Prefork.

Configuring Prefork MPM for handling more number of requests

< IfModule mpm_prefork_module >
StartServers 10
MinSpareServers 10
MaxSpareServers 20
MaxClients 512
ServerLimit 512
MaxRequestsPerChild 0
< /IfModule>

Max value of MaxClient can go to 256 untill you set ServerLimit also, but once you set ServetLimitm you can increase MaxClients, in general people have reported that Apache can handle max of 4000 requests after that it dies.

You should reduce KeepAliveRequests in apache, because this cause more memory usages by Apache and set the KeepAliveTimeOut for sure.

MaxKeepAliveRequests 50
KeepAliveTimeout 15


Now...
If you are using AJP protocol to connect tomcat, you should configure AJP connector, if you are using HTTP you should configure HTTP connector. In case of mod_proxy, Apache will redirect the request to tomcat in HTTP form, in case of mod_jk Apache will redirect the request to tomcat usin AJP.

< connector port="8009" maxthreads="512" minsparethreads="25" maxsparethreads="75" connectiontimeout="10000" enablelookups="false" redirectport="8443" protocol="AJP/1.3">


< connector port="8181" maxhttpheadersize="8192" maxthreads="50" minsparethreads="10" maxsparethreads="20" enablelookups="false" redirectport="8443" acceptcount="100" connectiontimeout="10000" disableuploadtimeout="true">

And you should make sure that you are able to serve the request from tomcat as fast as possible, I have seen as google give good value, if you can serve request in less than 80ms.

Ref:
1. http://httpd.apache.org/docs/2.0/mod/mpm_common.html#serverlimit
2. http://tomcat.apache.org/tomcat-5.5-doc/config/ajp.html
3. http://www.howtoforge.com/configuring_apache_for_maximum_performance
4. http://af-design.com/blog/2009/02/17/increasing-apache-serverlimit-on-ubuntu/