Parsing and Analyzing Apache Log file using linux commands

This content has been moved to a new URL.

Click here to read the actual article

RSA server certificate CommonName (CN) 'www.domain.com' does NOT match server name

Hi,

I was getting this error on restarting of my Apache,

"I receive the Error: "RSA server certificate CommonName (CN) 'www.domain.com' does NOT match server name"?"

And there was on website which was able to tell me that is something wrong, which is http://www.sslshopper.com/ssl-checker.html

In my case, I had missed an entry of "ServerName" directive in Apache virtual host configuration.

<VirtualHost _default_:443>
        ServerAdmin webmaster@domain.com
        ServerName www.domain.com

SSLEngine on
SSLCertificateFile /root/ssl/domain.crt
SSLCertificateKeyFile /root/ssl/server.key
SSLCertificateChainFile /root/ssl/bundle.crt

</VirtualHost>

So after entering the ServerName, apache error.log file stopped showing the error message. And the website sslshopper also, started getting the certificate of diagnosis, which made me believe, that issue is resolved.

In general, you should also check
1. DNS entry (host name and IP is correct)
2. /etc/hosts file
3. while creating CSR file, did you miss the common name by any chance, In this case create a new CSR and Re-key the certificate and deploy new certificates.
4. Check in Apache or Nginx, ServerName must match, certificates are issued for a fix domain.

Load Balancing using Apache mod_jk

After installing apache, install mod_jk module.

1. $ apt-get install libapache2-mod-jk

2. create a file jk.conf(if not present) in mods-available directory and write these lines
JkWorkersFile   /etc/apache2/workers.properties
JkLogFile       /var/log/apache2/mod_jk.log
JkShmFile       /var/log/apache2/mod_jk.shm
JkLogLevel      error

After creating file, may be you need to do symbol linking to mods-enabled directory. Or just disabling and enabling the mod_jk will do.

$ a2dismod jk
$ a2enmod jk

3. create a file /etc/apache2/workers.properties
and write lines for load balancing, creating a load balancer and workers and assembling the workers in load balancer. In below configuration server1 will take 60% load and server2 will take 40%.
#
worker.list=loadbalancer

worker.server1.port=8009 (-- The product server port where you want to forward the request)
worker.server1.host=server1 IP Address (-- The proxy server IP where you want to forward the request)
worker.server1.type=ajp13 (-- Protocol setting, in case of mod_jk module, it will be always ajp13)
worker.server1.lbfactor=60 (-- This parameter to set, how much load this server1 will be given)

worker.server2.port=8009
worker.server2.host=server2 IP Address
worker.server2.type=ajp13
worker.server2.lbfactor=40

worker.loadbalancer.type=lb
worker.loadbalancer.sticky_session=true
worker.loadbalancer.balance_workers=server1,server2

4. In virtual host configuration of Apache
In below example, all the request starting with /web/static/css(js)(images), will be JkUnMounted so , they will be served from apache document root directory(which is /var/www ), and rest all requests will be JkMounted, so they will be forwarded to load balancer and load balancer will forward the reuqest either to server1 or server2. This is done using AJP protocol, so make sure that you have configured AJP protocol in your application server like tomcat. TO configure the tomcat, you can check this url Configuration Apache and Tomcat to user Mod_jk connector for proxy passing


  ServerAlias www.example.com
  DocumentRoot /var/www
  ServerName example.com

  DocumentRoot /var/www/

  JkMount /* loadbalancer
  JkUnMount /web/static/css/* loadbalancer
  JkUnMount /web/static/js/* loadbalancer
  JkUnMount /web/static/images/* loadbalancer

  ErrorLog /var/log/apache/example-com-error_log
  CustomLog /var/log/apache/example-com-access_log combined

Apache performance tuning and security tuning

MaxKeepAliveRequests

It's actually the maximum number of requests to serve on a TCP connection. If you set it up to 100, clients with keepalive support will be forced to reconnect after downloading 100 items. Default in Apache is 100, you can increase it if you have enough memory on the system. If you are serving a page which contain high number of images then keeping is high is better because then it utilize the alive connections to serve the image requests.

KeepAliveTimeout

KeepAliveTimeout determines how long to wait for the next request. Set this to a low value, perhaps between two to five seconds. If it is set too high, child processed are tied up waiting for the client when they could be used for serving new clients.

MaxRequestsPerChild

The MaxRequestsPerChild directive sets the limit on the number of requests that an individual child server process will handle. After MaxRequestsPerChild requests, the child process will die. It's set to 0 by default, the child process will never expire. It is appropriate to set this to a value of few thousands. This can help prevent memory leakage, since the process dies after serving a certain number of requests. Don't set this too low, since creating new processes does have overhead.

Proper user of MPM (Multi-Processing Module)

This I have already explain at this URLConfiguring Apache/Tomcat for serving Maximum number of requests

Security tweaks

1. ServerTokens
This directive configures what you return as the Server HTTP response
Header. The default is 'Full' which sends information about the OS-Type and compiled in modules.
# Set to one of:  Full | OS | Minimal | Minor | Major | Prod
where Full conveys the most information, and Prod the least, you can also set it to "ProductOnly" which is best

ServerTokens ProductOnly

2. ServerSignature
Optionally add a line containing the server version and virtual host
# Set to one of:  On | Off | EMail
You can Set to "EMail" to also include a mailto: link to the ServerAdmin, better to set it to Off

ServerSignature Off

3. TraceEnable 
This Allow TRACE method to enable/disabled
# Set to one of:  On | Off | extended
Set to "extended" to also reflect the request body, best it to make it Off

TraceEnable Off

Secure your website with SSL - guidelines and experience

1.
First generate the key file
$ openssl genrsa -des3 -out server.key 2048
It will ask for a pass phrase, which will be further used to start the web server, so save it properly

2.
Now generate the CSR (Certificate Signing Request) file
$ openssl req -new -key server.key -out server.csr

This ask informations like, Location, Company Name, Common Name. Its better to ignore the "challenge password". Be careful with entering common name, which has to be your domain name.

If you serve your users with www.example.com, common name should be "www.example.com". Once certificate is issued for www.example.com, it won't be valid for example.com. If you want to secure with and without www, there is a certain preference you'll have to choose at the time of buying the certificate. If you want to secure all subdomains, there will be different prerefernece as well. Depending of number of sub domains you are looking for to make secure, cost will also vary. As of today verisign charges $ 400 USD for one domain, $ 600 for with and without www, and around $ 1500 USD for securing infinite sub domains.

3. Now use this CSR and avail the certificates which is crt file from any CA (certificate authority) company like verisign(costliest), go daddy cheapest (may be $ 10 USD)

4. Once you buy the SSL certificate, the product management will guide you on how to get the certificates. Its very simple.

5. In case of verisign, they will take average of 2 to 4 days for the entire process execution, as they will validate "CSR Verification", "Proof of Organization" and "Proof of Domain Registration".
They would ask company registration certificates also as a part of process. But if you buy from go daddy, no verification process, only based on CSR file they will issue you the certificates within a minute.

6. At the time of downloading the certificates makes sure that you also download the intermediate certificate. Intermediate certificates are connecting the certificate chains. In few browsers(without having intermediate certificate), some users might face unwanted error message.

7. Deploying the certificates, copy these 3 files at the following places and restart Apache

$ cp server.key /etc/ssl/private/
$ cp example.com.crt /etc/ssl/certs/
$ cp intermediate.crt /etc/ssl/certs/


8. Now change in apache
Enable the ssl module, if you are on debian(Ubuntu, RedHat) systems then you can use command a2enmod ssl.
Go to virtual host configuration and write these lines

SSLEngine on
SSLProtocol -all +TLSv1 +SSLv3

SSLCertificateKeyFile /etc/ssl/private/server.key
SSLCertificateFile /etc/ssl/certs/example.com.crt
SSLCertificateChainFile /etc/ssl/certs/intermediate.crt
     
$ /etc/init.d/apache2 restart (It will ask for the pass phrase that you created at step 1)
- and its Done :)
8. To validate everything done properly or not there are several websites to check one is, http://www.sslshopper.com/ssl-checker.html

Make your development system up on Mac

This post is about persisting the experience while installing Mac 10.6.3 (SnowLeopard) on my personal computer.

First of all you should decide if you want to install by using binary then I have no idea. I can tell you to install so many things easily using MacPort software manager.

So first of all follow this link to download and install MacRport http://www.macports.org/install.php

Basic command usages for MacPort
---------------------------
port list variant:no_ssl
port uninstall name:sql
port echo depof:mysql5
port echo apache*
port install mysql5
--------------------------

Now you need Apache + PHP + Mysql
Step 1: sudo port selfupdate
Step 2: sudo port install gawk
Step 3: sudo port install nawk (For me this step was failed, but I gave a damn !!!)
Step 4: sudo port install php5 +apache2 +mysql5-server

Now use the command below to setup the mysql db.
$ sudo /opt/local/lib/mysql5/bin/mysql_install_db --user=mysql

You can change mysql root password using this command
$ /opt/local/lib/mysql5/bin/mysqladmin -u root password 'new-password'

Next, you can configure apache and mysql to start automatically by enter the command below:-
$ sudo launchctl load -w /Library/LaunchDaemons/org.macports.apache2.plist
$ sudo launchctl load -w /Library/LaunchDaemons/org.macports.mysql5.plist

Now you need to configure apache to load php file, open /opt/local/apache2/conf/httpd.conf and add this 2 line:-
LoadModule php5_module modules/libphp5.so
AddType application/x-httpd-php .php

Once saved, you may restart your apache

To start your apache2 and mysql5 manually, type the command below:-
$ sudo /opt/local/etc/LaunchDaemons/org.macports.apache2/apache2.wrapper start
$ sudo /opt/local/etc/LaunchDaemons/org.macports.mysql5/mysql5.wrapper start


Now you'll want Java
Java is usually installed with Mac OS and XCode (and yes you need to install XCode at the very very first thing of above all, you can locate XCode in any of your given DVD with Mac, or you can download also). You can usually find java and javac commands are usually working. The working java home would be usually at /Library/Java/Home

Eclipse : Download it from here http://eclipse.org/downloads/

Tomcat :
Step 1: Download it from here http://tomcat.apache.org/download-60.cgi
Step 2: After downloading tomcat, extract the file and go to bin
Step 3: Create a file setenv.sh and write this line JAVA_HOME=/Lbrary/Java/Home (or whatever java home you have)
Step 4: Save the file, and run "chmod 777 setenv.sh" and you are ready with tomcat
Step 5: ./startup.sh will run the tomcat, you can see the log at ~tomcat_home/logs/catalina.out

Configuring Apache/Tomcat for serving Maximum number of requests

In Apache if you get to receive more number of request then it's time to configure your web server.

Apache has got MPM (Multi-Processing-Module) which has got many implementations, but mostly people are using 2 models, either prefork model or worker model. Prefork model is based on creating process as per request comes and the all the existing processes (as a pool) are like shared socket of master socket, So the delegation of request to any respective process is been handled by kernel. Worker model has got threads to handle the request. Prefork is stable in terms of No multi-threading so no clash or ambiguity of any shared library of Apache, though any unknown library can behave erroneous in case of worker model. But at the same time prefork model consumes more memory than worker model.

Using PHP, people suggest to use worker model, in case you are using AJP and then Tomcat, people suggest to use Prefork.

Configuring Prefork MPM for handling more number of requests

< IfModule mpm_prefork_module >
StartServers 10
MinSpareServers 10
MaxSpareServers 20
MaxClients 512
ServerLimit 512
MaxRequestsPerChild 0
< /IfModule>

Max value of MaxClient can go to 256 untill you set ServerLimit also, but once you set ServetLimitm you can increase MaxClients, in general people have reported that Apache can handle max of 4000 requests after that it dies.

You should reduce KeepAliveRequests in apache, because this cause more memory usages by Apache and set the KeepAliveTimeOut for sure.

MaxKeepAliveRequests 50
KeepAliveTimeout 15


Now...
If you are using AJP protocol to connect tomcat, you should configure AJP connector, if you are using HTTP you should configure HTTP connector. In case of mod_proxy, Apache will redirect the request to tomcat in HTTP form, in case of mod_jk Apache will redirect the request to tomcat usin AJP.

< connector port="8009" maxthreads="512" minsparethreads="25" maxsparethreads="75" connectiontimeout="10000" enablelookups="false" redirectport="8443" protocol="AJP/1.3">


< connector port="8181" maxhttpheadersize="8192" maxthreads="50" minsparethreads="10" maxsparethreads="20" enablelookups="false" redirectport="8443" acceptcount="100" connectiontimeout="10000" disableuploadtimeout="true">

And you should make sure that you are able to serve the request from tomcat as fast as possible, I have seen as google give good value, if you can serve request in less than 80ms.

Ref:
1. http://httpd.apache.org/docs/2.0/mod/mpm_common.html#serverlimit
2. http://tomcat.apache.org/tomcat-5.5-doc/config/ajp.html
3. http://www.howtoforge.com/configuring_apache_for_maximum_performance
4. http://af-design.com/blog/2009/02/17/increasing-apache-serverlimit-on-ubuntu/