Apr 29

How to monitoring Apache server performance and status using mod_status

Apache Comments Off on How to monitoring Apache server performance and status using mod_status

Okay it’s occur to me how i can monitor the specific process that happen on apache server, yes we can use top, ps aux , lsof etc command to monitor the process but we can only see a general info on it. And i found mod_status.

What is mod_status ?

mod_status is a built in Apache web server modules (by default) and with that we can  get server status from a web browser. With this module you can easily find out how well your server is preforming. All reports are generated in a html format. You can easily find out following type of information:

  1. Total number of worker serving requests
  2. Total number of of idle worker
  3. Find the status of each worker, the number of requests that worker has performed and the total number of bytes served by the worker
  4. Total number byte count served
  5. Other information such as CPU usage, number of requests per second,current hosts and requests being processed etc.

Setting the mod_status

Btw i use CentOS release 5.3 (Final) , httpd-manual-2.2.3-11.el5_1.centos.3 and httpd-2.2.3-11.el5_1.centos.3 for this tutorial. Okay let’s start the set up:

1. Go to httpd.conf or /apache2.conf on  /etc/httpd/conf/httpd.conf or /etc/apache2/apache2.conf  and add this

<Location /server-status>
    SetHandler server-status
    Order deny,allow
    Deny from all
    Allow from .your_domain.com
</Location>

it means :

server status will be acessible at location http://servername/server-status, access will be denied to everybody but people connecting from domain your_domain.com and any sub-domain from your_domain.com.

you could be using localhost instead of 127.0.0.1 for the only access for your http://servername/server-status. here is the settings:

<Location /server-status>
    SetHandler server-status
    Order deny,allow
    Deny from all
    Allow from 127.0.0.1
</Location>

If you did not want any resctriction you can set allow from all like this:

<Location /server-status>
    SetHandler server-status
    Order deny,allow
    Allow from all
</Location>

2.  Check your configuration and restart the settings

# service httpd restart

3. Access your server-status on your http://servername/server-status

mod1

3. You will see a standard info on the server-status, If you want to get more information from your server activity, you can add this on your httpd.conf/apache2.conf:

ExtendedStatus On

just above the <Location /server-status> directive. But keep in mind that this cannot be set on a per virtualhost basis and will use more resources.
But in another hand, you will be able to access more informations such as:

  • Total accesses
  • CPU usage
  • number of requests/sec, Bytes/sec and Bytes/request
  • See the different clients connected to your server, on which virtual host as well as the page there are requesting, you can even access the time it took to request a specific page as well as the resources required.

Then don’t forget to restart the httpd /apache service

# service httpd restart 

and you will see more info

mod2

and one more thing You can get the status page to update itself automatically if you have a browser that supports “refresh”. Access the page http://servername/server-status?refresh=N to refresh the page every N seconds.

http://localhost/server-status?refresh=2

Here is the detail info of the  http://servername/server-status

Apache Server Status for somedomain.com
Server Version: Apache/1.3.9 (Unix) PHP/4.0b3
Server Built: Mar 4 2000 17:01:01

The first few lines identify and provide a brief description of your server. The server version information includes an incomplete list of some of the modules compiled into your server. Our example server is running on a Unix system and has been compiled with support for the PHP scripting language. (The level of detail provided by the server version line may be limited by theServerTokens configuration directive.)

Current Time: Thursday, 13-Apr-2000 17:22:36 PDT
Restart Time: Thursday, 13-Apr-2000 17:15:26 PDT
Parent Server Generation: 14
Server uptime: 7 minutes 10 seconds
Total accesses: 42 - Total Traffic: 187 kB
CPU Usage: u.1 s.1 cu0 cs0 - .0465% CPU load
.0977 requests/sec - 445 B/second - 4559 B/request
3 requests currently being processed, 5 idle servers

The next block represents the server’s current state. Our example server has only been up for a few minutes and hasn’t yet seen much activity. It is currently dealing with three requests, one of which is my request for the server status itself. The message that five servers are idle servers is a clue that this server is configured to maintain a pool of at least five spare child processes ready to spring into action should the need arise.

K___K_W_........................................................
................................................................
................................................................
................................................................

Scoreboard Key:
   "_" Waiting for Connection, "S" Starting up, "R" Reading Request,
   "W" Sending Reply, "K" KeepAlive (read), "D" DNS Lookup, "L" Logging,
   "G" Gracefully finishing, "." Open slot with no current process

No, that’s not boring morse-code; it’s the “scoreboard,” a pseudo-graphical representation of the state of the server’s child processes. According to the included Scoreboard Key, our server is replying to one request, maintaining two KeepAlive connections, and is maintaining five idle processes. A busier server’s scoreboard would look more like:

WWKW__WW_KKKWK__KKKKWKKKKK_WKKK_KK__KRWKKK__KK___K____WKK__KWWKK
_K___K___WWKWWW_W_W_WWWK_WW_WWWLWWW_KWWKKWKWWKWWKKWW_KWKKKKW__WK
WKWWW_KKWKKKWK_KW_KKKK__KK_KKKWWK_KW__K_KKK_K..........W........
................................................................

For more on pool regulation and KeepAlive, see my earlier HTTP Wrangler column, “An Amble Through Apache Configuration.”

Srv  PID   Acc     M CPU  SS Req Conn Child Slot
0-14 29987 0/24/24 W 0.09 2  0   0.0  0.16  0.16

Client     VHost Request     Request
127.0.0.1  www.mydomain.net  GET /server-status HTTP/1.0

In addition to a more general overview of your server’s activity, mod_status gets down to the nitty-gritty, displaying a snapshot of the individual requests it is currently handling. Let’s take a gander at a fairly representative request. Please note that the output above has been split in half for display purposes.

0-14 Srv
The ID of the child process and its generation. The generation increases each time a child process is restarted, whether due to a server-restart or a limit placed on the number of processes a child is allowed to handle. See the MaxRequestsPerChild directive.
29987 PID
The child’s process ID.
0/24/24 Acc
The first number in this trio is the number of accesses or requests using this connection. For non-KeepAlive connections, this will be 0 since each request makes its own connection and so is always the first (and last). The second is the number of requests handled thus far by this child. The third is the number of requests handled by this slot; the child may have come and gone, its slot taken by another.
W Mode
The child’s mode of operation; one of the following possibilities:  

 

“_” Waiting for Connection, “S” Starting up,

“R” Reading Request, “W” Sending Reply,

 “K” KeepAlive (read),

“D” DNS Lookup, “L” Logging, “G” Gracefully finishing,

“.” Open slot with no current process

0.09
2
0
0.0
0.16
0.16
CPU SS Req Conn Child Slot
Some of the less useful bits and pieces…   

 

CPU: The child’s CPU usage in number of seconds. 
SS: Seconds elapsed since the beginning of the request. 
Req: Milliseconds taken to process the request. 
Conn: Kilobytes transferred across this connection. 
Child: Megabytes transferred by this child process. 
Slot: Megabytes transferred by this slot, across children.

www.mydomain.net VHost
Perhaps your server hosts multiple virtual domains; how would you determine which page is being requested by GET /index.html?. The VHost column helps you sort out which request is coming to which virtual host — in this example, www.mydomain.net.
GET /server-status HTTP/1.0 Request
This particular hit is my request for server-status. The GET bit indicates a simple request for a document (as opposed to sending data to the server using POST). The browser (in this case the Unix command-line wget program) is using HTTP version 1.0.  

 

For more on HTTP, see my earlier HTTP Wrangler column, “Introducing Apache.”

Apr 27

How to Benchmark your Apache and IIS ?

Apache Comments Off on How to Benchmark your Apache and IIS ?

Just several days ago someone ask me , can we benchmark the apache server ? can we set a custom environment to test the apache and server performance ? and i found this tool called ab.

What is Ab ?

ab is a tool for benchmarking your Apache Hypertext Transfer Protocol (HTTP) server. It is designed to give you an impression of how your current Apache installation performs

 

Apache Benchmark Procedures

  • You need to use same hardware configuration and kernel (OS) for all tests
  • You need to use same network configuration. For example, use 100Mbps port for all tests
  • First record server load using top or uptime command
  • Take at least 3-5 readings and use the best result
  • After each test reboot the server and carry out test on next configuration (web server)
  • Again record server load using top or uptime command
  • Carry on test using static html/php files and dynamic pages
  • It also important to carry out test using the Non-KeepAlive and KeepAlive (the Keep-Alive extension to provide long-lived HTTP sessions, which allow multiple requests to be sent over the same TCP connection) features
  • Also don’t forget to carry out test using fast-cgi and/or perl tests

Okay let’ start the benchmark

 

    Benchmarking using HTML only script:

Creat a simple html script

#vim tes.html

<!DOCTYPE HTML PUBLIC “-//W3C//DTD HTML 4.0 Transitional//EN”>

<html>

<head>

<title>Ab test </title>

</head>

<body>

Test using Ab.

</body>

</html>

# ab -n 1000 -c 5 http://202.58.111.111/tes.html

Where,

  •   -n 1000: ab will send 1000 number of requests to server 202.58.111.111 ( you can changes it to your own ip server) in order to perform for the benchmarking session
  •   -c 5 : 5 is concurrency number i.e. ab will send 5 number of multiple requests to perform at a time to server 202.58.111.111

Output:

This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0

Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/

Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking 202.58.182.129 (be patient)

Completed 100 requests

Completed 200 requests

Completed 300 requests

Completed 400 requests

Completed 500 requests

Completed 600 requests

Completed 700 requests

Completed 800 requests

Completed 900 requests

Finished 1000 requests

 

Server Software:        Apache/2.2.8

Server Hostname:        202.58.182.129

Server Port:            80

 

Document Path:          /tes.html

Document Length:        170 bytes

 

Concurrency Level:      5

Time taken for tests:   0.168989 seconds

Complete requests:      1000

Failed requests:        0

Write errors:           0

Total transferred:      531531 bytes

HTML transferred:       170170 bytes

Requests per second:    5917.54 [#/sec] (mean)

Time per request:       0.845 [ms] (mean)

Time per request:       0.169 [ms] (mean, across all concurrent requests)

Transfer rate:          3071.21 [Kbytes/sec] received

Connection Times (ms)

              min  mean[+/-sd] median   max

Connect:        0    0   0.0      0       0

Processing:     0    0   0.5      0       6

Waiting:        0    0   0.3      0       6

Total:          0    0   0.5      0       6

 

Percentage of the requests served within a certain time (ms)

  50%      0

  66%      0

  75%      0

  80%      0

  90%      1

  95%      1

  98%      1

  99%      1

 100%      6 (longest request)

       Benchmarking using php-mysql script

Create the script first

# vim phpsql.php

<?php

   $link = mysql_connect(“localhost”, “USERNAME”, “PASSWORD”);

   mysql_select_db(“DATABASE”);

   $query = “SELECT * FROM TABLENAME”;

   $result = mysql_query($query);

   while ($line = mysql_fetch_array($result))

   {

      foreach ($line as $value)

       {

         print “$value\n“;

      }

   }

    mysql_close($link);

?>

Where,

  • “localhost”, “USERNAME”, “PASSWORD” and  “TABLENAME” are refer to your mysql db settings

Output:

Test benchmark 1:

ab -n 1000 -c 5 http:// 202.58.111.111/phpsql.php

where,

  •          -n 1000 -> send request to server as much  1000 request
  •          -c 300  -> will send 300 number of multiple requests to perform at a time to server 202.58.111.111

# ab -n 1000 -c 300 http:// 202.58.111.111/phpsql.php

This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0

Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/

Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking 202.58.182.129 (be patient)

Completed 100 requests

Completed 200 requests

Completed 300 requests

Completed 400 requests

Completed 500 requests

Completed 600 requests

Completed 700 requests

Completed 800 requests

Completed 900 requests

Finished 1000 requests

 

Server Software:        Apache/2.2.8

Server Hostname:        202.58.182.129

Server Port:            80

Document Path:          /psql.php

Document Length:        157315 bytes

Concurrency Level:      300

Time taken for tests:   7.890645 seconds

Complete requests:      1000

Failed requests:        37

   (Connect: 0, Length: 37, Exceptions: 0)

Write errors:           0

Total transferred:      152002245 bytes

HTML transferred:       151726883 bytes

Requests per second:    126.73 [#/sec] (mean)

Time per request:       2367.194 [ms] (mean)

Time per request:       7.891 [ms] (mean, across all concurrent requests)

Transfer rate:          18812.02 [Kbytes/sec] received

Connection Times (ms)

              min  mean[+/-sd] median   max

Connect:        0  260 843.5      0    3017

Processing:    10 1570 1692.6   1070    7880

Waiting:        9 1531 1699.7   1027    7861

Total:         52 1830 2008.5   1074    7886

Percentage of the requests served within a certain time (ms)

  50%   1074

  66%   1106

  75%   1134

  80%   1218

  90%   4792

  95%   7494

  98%   7605

  99%   7810

 100%   7886 (longest request)

On this test there were failed request =  37 failed length it can be because of the dynamic script, ab  needs the content to be returned of identical length from one request to the next, so if your

content is dynamic in any way, it may “fail”. If there are any real Apache::ASP errors, they should show up  in your apache error_log.

The server performance:

# top

top – 15:21:37 up 33 days, 19:42,  3 users,  load average: 2.12, 1.96, 1.48

Tasks: 133 total,   2 running, 130 sleeping,   0 stopped,   1 zombie

Cpu(s):  2.3%us,  0.5%sy,  0.0%ni, 97.0%id,  0.0%wa,  0.0%hi,  0.2%si,  0.0%st

Mem:   4153976k total,  2800284k used,  1353692k free,   689188k buffers

Swap:  7783288k total,        0k used,  7783288k free,  1728880k cached

Load average pull up become  2.12  , memory also raising to 2800284k  but the server still okay

Test benchmark 2:

 

ab -n 10000 -c 300 http:// 202.58.111.111/phpsql.php

where,

  •          -n 1000 -> send request to server as much  10000 request
  •          -c 300  -> will send 300 number of multiple requests to perform at a time to server 202.58.111.111

ab -n 10000 -c 300 http:// 202.58.111.111/phpsql.php

 

This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0

Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/

Copyright 2006 The Apache Software Foundation, http://www.apache.org/

 

Benchmarking 202.58.182.129 (be patient)

Completed 1000 requests

Completed 2000 requests

Completed 3000 requests

Completed 4000 requests

Completed 5000 requests

Completed 6000 requests

Completed 7000 requests

Completed 8000 requests

Completed 9000 requests

Finished 10000 requests

 

 

Server Software:        Apache/2.2.8

Server Hostname:        202.58.182.129

Server Port:            80

 

Document Path:          /psql.php

Document Length:        157315 bytes

 

Concurrency Level:      300

Time taken for tests:   41.466140 seconds

Complete requests:      10000

Failed requests:        5158

   (Connect: 0, Length: 5158, Exceptions: 0)

Write errors:           0

Total transferred:      772220127 bytes

HTML transferred:       769366660 bytes

Requests per second:    241.16 [#/sec] (mean)

Time per request:       1243.984 [ms] (mean)

Time per request:       4.147 [ms] (mean, across all concurrent requests)

Transfer rate:          18186.43 [Kbytes/sec] received

 

Connection Times (ms)

              min  mean[+/-sd] median   max

Connect:        0  114 389.0     23    3419

Processing:    15 1071 1492.6    707   20874

Waiting:       13  813 1376.7    586   20854

Total:         30 1186 1539.6    839   21036

 

Percentage of the requests served within a certain time (ms)

  50%    839

  66%   1414

  75%   1720

  80%   1846

  90%   2088

  95%   3188

  98%   6012

  99%  10190

 100%  21036 (longest request)

 

Server load performance

# top

top – 15:15:04 up 33 days, 19:35,  3 users,  load average: 12.57, 4.27, 1.80

Tasks: 211 total,   1 running, 209 sleeping,   0 stopped,   1 zombie

Cpu(s): 40.5%us,  2.9%sy,  0.0%ni, 56.0%id,  0.0%wa,  0.0%hi,  0.5%si,  0.0%st

Mem:   4153976k total,  3063280k used,  1090696k free,   689184k buffers

Swap:  7783288k total,        0k used,  7783288k free,  1728440k cached

 

Load average raise to 12.57 and the memory also raising becoming 3063280k but the server still okay

uptime

 15:23:45 up 33 days, 19:44,  3 users,  load average: 0.28, 1.28, 1.28

Take at least 3-5 readings and use the best result

Okay that’s for apache how about for windows ( IIS) ?

You can use microsoft tools called wast , 

  • Web Application Stress Tool (WAST): You can use WAST to simulate the following:
    • Simulate the effect of multiple web browsers concurrently connecting to IIS to run applications
    • Simulate the effect of multiple web browsers concurrently connecting to IIS to download Web content
    • Simulate different loads so that you can determine the effect on the Web server. 

As for the settings you can refer here http://www.west-wind.com/presentations/webstress/webstress.htm