Regular expression for Apache log parsing

Generally, there are two commonly used formats for Apache log file.

Common log format example: – frank [10/Oct/2000: 13:55:36 -0700] “GET / apache_pb.gif HTTP/1.0” 200 2326

Combined log format example: – frank [10/Oct/2000: 13:55:36 -0700] “GET / apache_pb.gif HTTP/1.0” 200 2326 “” “Mozilla / 4.08 [en] (Win98; I; Nav) ”

As you can see, combined log format has two more request header information than the common log format. Use the combined log file as example, the meaning for each part is defined as follows (for more information see the Apache documentation)

  1.  ( This is the IP address of the client (remote host) which made the request to the server.
  2.  (-) The RFC 1413 identity of the client. The “hyphen” in the output indicate that the requested piece of information is not available.
  3. (frank) The userid of the person request the document as determined by HTTP authentication.
  4. ([10/Oct/2000:13:55:36  -0700] The time that the request was received. The format is: [day/month/year:hour:minute:second  zone]
  5. (“GET /apache_pb.gif  HTTP/1.0”) The request line from the client is given in double quotes. The request line contains a great deal of useful information, including method used by the client (GET), the resource requested by the client (/apache_pb.gif) and the protocol used by the client (HTTP/1.0).
  6. (200) This is the status code that the server sends back to the client. A successful response (codes beginning in 2), a redirection (codes beginning in 3), an error caused by the client (codes beginning in 4), or an error in the server (codes beginning in 5). The full list of possible status codes can be found in the HTTP specification (RFC2616 section 10).
  7. (2326) The size of the object returned to the client.
  8. (“”) The “Referer” HTTP request header. This gives the site that the client reports having been referred from.
  9. (“Mozilla/4.08  [en]  (Win98;  I  ;Nav)”) The User-Agent HTTP request header. This is the identifying information that the client browser reports about itself.

Continue reading