Analyze Access Logs

This Solution describes how to analyze web access logs. You can graph and alert on error rates, view the most commonly requested pages, and more.


1. The Scalyr Agent should be installed on the server(s) you want to monitor, and should be configured to upload your access logs. In the Scalyr Agent configuration file, each access log should be tagged with parser: "accessLog".

To verify:

  • In the navigation bar, click Overview.
  • Find each server in the list at the bottom of the page.
  • Verify that your web server's access log is listed next to each server.
  • Click the link for each log, and click on a log message. A box will appear showing details for that message. Verify that parsed fields such as "agent", "authUser", and "bytes" are listed.

If any server is not listed:

If any log file is not listed:

If the log view does not show parsed attributes:

  • Open the Scalyr Agent configuration file (agent.json) on the affected server, and find the entry that refers to your web access log. Check whether it specifies parser: "accessLog". If not, update the file accordingly. The relevant configuration entry should look something like this:
    path: "/var/log/httpd/access*",
    attributes: {parser: "accessLog"}


1. To see an overview of traffic to all of your web servers, click Dashboards in the navigation bar, and select the "Paths" dashboard.

This will display a list of each unique request page in your access logs, with the most frequently requested pages at the top. Click on any numeric value to see a graph of the data behind that number. For instance, the "2xx" column shows the number of requests for that page in the last hour which yielded a successful response (HTTP status 200 through 299). Click on the number to see a graph of successful responses per second.

2. To see graphs of traffic for an individual server, click Dashboards in the navigation bar, and select the "WebServer" dashboard.

This will display a set of graphs summarizing traffic to a particular server. Use the Host dropdown to view different servers.

3. To see the pages which most frequently trigger a server error (HTTP status 500 through 599):

  • In the navigation bar, click Search.
  • In the Search box, enter dataset='accesslog' status >= 500
  • Click the Search button
  • Switch to the Facets tab
  • If your web servers haven't had any errors in the last few hours, congratulations! You'll see a "No matching events" message. Otherwise, you'll see a breakdown of all fields in your web access logs for failed request. Scroll down to the section for the uriPath field to see the most common error pages. If more than 10 pages have generated errors, click the see all values link to view more.

Further Reading

For additional search options, see the Search overview page, and the Query Language page.

For instructions on viewing which IP addresses generate the most traffic to your web servers, or related queries such as the most common user agent, see the Display Common IP Addresses solution.