HTTP Monitor

This agent plugin periodically fetches specified HTTP or HTTPS URL (or POSTs to it), and records the response. The URL is retrieved from the agent process, so any host and port reachable from the machine on which the agent is running can be monitored.

An agent monitor plugin is a component of the Scalyr Agent. To use a plugin, simply add it to the monitors section of the Scalyr Agent configuration file (/etc/scalyr/agent.json). For more information, see Agent Plugins.

Sample Configuration

Here is a simple configuration fragment showing use of the url_monitor plugin. This sample will record the instance type of the Amazon EC2 server on which the agent is running.

monitors: [
    module:  "scalyr_agent.builtin_monitors.url_monitor",
    id:      "instance-type",
    url:     ""

Here is a configuration fragment showing how to have the plugin issue a POST request while setting some of the HTTP Headers and setting custom request data:

monitors: [
    module: "scalyr_agent.builtin_monitors.url_monitor",
    id: "post-example",
    url: "http://your-url-to-make-post-request",
    request_method: "POST",
    request_headers: [{"header": "Accept-Encoding", "value": "gzip"}],
    request_data: "your-request-data"

To monitor more than one URL, use several copies of the url_monitor plugin in your configuration.

Viewing Data

After adding this plugin to the agent configuration file, wait one minute for data to begin recording. Then go to the Search page and search for $monitor = 'url_monitor'. This will show all data collected by this plugin, across all servers. You can use the Refine search by dropdown to narrow your search to specific servers and monitors.

The Search overview page describes the tools you can use to view and analyze log data. Query Language lists the operators you can use to select specific metrics and values. You can also use this data in Dashboards and Alerts.

Configuration Reference

Option Usage
module Always scalyr_agent.builtin_monitors.url_monitor
id Included in each log message generated by this monitor, as a field named instance. Allows you to distinguish between values recorded by different monitors.
url The URL to fetch. Must be an http or https URL.
request_method Optional (defaults to GET): The HTTP request method to make the request
request_headers Optional: The HTTP Headers to set when making the request. It is a list of dictionaries with keys header (header name) and value (header value)
request_data Optional: The HTTP request data to pass with the request. Only used with request_method POST.
timeout Optional (defaults to 10): the maximum amount of time, in seconds, to wait for the URL to load.
extract Optional: a regular expression to apply to the command output. If defined, this expression must contain a matching group (i.e. a subexpression enclosed in parentheses). The monitor will record only the content of that matching group. This allows you to discard unnecessary portions of the command output and extract the information you need.
log_all_lines Optional (defaults to false). If true, the monitor will record the entire command output; otherwise, it only records the first line.
max_characters Optional (defaults to 200). At most this many characters of output are recorded. You may specify a value up to 10000, but the Scalyr server currently truncates all fields to 3500 characters.

Log Reference

Each event recorded by this plugin will have the following fields:

Field Meaning
monitor Always url_monitor
metric Always response
instance The id value from the monitor configuration, e.g. instance-type
url The URL that was retrieved, e.g.
status The HTTP response code, e.g. 200 or 404
length The length of the HTTP response
value The body of the HTTP response