Query Language

The Scalyr query language is used to select a set of events from your log data. It is used in data views, alert triggers, report definitions, and elsewhere. Examples:

All log messages containing the word "error":

error

Text containing spaces, digits, or punctuation must be enclosed in quotes:

"production-database"

Requests in the access log for "/index.html", for which at least 5000 bytes were returned:

$logfile contains 'access_log' uriPath = '/index.html' bytes >= 5000

Log messages containing the phrase "deadline exceeded", from servers tagged as part of a database tier:

"deadline exceeded" $serverTier = "database"

Query Structure

A query can contain any number of terms. To select events matching all of the terms (an "AND" query), you can simply enter the terms next to one another:

$logfile contains 'access_log' uriPath = '/index.html'

You can also use explicit AND, OR, and NOT keywords to combine terms:

$logfile contains 'access_log' and not (uriPath = '/home' or path = "/away")

The operators &&, ||, and ! can be used as synonyms for AND, OR, and NOT:

$logfile contains 'access_log' && !(uriPath = '/home' || uriPath = "/away")

Text search

To search for a word, simply type that word. That forms a search term, which can be combined with other search terms as described above. For example, to search for all events containing the word "hello" and at least one of "sir" or "madam":

hello (sir || madam)

To search for a more complex string, enclose it in single or double quotes:

"cache miss"
'***critical error***'

You can also search using regular expressions. Enclose the expression in double quotes, preceeded by a $:

$"/images/.*\.png"

All of these terms search in the "message" field of an event. For logs uploaded by the Scalyr Agent, this field contains the complete text of the log message. However, you can also search in other fields. To perform a string match, use the "contains" keyword:

$logfile contains 'access_log'

For a regular expression match, use "matches":

uriPath matches '\\.png$'

All text search is case-insensitive.

Field Comparison

The most powerful searches rely on event fields. (For a review of fields, refer back to the Getting Started page.) You can select events that do or do not have a particular value, using the == and != operators. = can be used as a synonym for ==:

uriPath = '/index.html'
status == 404
client != "localhost"

You can also use the <, <=, >, and >= operators to compare values. For instance, this query matches all requests with a status in the range 400-499:

status >= 400 status <= 499

For field comparison operators, strings are treated as case sensitive.

When using a field which is associated with a server or log file, place a $ before the field name:

$serverHost = 'frontend-1'

For fields that come directly from the log event, the $ is optional.

If a field name contains spaces or punctuation, use a backslash to escape those characters. For instance, to display events with field service-name equal to "memcache":

service\-name == "memcache"

Finally, you can compare a field with the special value * to match events with any value in that field:

error == *

Searching For Multiple Values

You can use the in operator to search for several values at once:

userid in ("aaa", "bbb", "ccc")

The in operator can be used to match numeric, text, or boolean (true / false) values, but can not match null / missing fields.

Graph Functions

The graph view differs from the search view in that it displays numerical values rather than events. By default, the graph view will show a rate graph tracking the number of matching events per second over the search's time range. You can graph a different set of values by using a function that is applied either to the rate of matching events, or to the values of a particular field:

Function Meaning
count The number of matching events
mean(value) Average value
min(value) Smallest value
max(value) Largest value
sumPerSecond(value) The "smoothed" sum of all values per second. For instance, if you have a field responseSize which records the number of bytes returned by some operation, then sumPerSecond(responsesize) will graph the bandwidth consumed by this operation, in bytes per second. (We divide the time period of your graph into a number of time spans, sum all values per time span, and then divide by the time span in seconds to get an average sum per second, per time span. Note that graphed values are exact over brief time periods (100 seconds, for example), and effectively smoothed over longer time periods.)
median(value) The median (50th percentile) value.
p10(value) The 10th percentile value.
p50(value) The 50th percentile value.
p90(value) The 90th percentile value.
p95(value) The 95th percentile value.
p99(value) The 99th percentile value.
p999(value) The 99.9th percentile value.
p(value, n) The Nth percentile value. For instance, p(value, 80) gives the 80th percentile.
fraction(expr) The fraction (from 0 to 1) of events which match the given expression. For instance, fraction(status >= 500 status <= 599) is the fraction of requests which have a status in the 5xx range. You can use any query expression, as documented in the earlier sections of this page.

Ratios and Complex Expressions

A graph can combine multiple numeric queries into a single expression. For example, this expression computes the ratio of log messages containing "error" to messages containing "success":

count(message contains "error") / count(message contains "success")

In this type of expression, you can use any number of numeric queries, combining them with the standard arithmetic operators +, -, *, and /, as well as parentheses. You can also use simple numbers. For instance, the following expression shows free disk space in gigabytes, converting from the kilobyte units reported by the Scalyr Agent:

mean(value where $serverHost='frontend-1' metric='df.1kblocks.free' mount='/') / (1024 * 1024)

The general form of a numeric query is as follows:

  FUNCTION([FIELD where] FILTER)

FUNCTION is one of the functions listed above. FIELD is the name of a numeric event field; if omitted, "value" is used. FILTER is a query. Examples:

count(message contains "error")

The number of log messages that contain the word "error".

mean(bytes where uriPath == "/home")

The average number of bytes returned in requests for /home.

p[95](time where uriPath == "/home")

95th percentile latency of requests for /home.