Graphs

This section gives a detailed guide to the Graph view.

Accessing Graphs

There are various ways to access Scalyr's graphing tools. The most common workflow involves using Search view to select the field or metric you wish to graph (see (1) through (4) below). Selecting any of the graphing options from the Field list will take you to Graph view.

Another way to enter Graph view is by clicking Expand Graph on the Search view page. This takes you to a graph of your log volume. You can then use the tools explained below to modify the graph, including the field and graph type.

If you have already saved a graph as an Alert, or to a Dashboard, click on Alerts or Dashboards to view that graph. From there you can also access the Dashboard or Alert JSON to add, change, or modify your existing graphs. See Editing Dashboards in JSON for a thorough discussion of scripting graphs in JSON.

Quick Reference

(1) To search for a specific word or phrase, type it here. Numbers, punctuation, or phrases must be enclosed in quotes. For example, error, "503", and "customer 1309". You can reference a field by just typing it, for example serverHost == 'appserver-4', or status>=500 status<=599. See Query Language for a full description of the Scalyr query language.

As you type your text is parsed and presented in an easy-to-read form. Fields, operators and values are differentiated via highlighting.

(2) This is the time range of your search. By default, the last four hours are displayed. You can customize this default.

Click the button to change the time range:

You can select a preset to quickly search a range, or enter a Custom range via the Start and End boxes. Hint: it's faster to build and refine your query over a short time range, then apply it to the full range of interest.

You can enter a time (e.g. 14:30 or 5:05 AM), a date (May 23), or date and time (5/14/2016 2:00 PM), using a wide variety of formats. Shortcuts like 5d/ 5h/ 5m/ 5s indicate five days/hours/minutes/seconds. The End time assumes NOW, so entering 5m for the Start time and hitting Enter will search the last five minutes. Using the + shortcut for the End time, for example +24h or +1d, will search from the Start time to one-day later.

See the Date/Time Reference for a complete list of options.

(3) Use these fields to search a specific server or log file. If you're using Kubernetes these will allow you to search cluster and controller name, respectively. You can use * as a wildcard to represent zero or more characters, wherever they appear.

(4) The Field list displays all parsed fields in the events matching your search. They are alphabetically arranged in a scrollable window, with a Filter box for searching. Fields referenced by your query in (1) and in (3) are bold. As you move your cursor over the list the selected field is highlighted. You can also use the up/down arrows on your keyboard to navigate the list.

The number next to each field indicates how many distinct values appear in that field. (If there are more than a few hundred distinct values, the number shown will be an estimate). Click on a field to bring up a list of its most common values:

(4a) Depending on the type of data, various graphing options appear as buttons:

  • Graph Values graphs the selected field over time.
  • #Matches graphs matching events per second, broken down by the selected field.
  • Distribution graphs a distribution of the selected field.

For more information on Scalyr's graphing tools, see Graphs.

(4b) You can click on a value to restrict your events to only that value. You can also use the </ ==/ !=/ > operators to restrict events to values greater than, equal to, not equal to, or lesser than the value. This is often done as a precursor to graphing specific values of interest.

(4c) The bars provide a visual indication of how often each value appears, while the numbers provide more precision. Note that these are often estimates due to sampling (see (4d)).

(4d) Information concerning estimated values is located here. (We sample to achieve a statistically valid number, using a two-pass method. In the first pass (first bullet point), we query a 1% subsample to estimate the total number of matches. In the second pass (second bullet point), we sample the data with a rate based on information obtained in the first pass.)

(4e) Up to 30 distinct values are displayed in the scrollable window. Click Check for More to show up to a maximum of 200 values.

(5) The name of the field you are graphing. To the upper-right of the graph is information concerning the time range of your graph, and the time zone.

(6) Click here to select the type of chart you'd like to display:

The Stacked Bar Chart option allows you to select the time interval for the bars. This chart type is a bar graph for single plots, and a stacked bar graph for multiple plots.

You may want to switch to a logarithmic scale if there is a large difference between high and low values in your graph.

(7) This area lists the functions which you can select for your graph. Check one or more boxes to select different functions of the graphed field. Calculations are over the entire time range of your graph. The available functions are:

Function Value
Mean The average of all values in each time period. For instance, if you are graphing server response times, this will show the average response time.
Minimum The smallest value in each time period.
Maximum The largest value in each time period.
Sum/sec The "smoothed" sum of all values per second. For instance, if you have a field responseSize which records the number of bytes returned by some operation, then sumPerSecond(responsesize) will graph the bandwidth consumed by this operation, in bytes per second. (We divide the time period of your graph into a number of time spans, sum all values per time span, and then divide by the time span in seconds to get an average sum per second, per time span. Note that graphed values are exact over brief time periods (100 seconds, for example), and effectively smoothed over longer time periods.)
10th %ile Shows the 10th percentile of all values in each time period.
50th %ile Shows the 50th percentile (median) of all values in each time period.
90th %ile Shows the 90th percentile of all values in each time period.
95th %ile Shows the 95th percentile of all values in each time period.
99th %ile Shows the 99th percentile of all values in each time period.
99.9th %ile Shows the 99.9th percentile of all values in each time period.

(8) Move your cursor over the graph to select and view a mean value for nearby events. The functions in (7) do not change, allowing comparisons between point values and the overall time range. (Most plots involve many events per second, so we present the mean value of nearby events, rather than a single value. For bar charts, we present the mean value over the time-span of the bar.)

You can also click and drag in the graph to select a time range. The functions in (7) will adjust to calculate values based on your selection. A Zoom In button will also appear; click this button to zoom in to the selected time range. Clicking outside the button returns to the graph as plotted, and the functions in (7) will again show values calculated over the entire time range of the graph.

(9) These statistics apply only when you click and drag in the graph to select a time range in (8). The deltas show information about the slope, or rate of change, of your selection. For instance, if you're viewing a graph of free disk space, the delta tells you how quickly disk space is being consumed.

  • delta shows the change in value from the beginning and end of the selection.
  • deltaHour shows the average change in value per hour.
  • deltaSec shows the average change in value per second.

Deltas are computed based on average values in the first and last time periods of the graph, even if you have chosen to display a different function (such as minimum or maximum).

(10) Click Break Down to create a breakdown graph. This graphs event volume broken down by a field, or a field broken down by another field. For example, when graphing data from a web access log, you could break it down by URL or user-agent.

The example below filters for logs where status == "failure" (10a), breaks the results down by server (10b), and then presents the information as a stacked bar chart with 1-hour intervals (10c):

Note that breakdown graphs can timeout when they require searching through large amounts of log data. Whenever possible, we recommend the use of standard graphs when saving to Alerts or Dashboards. See Timeout Tips for more information.

(11) Scalyr can graph the distribution of values for a numerical field. Access to this type of graph is via the Field list, mentioned in (4a). Distribution graphs are particularly useful for summarizing the frequency of occurrence of values in a field.

(11a) In the above example we have searched for events where dataset="accesslog", and we are plotting the distribution of the time field.

(11b) The Y-axis of a distribution is the number of times a particular field value occurs. For example, this distribution shows a little over 25,000 events where the time field has a value of approximately 5.8.

If you move your mouse over the graph, Value and Count data is visible for the individual "bins" in the distribution.

(11c) The X-axis of a distribution is the range of numeric values in a field, in this case time. Note that the scale of the axis, which is logarithmic in this example, depends on the range of values for the field.

(11d) You can click and move your mouse over the graph to select a time range. Selection and Count information for your selection becomes visible to the upper-right.

(12) Use these buttons to move forward or backward one half-graph at a time.

(13) Click the Save button to display the following actions for your current search:

  • Save Graph: Opens a dialog box that lets you save the graph to either your personal or team's list of saved graphs, which are also available in the main Search menu at the top of the page.
  • Save as Alert: Create a new alerting rule, which will trigger if the number of matches to your current search goes above or below a level you specify.
  • Save to Dashboard: Add this search to an existing dashboard, or start a new dashboard with this search.
  • Download as PNG: Saves the current graph as a PNG file and downloads it to your default Downloads folder.

When possible, we recommend the use of standard graphs when saving to Alerts or Dashboards. See Timeout Tips for more information.

(14) Click the Share button in the left-center of the search bar to display the following Share actions for your current search:

  • Copy link to graph: Opens a modal window where you can copy a link to this search with relative time references replaced by absolute (e.g., instead of the searching the previous hour, it would search 8 a.m. to 9 a.m.).
  • Post graph to slack: Opens a dialog box that lets you select a Slack channel, compose a Slack message, and post the graph to Slack. See Post to Slack Reference for instructions.
  • Add graph to shared search list: Opens a dialog box that lets you save the active search query to either your personal or team's list of saved searches; the team list is selected by default. Saved searches are available in the Search menu.

(15) Click Compare to graph a previous period alongside the current period.

(16) Click Show Logs to return to Search view, where you can view individual events.