This section gives a detailed guide to the Graph view, which is where you can graph values in your logs, such as server response times or page sizes. You can graph values from an individual log, aggregate multiple servers / logs, and use search terms to narrow the log messages to graph.
There are various ways to access Scalyr's graphing tools. The most common workflow involves using Search view to select the field or metric you wish to graph (see (1) through (3), (6), and (7) below). Selecting any of the graphing options from the Field list will take you to the Graph view (see (7a) below).
Another way to enter Graph view is by clicking Expand Graph on the Search view page. This takes you to a graph of your log volume (matches per second). You can then use the tools explained below to modify the graph, including the field and graph type.
If you have already saved a graph as an alert, or to a dashboard, click on Alerts or Dashboards to view that graph. From there you can also access the Dashboard or Alert JSON to add, change, or modify your existing graphs. See Editing Dashboards in JSON for a thorough discussion of scripting graphs in JSON.
(1) To search for a specific word or phrase, type it here. This determines which log messages are reflected in the graph. Numbers, punctuation, or phrases must be enclosed in quotes. Sample searches:
|error||To search for a word or part of a word, just type it|
|"/blog"||Punctuation must be enclosed in quotes|
|"customer 1309"||Multi-word phrases must also be enclosed in quotes|
|userId = 1309||Matching on a parsed field|
|time > 0.5||Numeric comparison on a parsed field|
See Query Language for a full description of the Scalyr query language.
(2) As you type search text into the box, it is parsed and presented in a form that makes your search easier to read and understand. Different parts of your search text such as fields, operators, and values are highlighted in order to visually differentiate them. For example, in the search text "bytes > 5000", each of the three components will be a different color.
(3) Click here to specify the time range to graph. The following options will appear:
(3a) Click on a preset to quickly graph that time range.
(3b) Enter the start time for your graph. You can enter a time (e.g. 14:30 or 5:05 AM), a date (May 23), or date and time (5/14/2016 2:00 PM), using a wide variety of date and time formats. You can also enter shortcuts like "5h" to indicate five hours ago. See the Date/Time Reference for a complete list of options.
(3c) Enter the end time for your graph. You can use any of the formats explained in (3b). You can also enter a shortcut beginning with + to specify the amount of time you'd like to search, e.g. +24h or +1d to graph a one-day period beginning at the From time.
(4) Use these buttons to move forward or backward one half-graph at a time.
(5) Use this button to view the raw log messages matching your search.
(6) Use these fields to search a specific server or log file. If you're using Kubernetes these will allow you to search cluster and controller name, respectively. You can use a single * as a wildcard at the beginning or end (but not the middle) of the server or log file name.
(7) This area lists the fields the parser found in the log messages matching your search. The top 100 fields are arranged alphabetically in a scrollable window (All Fields). Click the dropdown and switch to Top Fields to view the most common fields first.
The number next to each field indicates how many distinct values appear in that field. (If there are more than a few hundred distinct values, the number shown will be an estimate.)
Click on a field to bring up a list of its most common values:
(7a) Depending on the type of data, various graphing options appear as buttons:
- Graph Values graphs the selected field over time.
- #Matches graphs matching events per second, broken down by the selected field.
- Distribution graphs a distribution of the selected field.
(7b) You can click on a value to restrict your graph to events having that field value. You can also use the `==` and `!=` symbols to include (or exclude) these values from your search.
(7c) The bars provide a visual indication of how often each value appears, while the numbers provide more precision. Note that these are often estimates due to sampling (see (7d) below).
You can click on a value to restrict your graph to events having that field value.
(7d) Information concerning estimated values is located here. (We sample to achieve a statistically valid number, using a two-pass method. In the first pass (first bullet point), we query a 1% subsample to estimate the total number of matches. In the second pass (second bullet point), we sample the data with a rate based on information obtained in the first pass.)
(7e) If the field has too many values to display on one screen, click See More to show up to a maximum of 200 values.
(8) This shows the name of the field you're graphing.
(9) This area shows a graph of the specified field.
(10) Use the chart type drop down to select the type of chart you'd like to see. If you choose Stacked Bar Chart you'll also be able to select the time interval for the bars, e.g. 1 minute or 1 hour. If you have a graph with a huge difference between highest and lowest values you may want to switch to a logarithmic y-axis.
(11) This area lists the functions which you can select for your graph. Check one or more boxes to select different functions of the graphed field. The available functions are:
|Average||The average of all values in each time period. For instance, if you are graphing server response times, this will show the average response time.|
|Minimum||The smallest value in each time period.|
|Maximum||The largest value in each time period.|
|Sum/sec||The "smoothed" sum of all values per second. For instance, if you have a field responseSize which records the number of bytes returned by some operation, then sumPerSecond(responsesize) will graph the bandwidth consumed by this operation, in bytes per second. (We divide the time period of your graph into a number of time spans, sum all values per time span, and then divide by the time span in seconds to get an average sum per second, per time span. Note that graphed values are exact over brief time periods (100 seconds, for example), and effectively smoothed over longer time periods.)|
|10th %ile||Shows the 10th percentile of all values in each time period.|
|50th %ile||Shows the 50th percentile (median) of all values in each time period.|
|90th %ile||Shows the 90th percentile of all values in each time period.|
|95th %ile||Shows the 95th percentile of all values in each time period.|
|99th %ile||Shows the 99th percentile of all values in each time period.|
|99.9th %ile||Shows the 99.9th percentile of all values in each time period.|
With your cursor outside the graph, the values in the legend are calculated over the time range of the graph. Move your cursor over the graph to view values for individual points or bars, depending on the type of graph you have selected.
You can also click and drag in the graph to select a time range. A Zoom In button will appear; click this button to zoom in to the selected time range. With your cursor outside the graph, the legend will now show values calculated over the time range of your selection.
(12) These statistics apply only when you have dragged to select a time range in the graph. The deltas show information about the slope, or rate of change, of your graph. For instance, if you're viewing a graph of free disk space, the delta tells you how quickly disk space is being consumed.
- Change shows the change in value from one end of the graph to the other.
- Change/hour shows the average change in value per hour.
- Change/sec shows the average change in value per second.
Deltas are computed based on average values in the first and last time periods of the graph, even if you have chosen to display a different function (such as minimum or maximum).
(13) Click the Compare dropdown to graph a previous period with the current period.
(14) Click Break Down By to create a breakdown graph. This graph type breaks down log volume or a plot of a field by another field. For example, when graphing data from a web access log, you could break it down by URL or user-agent. The breakdown graph below filters for logs where `$status == "failure"` (14a), breaks the results down by server (14b), and then presents the information as a stacked bar chart (14c):
Note that breakdown graphs can timeout when they require searching through large amounts of log data. Whenever possible, we recommend the use of standard graphs when saving to Alerts or Dashboards. See Timeout Tips for more information.
(15) Click the Save button to display the following actions for your current search:
- Save Graph: Opens a dialog box that lets you save the graph to either your personal or team's list of saved graphs, which are also available in the main Search menu at the top of the page.
- Save as Alert: Create a new alerting rule, which will trigger if the number of matches to your current search goes above or below a level you specify.
- Save to Dashboard: Add this search to an existing dashboard, or start a new dashboard with this search.
- Download as PNG: Saves the current graph as a PNG file and downloads it to your default Downloads folder.
When possible, we recommend the use of standard graphs when saving to Alerts or Dashboards. See Timeout Tips for more information.
(16) Click the Share button in the left-center of the search bar to display the following Share actions for your current search:
- Copy link to graph: Opens a modal window where you can copy a link to this search with relative time references replaced by absolute (e.g., instead of the searching the previous hour, it would search 8 a.m. to 9 a.m.).
- Post graph to slack: Opens a dialog box that lets you select a Slack channel, compose a Slack message, and post the graph to Slack. See Post to Slack Reference for instructions.
- Add graph to shared search list: Opens a dialog box that lets you save the active search query to either your personal or team's list of saved searches; the team list is selected by default. Saved searches are available in the Search menu.
Distribution graphs are particularly useful for summarizing the frequency of occurrence of numeric fields such as server response times or page sizes. A distribution shows the range of values in a numeric field, and which values are most common. You can view distributions for a single server or log file, or aggregated over multiple servers and log files, and you can use search terms to filter your log messages.
(1) You can plot the distribution of any numeric field by selecting it from the Field List and then clicking the Distribution button. Note that the Distribution button only appears for numeric fields. See **(7)** above for more information on this process.
(2) Use the Search box to filter your data. In this case, we are plotting the distribution of the time field only for logs where dataset="accesslog".
(3) You can easily filter for a specific log file or server here. If you're using Kubernetes these will allow you to search cluster and controller name, respectively.
(4) Select the time range of your distribution here.
(5) The Y-axis of a distribution is the number of times a particular field value occurs. For example, this distribution shows a little over 50,000 instances where the time field had a value of approximately 4.
(6) The X-axis of a distribution is the range of numeric values in a field, in this case time. Note that the scale of the axis, which is logarithmic in this example, depends on the range of values for the field.
(7) If you move your mouse over the graph, Value and Count data is visible for the individual "bins" in the distribution.
(8) The Functions Panel provides useful summary statistics of the field you are graphing.