Introduction
Import / Monitor
The Scalyr UI
- Search
- PowerQueries
- Graphs
- Dashboards
- Alerts
View / Analyze
Other Topics
API

Environment-Aware Variables (Scalyr Agent)

Environment-aware variables are Scalyr Agent configuration variables that are mapped to system environment variables. If not defined in the agent.json file (or agent.d fragments), the agent falls back to the value set in the environment.

An environment-aware variable usually maps to a system environment variable of the same name, but prefixed with SCALYR_. Both upper-case and lower-case versions are recognized by the agent.

E.g., the global api_key environment-aware config variable maps to the SCALYR_API_KEY environment variable.

For aesthetics, legacy reasons, or proper association, some environment variables do not follow the SCALYR prefix format. For example:

The global scalyr_server config variable maps to just SCALYR_SERVER (no extra prefix).
The Kubernetes event monitor leader_node has an additional K8S prefix (maps to SCALYR_K8S_LEADER_NODE).

Commonly Modified

The following environment-aware variables are required by the Scalyr Agent and are usually modified by customers. (Note: <prefix format> is short-form for the `SCALYR_` prefix described above.)

Context	Config variable	Environment variable	Default value	Description
Global	api_key	<prefix format>	<placeholder>	The write logs access API key
Global	scalyr_server	SCALYR_SERVER	agent.scalyr.com	Scalyr backend server
Docker monitor	docker_raw_logs	<prefix format>	true	If true, the docker monitor will use the raw log files on disk to read logs. The location of the raw log file is obtained by querying the path from the Docker API. If false, the logs will be streamed over the Docker API. (This value defaults to false to preserve legacy behavior. It is recommended to set this to true for all new deployments.)

Occasionally Modified

You may need to modify the following environment-aware variables:

Context	Config variable	Environment variable	Default value	Description
Global	max_line_size	<prefix format>	9900	Maximum size for a line when reading from a log file. The Scalyr Agent will truncate lines longer than this limit when possible.
Global	read_page_size	<prefix format>	64*1024	Number log data bytes to be buffered at a time. Must always be greater than max_line_size.
Global	log_rotation_backup_count	<prefix format>	2	Number of log rotations
Global	log_rotation_max_bytes	<prefix format>	2010241024	Size of each log rotation file.
Global	debug_level	<prefix format>	0	The debug level must be between 0 and 5 inclusive. Level 0 messages go into agent.log and are uploaded. Level 1 and above are finer grained and are not.
Global	ca_cert_path	<prefix format>	TODO
Global	verify_server_certificate	<prefix format>	true
Global	http_proxy	<prefix format>	None
Global	https_proxy	<prefix format>	None
Global	k8s_ignore_namespaces	<prefix format>	kube-system	List of namespaces whose pods are to be ignored.
Global	k8s_api_url	<prefix format>	https://kubernetes.default
Global	k8s_verify_api_queries	<prefix format>	true
Global	k8s_cache_expiry_secs	<prefix format>	30
Global	k8s_cache_purge_secs	<prefix format>	300
Global	k8s_events_disable	<prefix format>	false	If true, will disable collection of Kubernetes events.
Global	k8s_cache_start_fuzz_secs	<prefix format>	0	Random delay (in seconds) when k8s monitor starts up to prevent all agents in the cluster from querying the API master simultaneously. (Typically only needed for very large clusters.)
Global	k8s_cache_expiry_fuzz_secs	<prefix format>	0	Periodic random delay (in seconds) designed to further distribute multiple-agent queries to the API master that occur every `k8s_cache_expiry_secs` seconds. (Typically only needed for very large clusters.)
Global	disable_send_requests	<prefix format>	false
Global	config_change_check_interval	<prefix format>	30
Global	user_agent_refresh_interval	<prefix format>	60
Global	garbage_collect_interval	<prefix format>	300
Global	implicit_agent_log_collection	<prefix format>	true
Global	implicit_metric_monitor	<prefix format>	true	min_request_spacing_interval
Global	implicit_agent_process_metrics_monitor	<prefix format>	true
Global	default_sessions_per_worker	<prefix format>	1	The default number of sessions per worker. See Multi-sessions configuration
Global	use_multiprocess_workers	<prefix format>	false	Start each session in a separate process. See Multi-session configuration
K8s monitor	container_globs	<prefix format>	None	Comma-separated list of glob patterns of container names to monitor.
K8s monitor	container_check_interval	<prefix format>	5	Polling interval for started or stopped containers.
K8s monitor	docker_max_parallel_stats	<prefix format>	20	Maximum stats requests to issue in parallel when retrieving container metrics using the Docker API.
K8s monitor	report_container_metrics	<prefix format>	true	If true, also collect metrics from containers whose logs are monitored.
K8s monitor	report_k8s_metrics	<prefix format>	true	If true and report_container_metrics is also true, then collect Kubernetes system metrics.
K8s monitor	k8s_ignore_pod_sandboxes	<prefix format>	true	If true, exclude logs from containers where label io.kubernetes.docker.type == podsandbox
K8s monitor	k8s_include_all_containers	<prefix format>	true	Controls the default behavior for inclusion/exclusion of container logs. See documentation on Kubernetes Monitor annotations for more details.
K8s events	check_labels	SCALYR_K8S_CHECK_LABELS	false
K8s events	log_flush_delay	SCALYR_K8S_LOG_FLUSH_DELAY	1.0
K8s events	leader_check_interval	SCALYR_K8S_LEADER_CHECK_INTERVAL	60
K8s events	leader_node	SCALYR_K8S_LEADER_NODE	None
K8s events	max_log_size	SCALYR_K8S_MAX_LOG_SIZE	None
K8s events	max_log_rotations	SCALYR_K8S_MAX_LOG_ROTATIONS	None
Docker monitor	container_globs_exclude	<prefix format>	None	A blacklist of container name glob patterns to exclude from monitoring. Any container whose name matches one of the glob patterns will not be monitored. If `None`, all container names matched by `container_globs` are monitored. This value is applied after `container_globs`.
Docker monitor	container_check_interval	<prefix format>	5	How often (in seconds) to check for new containers.
Docker monitor	report_container_metrics	<prefix format>	true	If true, metrics will be collected from the container and reported to Scalyr.
Docker monitor	labels_as_attributes	<prefix format>	false	If true, the docker monitor will add any labels found on the container as log attributes, after applying label_include_globs and label_exclude_globs.
Docker monitor	label_include_globs	<prefix format>	['*']	If labels_as_attributes is true then this option is a list of glob strings used to include labels that should be uploaded as log attributes. The docker monitor first gets all container labels that match any glob in this list and then filters out any labels that match label_exclude_globs, and the final list is then uploaded as log attributes.
Docker monitor	label_exclude_globs	<prefix format>	['com.scalyr.config.*']	If labels_as_attributes is true, then this is a list of glob strings used to exclude labels from being uploaded as log attributes. Any label whose key matches any glob on this list will not be added as a log attribute. Note: the globs in this list are applied after label_include_globs.

Rarely Modified

The following environment-aware variables are rarely modified and usually by power users or under guidance from Scalyr support.

Context	Config variable	Environment variable	Default value	Description
Global	allow_http	<prefix format>	false
Global	agent_log_path	<prefix format>	/var/log/scalyr-agent-2	Agent log files dir
Global	agent_data_path	<prefix format>	/var/lib/scalyr-agent-2	Agent metadata dir (for checkpoints etc.)
Global	config_directory	<prefix format>	agent.d	Agent config dir (located in same parent directory as agent.json file)
Global	debug_init	<prefix format>	false
Global	additional_monitor_module_paths	<prefix format>	<empty string>
Global	compression_type	<prefix format>	bz2	Compression type for data uploaded to Scalyr. One of bz2 or deflate (which uses zlib compress).
Global	compression_level	<prefix format>	9
Global	check_remote_if_no_tty	<prefix format>	true
Global	close_old_files_duration_in_seconds	<prefix format>	3600
Global	copying_thread_profile_interval	<prefix format>	0	min_allowed_request_size
Global	copying_thread_profile_output_path	<prefix format>	/tmp/copying_thread_profiles_
Global	copy_staleness_threshold	<prefix format>	900
Global	full_checkpoint_interval_in_seconds	<prefix format>	60
Global	global_monitor_sample_interval	<prefix format>	30
Global	max_allowed_request_size	<prefix format>	1024*1024
Global	min_allowed_request_size	<prefix format>	100*1024
Global	min_request_spacing_interval	<prefix format>	1
Global	max_request_spacing_interval	<prefix format>	5	Max amount of time to wait between sending requests
Global	max_error_request_spacing_interval	<prefix format>	30
Global	minimum_scan_interval	<prefix format>	None
Global	low_water_bytes_sent	<prefix format>	20*1024
Global	high_water_bytes_sent	<prefix format>	100*1024
Global	low_water_request_spacing_adjustment	<prefix format>	1.5
Global	high_water_request_spacing_adjustment	<prefix format>	0.6
Global	failure_request_spacing_adjustment	<prefix format>	1.5
Global	request_too_large_adjustment	<prefix format>	0.5
Global	log_deletion_delay	<prefix format>	600	Min seconds the agent waits for file to reappear on a file system after removal before it's considered deleted.
Global	line_completion_wait_time	<prefix format>	5	Max number of seconds the agent waits before returning partial lines. (A partial line is the last log line in a file not ending in a new line.)
Global	max_new_log_detection_time	<prefix format>	60
Global	max_log_offset_size	<prefix format>	510241024	How far to search back when getting a new log
Global	max_existing_log_offset_size	<prefix format>	10010241024	How far to search back when getting an existing log
Global	max_sequence_number	<prefix format>	1024^4
Global	pipeline_threshold	<prefix format>	1.1	Size required for a message to trigger pipelining the next addEvents request, expressed as a multiple of max_allowed_request_size.
Global	pidfile_advanced_reuse_guard	<prefix format>	false
Global	request_deadline	<prefix format>	60.0
Global	strip_domain_from_default_server_host	<prefix format>	false
Global	use_requests_lib	<prefix format>	false
Global	use_unsafe_debugging	<prefix format>	false
Global	enable_profiling	<prefix format>	false	If true, the agent will log performance profiling data about itself into a log file.
Global	profile_log_name	<prefix format>	'agent.callgrind'	Profile data log file name.
Global	profile_duration_minutes	<prefix format>	2	Duration (in minutes) to gather profiling information.
Global	max_profile_interval_minutes	<prefix format>	60	Frequency (in minutes) between consecutive runs of the profiler.
Global	profile_clock	<prefix format>	'random'	The clock used for measuring profiling performance. One of ['random', 'wall', 'cpu'].
K8s monitor	docker_percpu_metrics	<prefix format>	false	If true, the agent emits CPU usage metrics per core. Note: this is disabled by default because it can lead to an excessive amount of metric data on CPUs with a large number of cores.
K8s monitor	k8s_parse_format	<prefix format>	'auto'
K8s monitor	k8s_always_use_cri	<prefix format>	false
K8s monitor	k8s_cri_query_filesystem	<prefix format>	false
K8s events	message_log	SCALYR_K8S_MESSAGE_LOG	kubernetes_events.log
K8s events	event_object_filter	SCALYR_K8S_EVENT_OBJECT_FILTER	'CronJob', 'DaemonSet', 'Deployment', 'Job', 'Node', 'Pod', 'ReplicaSet', 'ReplicationController', 'StatefulSet'
K8s events	ignore_master	SCALYR_K8S_IGNORE_MASTER	true
Docker monitor	docker_percpu_metrics	<prefix format>	false	If true, the agent emits CPU usage metrics per core. Note: this is disabled by default because it can lead to an excessive amount of metric data on CPUs with a large number of cores.
Docker monitor	log_mode	SCALYR_DOCKER_LOG_MODE	'docker_api'	Determines which method is used to gather logs from local containers. If docker_api, then the agent will query container logs over the Docker api. If syslog, then the agent expects the other containers to push logs to this one using the Docker syslog logging driver. This option is to protect legacy behavior; the preferred method is to leave this value set to default (docker_api) and then set docker_raw_logs to true.
Docker monitor	docker_api_version	<prefix format>	'auto'	The version of the Docker API to use. WARNING, if you have log_mode set to syslog, you must also be running the syslog monitor and must set the docker_api_version configuration option in the syslog monitor to this same value.
Docker monitor	docker_log_prefix	<prefix format>	'docker'	Prefix added to the start of all docker log file names. Only applies if docker_raw_logs if false.
Docker monitor	metrics_only	SCALYR_DOCKER_METRICS_ONLY	false	If true, the docker monitor will only log docker metrics and not any other logs from running containers. If set to true, this value overrides the config variable 'report_container_metrics'.
Docker monitor	container_globs	<prefix format>	None	A whitelist of container name glob patterns to monitor. Only containers whose name matches one of the glob patterns will be monitored. If None, all container names are matched. This value is applied before container_globs_exclude.
Docker monitor	label_prefix	<prefix format>	<empty string>	If labels_as_attributes is true, then append this prefix to the start of each label before adding it to the log attributes.
Docker monitor	use_labels_for_log_config	<prefix format>	true	If true, the docker monitor will check each container for any labels that begin with com.scalyr.config.log. and use those labels (minus the prefix) as fields in the containers log_config. Keys that contain hyphens will automatically be converted to underscores.