operator

The Istio operator.

FlagsDescription
—kubeconfig <string>Paths to a kubeconfig. Only required if out-of-cluster. (default ``)

operator server

Starts the Istio operator server

  1. operator server [flags]
FlagsDescription
—ctrlz_address <string>The IP Address to listen on for the ControlZ introspection facility. Use ‘*’ to indicate all addresses. (default localhost)
—ctrlz_port <uint16>The IP port to use for the ControlZ introspection facility (default 9876)
—kubeconfig <string>Paths to a kubeconfig. Only required if out-of-cluster. (default )</td></tr><tr><td><code>--log_as_json</code></td><td>Whether to format output as JSON or in plain console-friendly format</td></tr><tr><td><code>--log_caller &lt;string&gt;</code></td><td>Comma-separated list of scopes for which to include caller information, scopes can be any of [ads, adsc, all, authn, authorization, ca, cache, default, installer, klog, kube, model, patch, spiffe, tpath, translator, util, validation, validationController, wle] (default)
—log_output_level <string>Comma-separated minimum per-scope logging level of messages to output, in the form of <scope>:<level>,<scope>:<level>,… where scope can be one of [ads, adsc, all, authn, authorization, ca, cache, default, installer, klog, kube, model, patch, spiffe, tpath, translator, util, validation, validationController, wle] and level can be one of [debug, info, warn, error, fatal, none] (default default:info)
—log_rotate <string>The path for the optional rotating log file (default `)</td></tr><tr><td><code>--log_rotate_max_age &lt;int&gt;</code></td><td>The maximum age in days of a log file beyond which the file is rotated (0 indicates no limit) (default30)</td></tr><tr><td><code>--log_rotate_max_backups &lt;int&gt;</code></td><td>The maximum number of log file backups to keep before older files are deleted (0 indicates no limit) (default1000)</td></tr><tr><td><code>--log_rotate_max_size &lt;int&gt;</code></td><td>The maximum size in megabytes of a log file beyond which the file is rotated (default104857600)</td></tr><tr><td><code>--log_stacktrace_level &lt;string&gt;</code></td><td>Comma-separated minimum per-scope logging level at which stack traces are captured, in the form of &lt;scope&gt;:&lt;level&gt;,&lt;scope:level&gt;,... where scope can be one of [ads, adsc, all, authn, authorization, ca, cache, default, installer, klog, kube, model, patch, spiffe, tpath, translator, util, validation, validationController, wle] and level can be one of [debug, info, warn, error, fatal, none] (defaultdefault:none)</td></tr><tr><td><code>--log_target &lt;stringArray&gt;</code></td><td>The set of paths where to output the log. This can be any path as well as the special values stdout and stderr (default[stdout]`)

operator version

Prints out build version information

  1. operator version [flags]
FlagsShorthandDescription
—kubeconfig <string>Paths to a kubeconfig. Only required if out-of-cluster. (default )</td></tr><tr><td><code>--output &lt;string&gt;</code></td><td><code>-o</code></td><td>One of 'yaml' or 'json'. (default)
—short-sUse —short=false to generate full version information

Environment variables

These environment variables affect the behavior of the operator command. Please use with caution as these environment variables are experimental and can change anytime.

Variable NameTypeDefault ValueDescription
CLUSTER_IDStringKubernetesDefines the cluster and service registry that this Istiod instance is belongs to
ENABLE_ADMIN_ENDPOINTSBooleanfalseIf this is set to true, dangerous admin endpoins will be exposed on the debug interface. Not recommended for production.
ENABLE_CA_SERVERBooleantrueIf this is set to false, will not create CA server in istiod.
ENABLE_DEBUG_ON_HTTPBooleantrueIf this is set to false, the debug interface will not be ebabled on Http, recommended for production
ENABLE_LEGACY_FSGROUP_INJECTIONBooleantrueIf true, Istiod will set the pod fsGroup to 1337 on injection. This is required for Kubernetes 1.18 and older (see https://github.com/kubernetes/kubernetes/issues/57923 for details) unless JWT_POLICY is “first-party-jwt”.
ENABLE_WASM_TELEMETRYBooleanfalseIf enabled, Wasm-based telemetry will be enabled.
EXTERNAL_ISTIODBooleanfalseIf this is set to true, one Istiod will control remote clusters including CA.
INJECTION_WEBHOOK_CONFIG_NAMEStringistio-sidecar-injectorName of the mutatingwebhookconfiguration to patch, if istioctl is not used.
ISTIOD_CUSTOM_HOSTStringCustom host name of istiod that istiod signs the server cert.
ISTIOD_ENABLE_SDS_SERVERBooleantrueIf enabled, Istiod will serve SDS for credentialName secrets (rather than in-proxy). To ensure proper security, PILOT_ENABLE_XDS_IDENTITY_CHECK=true is required as well.
ISTIO_AGENT_ENABLE_WASM_REMOTE_LOAD_CONVERSIONBooleantrueIf enabled, Istio agent will intercept ECDS resource update, downloads Wasm module, and replaces Wasm module remote load with downloaded local module file.
ISTIO_DEFAULT_REQUEST_TIMEOUTTime Duration0sDefault Http and gRPC Request timeout
ISTIO_GPRC_MAXRECVMSGSIZEInteger4194304Sets the max receive buffer size of gRPC stream in bytes.
ISTIO_GPRC_MAXSTREAMSInteger100000Sets the maximum number of concurrent grpc streams.
JWT_POLICYStringthird-party-jwtThe JWT validation policy.
K8S_INGRESS_NSString
PILOT_ALLOW_METADATA_CERTS_DR_MUTUAL_TLSBooleanfalseIf true, Pilot will allow certs specified in Metadata to override DR certs in MUTUAL TLS mode. This is only enabled for migration and will be removed soon.
PILOT_CERT_PROVIDERStringistiodThe provider of Pilot DNS certificate.
PILOT_DEBOUNCE_AFTERTime Duration100msThe delay added to config/registry events for debouncing. This will delay the push by at least this internal. If no change is detected within this period, the push will happen, otherwise we’ll keep delaying until things settle, up to a max of PILOT_DEBOUNCE_MAX.
PILOT_DEBOUNCE_MAXTime Duration10sThe maximum amount of time to wait for events while debouncing. If events keep showing up with no breaks for this time, we’ll trigger a push.
PILOT_DISTRIBUTION_HISTORY_RETENTIONTime Duration1m0sIf enabled, Pilot will keep track of old versions of distributed config for this duration.
PILOT_ENABLED_SERVICE_APISBooleanfalseIf this is set to true, support for Kubernetes service-apis (github.com/kubernetes-sigs/service-apis) will be enabled. This feature is currently experimental, and is off by default.
PILOT_ENABLE_ANALYSISBooleanfalseIf enabled, pilot will run istio analyzers and write analysis errors to the Status field of any Istio Resources
PILOT_ENABLE_CONFIG_DISTRIBUTION_TRACKINGBooleantrueIf enabled, Pilot will assign meaningful nonces to each Envoy configuration message, and allow users to interrogate which envoy has which config from the debug interface.
PILOT_ENABLE_CRD_VALIDATIONBooleanfalseIf enabled, pilot will validate CRDs while retrieving CRDs from kubernetes cache.Use this flag to enable validation of CRDs in Pilot, especially in deployments that do not have galley installed.
PILOT_ENABLE_CROSS_CLUSTER_WORKLOAD_ENTRYBooleanfalseIf enabled, pilot will read WorkloadEntry from other clusters, selectable by Services in that cluster.
PILOT_ENABLE_DESTINATION_RULE_INHERITANCEBooleanfalseIf set, workload specific DestinationRules will inherit configurations settings from mesh and namespace level rules
PILOT_ENABLE_EDS_DEBOUNCEBooleantrueIf enabled, Pilot will include EDS pushes in the push debouncing, configured by PILOT_DEBOUNCE_AFTER and PILOT_DEBOUNCE_MAX. EDS pushes may be delayed, but there will be fewer pushes. By default this is enabled
PILOT_ENABLE_EDS_FOR_HEADLESS_SERVICESBooleanfalseIf enabled, for headless service in Kubernetes, pilot will send endpoints over EDS, allowing the sidecar to load balance among pods in the headless service. This feature should be enabled if applications access all services explicitly via a HTTP proxy port in the sidecar.
PILOT_ENABLE_FLOW_CONTROLBooleanfalseIf enabled, pilot will wait for the completion of a receive operation beforeexecuting a push operation. This is a form of flow control and is useful inenvironments with high rates of push requests to each gateway. By default,this is false.
PILOT_ENABLE_HEADLESS_SERVICE_POD_LISTENERSBooleantrueIf enabled, for a headless service/stateful set in Kubernetes, pilot will generate an outbound listener for each pod in a headless service. This feature should be disabled if headless services have a large number of pods.
PILOT_ENABLE_ISTIO_TAGSBooleantrueDetermines whether or not trace spans generated by Envoy will include Istio-specific tags.
PILOT_ENABLE_K8S_SELECT_WORKLOAD_ENTRIESBooleantrueIf enabled, Kubernetes services with selectors will select workload entries with matching labels. It is safe to disable it if you are quite sure you don’t need this feature
PILOT_ENABLE_LOOP_BLOCKERBooleantrueIf enabled, Envoy will be configured to prevent traffic directly the the inbound/outbound ports (15001/15006). This prevents traffic loops. This option will be removed, and considered always enabled, in 1.9.
PILOT_ENABLE_MYSQL_FILTERBooleanfalseEnableMysqlFilter enables injection of envoy.filters.network.mysql_proxy in the filter chain.
PILOT_ENABLE_PROTOCOL_SNIFFING_FOR_INBOUNDBooleantrueIf enabled, protocol sniffing will be used for inbound listeners whose port protocol is not specified or unsupported
PILOT_ENABLE_PROTOCOL_SNIFFING_FOR_OUTBOUNDBooleantrueIf enabled, protocol sniffing will be used for outbound listeners whose port protocol is not specified or unsupported
PILOT_ENABLE_REDIS_FILTERBooleanfalseEnableRedisFilter enables injection of envoy.filters.network.redis_proxy in the filter chain.
PILOT_ENABLE_SERVICEENTRY_SELECT_PODSBooleantrueIf enabled, service entries with selectors will select pods from the cluster. It is safe to disable it if you are quite sure you don’t need this feature
PILOT_ENABLE_STATUSBooleanfalseIf enabled, pilot will update the CRD Status field of all istio resources with reconciliation status.
PILOT_ENABLE_THRIFT_FILTERBooleanfalseEnableThriftFilter enables injection of envoy.filters.network.thrift_proxy in the filter chain.
PILOT_ENABLE_VIRTUAL_SERVICE_DELEGATEBooleantrueIf set to false, virtualService delegate will not be supported.
PILOT_ENABLE_WORKLOAD_ENTRY_AUTOREGISTRATIONBooleanfalseEnables auto-registering WorkloadEntries based on associated WorkloadGroups upon XDS connection by the workload.
PILOT_ENABLE_WORKLOAD_ENTRY_HEALTHCHECKSBooleanfalseEnables automatic health checks of WorkloadEntries based on the config provided in the associated WorkloadGroup
PILOT_ENABLE_XDS_CACHEBooleantrueIf true, Pilot will cache XDS responses.
PILOT_ENABLE_XDS_IDENTITY_CHECKBooleantrueIf enabled, pilot will authorize XDS clients, to ensure they are acting only as namespaces they have permissions for.
PILOT_ENDPOINT_TELEMETRY_LABELBooleantrueIf true, pilot will add telemetry related metadata to Endpoint resource, which will be consumed by telemetry filter.
PILOT_FILTER_GATEWAY_CLUSTER_CONFIGBooleanfalse
PILOT_FLOW_CONTROL_TIMEOUTTime Duration15sIf set, the max amount of time to delay a push by. Depends on PILOT_ENABLE_FLOW_CONTROL.
PILOT_HTTP10BooleanfalseEnables the use of HTTP 1.0 in the outbound HTTP listeners, to support legacy applications.
PILOT_INBOUND_PROTOCOL_DETECTION_TIMEOUTTime Duration1sProtocol detection timeout for inbound listener
PILOT_INITIAL_FETCH_TIMEOUTTime Duration0sSpecifies the initial_fetch_timeout for config. If this time is reached without a response to the config requested by Envoy, the Envoy will move on with the init phase. This prevents envoy from getting stuck waiting on config during startup.
PILOT_PUSH_THROTTLEInteger100Limits the number of concurrent pushes allowed. On larger machines this can be increased for faster pushes
PILOT_SCOPE_GATEWAY_TO_NAMESPACEBooleanfalseIf enabled, a gateway workload can only select gateway resources in the same namespace. Gateways with same selectors in different namespaces will not be applicable.
PILOT_SIDECAR_ENABLE_INBOUND_TLS_V2BooleantrueIf true, Pilot will set the TLS version on server side as TLSv1_2 and also enforce strong cipher suites
PILOT_SIDECAR_USE_REMOTE_ADDRESSBooleanfalseUseRemoteAddress sets useRemoteAddress to true for side car outbound listeners.
PILOT_SKIP_VALIDATE_TRUST_DOMAINBooleanfalseSkip validating the peer is from the same trust domain when mTLS is enabled in authentication policy
PILOT_STATUS_BURSTInteger500If status is enabled, controls the Burst rate with which status will be updated. See https://godoc.org/k8s.io/client-go/rest#Config Burst
PILOT_STATUS_MAX_WORKERSInteger100The maximum number of workers Pilot will use to keep configuration status up to date. Smaller numbers will result in higher status latency, but larger numbers may impact CPU in high scale environments.
PILOT_STATUS_QPSFloating-Point100If status is enabled, controls the QPS with which status will be updated. See https://godoc.org/k8s.io/client-go/rest#Config QPS
PILOT_TRACE_SAMPLINGFloating-Point1Sets the mesh-wide trace sampling percentage. Should be 0.0 - 100.0. Precision to 0.01. Default is 1.0.
PILOT_USE_ENDPOINT_SLICEBooleanfalseIf enabled, Pilot will use EndpointSlices as the source of endpoints for Kubernetes services. By default, this is false, and Endpoints will be used. This requires the Kubernetes EndpointSlice controller to be enabled. Currently this is mutual exclusive - either Endpoints or EndpointSlices will be used
PILOT_WORKLOAD_ENTRY_GRACE_PERIODTime Duration10sThe amount of time an auto-registered workload can remain disconnected from all Pilot instances before the associated WorkloadEntry is cleaned up.
PILOT_XDS_CACHE_SIZEInteger20000The maximum number of cache entries for the XDS cache. If the size is <= 0, the cache will have no upper bound.
PILOT_XDS_CACHE_STATSBooleanfalseIf true, Pilot will collect metrics for XDS cache efficiency.
PILOT_XDS_SEND_TIMEOUTTime Duration5sThe timeout to send the XDS configuration to proxies. After this timeout is reached, Pilot will discard that push.
POD_NAMEString
REQUIRE_3P_TOKENBooleanfalseReject k8s default tokens, without audience. If false, default K8S token will be accepted
SPIFFE_BUNDLE_ENDPOINTSStringThe SPIFFE bundle trust domain to endpoint mappings. Istiod retrieves the root certificate from each SPIFFE bundle endpoint and uses it to verify client certifiates from that trust domain. The endpoint must be compliant to the SPIFFE Bundle Endpoint standard. For details, please refer to https://github.com/spiffe/spiffe/blob/master/standards/SPIFFE_Trust_Domain_and_Bundle.md . No need to configure this for root certificates issued via Istiod or web-PKI based root certificates. Use || between <trustdomain, endpoint> tuples. Use | as delimiter between trust domain and endpoint in each tuple. For example: foo|https://url/for/foo||bar|https://url/for/bar
TERMINATION_DRAIN_DURATION_SECONDSInteger5The amount of time allowed for connections to complete on pilot-agent shutdown. On receiving SIGTERM or SIGINT, pilot-agent tells the active Envoy to start draining, preventing any new connections and allowing existing connections to complete. It then sleeps for the TerminationDrainDuration and then kills any remaining active Envoy processes.
TOKEN_AUDIENCESStringistio-caA list of comma separated audiences to check in the JWT token before issuing a certificate. The token is accepted if it matches with one of the audiences
XDS_AUTHBooleantrueIf true, will authenticate XDS clients.

Exported metrics

Metric NameTypeDescription
cache_flush_totalSumnumber of times operator cache was flushed
cr_deletion_totalSumNumber of IstioOperator CR deleted
cr_merge_failure_totalSumNumber of IstioOperator CR merge failures
cr_validation_error_totalSumNumber of IstioOperator CR validation failures
endpoint_no_podLastValueEndpoints without an associated pod.
galley_validation_config_delete_errorCountk8s webhook configuration delete error
galley_validation_config_loadCountk8s webhook configuration (re)loads
galley_validation_config_load_errorCountk8s webhook configuration (re)load error
galley_validation_config_update_errorCountk8s webhook configuration update error
galley_validation_config_updatesCountk8s webhook configuration updates
get_cr_error_totalSumNumber of times fetching CR from apiserver failed
istio_buildLastValueIstio component build info
legacy_path_translation_totalSumNumber of times a legacy API path is translated
manifest_patch_error_totalSumNumber of times K8S patch overlays failed
manifest_render_error_totalSumNumber of times error occurred during rendering output manifest
num_failed_outgoing_requestsSumNumber of failed outgoing requests (e.g. to a token exchange server, CA, etc.)
num_file_secret_failures_totalSumNumber of times secret generation failed for files
num_file_watcher_failures_totalSumNumber of times file watcher failed to add watchers
num_outgoing_requestsSumNumber of total outgoing requests (e.g. to a token exchange server, CA, etc.)
num_outgoing_retriesSumNumber of outgoing retry requests (e.g. to a token exchange server, CA, etc.)
outgoing_latencySumThe latency of outgoing requests (e.g. to a token exchange server, CA, etc.) in milliseconds.
owned_resource_totalLastValueNumber of resources currently owned by the operator
pilot_conflict_inbound_listenerLastValueNumber of conflicting inbound listeners.
pilot_conflict_outbound_listener_http_over_current_tcpLastValueNumber of conflicting wildcard http listeners with current wildcard tcp listener.
pilot_conflict_outbound_listener_tcp_over_current_httpLastValueNumber of conflicting wildcard tcp listeners with current wildcard http listener.
pilot_conflict_outbound_listener_tcp_over_current_tcpLastValueNumber of conflicting tcp listeners with current tcp listener.
pilot_destrule_subsetsLastValueDuplicate subsets across destination rules for same host
pilot_duplicate_envoy_clustersLastValueDuplicate envoy clusters caused by service entries with same hostname
pilot_eds_no_instancesLastValueNumber of clusters without instances.
pilot_endpoint_not_readyLastValueEndpoint found in unready state.
pilot_inbound_updatesSumTotal number of updates received by pilot.
pilot_invalid_out_listenersLastValueNumber of invalid outbound listeners.
pilot_jwks_resolver_network_fetch_fail_totalSumTotal number of failed network fetch by pilot jwks resolver
pilot_jwks_resolver_network_fetch_success_totalSumTotal number of successfully network fetch by pilot jwks resolver
pilot_k8s_cfg_eventsSumEvents from k8s config.
pilot_k8s_endpoints_pending_podLastValueNumber of endpoints that do not currently have any corresponding pods.
pilot_k8s_endpoints_with_no_podsSumEndpoints that does not have any corresponding pods.
pilot_k8s_object_errorsLastValueErrors converting k8s CRDs
pilot_k8s_reg_eventsSumEvents from k8s registry.
pilot_no_ipLastValuePods not found in the endpoint table, possibly invalid.
pilot_proxy_convergence_timeDistributionDelay in seconds between config change and a proxy receiving all required configuration.
pilot_proxy_queue_timeDistributionTime in seconds, a proxy is in the push queue before being dequeued.
pilot_push_triggersSumTotal number of times a push was triggered, labeled by reason for the push.
pilot_servicesLastValueTotal services known to pilot.
pilot_total_k8s_object_errorsSumTotal Errors converting k8s CRDs
pilot_total_rejected_configsSumTotal number of configs that Pilot had to reject or ignore.
pilot_total_xds_internal_errorsSumTotal number of internal XDS errors in pilot.
pilot_total_xds_rejectsSumTotal number of XDS responses from pilot rejected by proxy.
pilot_virt_servicesLastValueTotal virtual services known to pilot.
pilot_vservice_dup_domainLastValueVirtual services with dup domains.
pilot_xdsLastValueNumber of endpoints connected to this pilot using XDS.
pilot_xds_cds_rejectLastValuePilot rejected CDS configs.
pilot_xds_delayed_push_timeouts_totalSumTotal number of XDS pushes that are delayed and timed out
pilot_xds_delayed_pushes_totalSumTotal number of XDS pushes that are delayed.
pilot_xds_eds_rejectLastValuePilot rejected EDS.
pilot_xds_expired_nonceSumTotal number of XDS requests with an expired nonce.
pilot_xds_lds_rejectLastValuePilot rejected LDS.
pilot_xds_push_context_errorsSumNumber of errors (timeouts) initiating push context.
pilot_xds_push_timeDistributionTotal time in seconds Pilot takes to push lds, rds, cds and eds.
pilot_xds_pushesSumPilot build and send errors for lds, rds, cds and eds.
pilot_xds_rds_rejectLastValuePilot rejected RDS.
pilot_xds_send_timeDistributionTotal time in seconds Pilot takes to send generated configuration.
pilot_xds_write_timeoutSumPilot XDS response write timeouts.
render_manifest_totalSumNumber of component manifests rendered
resource_creation_totalSumNumber of resources created by the operator
resource_deletion_totalSumNumber of resources deleted by the operator
resource_prune_totalSumNumber of resources pruned by the operator
resource_update_totalSumNumber of resources updated by the operator
versionLastValueVersion of operator binary
xds_cache_evictionsSumTotal number of xds cache evictions.
xds_cache_readsSumTotal number of xds cache xdsCacheReads.
xds_cache_sizeLastValueCurrent size of xds cache