Module: INFRA

Module: INFRA

More observability stack and database runtime

Pigsty has a battery-included, production-ready INFRA module, to provide ultimate observability.

Overview

Each Pigsty deployment requires a set of infrastructure components to work properly. which including:

Component	Port	Domain	Description
Nginx	80	`h.pigsty`	Web Service Portal (Also used as Yum Repo)
AlertManager	9093	`a.pigsty`	Alert Aggregation and delivery
Prometheus	9090	`p.pigsty`	Monitoring Time Series Database
Grafana	3000	`g.pigsty`	Visualization Platform
Loki	3100	-	Logging Collection Server
PushGateway	3100	-	Logging Collection Server
BlackboxExporter	3100	-	Logging Collection Server
Dnsmasq	53	-	DNS Server (optional)
Chronyd	123	-	NTP Time Server (optional)
PostgreSQL	5432	-	Pigsty CMDB & default database
Ansible	-	-	Run playbooks

Pigsty will set up these components for you on infra nodes. You can expose them to the outside world by configuring the infra_portal parameter.

infra_portal:  # domain names and upstream servers
  home         : { domain: h.pigsty }
  grafana      : { domain: g.pigsty ,endpoint: "${admin_ip}:3000" , websocket: true }
  prometheus   : { domain: p.pigsty ,endpoint: "${admin_ip}:9090" }
  alertmanager : { domain: a.pigsty ,endpoint: "${admin_ip}:9093" }
  blackbox     : { endpoint: "${admin_ip}:9115" }
  loki         : { endpoint: "${admin_ip}:3100" }
  minio        : { domain: sss.pigsty  ,endpoint: "${admin_ip}:9001" ,scheme: https ,websocket: true }

Playbooks

install.yml : Install Pigsty on current node in one-pass
infra.yml : Init pigsty infrastructure on infra nodes
infra-rm.yml : Remove infrastructure components from infra nodes

Dashboards

INFRA Overview : Overview of all infra components
Nginx Overview : Nginx metrics & logs
Grafana Overview: Grafana metrics & logs
Prometheus Overview: Prometheus metrics & logs
Loki Overview: Loki metrics & logs
Logs Instance: Logs for a single instance
CMDB Overview: CMDB visualization
ETCD Overview: etcd metrics & logs

Parameters

API Reference for INFRA module:

META: infra meta data
CA: self-signed CA
INFRA_ID : Portals and identity
REPO: local yum repo
INFRA_PACKAGE : packages to be installed
NGINX : nginx web server
DNS: dnsmasq nameserver
PROMETHEUS : prometheus, alertmanager, pushgateway & blackbox_exporter
GRAFANA : Grafana, the visualization platform
LOKI : Loki, the logging server

Parameters

Parameter	Section	Type	Level	Comment
version	META	string	G	pigsty version string
admin_ip	META	ip	G	admin node ip address
region	META	enum	G	upstream mirror region: default,china,europe
proxy_env	META	dict	G	global proxy env when downloading packages
ca_method	CA	enum	G	create,recreate,copy, create by default
ca_cn	CA	string	G	ca common name, fixed as pigsty-ca
cert_validity	CA	interval	G	cert validity, 20 years by default
infra_seq	INFRA_ID	int	I	infra node identity, REQUIRED
infra_portal	INFRA_ID	dict	G	infra services exposed via portal
repo_enabled	REPO	bool	G/I	create a yum repo on this infra node?
repo_home	REPO	path	G	repo home dir, `/www` by default
repo_name	REPO	string	G	repo name, pigsty by default
repo_endpoint	REPO	url	G	access point to this repo by domain or ip:port
repo_remove	REPO	bool	G/A	remove existing upstream repo
repo_upstream	REPO	upstream[]	G	where to download upstream packages
repo_packages	REPO	string[]	G	which packages to be included
repo_url_packages	REPO	string[]	G	extra packages from url
infra_packages	INFRA_PACKAGE	string[]	G	packages to be installed on infra nodes
infra_packages_pip	INFRA_PACKAGE	string	G	pip installed packages for infra nodes
nginx_enabled	NGINX	bool	G/I	enable nginx on this infra node?
nginx_sslmode	NGINX	enum	G	nginx ssl mode? disable,enable,enforce
nginx_home	NGINX	path	G	nginx content dir, `/www` by default
nginx_port	NGINX	port	G	nginx listen port, 80 by default
nginx_ssl_port	NGINX	port	G	nginx ssl listen port, 443 by default
nginx_navbar	NGINX	index[]	G	nginx index page navigation links
dns_enabled	DNS	bool	G/I	setup dnsmasq on this infra node?
dns_port	DNS	port	G	dns server listen port, 53 by default
dns_records	DNS	string[]	G	dynamic dns records resolved by dnsmasq
prometheus_enabled	PROMETHEUS	bool	G/I	enable prometheus on this infra node?
prometheus_clean	PROMETHEUS	bool	G/A	clean prometheus data during init?
prometheus_data	PROMETHEUS	path	G	prometheus data dir, `/data/prometheus` by default
prometheus_sd_interval	PROMETHEUS	interval	G	prometheus target refresh interval, 5s by default
prometheus_scrape_interval	PROMETHEUS	interval	G	prometheus scrape & eval interval, 10s by default
prometheus_scrape_timeout	PROMETHEUS	interval	G	prometheus global scrape timeout, 8s by default
prometheus_options	PROMETHEUS	arg	G	prometheus extra server options
pushgateway_enabled	PROMETHEUS	bool	G/I	setup pushgateway on this infra node?
pushgateway_options	PROMETHEUS	arg	G	pushgateway extra server options
blackbox_enabled	PROMETHEUS	bool	G/I	setup blackbox_exporter on this infra node?
blackbox_options	PROMETHEUS	arg	G	blackbox_exporter extra server options
alertmanager_enabled	PROMETHEUS	bool	G/I	setup alertmanager on this infra node?
alertmanager_options	PROMETHEUS	arg	G	alertmanager extra server options
exporter_metrics_path	PROMETHEUS	path	G	exporter metric path, `/metrics` by default
exporter_install	PROMETHEUS	enum	G	how to install exporter? none,yum,binary
exporter_repo_url	PROMETHEUS	url	G	exporter repo file url if install exporter via yum
grafana_enabled	GRAFANA	bool	G/I	enable grafana on this infra node?
grafana_clean	GRAFANA	bool	G/A	clean grafana data during init?
grafana_admin_username	GRAFANA	username	G	grafana admin username, `admin` by default
grafana_admin_password	GRAFANA	password	G	grafana admin password, `pigsty` by default
grafana_plugin_cache	GRAFANA	path	G	path to grafana plugins cache tarball
grafana_plugin_list	GRAFANA	string[]	G	grafana plugins to be downloaded with grafana-cli
loki_enabled	LOKI	bool	G/I	enable loki on this infra node?
loki_clean	LOKI	bool	G/A	whether remove existing loki data?
loki_data	LOKI	path	G	loki data dir, `/data/loki` by default
loki_retention	LOKI	interval	G	loki log retention period, 15d by default

Last modified 2023-02-27: add v2.0 images and docs (5b09f12)