Linux 监控 - Prometheus + Grafana

Linux 监控 - Prometheus + Grafana

Grafana 是界面,Prometheus是数据源,通过Http 监控状态,不需要任何SDK或其他集成。Http接口是 exporter

安装Prometheus

我自己的环境是Amazon Linux,以下应该同样适合CentOS
yum update
yum install wget

下载 prometheus
wget https://github.com/prometheus/prometheus/releases/download/v2.40.2/prometheus-2.40.2.linux-amd64.tar.gz
tar zxf prometheus-2.40.2.linux-amd64.tar.gz -C /opt
mv /opt/prometheus-2.40.2.linux-amd64 /opt/prometheus

配置开机启动
vim /usr/lib/systemd/system/prometheus.service
[Unit]
Description=prometheus service

[Service]
User=root
ExecStart=/opt/prometheus/prometheus --config.file=/opt/prometheus/prometheus.yml --storage.tsdb.path=/opt/prometheus/data

TimeoutStopSec=10
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

systemctl daemon-reload
systemctl enable prometheus —now
systemctl status prometheus

防火墙9090放行,linux本身与网络策略都要放行
访问 http://ip:9090 试验

安装Grafana 官方源(需要科学上网环境)
vim /etc/yum.repos.d/grafana.repo
[grafana]
name=grafana
baseurl=https://packages.grafana.com/oss/rpm
repo_gpgcheck=1
enabled=1
gpgcheck=1
gpgkey=https://packages.grafana.com/gpg.key
sslverify=1
sslcacert=/etc/pki/tls/certs/ca-bundle.crt

install
yum makecache -y
yum install grafana -y
systemctl daemon-reload
systemctl enable grafana-server —now
systemctl status grafana-server
防火墙端口 3000 放行
访问 http://ip:3000, 默认用户名密码都是 admin, 初始登录要求更改
添加 Prometheus 数据源
“Add your first data source” 选择 Prometheus
Dashboards 选择 “Prometheus 2.0 Starts” 进行 Import
Settings 页面填写 Prometheus 地址保存 http://ip:9090
控制端完成

被控端安装 node_exporter
node_exporter
wget https://github.com/prometheus/node_exporter/releases/download/v1.4.0/node_exporter-1.4.0.linux-amd64.tar.gz
tar xf node_exporter-1.4.0.linux-amd64.tar.gz -C /opt/

mv /opt/node_exporter-1.4.0.linux-amd64/ /opt/node_exporter

创建 node_exporter 系统服务启动文件
vim /usr/lib/systemd/system/node_exporter.service
[Unit]
Description=Node Exporter
Wants=network-online.target
After=network-online.target

[Service]
User=root
ExecStart=/opt/node_exporter/node_exporter

[Install]
WantedBy=default.target

启动node_exporter
systemctl daemon-reload
systemctl enable node_exporter —now
systemctl status node_exporter

# 服务启动后可以用 http://localhost:9100/metrics 

防火墙放行 9100
浏览器里访问 http://ip:9100/metrics 测试 node_exporter 是否获取到节点的监控指标

Prometheus配置文件添加监控项
vim /opt/prometheus/prometheus.yml

- job_name: 'linux-master'
    static_configs:
    - targets: ['ip:9100']
      labels:
        instance: linux-master

务必注意格式,空格要一样
重启prometheus
systemctl reload prometheus.service

回到Grafana的主界面,导入dashboard
下载json
import

安装插件
grafana-cli plugins install grafana-piechart-panel
grafana-cli plugins install digiapulssi-breadcrumb-panel
grafana-cli plugins install grafana-polystat-panel

systemctl restart grafana-server

查看已安装插件
/usr/sbin/grafana-cli plugins ls
installed plugins:
grafana-piechart-panel @ 1.3.3

有多台被控端,就各自安装 node_exporter, 然后 编辑 Prometheus主控端
/opt/prometheus/prometheus.yml

最后重启 prometheus
systemctl reload prometheus