So, how can we be able to monitor Docker swarm metrics using Grafana, coupled with Prometheus, cAdvisor?<\/p>\n\n\n\n
We will run Grafana and Prometheus as global swarm services, while cAdvisor as docker containers each on a swarm node.<\/p>\n\n\n\n
Install Docker Engine<\/h3>\n\n\n\n
If you have not already installed Docker Engine, please refer to appropriate guides in the link below;<\/p>\n\n\n\n
Install Docker in Linux<\/a><\/p>\n\n\n\n Since we are dealing swarm services, you need to have setup swarm cluster that whose metrics need to be monitored.<\/p>\n\n\n\n If you haven’t, please refer to this guide to learn how to setup swarm cluster.<\/p>\n\n\n\n How to Setup Docker Swarm Cluster on Ubuntu<\/a><\/p>\n\n\n\n In order for the monitoring tools to communicate with each other, they should be in the same network. In a Docker Swarm, overlay networks can be used to provide communication between services, hence, create the network as follows.<\/p>\n\n\n\n The command below creates a Docker swarm network called Since we also need to attach Docker containers to this network, make it attachable<\/strong>.<\/p>\n\n\n\n You check available Docker networks using the command below;<\/p>\n\n\n\n To begin with, let’s deploy cAdvisor as a Docker container on each node<\/strong> in the cluster so that it can collect swarm services runtime metrics from each node.<\/p>\n\n\n\n We have three nodes in our swarm cluster;<\/p>\n\n\n\n You can simply run the command below to deploy cAdvisor and attach it to our custom Docker network above;<\/p>\n\n\n\n By default, cAdvisor exposes metrics on port 8080. This port will be reachable within container networks.<\/p>\n\n\n\n On the rest of the nodes, run the command, replacing the name of the cadvisor on each node.<\/p>\n\n\n\n cAdvisor container should now be up and running. For example on swarm node 2;<\/p>\n\n\n\n Next, let’s deploy Prometheus as a Docker swarm service to scrape collected swarm service metrics from cAdvisor and store them in its internal time series database which can later be read and visualized via Grafana.<\/p>\n\n\n\n First of all, before you can deploy Prometheus swarm service, create a Prometheus configuration file.<\/p>\n\n\n\n In our setup, we placed the Prometheus configuration file under In the configuration above, we use default settings for Prometheus and configured it to scrape the Docker swarm service metrics collected by cAdvisor container, running on each swarm node.<\/p>\n\n\n\n Prometheus will publish the metrics it scrapes on port 9090.<\/p>\n\n\n\n Thus, let’s create Prometheus Docker container using the official Prometheus Docker image and configure it to use our custom network created above, Your Prometheus container should now be running and exposed via port 9090\/tcp.<\/p>\n\n\n\n Similarly, ensure you allow port 9090\/tcp on firewall to allow you access the metrics externally!<\/p>\n\n\n\n You can now navigate to Targets;<\/p>\n\n\n\n Let’s check which container metrics are available.<\/p>\n\n\n\n Thus, on the dashboard, click Prometheus,<\/strong> and open metrics explorer.<\/p>\n\n\n\n Just type container<\/strong> and you should see quite a number of metrics;<\/p>\n\n\n\n You can filter running Swarm services using the query;<\/p>\n\n\n\n To get a better visualization of the Docker container metrics, you can deploy Grafana Docker swarm service across all the nodes in the cluster.<\/p>\n\n\n\n Create Grafana Data, Configuration and home directories volumes. Using a volume for Grafana data ensures that the dashboards, users, and other settings you create are persisted across container restarts. This is important if you want to keep your dashboards and other Grafana settings even if the container is updated or recreated.<\/p>\n\n\n\n To deploy Grafana Docker container using the official image, Grafana is now running as a single replica.<\/p>\n\n\n\n If you run it as global or multi-replicated, you may not be able to login, getting Unauthorized, user token not found issue on the logs!<\/p>\n\n\n\n Open port 3000\/tcp on firewall to allow external access to Grafana;<\/p>\n\n\n\n You can now access Grafana web interface http:\/\/docker-host-IP:3000.<\/p>\n\n\n\n Default credentials are admin\/admin<\/strong>. Reset the password and proceed to the Dashboard;<\/p>\n\n\n\n You can now integrate Prometheus with Grafana by adding Prometheus data source to Grafana. Check the link below;<\/p>\n\n\n\n Integrate Prometheus with Grafana For Monitoring<\/a><\/p>\n\n\n\n Data sources, once you have added the Prometheus data source.<\/p>\n\n\n\n As you can see, we specified the name of the Prometheus container on the data source URL because both Grafana and Prometheus are on same network.<\/p>\n\n\n\n You can now create your own Grafana visualization dashboards for your Docker swarm cluster.<\/p>\n\n\n\n As a simple example, let’s create a simple visualization to display running, swarm cluster service tasks. Note that a task is a running instance of a service.<\/p>\n\n\n\n For example, using docker command, you can get information about the tasks running on your Swarm cluster using the e.g<\/p>\n\n\n\n To create a Grafana dashboard, navigate to dashboards menu > New dashboard.<\/p>\n\n\n\n Add new panel;<\/p>\n\n\n\n You have three sections on the dashboard panel;<\/p>\n\n\n\n So, to create a visualization of specific metric, you need a query to fetch those metrics from the datasource, which in this case is our Preom<\/p>\n\n\n\n The query will count total number of running swarm service in last 5 minute, Under the Panel options;<\/p>\n\n\n\n When done, click Save\/Apply<\/strong> to save the dashboard\/visualization.<\/p>\n\n\n\n Docker container total CPU usage, RAM usage, Network I\/O, Block I\/O<\/p>\n\n\n\n Sample dashboard;<\/p>\n\n\n\n Here is a sample dashboard JSON file;<\/p>\n\n\n\n You can also check community created dashboards<\/a> an import any that might impress you.<\/p>\n\n\n\n That marks the end of our guide on how to monitoring Docker swarm service metrics using Grafana\/Prometheus and cAdvisor.<\/p>\n\n\n\n Monitor Docker Containers Metrics using Grafana<\/a><\/p>\n\n\n\nSetup Docker Swarm Cluster<\/h3>\n\n\n\n
Create Docker Swarm Network to Interconnect Monitoring Tools <\/h3>\n\n\n\n
monitoring_stack<\/code><\/strong>. You can use any name of your preference.<\/p>\n\n\n\ndocker network create --driver overlay --attachable monitoring_stack<\/code><\/pre>\n\n\n\ndocker network ls<\/code><\/pre>\n\n\n\n\nNETWORK ID NAME DRIVER SCOPE\n816c7f48bbb3 bridge bridge local\n841640b7368f docker_gwbridge bridge local\n45c3672051a1 host host local\nsxnbs83rwn5f ingress overlay swarm\ne5mklugaqapm monitoring_stack overlay swarm<\/strong>\n05990914584e none null local\n<\/code><\/pre>\n\n\n\nDeploy cAdvisor Docker Container<\/h3>\n\n\n\n
docker node ls<\/code><\/pre>\n\n\n\nID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION\nhim49eblt74amba1nghv2f8z1 * swarm01 Ready Active Leader 20.10.22\n4lxyd0f6d9h039y5itjiffqax swarm02 Ready Active Reachable 20.10.12\ndfmdl4wy4e7ouklu8mu7nqqh9 swarm03 Ready Active Reachable 20.10.22<\/code><\/pre>\n\n\n\n\ndocker run -d \\\n --name cadvisor-swarm01 \\\n --restart unless-stopped \\\n --network monitoring_stack \\\n --volume \/:\/rootfs:ro \\\n --volume \/var\/run:\/var\/run:rw \\\n --volume \/sys:\/sys:ro \\\n --volume \/var\/lib\/docker\/:\/var\/lib\/docker:ro \\\n gcr.io\/cadvisor\/cadvisor:v0.47.1\n<\/code><\/pre>\n\n\n\ndocker ps <\/code><\/pre>\n\n\n\n\nCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES\n6ed4926e9acf gcr.io\/cadvisor\/cadvisor:v0.47.1 \"\/usr\/bin\/cadvisor -\u2026\" 46 seconds ago Up 44 seconds (healthy) 8080\/tcp cadvisor-swarm02\n41612d395de3 wordpress:latest \"docker-entrypoint.s\u2026\" 24 minutes ago Up 24 minutes 80\/tcp wordpress.3.8kma6ye90bw6e9j2uaj6er25v\n5466529b2b4f jasonrivers\/nagios:latest \"\/usr\/local\/bin\/star\u2026\" 25 minutes ago Up 25 minutes 80\/tcp, 5667\/tcp nagios-server.2.3ueqfxnm2m8t1tclfgy93ebgt\n<\/code><\/pre>\n\n\n\nDeploy Prometheus Docker Swarm Service<\/h3>\n\n\n\n
\/opt\/prometheus<\/strong><\/code> directory. While creating Prometheus swarm service, we will mount this configuration file to the default Prometheus configuration file, \/etc\/prometheus\/prometheus.yml<\/code><\/strong>.<\/p>\n\n\n\nvim \/opt\/prometheus\/prometheus.yml<\/code><\/pre>\n\n\n\n\nglobal:\n scrape_interval: 5s\n evaluation_interval: 10s\n\nscrape_configs:\n - job_name: 'cadvisor'\n metrics_path: '\/metrics'\n static_configs:\n - targets: ['cadvisor-swarm01:8080','cadvisor-swarm02:8080','cadvisor-swarm03:8080']\n<\/code><\/pre>\n\n\n\nmonitoring_stack<\/code>.<\/p>\n\n\n\n\ndocker service create \\\n-p 9090:9090 \\\n--mode global \\\n--network monitoring_stack \\\n--mount type=bind,src=\/opt\/prometheus\/prometheus.yml,dst=\/etc\/prometheus\/prometheus.yml \\\n--name prometheus \\\nprom\/prometheus\n<\/code><\/pre>\n\n\n\ndocker service ls --format '{{.Name}}\\t{{.Ports}}'<\/code><\/pre>\n\n\n\n\nnagios-server\t*:8081->80\/tcp\nprometheus\t*:9090->9090\/tcp<\/strong>\nwordpress\t*:8880->80\/tcp\n<\/code><\/pre>\n\n\n\nVerify Prometheus Targets\/Metrics from Prometheus Dashboard<\/h3>\n\n\n\n
http:\/\/docker-host-IP:9090\/targets<\/code><\/strong> to see Prometheus targets,<\/p>\n\n\n\ndocker-host-IP<\/code><\/strong> can be IP of any of the swarm node.<\/p>\n\n\n\n![\"Monitor](\"https:\/\/kifarunix.com\/wp-content\/uploads\/2023\/04\/prometheus-cadvisor-targets.png\" "\\"\\"")
<\/figure><\/a><\/div>\n\n\n\nCheck Docker Container Metrics on Prometheus<\/h3>\n\n\n\n
![\"\"](\"https:\/\/kifarunix.com\/wp-content\/uploads\/2023\/04\/metric-explorer.png\" "\\"\\"")
<\/figure>\n\n\n\ncontainer_tasks_state{container_label_com_docker_swarm_service_name=~\".+.\",state=\"running\"}<\/code><\/pre>\n\n\n\n![\"\"](\"https:\/\/kifarunix.com\/wp-content\/uploads\/2023\/04\/get-running-swarm-services.png\" "\\"\\"")
<\/figure><\/a><\/div>\n\n\n\nDeploy Grafana Docker Swarm Service<\/h3>\n\n\n\n
for i in conf data home; do docker volume create grafana-$i; done<\/code><\/pre>\n\n\n\ngrafana\/grafana-oss<\/code><\/strong> and attach it to the custom network created above.<\/p>\n\n\n\n\ndocker service create \\\n --name grafana \\\n --network monitoring_stack \\\n --mount type=volume,source=grafana-data,target=\/var\/lib\/grafana \\\n --mount type=volume,source=grafana-home,target=\/usr\/share\/grafana \\\n --mount type=volume,source=grafana-config,target=\/etc\/grafana \\\n --publish published=3000,target=3000 \\\n grafana\/grafana\n<\/code><\/pre>\n\n\n\nufw allow 3000\/tcp<\/code><\/pre>\n\n\n\nAccessing Grafana Web Interface<\/h3>\n\n\n\n
![\"Monitor](\"https:\/\/kifarunix.com\/wp-content\/uploads\/2023\/02\/grafana-dashboard.png\" "\\"\\"")
<\/figure><\/a><\/div>\n\n\n\nIntegrate Prometheus with Grafana For Monitoring<\/h3>\n\n\n\n
![\"Monitor](\"https:\/\/kifarunix.com\/wp-content\/uploads\/2023\/02\/grafana-prometheus-docker-container-data-source.png\" "\\"Monitor")
<\/figure><\/a><\/div>\n\n\n\nCreate Docker Swarm Metrics Dashboards on Grafana<\/h3>\n\n\n\n
docker service ps<\/code> command, which shows the current status, node, and container ID of each task, …<\/p>\n\n\n\ndocker service ps grafana<\/code><\/pre>\n\n\n\n![\"\"](\"https:\/\/kifarunix.com\/wp-content\/uploads\/2023\/02\/grafana-dashboard-menu.png\" "\\"\\"")
<\/figure><\/a><\/div>\n\n\n\n![\"\"](\"https:\/\/kifarunix.com\/wp-content\/uploads\/2023\/02\/new-dashboard-panel.png\" "\\"\\"")
<\/figure><\/a><\/div>\n\n\n\n\n
\n
![\"\"](\"https:\/\/kifarunix.com\/wp-content\/uploads\/2023\/04\/query-builder-for-running-swarm-services.png\" "\\"\\"")
<\/li>\n\n\n\n![\"Monitor](\"https:\/\/kifarunix.com\/wp-content\/uploads\/2023\/04\/query-code-for-running-swarm-services.png\" "\\"\\"")
<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\ncount(rate(container_last_seen{image=~\".+\",container_label_com_docker_swarm_service_id=~\".+\"}[5m]))<\/code><\/strong>.<\/p>\n\n\n\n\n
![\"Monitor](\"https:\/\/kifarunix.com\/wp-content\/uploads\/2023\/04\/running-swarm-services-number.png\" "\\"\\"")
<\/figure><\/a><\/div>\n\n\n\n![\"Monitor](\"https:\/\/kifarunix.com\/wp-content\/uploads\/2023\/04\/swarm-cluster-monitoring.png\" "\\"\\"")
<\/figure><\/a><\/div>\n\n\n\n![\"Monitor](\"https:\/\/kifarunix.com\/wp-content\/uploads\/2023\/04\/swarm-cluster-monitoring-1.png\" "\\"\\"")
<\/figure><\/a><\/div>\n\n\n\n![\"Monitor](\"https:\/\/kifarunix.com\/wp-content\/uploads\/2023\/04\/swarm-cluster-monitoring-2.png\" "\\"\\"")
<\/figure><\/a><\/div>\n\n\n\n\n{\n \"annotations\": {\n \"list\": [\n {\n \"builtIn\": 1,\n \"datasource\": {\n \"type\": \"datasource\",\n \"uid\": \"grafana\"\n },\n \"enable\": true,\n \"hide\": true,\n \"iconColor\": \"rgba(0, 211, 255, 1)\",\n \"name\": \"Annotations & Alerts\",\n \"target\": {\n \"limit\": 100,\n \"matchAny\": false,\n \"tags\": [],\n \"type\": \"dashboard\"\n },\n \"type\": \"dashboard\"\n }\n ]\n },\n \"description\": \"Sample Docker Swarm Cluster\",\n \"editable\": true,\n \"fiscalYearStartMonth\": 0,\n \"gnetId\": 7007,\n \"graphTooltip\": 1,\n \"id\": 2,\n \"links\": [],\n \"liveNow\": false,\n \"panels\": [\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"GbRl4cL4k\"\n },\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"thresholds\"\n },\n \"custom\": {\n \"align\": \"left\",\n \"cellOptions\": {\n \"type\": \"auto\"\n },\n \"inspect\": false\n },\n \"mappings\": [\n {\n \"options\": {\n \"pattern\": \"\\\\w+-(\\\\S+):\\\\d+\",\n \"result\": {\n \"index\": 0,\n \"text\": \"$1\"\n }\n },\n \"type\": \"regex\"\n }\n ],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\",\n \"value\": null\n },\n {\n \"color\": \"red\",\n \"value\": 80\n }\n ]\n }\n },\n \"overrides\": [\n {\n \"matcher\": {\n \"id\": \"byName\",\n \"options\": \"container_label_com_docker_swarm_service_name\"\n },\n \"properties\": [\n {\n \"id\": \"displayName\",\n \"value\": \"Swarm Service\"\n }\n ]\n },\n {\n \"matcher\": {\n \"id\": \"byName\",\n \"options\": \"instance\"\n },\n \"properties\": [\n {\n \"id\": \"displayName\",\n \"value\": \"Swarm Node\"\n }\n ]\n },\n {\n \"matcher\": {\n \"id\": \"byName\",\n \"options\": \"Value\"\n },\n \"properties\": [\n {\n \"id\": \"displayName\",\n \"value\": \"No. Services\"\n }\n ]\n }\n ]\n },\n \"gridPos\": {\n \"h\": 8,\n \"w\": 14,\n \"x\": 0,\n \"y\": 0\n },\n \"id\": 13,\n \"options\": {\n \"footer\": {\n \"countRows\": false,\n \"fields\": [\n \"Value\"\n ],\n \"reducer\": [],\n \"show\": true\n },\n \"showHeader\": true,\n \"sortBy\": [\n {\n \"desc\": true,\n \"displayName\": \"No. Services\"\n }\n ]\n },\n \"pluginVersion\": \"9.4.7\",\n \"targets\": [\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"GbRl4cL4k\"\n },\n \"editorMode\": \"code\",\n \"exemplar\": false,\n \"expr\": \"count(container_last_seen{image=~\\\".+\\\",container_label_com_docker_swarm_service_id=~\\\".+\\\"}) by (instance,container_label_com_docker_swarm_service_name)\",\n \"format\": \"table\",\n \"instant\": true,\n \"legendFormat\": \"{{instance}}\",\n \"range\": false,\n \"refId\": \"A\"\n }\n ],\n \"title\": \"Swarm Services Per Node\",\n \"transformations\": [\n {\n \"id\": \"organize\",\n \"options\": {\n \"excludeByName\": {\n \"Time\": true\n },\n \"indexByName\": {\n \"Time\": 0,\n \"Value\": 3,\n \"container_label_com_docker_swarm_service_name\": 2,\n \"instance\": 1\n },\n \"renameByName\": {}\n }\n }\n ],\n \"type\": \"table\"\n },\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"GbRl4cL4k\"\n },\n \"description\": \"These include both docker containers and swarm services tasks\",\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"thresholds\"\n },\n \"mappings\": [\n {\n \"options\": {\n \"match\": \"null\",\n \"result\": {\n \"text\": \"N\/A\"\n }\n },\n \"type\": \"special\"\n }\n ],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\",\n \"value\": null\n },\n {\n \"color\": \"red\",\n \"value\": 80\n }\n ]\n },\n \"unit\": \"none\"\n },\n \"overrides\": []\n },\n \"gridPos\": {\n \"h\": 4,\n \"w\": 6,\n \"x\": 14,\n \"y\": 0\n },\n \"id\": 3,\n \"links\": [],\n \"maxDataPoints\": 100,\n \"options\": {\n \"colorMode\": \"value\",\n \"graphMode\": \"none\",\n \"justifyMode\": \"auto\",\n \"orientation\": \"horizontal\",\n \"reduceOptions\": {\n \"calcs\": [\n \"lastNotNull\"\n ],\n \"fields\": \"\",\n \"values\": false\n },\n \"textMode\": \"auto\"\n },\n \"pluginVersion\": \"9.4.7\",\n \"targets\": [\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"GbRl4cL4k\"\n },\n \"editorMode\": \"code\",\n \"expr\": \"count(rate(container_last_seen{image=~\\\".+\\\"}[$interval]))\",\n \"format\": \"time_series\",\n \"intervalFactor\": 2,\n \"range\": true,\n \"refId\": \"B\",\n \"step\": 4\n }\n ],\n \"title\": \"Total Swarm Cluster Containers\",\n \"type\": \"stat\"\n },\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"GbRl4cL4k\"\n },\n \"description\": \"These are swarm service containers\/tasks\",\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"thresholds\"\n },\n \"mappings\": [\n {\n \"options\": {\n \"match\": \"null\",\n \"result\": {\n \"text\": \"N\/A\"\n }\n },\n \"type\": \"special\"\n }\n ],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\",\n \"value\": null\n },\n {\n \"color\": \"red\",\n \"value\": 80\n }\n ]\n },\n \"unit\": \"none\"\n },\n \"overrides\": []\n },\n \"gridPos\": {\n \"h\": 4,\n \"w\": 6,\n \"x\": 14,\n \"y\": 4\n },\n \"id\": 11,\n \"links\": [],\n \"maxDataPoints\": 100,\n \"options\": {\n \"colorMode\": \"value\",\n \"graphMode\": \"none\",\n \"justifyMode\": \"auto\",\n \"orientation\": \"horizontal\",\n \"reduceOptions\": {\n \"calcs\": [\n \"lastNotNull\"\n ],\n \"fields\": \"\",\n \"values\": false\n },\n \"textMode\": \"auto\"\n },\n \"pluginVersion\": \"9.4.7\",\n \"targets\": [\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"GbRl4cL4k\"\n },\n \"editorMode\": \"code\",\n \"expr\": \"count(rate(container_last_seen{image=~\\\".+\\\",container_label_com_docker_swarm_service_id=~\\\".+\\\"}[$__rate_interval]))\",\n \"format\": \"time_series\",\n \"intervalFactor\": 2,\n \"range\": true,\n \"refId\": \"B\",\n \"step\": 4\n }\n ],\n \"title\": \"Total Swarm Tasks\/Services\",\n \"type\": \"stat\"\n },\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"GbRl4cL4k\"\n },\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"palette-classic\"\n },\n \"custom\": {\n \"axisCenteredZero\": false,\n \"axisColorMode\": \"series\",\n \"axisLabel\": \"\",\n \"axisPlacement\": \"auto\",\n \"barAlignment\": 0,\n \"drawStyle\": \"line\",\n \"fillOpacity\": 5,\n \"gradientMode\": \"none\",\n \"hideFrom\": {\n \"legend\": false,\n \"tooltip\": false,\n \"viz\": false\n },\n \"lineInterpolation\": \"linear\",\n \"lineStyle\": {\n \"fill\": \"solid\"\n },\n \"lineWidth\": 1,\n \"pointSize\": 5,\n \"scaleDistribution\": {\n \"type\": \"linear\"\n },\n \"showPoints\": \"never\",\n \"spanNulls\": false,\n \"stacking\": {\n \"group\": \"A\",\n \"mode\": \"none\"\n },\n \"thresholdsStyle\": {\n \"mode\": \"off\"\n }\n },\n \"mappings\": [],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\",\n \"value\": null\n },\n {\n \"color\": \"red\",\n \"value\": 80\n }\n ]\n },\n \"unit\": \"percentunit\"\n },\n \"overrides\": []\n },\n \"gridPos\": {\n \"h\": 9,\n \"w\": 24,\n \"x\": 0,\n \"y\": 8\n },\n \"id\": 19,\n \"options\": {\n \"legend\": {\n \"calcs\": [\n \"lastNotNull\"\n ],\n \"displayMode\": \"table\",\n \"placement\": \"right\",\n \"showLegend\": true\n },\n \"tooltip\": {\n \"mode\": \"multi\",\n \"sort\": \"none\"\n }\n },\n \"targets\": [\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"GbRl4cL4k\"\n },\n \"editorMode\": \"code\",\n \"expr\": \"sum(irate(container_cpu_usage_seconds_total{container_label_com_docker_swarm_service_name=~\\\".+\\\"}[$__rate_interval])) by (container_label_com_docker_swarm_service_name)\",\n \"legendFormat\": \"{{container_label_com_docker_swarm_service_name}}\",\n \"range\": true,\n \"refId\": \"A\"\n }\n ],\n \"title\": \"Swarm Service CPU Usage\",\n \"type\": \"timeseries\"\n },\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"GbRl4cL4k\"\n },\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"palette-classic\"\n },\n \"custom\": {\n \"axisCenteredZero\": false,\n \"axisColorMode\": \"series\",\n \"axisLabel\": \"\",\n \"axisPlacement\": \"auto\",\n \"barAlignment\": 0,\n \"drawStyle\": \"line\",\n \"fillOpacity\": 0,\n \"gradientMode\": \"none\",\n \"hideFrom\": {\n \"legend\": false,\n \"tooltip\": false,\n \"viz\": false\n },\n \"lineInterpolation\": \"smooth\",\n \"lineStyle\": {\n \"fill\": \"solid\"\n },\n \"lineWidth\": 1,\n \"pointSize\": 5,\n \"scaleDistribution\": {\n \"type\": \"linear\"\n },\n \"showPoints\": \"never\",\n \"spanNulls\": false,\n \"stacking\": {\n \"group\": \"A\",\n \"mode\": \"none\"\n },\n \"thresholdsStyle\": {\n \"mode\": \"off\"\n }\n },\n \"mappings\": [],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\",\n \"value\": null\n },\n {\n \"color\": \"red\",\n \"value\": 80\n }\n ]\n },\n \"unit\": \"bytes\"\n },\n \"overrides\": []\n },\n \"gridPos\": {\n \"h\": 9,\n \"w\": 24,\n \"x\": 0,\n \"y\": 17\n },\n \"id\": 20,\n \"options\": {\n \"legend\": {\n \"calcs\": [\n \"lastNotNull\"\n ],\n \"displayMode\": \"table\",\n \"placement\": \"right\",\n \"showLegend\": true\n },\n \"tooltip\": {\n \"mode\": \"multi\",\n \"sort\": \"none\"\n }\n },\n \"targets\": [\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"GbRl4cL4k\"\n },\n \"editorMode\": \"code\",\n \"expr\": \"sum(container_memory_rss{container_label_com_docker_swarm_service_name=~\\\".+\\\"}) by (container_label_com_docker_swarm_service_name)\",\n \"legendFormat\": \"{{container_label_com_docker_swarm_service_name}}\",\n \"range\": true,\n \"refId\": \"A\"\n }\n ],\n \"title\": \"Swarm Service Memory Usage\",\n \"type\": \"timeseries\"\n },\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"GbRl4cL4k\"\n },\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"palette-classic\"\n },\n \"custom\": {\n \"axisCenteredZero\": false,\n \"axisColorMode\": \"series\",\n \"axisLabel\": \"\",\n \"axisPlacement\": \"auto\",\n \"barAlignment\": 0,\n \"drawStyle\": \"line\",\n \"fillOpacity\": 10,\n \"gradientMode\": \"none\",\n \"hideFrom\": {\n \"legend\": false,\n \"tooltip\": false,\n \"viz\": false\n },\n \"lineInterpolation\": \"linear\",\n \"lineWidth\": 2,\n \"pointSize\": 5,\n \"scaleDistribution\": {\n \"type\": \"linear\"\n },\n \"showPoints\": \"never\",\n \"spanNulls\": false,\n \"stacking\": {\n \"group\": \"A\",\n \"mode\": \"none\"\n },\n \"thresholdsStyle\": {\n \"mode\": \"off\"\n }\n },\n \"mappings\": [],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\",\n \"value\": null\n },\n {\n \"color\": \"red\",\n \"value\": 80\n }\n ]\n },\n \"unit\": \"Bps\"\n },\n \"overrides\": []\n },\n \"gridPos\": {\n \"h\": 7,\n \"w\": 24,\n \"x\": 0,\n \"y\": 26\n },\n \"id\": 21,\n \"links\": [],\n \"options\": {\n \"legend\": {\n \"calcs\": [\n \"lastNotNull\"\n ],\n \"displayMode\": \"table\",\n \"placement\": \"right\",\n \"showLegend\": true\n },\n \"tooltip\": {\n \"mode\": \"multi\",\n \"sort\": \"desc\"\n }\n },\n \"pluginVersion\": \"9.4.7\",\n \"targets\": [\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"GbRl4cL4k\"\n },\n \"editorMode\": \"code\",\n \"expr\": \"sum(rate(container_network_receive_bytes_total{container_label_com_docker_swarm_service_name=~\\\".+\\\"}[$__rate_interval])) by (container_label_com_docker_swarm_service_name)\",\n \"format\": \"time_series\",\n \"intervalFactor\": 2,\n \"legendFormat\": \"{{ container_label_com_docker_swarm_service_name }}\",\n \"range\": true,\n \"refId\": \"A\",\n \"step\": 2\n }\n ],\n \"title\": \"Incoming Network Traffic per Swarm Service\",\n \"type\": \"timeseries\"\n },\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"GbRl4cL4k\"\n },\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"palette-classic\"\n },\n \"custom\": {\n \"axisCenteredZero\": false,\n \"axisColorMode\": \"text\",\n \"axisLabel\": \"\",\n \"axisPlacement\": \"auto\",\n \"barAlignment\": 0,\n \"drawStyle\": \"line\",\n \"fillOpacity\": 10,\n \"gradientMode\": \"none\",\n \"hideFrom\": {\n \"legend\": false,\n \"tooltip\": false,\n \"viz\": false\n },\n \"lineInterpolation\": \"linear\",\n \"lineWidth\": 1,\n \"pointSize\": 5,\n \"scaleDistribution\": {\n \"type\": \"linear\"\n },\n \"showPoints\": \"never\",\n \"spanNulls\": false,\n \"stacking\": {\n \"group\": \"A\",\n \"mode\": \"none\"\n },\n \"thresholdsStyle\": {\n \"mode\": \"off\"\n }\n },\n \"mappings\": [],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\",\n \"value\": null\n },\n {\n \"color\": \"red\",\n \"value\": 80\n }\n ]\n },\n \"unit\": \"Bps\"\n },\n \"overrides\": []\n },\n \"gridPos\": {\n \"h\": 7,\n \"w\": 24,\n \"x\": 0,\n \"y\": 33\n },\n \"id\": 8,\n \"links\": [],\n \"options\": {\n \"legend\": {\n \"calcs\": [\n \"lastNotNull\"\n ],\n \"displayMode\": \"table\",\n \"placement\": \"right\",\n \"showLegend\": true\n },\n \"tooltip\": {\n \"mode\": \"multi\",\n \"sort\": \"none\"\n }\n },\n \"pluginVersion\": \"9.4.7\",\n \"targets\": [\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"GbRl4cL4k\"\n },\n \"editorMode\": \"code\",\n \"expr\": \"sum(rate(container_network_transmit_bytes_total{container_label_com_docker_swarm_service_name=~\\\".+\\\"}[$__rate_interval])) by (container_label_com_docker_swarm_service_name)\",\n \"format\": \"time_series\",\n \"intervalFactor\": 2,\n \"legendFormat\": \"{{ container_label_com_docker_swarm_service_name }}\",\n \"range\": true,\n \"refId\": \"A\",\n \"step\": 2\n }\n ],\n \"title\": \"Outgoing Network Traffic per Swarm Service\",\n \"type\": \"timeseries\"\n }\n ],\n \"refresh\": false,\n \"revision\": 1,\n \"schemaVersion\": 38,\n \"style\": \"dark\",\n \"tags\": [\n \"prometheus\",\n \"cAdvisor\",\n \"node-exporter\",\n \"alertmanager\"\n ],\n \"templating\": {\n \"list\": [\n {\n \"current\": {\n \"isNone\": true,\n \"selected\": false,\n \"text\": \"None\",\n \"value\": \"\"\n },\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"GbRl4cL4k\"\n },\n \"definition\": \"query_result(count(container_last_seen{container_label_com_docker_swarm_service_name=~\\\".+\\\"}) by (instance))\",\n \"hide\": 2,\n \"includeAll\": false,\n \"label\": \"Swarm Node\",\n \"multi\": false,\n \"name\": \"node\",\n \"options\": [],\n \"query\": {\n \"query\": \"query_result(count(container_last_seen{container_label_com_docker_swarm_service_name=~\\\".+\\\"}) by (instance))\",\n \"refId\": \"StandardVariableQuery\"\n },\n \"refresh\": 1,\n \"regex\": \"\/instance=\\\"\\\\w+-(\\\\S+):\\\\d+\\\"\/\",\n \"skipUrlSync\": false,\n \"sort\": 2,\n \"tagValuesQuery\": \"\",\n \"tagsQuery\": \"\",\n \"type\": \"query\",\n \"useTags\": false\n },\n {\n \"auto\": true,\n \"auto_count\": 50,\n \"auto_min\": \"50s\",\n \"current\": {\n \"selected\": false,\n \"text\": \"auto\",\n \"value\": \"$__auto_interval_interval\"\n },\n \"hide\": 0,\n \"label\": \"Interval\",\n \"name\": \"interval\",\n \"options\": [\n {\n \"selected\": true,\n \"text\": \"auto\",\n \"value\": \"$__auto_interval_interval\"\n },\n {\n \"selected\": false,\n \"text\": \"30s\",\n \"value\": \"30s\"\n },\n {\n \"selected\": false,\n \"text\": \"1m\",\n \"value\": \"1m\"\n },\n {\n \"selected\": false,\n \"text\": \"2m\",\n \"value\": \"2m\"\n },\n {\n \"selected\": false,\n \"text\": \"3m\",\n \"value\": \"3m\"\n },\n {\n \"selected\": false,\n \"text\": \"5m\",\n \"value\": \"5m\"\n },\n {\n \"selected\": false,\n \"text\": \"7m\",\n \"value\": \"7m\"\n },\n {\n \"selected\": false,\n \"text\": \"10m\",\n \"value\": \"10m\"\n },\n {\n \"selected\": false,\n \"text\": \"30m\",\n \"value\": \"30m\"\n },\n {\n \"selected\": false,\n \"text\": \"1h\",\n \"value\": \"1h\"\n },\n {\n \"selected\": false,\n \"text\": \"6h\",\n \"value\": \"6h\"\n },\n {\n \"selected\": false,\n \"text\": \"12h\",\n \"value\": \"12h\"\n },\n {\n \"selected\": false,\n \"text\": \"1d\",\n \"value\": \"1d\"\n },\n {\n \"selected\": false,\n \"text\": \"7d\",\n \"value\": \"7d\"\n },\n {\n \"selected\": false,\n \"text\": \"14d\",\n \"value\": \"14d\"\n },\n {\n \"selected\": false,\n \"text\": \"30d\",\n \"value\": \"30d\"\n }\n ],\n \"query\": \"30s,1m,2m,3m,5m,7m,10m,30m,1h,6h,12h,1d,7d,14d,30d\",\n \"queryValue\": \"\",\n \"refresh\": 2,\n \"skipUrlSync\": false,\n \"type\": \"interval\"\n }\n ]\n },\n \"time\": {\n \"from\": \"now-1h\",\n \"to\": \"now\"\n },\n \"timepicker\": {\n \"refresh_intervals\": [\n \"5s\",\n \"10s\",\n \"30s\",\n \"1m\",\n \"5m\",\n \"15m\",\n \"30m\",\n \"1h\",\n \"2h\",\n \"1d\"\n ],\n \"time_options\": [\n \"5m\",\n \"15m\",\n \"1h\",\n \"6h\",\n \"12h\",\n \"24h\",\n \"2d\",\n \"7d\",\n \"30d\"\n ]\n },\n \"timezone\": \"browser\",\n \"title\": \"Swarm Cluster Monitoring\",\n \"uid\": \"364ipPOizz\",\n \"version\": 8,\n \"weekStart\": \"\"\n}\n<\/code><\/pre>\n\n\n\nOther Tutorials<\/h3>\n\n\n\n