Just today, I had to run an Ansible playbook on a large number of hosts. By default, Ansible will run each task on all the hosts affected by the play using 5 forks. It will only move to the next task when all the hosts have completed the running task.
In my case, this was not optimal, since I had to stop an important service on every host in the beginning of the playbook and only start it at the end. That would mean the service would be down on all the hosts while the playbook would be run on a big number of hosts. Which would be a very long time.
Of course, I could just run the playbook on one host with the –limit argument. Which is the obvious choice when you only want the task to run on one host. But then I would have to either sit there and restart the run for every host or write a script to do it for me. Surely, Ansible can do this more intelligently? Below you can see what I found in my research effort.
Run the entire playbook on only one host at a time
There are a bunch of options to control how many hosts the playbook is executed on. The one I decided to use was the serial keyword at the play level.
The serial keyword defines how many hosts the playbook is run on at a time. Once the playbook execution is finished, it will be run again on a new batch of hosts.
You can use the serial keyword with a different batch size than 1. Say you have multiple hosts you want to execute the playbook on. But only two at a time. Then you would set serial to 2.
Here you can see how you would define a playbook where the playbook will run on just one host at a time.
---
- name: Run play on some remote hosts
hosts: all
serial: 1
tasks:
- name: first task
command: echo "This is the first task"
- name: second task
command: echo "This is the second task"
The serial keyword has some other ways to affect play execution, such as defining multiple batch sizes and defining a percentage of hosts to run in the same batch.
You can define an arbitrary percentage to run the play on at a time. Let’s say you want a play to be executed by 30% of the hosts before moving on to the next host. Then you would add serial: “30%” like so:
--- - name: Run play on some remote hosts hosts: all serial: 1
Don’t worry. You don’t have to divide equally. If there are less than 30% of the hosts left. The final pass will run the play on all the remaining hosts.
You can also list multiple batch sizes to run the play on. As an example, if we want to run the task on a single host to see if it is successful before we move on to the following batch. If the task fails we can break the execution and fix the problem.
Then we would create a list of batches where the first batch is set to one.
---
- name: Run play on some remote hosts
hosts: all
serial:
- 1
- 10
Run just one task, one host at a time
How about if you want to do this on a task level?
Then you would use the throttle keyword. This keyword will instruct Ansible to run just one specific task on one or more hosts at a time. Depending on the value you define.
This way, Ansible executes all other tasks as it would normally do.
Let’s try the same playbook as we did before. Only this time, we will skip the serial keyword and add the throttle keyword.
---
- name: Run play on some remote machines
hosts: all
tasks:
- name: Task runs on one host at a time
command: echo "This is the first task"
throttle: 1
- name: Task is run in parallel
command: echo "This is the second task"
If we had five hosts in our inventory and executed this playbook. Ansible would run the first task, one single host at a time. When the last remaining host is done it would run the second one on all hosts in parallel. As is the default behavior.
Run a playbook without forking from the command line
There is also one more way to limit the number of hosts Ansible tries to run at a time. That is the forks setting. This controls how many forks ansible will start to run tasks in parallel. Ansible will still run each task on the same number of hosts, but it will either slow down or speed up the process depending on how you set it.
This could be very beneficial if your ansible control node has a lot of processing power available and you want to get through the playbook execution as fast as possible. If your control node doesn’t have as much processing power or you want to let the playbook run slowly in the background, you can set it to a lower setting.
There are two ways to change this setting. In your ansible.cfg file or as a parameter on the command line.
To change it in your configuration file, all you need to do is add forks=n to ansible.cfg like so:
[defaults] forks = 2
In the above example, Ansible will spin up 2 forks each time you run a playbook. The default behavior is to use 5 forks. So set the forks variable to a number higher than 5 to speed up and lower if you want to slow down.
This can also be set every time you run a playbook with the Ansible playbook command. For this, you would use the –forks or -f parameters.
This example would limit the number of threads to 1:
ansible-playbook --forks 1 myplaybook.yml
And if you want to shorten the command, using -f will do the exact same thing:
ansible-playbook -f 1 myplaybook.yml
Limit a task to run only on one host with run_once
I think we have covered how to control parallelism in Ansible quite well. But what if you want a task to run on just the first host and then skip it on all the other hosts in the play completely?
That’s where the run_once keyword comes into play. When you add run_once: true to a task, the task will only run on the first host in the batch and skip the remaining hosts.
Have a look at this example:
- name: Run on all remote machines but not the first task.
hosts: webservers
tasks:
- name: This will be run on the first host
command: echo "This is the first task"
run_once: true
- name: Second task
command: echo "This is the second one"
In this example, Ansible will only attempt the execution of the first task on one host. I will then run the second task on all the hosts in the current batch.
You can even delegate the task to some specific host. It would be beneficial if you had to add DNS records for all the hosts, change load balancer settings, etc.
Just add the delegate_to keyword to your task, and it will only be run on the delegated host.
- name: Delegate one task
hosts: webservers
tasks:
- name: First task that will only be run on dns.example.com
command: echo "This is the first task"
run_once: true
delegate_to: dns.example.com
- name: Run on all including the original host
command: echo "This is the second task"
Ordering hosts
I have shown you several keywords to control how much is done in parallel. It could also be helpful to be able to order the hosts in some way.
The order keyword
You can set the order of your hosts at the playbook level. The default behavior is to run the plays on hosts in the order they appear in your inventory. Simply add the order keyword with one of these values:
- reverse_inventory: The reverse of the order that the inventory is in
- sorted: Sort alphabetically
- reverse_sorted: Sort alphabetically and reverse the result
- shuffle: Shuffle the order on each run
The strategy keyword
The default strategy that is used is the linear strategy. You can use a different strategy plugin by using the strategy keyword in your playbook.
- linear (default): Run each task on as many hosts as specified by the forks setting and then move on to the next batch of hosts
- free: Same as above but don’t wait for hosts to finish their task before queuing up the next one.
- debug: Same as linear but is controlled by an interactive debug session
- host_pinned: Same as linear but never start a task on a host unless other tasks can be completed without waiting for other hosts.
That’s it for now
If you got it this far you should know a lot more about controlling executions in Ansible. In my personal experience, these are all necessary skills to know when using Ansible in a large environment.
Most tasks won’t require you to give this much thought to the execution order and speed. But when it is needed it is usually important.