How to Design an End-to-End Ansible Automation Lab with Playbooks, Inventories, Roles, Vault, Dynamic Inventory, and Custom Modules
In this tutorial, we construct a whole Ansible lab that runs end-to-end in Google Colab or any Linux atmosphere. We begin by putting in ansible-core, organising a neighborhood workspace, creating an Ansible configuration file, and defining each static and dynamic inventories. We then discover key Ansible ideas, together with group variables, host variables, variable priority, advert hoc instructions, playbooks, loops, conditionals, registered outputs, info, templates, customized filters, customized modules, roles, handlers, tags, dry runs, idempotency, and Ansible Vault. Since each host runs regionally, we apply these ideas safely with no need SSH keys, distant servers, or cloud infrastructure.
import os, sys, subprocess, textwrap, stat
BASE = "/content material/ansible_lab" if os.path.isdir("/content material") else os.path.expanduser("~/ansible_lab")
os.makedirs(BASE, exist_ok=True)
ENV = os.environ.copy()
ENV["ANSIBLE_CONFIG"] = os.path.be part of(BASE, "ansible.cfg")
ENV["ANSIBLE_FORCE_COLOR"] = "1"
ENV["PY_COLORS"] = "0"
def banner(title):
print("n" + "=" * 78 + f"n {title}n" + "=" * 78)
def write(relpath, content material):
"""Write a dedented file underneath BASE, creating dad or mum dirs."""
path = os.path.be part of(BASE, relpath)
os.makedirs(os.path.dirname(path), exist_ok=True)
with open(path, "w") as f:
f.write(textwrap.dedent(content material).lstrip("n"))
return path
def sh(cmd, title=None):
"""Run a shell command from BASE, stream stdout, by no means increase."""
if title:
banner(title)
print(f"$ {cmd}n")
p = subprocess.run(cmd, shell=True, cwd=BASE, env=ENV,
stdout=subprocess.PIPE, stderr=subprocess.STDOUT, textual content=True)
print(p.stdout)
return p.returncode
banner("STEP 1 — Installing ansible-core")
subprocess.run([sys.executable, "-m", "pip", "install", "-q", "ansible-core"], verify=True)
sh("ansible --version")
write("ansible.cfg", """
[defaults]
stock = ./stock.ini
roles_path = ./roles
library = ./library
filter_plugins = ./filter_plugins
vault_password_file = ./vault_pass.txt
host_key_checking = False
retry_files_enabled = False
interpreter_python = auto_silent
callback_result_format = yaml
deprecation_warnings = False
localhost_warning = False
nocows = 1
[privilege_escalation]
grow to be = False
""")
write("stock.ini", """
[webservers]
web1 ansible_connection=native
web2 ansible_connection=native
[dbservers]
db1 ansible_connection=native
[datacenter:children]
webservers
dbservers
""")We begin by making ready the Ansible workspace, setting atmosphere variables, and defining helper features that make the tutorial simpler to run. We set up ansible-core, confirm the set up, and create the primary Ansible configuration file. We additionally outline a static stock with native internet and database host teams in order that we will apply Ansible ideas with out utilizing distant servers.
write("group_vars/all.yml", """
---
app_name: "Colab Demo App"
app_version: "2.0.1"
admin_email: "[email protected]"
packages:
- nginx
- git
- htop
feature_flags:
enable_cache: true
enable_metrics: false
""")
write("host_vars/web1.yml", """
---
server_id: 101
max_connections: 512
""")
write("filter_plugins/custom_filters.py", '''
import re
def to_slug(worth):
return re.sub(r"[^a-z0-9]+", "-", str(worth).decrease()).strip("-")
def human_bytes(worth):
n = float(worth)
for unit in ["B", "KB", "MB", "GB", "TB"]:
if n < 1024:
return f"{n:.1f}{unit}"
n /= 1024
return f"{n:.1f}PB"
class FilterModule(object):
def filters(self):
return {"to_slug": to_slug, "human_bytes": human_bytes}
''')
write("library/system_report.py", '''
#!/usr/bin/python
from ansible.module_utils.primary import AnsibleModule
import platform, os
def most important():
module = AnsibleModule(
argument_spec=dict(
label=dict(kind="str", required=True),
threshold=dict(kind="int", required=False, default=80),
),
supports_check_mode=True,
)
report = {
"label": module.params["label"],
"system": platform.system(),
"launch": platform.launch(),
"python": platform.python_version(),
"cpu_count": os.cpu_count(),
"threshold": module.params["threshold"],
}
module.exit_json(modified=False,
report=report,
message="Report generated for %s" % module.params["label"])
if __name__ == "__main__":
most important()
''')We outline shared group variables and host-specific variables to present how Ansible manages configuration information and applies variable priority. We then create a customized Jinja2 filter plugin that converts textual content into slugs and codecs byte values into readable models. We additionally constructed a customized Python-based Ansible module that generates a easy system report for every host.
write("roles/webserver/defaults/most important.yml", """
---
listen_port: 8080
""")
write("roles/webserver/vars/most important.yml", """
---
doc_root: "/tmp/www"
""")
write("roles/webserver/duties/most important.yml", """
---
- identify: Ensure docroot exists
ansible.builtin.file:
path: "{{ doc_root }}"
state: listing
mode: "0755"
- identify: Deploy index.html from a Jinja2 template
ansible.builtin.template:
src: index.html.j2
dest: "{{ doc_root }}/index.html"
notify: Restart internet service
- identify: Run handlers instantly (as a substitute of finish of play)
ansible.builtin.meta: flush_handlers
""")
write("roles/webserver/handlers/most important.yml", """
---
- identify: Restart internet service
ansible.builtin.debug:
msg: "(simulated) restarting internet service on port {{ listen_port }}"
""")
write("roles/webserver/templates/index.html.j2", """
<!DOCTYPE html>
<html>
<head><title>{{ app_name }}</title></head>
<physique>
<h1>{{ app_name }} v{{ app_version }}</h1>
<p>Served on port {{ listen_port }} from {{ doc_root }}</p>
<p>Host: {{ inventory_hostname }}</p>
</physique>
</html>
""")
write("templates/report.txt.j2", """
Deployment Report
=================
App: {{ app_name }} ({{ app_version }})
Host: {{ inventory_hostname }}
Generated: { default('n/a') }
Slug: { to_slug }
Packages:
{% for p in packages %}
- {{ p }}
{% endfor %}
Cache enabled: {{ feature_flags.enable_cache }}
Metrics enabled: {{ feature_flags.enable_metrics }}
""")
dyn = write("dynamic_inventory.py", '''
#!/usr/bin/env python3
import json, sys
INV = {
"webservers": {"hosts": ["web1", "web2"], "vars": {"function": "frontend"}},
"dbservers": {"hosts": ["db1"], "vars": {"function": "backend"}},
"_meta": {
"hostvars": {
"web1": {"ansible_connection": "native", "tier": "gold"},
"web2": {"ansible_connection": "native", "tier": "silver"},
"db1": {"ansible_connection": "native", "tier": "gold"},
}
},
}
if "--host" in sys.argv:
print(json.dumps({}))
else:
print(json.dumps(INV, indent=2))
''')
os.chmod(dyn, os.stat(dyn).st_mode | stat.S_IEXEC | stat.S_IXGRP | stat.S_IXOTH)We create a whole internet server function with defaults, variables, duties, handlers, and templates to exhibit how to construct reusable Ansible automation. We use Jinja2 templates to generate an HTML web page and a deployment report from Ansible variables. We additionally add a dynamic stock script that returns host and group info in JSON format.
write("playbook.yml", """
---
- identify: Advanced ideas demo
hosts: webservers
gather_facts: true
vars:
deploy_user: colab
duties:
- identify: Merged variables (group_vars + host_vars priority)
ansible.builtin.debug:
msg: "App={{ app_name }} v{{ app_version }} | server_id={ default('n/a') }"
- identify: CUSTOM filter -> to_slug
ansible.builtin.debug:
msg: "slug => { to_slug }"
- identify: CUSTOM filter -> human_bytes
ansible.builtin.debug:
msg: "measurement => { human_bytes }"
- identify: LOOP with an index variable
ansible.builtin.debug:
msg: "bundle #{{ idx + 1 }} = {{ merchandise }}"
loop: "{{ packages }}"
loop_control:
index_var: idx
- identify: CONDITIONAL (when) — provided that caching is enabled
ansible.builtin.debug:
msg: "cache is ON"
when: feature_flags.enable_cache | bool
- identify: Run a command and REGISTER its output
ansible.builtin.command: date +%Y-%m-%d
register: date_out
changed_when: false
- identify: SET a derived reality from the registered worth
ansible.builtin.set_fact:
deploy_stamp: "{ to_slug }-{{ date_out.stdout }}"
- identify: Show the derived reality
ansible.builtin.debug:
var: deploy_stamp
- identify: Run our CUSTOM MODULE (system_report)
system_report:
label: "{{ inventory_hostname }}"
threshold: 90
register: sysrep
- identify: Show customized module output
ansible.builtin.debug:
var: sysrep.report
- identify: BLOCK with rescue/all the time (error dealing with)
block:
- identify: This fails on objective
ansible.builtin.command: /bin/false
- identify: Never reached
ansible.builtin.debug:
msg: "unreachable"
rescue:
- identify: Recover gracefully
ansible.builtin.debug:
msg: "caught the failure — recovering"
all the time:
- identify: Always run cleanup
ansible.builtin.debug:
msg: "cleanup runs it doesn't matter what"
- identify: Use a VAULT-encrypted secret (decrypted at runtime)
ansible.builtin.debug:
msg: "token prefix={{ api_secret_token[:3] }}*** len={ size }"
- identify: TEMPLATE a report file (tagged 'report')
ansible.builtin.template:
src: report.txt.j2
dest: "/tmp/{{ inventory_hostname }}_report.txt"
tags: [report]
- identify: Role demo
hosts: web1
gather_facts: false
roles:
- function: webserver
""")We write the primary playbook that brings collectively variables, customized filters, loops, conditionals, registered outputs, derived info, and a customized module. We deliberately embody a failing command to exhibit error dealing with by means of block, rescue, and all the time. We additionally use a Vault-encrypted secret and apply the net server function to exhibit how role-based automation works in an actual workflow.
banner("STEP 2 — Ansible Vault: encrypting an inline secret")
write("vault_pass.txt", "colab-demo-vault-passn")
enc = subprocess.run(
"ansible-vault encrypt_string 'S3cr3t-Token-42' --name 'api_secret_token'",
shell=True, cwd=BASE, env=ENV, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, textual content=True
).stdout
with open(os.path.be part of(BASE, "group_vars/webservers.yml"), "w") as f:
f.write("---n")
f.write(enc)
print("group_vars/webservers.yml now incorporates:n")
print(open(os.path.be part of(BASE, "group_vars/webservers.yml")).learn())
sh("ansible-inventory -i stock.ini --graph", "STEP 3 — Static stock graph")
sh("ansible-inventory -i dynamic_inventory.py --list", "STEP 4 — Dynamic stock (JSON)")
sh("ansible all -m ping", "STEP 5 — Ad-hoc: ping all hosts")
sh("ansible web1 -m setup -a 'filter=ansible_python_version'",
"STEP 6 — Ad-hoc: collect a single reality")We create a Vault password file and encrypt an inline secret that Ansible decrypts robotically when the playbook runs. We examine each the static and dynamic inventories to perceive how Ansible reads hosts, teams, and metadata. We then run ad-hoc instructions to ping all hosts and collect a particular Python model reality from web1.
sh("ansible-playbook playbook.yml --check --diff", "STEP 7 — Dry run (--check)")
sh("ansible-playbook playbook.yml", "STEP 8 — Real run")
sh("ansible-playbook playbook.yml", "STEP 9 — Re-run (idempotency: anticipate 0 modified)")
sh("ansible-playbook playbook.yml --tags report", "STEP 10 — Run solely duties tagged 'report'")
sh("echo '--- /tmp/www/index.html ---'; cat /tmp/www/index.html; "
"echo; echo '--- /tmp/web1_report.txt ---'; cat /tmp/web1_report.txt",
"STEP 11 — Generated recordsdata")
sh('ansible webservers --limit web1 -m debug -a "var=api_secret_token"',
"STEP 12a — Inline vault secret decrypted at runtime")
write("secrets and techniques.yml", """
---
db_password: full-file-secret-99
api_key: abc123
""")
sh("ansible-vault encrypt secrets and techniques.yml", "STEP 12b — Encrypt a WHOLE file")
sh("head -c 60 secrets and techniques.yml; echo ' ...'")
sh("ansible-vault view secrets and techniques.yml", "STEP 12c — View the fully-encrypted file")
banner("DONE — you now have a working superior Ansible lab in Colab")
print(f"Workspace: {BASE}nEdit any file there and re-run a step with the sh() helper.")We run the playbook in verify mode, execute it for actual, and rerun it to affirm that the workflow is idempotent. We use tags to run solely the report-related job and then examine the generated HTML and textual content report recordsdata. We additionally exhibit full-file Vault encryption, safely view the encrypted file, and full the superior Ansible lab.
In conclusion, we now have a working Ansible lab that demonstrates how automation workflows are structured and executed in actual tasks. We created reusable roles, generated recordsdata from Jinja2 templates, ran customized Python-based Ansible modules, dealt with errors with rescue and all the time blocks, encrypted secrets and techniques with Ansible Vault, and validated our setup by means of dry runs and repeated idempotent executions. We additionally realized how static and dynamic inventories work, how tags assist us run chosen duties, and how Ansible organizes infrastructure automation in a clear, repeatable, and production-friendly approach.
Check out the Full Codes with Notebook here. Also, be at liberty to observe us on Twitter and don’t overlook to be part of our 150k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
Need to associate with us for selling your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar and so forth.? Connect with us
The put up How to Design an End-to-End Ansible Automation Lab with Playbooks, Inventories, Roles, Vault, Dynamic Inventory, and Custom Modules appeared first on MarkTechPost.
