Ansible playbook deployment in production using AWX

 Question : we have two branch in azure deveop one is dev and other is master and want to deploy playbook using awx and it is already tested in preprod through test awx and want to deploy on prod awx , please explain steps

To deploy your Ansible playbooks from Azure DevOps (dev and master) to Production AWX, you need to connect your Azure DevOps pipeline to the Prod AWX server using a webhook or API call.

Prerequisites

Prod AWX API Access: A service account or token with permissions to launch job templates in Prod AWX.
AWX Job Template: A job template configured in Prod AWX that points to your playbook and inventory.
Service Connection: Credentials stored in Azure DevOps to authenticate with Prod AWX securely.

---

Step 1: Configure Prod AWX Job Template

Create or verify a Job Template in Prod AWX that points to the correct production inventory and playbook.
Note the Job Template ID from the AWX URL (e.g., /#/templates/job_template/15 means ID is 15).
Enable Webhook on the job template if you want to trigger it via webhook, or use the AWX CLI (ansible-tower-cli) or curl via a REST API call in your pipeline.

Step 2: Store AWX Credentials in Azure DevOps

Go to your Azure DevOps project settings.
Select Service connections and click New service connection.
Choose Generic or Incoming Webhook (or use secret variables for AWX URL and API Token/User credentials).
Save your Prod AWX URL, username, and password (or personal access token) as Secret variables in your Azure DevOps pipeline or Library variable group.

Step 3: Create the Azure DevOps Pipeline

Create a YAML pipeline (e.g., azure-pipelines.yml) in your master branch (or configured to trigger on merges/pull requests to master).

Use an Azure DevOps script task to trigger the Prod AWX job using curl and the AWX REST API.

yaml

trigger:
  branches:
    include:
      - master

pool:
  vmImage: 'ubuntu-latest'

steps:
- task: Bash@TF
  displayName: 'Trigger Prod AWX Job'
  script: |
    # Define variables
    AWX_URL="https://yourcompany.com"
    JOB_TEMPLATE_ID="15"
    
    # Send API request to launch the job template
    response=$(curl -s -k -X POST \
      -H "Content-Type: application/json" \
      -u "$(AWX_USER):$(AWX_PASSWORD)" \
      --data '{"extra_vars": {"branch": "master"}}' \
      "$AWX_URL/api/v2/job_templates/$JOB_TEMPLATE_ID/launch/")
    
    echo "Response: $response"
    
    # Extract Job ID and check status (optional logic)
    job_id=$(echo $response | grep -o '"id":[0-9]*' | head -n1 | cut -d':' -f2)
    echo "Started AWX Job ID: $job_id"

Step 4: Add Approval Gates for Production

Go to your Azure DevOps Environments.
Create a production environment named prod-awx.
Add Approvals and checks to this environment, adding the specific team members or leads who must approve the deployment before curl executes against Prod AWX.
Reference the environment in your pipeline job:

yaml

jobs:
- deployment: DeployToProd
  environment: 'prod-awx'
  strategy:
    runOnce:
      deploy:
        steps:
        - script: echo "Running deployment via AWX..."

Step-by-Step Deployment Guide

[Azure DevOps (Dev/Master)] ➔ [Webhook / Sync] ➔ [Prod AWX] ➔ [Target Infrastructure]

Step 1: Freeze and Merge the Code

Create a Pull Request (PR) from dev to master in Azure DevOps.
Complete the peer review.
Merge the PR to make master your production-ready baseline.

Step 2: Configure Prod AWX Credentials

Log into Prod AWX.
Navigate to Credentials -> Add.
Create a Source Control credential.
Use a Personal Access Token (PAT) from Azure DevOps with Code (Read) scopes.

Step 3: Create the Project in Prod AWX

Navigate to Projects -> Add.
Name the project (e.g., Prod-Infrastructure-Playbooks).
Set Source Control Type to Git.
Paste your Azure DevOps master branch clone URL.
Select the Azure DevOps credential created in Step 2.
Set the SCM Branch explicitly to master.
Enable Update Revision on Launch to pull fresh code before every run.

Step 4: Set Up Inventory and Machine Credentials

Create your Production Inventory in AWX.
Add production target hosts or configure dynamic cloud plugins (Azure, AWS).
Add Machine Credentials (SSH keys or WinRM passwords) to access production servers.

Step 5: Create the Job Template

Navigate to Templates -> Add -> Job Template.
Link the Production Inventory, the Project (Step 3), and the Machine Credentials.
Select the specific playbook you want to run from the dropdown.
Save and click the Rocket Icon to launch.

---

Top 3 Interview Questions & Answers

Q1: How do you ensure that changes tested in Pre-Prod AWX don't break Prod AWX?

Answer: We enforce a strict Git branching strategy and environment separation. Pre-Prod AWX points exclusively to the dev branch, while Prod AWX points to the master branch. No code reaches production without passing code reviews and automated tests in Pre-Prod. We also use AWX Survey Prompts to double-check variables before execution.

Q2: How do you automate the sync between Azure DevOps and AWX so you don't have to click "Update Project" manually?

Answer: We configure an AWX Webhook. In Azure DevOps, we set up a Service Hook that triggers on code pushes to the master branch. This sends a payload to the Prod AWX webhook URL, automatically refreshing the project baseline.

Q3: How do you manage sensitive production data like passwords and API keys?

Answer: We never hardcode secrets in Git. We encrypt sensitive variables using Ansible Vault inside the repository or inject them securely at runtime using AWX Credential Types integrated with enterprise vaults (like CyberArk or Azure Key Vault).

---

Production Challenges & How to Troubleshoot Them

1. Authentication Failures with Azure DevOps

Challenge: Prod AWX fails to sync the project, throwing Authentication failed or Repository not found errors.
Troubleshooting: Check the Azure DevOps PAT expiration date. Ensure the PAT user has explicitly granted read permissions to the specific repository containing the playbooks.

2. Environment Drift (Pre-Prod vs. Prod)

Challenge: The playbook succeeds perfectly in Pre-Prod but fails in Prod due to subtle environment differences.
Troubleshooting: Compare host facts. Ensure python dependencies (pip packages) and Ansible collections match perfectly across both AWX cluster instances. Use execution environments (containers) to lock down dependencies.

3. Network and Firewall Blocks

Challenge: Prod AWX cannot reach the target production servers, resulting in UNREACHABLE! errors.
Troubleshooting: Validate network security groups (NSGs) and firewalls. Prod AWX often sits in a different subnet or VPC than Pre-Prod AWX. Ensure port 22 (SSH) or 5986 (WinRM) is open from the Prod AWX execution nodes to the target hosts.