Introduction
This blog walks through building an Azure infrastructure deployment pipeline using Terraform and Azure DevOps. It covers project setup, repository import, self-hosted agent installation, Terraform configuration, service principal creation, remote state backend setup, RBAC, pipeline execution (apply/destroy), and cleanup. Screenshots are useful for each step — placeholders are provided throughout so you can add captures of your environment.
This blog walks through building an Azure infrastructure deployment pipeline using Terraform and Azure DevOps. It covers project setup, repository import, self-hosted agent installation, Terraform configuration, service principal creation, remote state backend setup, RBAC, pipeline execution (apply/destroy), and cleanup. Screenshots are useful for each step — placeholders are provided throughout so you can add captures of your environment.
Project setup and repository import
- Sign in to: https://dev.azure.com/vallabhdarole/ and create a new Azure DevOps project
- The project home shows Overview, Summary, Dashboard, Wiki, Boards, Repos, Pipelines, Test Plans, Artifacts, and Project Settings.
- Import the Git repo: https://github.com/vdarole/Terraform_project into Repos → Files → Import repository. Verify the branch (main) and files post-import.
Pipeline creation
- Create a new Pipeline and choose YAML to keep the pipeline definition in the repo. Choose Azure Repos Git as the source and review the azure-pipelines.yml content before saving.
Agent pool and self-hosted agent
- Navigate to Project settings → Agent pools → Default (or create a dedicated pool). Choose New agent to get platform-specific install instructions.
Suggestion: Use a dedicated agent pool for terraform workloads to control concurrency and isolation.
- Provision a VM for the agent. Recommended minimum size: Standard_DS1_v2. Create a non-root user account for the agent (example: ansible).
Screenshot placeholder
Suggestion: Choose a VM size with enough CPU, memory, and disk for terraform operations and provider plugins.
Agent install and config (Linux example)
- On the VM, prepare the agent folder, download the agent, extract, and run configuration. Use a Personal Access Token (PAT) to register the agent.
Screenshot placeholder
Suggestion: Run these commands as the agent user and confirm network connectivity to dev.azure.com.
```bash
[ansible@Linux01 myagent]$ mkdir myagent && cd myagent
[ansible@Linux01 myagent]$ wget https://download.agent.dev.azure.com/agent/4.261.0/vsts-agent-linux-x64-4.261.0.tar.gz
[ansible@Linux01 myagent]$ tar -zxvf vsts-agent-linux-x64-4.261.0.tar.gz
[ansible@Linux01 myagent]$ ./config.sh
```
Suggestion: Use a descriptive agent name and a dedicated working folder (for example _work).
- Follow the prompts: accept the agreement, enter server URL (https://dev.azure.com/vallabhdarole/), choose authentication type PAT, and paste the PAT.
Screenshot placeholder
Suggestion: Store the PAT securely; do not commit it to source control.
- Install and start the agent service so it runs and restarts automatically:
Screenshot placeholder
```bash
[ansible@Linux01 myagent]$ sudo ./svc.sh install
[ansible@Linux01 myagent]$ sudo ./svc.sh start
[ansible@Linux01 myagent]$ sudo ./svc.sh status
```
Suggestion: Check agent logs if status is not healthy.
Install Terraform on the agent VM
- Install Terraform using HashiCorp’s repo and confirm the version. Example includes terraform v1.13.2.
Screenshot placeholder
```bash
[ansible@Linux01 myagent]$ sudo yum install -y yum-utils
[ansible@Linux01 myagent]$ sudo yum-config-manager --add-repo https://rpm.releases.hashicorp.com/RHEL/hashicorp.repo
[ansible@Linux01 myagent]$ sudo yum install -y terraform
[ansible@Linux01 myagent]$ terraform --version
Terraform v1.13.2
on linux_amd64
[ansible@Linux01 myagent]$
```
Suggestion: Pin the Terraform version in CI to avoid unintended upgrades.
Azure App Registration and service principal
- In Azure Portal, go to App registrations and create a new app: Name: Terraform-App-Registration, Supported account type: Single tenant. Copy Application (client) ID and Directory (tenant) ID.
Screenshot placeholder
Suggestion: Add a clear description so the intent is visible in the portal.
- Create a client secret: Description Terraform-Secret, Expiry 6 months. Copy the secret value (it’s shown once). Treat it as highly sensitive.
Screenshot placeholder
Suggestion: Store secret values in Azure Key Vault or Azure DevOps secret variables and rotate them before expiry.
Notes about provided secret and IDs
- The example includes a secret value and IDs. Use the client secret value (not the secret ID) when configuring ARM_CLIENT_SECRET in your CI.
Screenshot placeholder
Suggestion: If secrets were exposed, rotate them immediately and remove them from shared documents.
Variable group in Azure DevOps
- In Pipelines → Library, create a variable group (example name Terraform_var or terraform-secrets) and add the following variables. Mark secrets as secret in the UI.
- ARM_CLIENT_ID: 8f94cd26-ddd8-4dbb-ab9c-ed6d4f62badb
Screenshot placeholder
- ARM_CLIENT_SECRET: AHR8Q~5pvoX9F9hR.l2cOBnRmyVTvK8zXCrEObWD (secret)
Screenshot placeholder
- ARM_SUBSCRIPTION_ID: e113ed96-0aeb-4c7d-aa00-8c8966c3e7f8
Screenshot placeholder
- ARM_TENANT_ID: 60e4e629-0e85-4cc5-b3d9-26d28597d8ec
Screenshot placeholder
Suggestion: Use variable group names that match what is referenced in azure-pipelines.yml and enforce pipeline access control.
Terraform backend (remote state) setup
- Create an Azure Storage Account (example name: vallabhterraformstorage) with LRS. Enable hierarchical namespace (ADLS Gen2) if required. Create a container (example name: azcontainer). Collect resource group name, storage account name, and container name.
Screenshot placeholder
Suggestion: Keep the container private and use RBAC to grant only the service principal access.
- Update backend.tf in the repo to reference your backend storage. Example backend block is provided below. After committing, run terraform init so the state is stored remotely.
Screenshot placeholder
Suggestion: Ensure the storage account and container exist before running terraform init.
RBAC assignments for the service principal
- Assign roles so the service principal can manage state and create resources. Recommended scopes and roles:
- Storage Blob Data Contributor on the container or storage account (for state operations).
Screenshot placeholder
- Storage Account Contributor or Storage Blob Data Contributor as needed for the storage account.
Screenshot placeholder
- Contributor role at subscription or resource-group scope if Terraform must create resources at that scope.
Screenshot placeholder
Suggestion: Prefer least privilege — grant permissions at the resource group level rather than subscription when possible. Allow a few minutes for RBAC to propagate.
Run the pipeline (apply)
- Trigger the pipeline from Pipelines → Run pipeline. Select parameters: environment (uat/prod) and action (apply/destroy). Approve any permission prompts (environment access). Monitor logs for terraform init, plan, and apply.
Screenshot placeholder
Suggestion: Add pre-check jobs such as terraform fmt -check, terraform validate, and tflint before plan/apply.
Environments and approvals
- Register environments in Azure DevOps (example name: prod). Add Approvals and checks to require manual approval for production deployments. When running a pipeline targeting prod, reviewers must approve it before the deployment job starts.
Screenshot placeholder
Suggestion: Add multiple approvers or group-based approvals depending on org policy.
Destroy resources using the pipeline
- Use the same pipeline with action destroy to tear down resources for a selected environment. Approvals are required for production destroy operations. Confirm terraform destroy runs successfully and the Azure resources are removed.
Screenshot placeholder
Suggestion: Keep tfstate access available; if state is lost, destroy may fail to detect resources.
Cleanup and cost control
- After validating the deployment, destroy all resources you no longer need. Remove self-hosted agents, revoke or rotate PATs and client secrets, and delete storage used for tfstate if appropriate.
Screenshot placeholder
Suggestion: Maintain a cleanup checklist and billing checks to avoid unexpected charges.
Appendix — repository code (from Terraform_project)
- azure-pipelines.yml (pipeline definition)
```yaml
trigger:
- main
parameters:
- name: environment
displayName: Select environment to create infra resources
type: string
values:
- uat
- prod
- name: action
displayName: Select Action - Apply or destroy resources
type: string
values:
- apply
- destroy
pool: default
variables:
- group: terraform-secrets
stages:
- stage: TerraformApply
displayName: Terraform plan deployment resource
condition: eq('${{ parameters.action }}', 'apply')
jobs:
- job: init_plan
displayName: Terraform plan deployment
steps:
- checkout: self
- script: |
terraform init
terraform workspace select ${{ parameters.environment }} || terraform workspace new ${{ parameters.environment }}
terraform plan -var-file=./envs/${{ parameters.environment }}/${{ parameters.environment }}-vars.tfvars
displayName: 'Terraform plan deployment'
env:
ARM_CLIENT_ID: $(ARM_CLIENT_ID)
ARM_CLIENT_SECRET: $(ARM_CLIENT_SECRET)
ARM_SUBSCRIPTION_ID: $(ARM_SUBSCRIPTION_ID)
ARM_TENANT_ID: $(ARM_TENANT_ID)
- deployment: apply
displayName: Terraform apply deployment
dependsOn: init_plan
condition: succeeded()
environment: ${{ parameters.environment }}
strategy:
runOnce:
deploy:
steps:
- checkout: self
- script: |
terraform init
terraform workspace select ${{ parameters.environment }}
terraform apply -var-file=./envs/${{ parameters.environment }}/${{ parameters.environment }}-vars.tfvars -auto-approve
displayName: 'Terraform apply deployment'
env:
ARM_CLIENT_ID: $(ARM_CLIENT_ID)
ARM_CLIENT_SECRET: $(ARM_CLIENT_SECRET)
ARM_SUBSCRIPTION_ID: $(ARM_SUBSCRIPTION_ID)
ARM_TENANT_ID: $(ARM_TENANT_ID)
- stage: TerraformDestroy
displayName: Terraform destroy deployment
condition: eq('${{ parameters.action }}', 'destroy')
jobs:
- deployment: destroy
displayName: Terraform destroy deployment
environment: ${{ parameters.environment }}
strategy:
runOnce:
deploy:
steps:
- checkout: self
- script: |
terraform init
terraform workspace select ${{ parameters.environment }}
terraform destroy -var-file=./envs/${{ parameters.environment }}/${{ parameters.environment }}-vars.tfvars -auto-approve
displayName: 'Terraform destroy deployment'
env:
ARM_CLIENT_ID: $(ARM_CLIENT_ID)
ARM_CLIENT_SECRET: $(ARM_CLIENT_SECRET)
ARM_SUBSCRIPTION_ID: $(ARM_SUBSCRIPTION_ID)
ARM_TENANT_ID: $(ARM_TENANT_ID)
```
- backend.tf
```hcl
/* make sure you bootstap the storage account and container before running the terraform init while
you mention the storage account details and container detials in backend block
*/
terraform {
backend "azurerm" {
resource_group_name = "rg-terraform-backend"
storage_account_name = "roytfstate123"
container_name = "tfstateenvs"
key = "terraform.tfstate"
}
}
```
- main.tf
```hcl
module "resource_group" {
source = "./modules/resource-group"
name = var.name
location = var.location
}
```
- providers.tf
```hcl
provider "azurerm" {
features {}
# subscription_id = var.subscription_id
# client_id = var.client_id ## Registered App's ID
# client_secret = var.client_secret ## client secret value
# tenant_id = var.tenant_id ## Registered App's tenant ID
}
```
- variables.tf
```hcl
/*
service principle credentials shouldnt be defined here
with sensitive as true if we are defining these sensitive variables
in azure devops pipeline or any other cicd pipelines as environment
variables else it wont work.
that is the reason why i have commented out here..
and reference variables in providers.tf file
*/
variable "name" {
type = string
description = "resource group name"
}
variable "location" {
type = string
description = "resource group location"
}
```
- modules/resource-group/main.tf
```hcl
resource "azurerm_resource_group" "rg-details" {
name = var.name
location = var.location
}
```
- modules/resource-group/outputs.tf
```hcl
output "resource_group_name" {
value = azurerm_resource_group.rg-details.name
}
```
- envs/prod/prod-vars.tfvars
```hcl
name = "rg-prod"
location = "East US"
```
- envs/uat/uat-vars.tfvars
```hcl
name = "rg-uat"
location = "East US"
```
Closing notes
- Keep sensitive values out of screenshots and repository commits. Rotate PATs and client secrets if they were shared. Review RBAC scope and pipeline access. Add CI checks for format and linting. Automate cleanup to avoid unwanted charges.
If you’d like, I can:
- Convert this blog into a formatted HTML page ready for your blog platform.
- Produce screenshots-ready templates with caption placeholders.
- Redact or flag any exposed secrets in the content.
Which of these would you prefer next?
No comments:
Post a Comment