Self-hosted Azure DevOps Agent Deployment with Terraform

Introduction

Setting up self-hosted Azure DevOps agents manually is slow, error-prone, and not scalable. In this guide, we will show you how we've setup the Self-hosted Azure DevOps Agent Deployment with Terraform for FFA TITAN 2.0. We will go over it step-by-step, using Key Vault for credentials, a secure VM extension, and full CI/CD pipeline authorization.

At Food For Analytics, we use Terraform to fully automate the rollout of self-hosted build agents inside our Azure environment. From creating agent pools to spinning up Linux VMs and connecting them to Azure DevOps. Everything happens in a single pipeline.

What is a Self-Hosted Azure DevOps Agent?

A self-hosted Azure DevOps agent is a build or deployment machine that you manage yourself and connect to Azure DevOps to run your pipelines from within a secured private azure network.

We deploy it in each FFA Titan environment.

Why a Self-Hosted Azure DevOps Agent?

While Microsoft-hosted agents are convenient for simple workloads, self-hosted Azure DevOps agents give you full control over performance, security, and customization. Here’s why teams choose to self-host their build agents:.

When Should You Use a PAT vs. Microsoft Entra ID Token?

As each FFA Titan instance runs it its own private vnet that cannot be access via the public internet we need a Self-hosted Azure DevOps Agent Deployment with Terraform within the private vnet. Otherwise we cannot deploy an FFA Titan instance in an automated and repeatable manner.

The Challenge: auto provision Self-Hosted Azure DevOps Agent.

Azure DevOps doesn't support fully automated self-hosted agent registration via Terraform. While agent pools and queues can be provisioned, the agent must be manually configured on the VM using a PAT from a service account. Without automation, this adds manual steps, increases security risks, and hinders scalability.

The Solution: Terraform and Shellscript

To overcome this challenge, we combine Terraform for infrastructure provisioning with a shellscript to securely install the self-hosted Azure DevOps agent on the VM and register it with the respective Azure Devops project . This ensures a fully automated and repeatable setup whenever we need it.

Terraforming Self-hosted Azure DevOps Agent

We want a self-hosted Azure DevOps agent that can automatically register itself to a private agent pool. This must happen securely, without manual token handling. The agent runs inside our FFA Titan 2.0 private network and connects to Azure DevOps using a Personal Access Token (PAT) from a scoped service account.

Before we dive in, let’s clarify which FFA Titan 2.0 platform components are involved:

Infrastructure-as-Code (IaC) Setup

To achieve secure and private connectivity between [1] and [2] we need to achieve the following in our Infrastructure-as-Code:

Step 1: Create Azure DevOps Agent Pool and Queue

This step creates the Azure Devops agent Pool and Queue for the respective Azure DevOps project for the FFA Titan CI/CD-pipeline

# Create AZDO Agent Pool
resource "azuredevops_agent_pool" "ffa_titan_azdo_agent_pool" {
  name           = var.azuredevopsagentpoolname
  auto_provision = false
  auto_update    = false

}

# Add AZDO agent queue to AZDO Agent Pool
resource "azuredevops_agent_queue" "ffa_titan_azdo_agent_queue" {
  project_id    = var.AZUREDEVOPSPROJECTUUID
  agent_pool_id = azuredevops_agent_pool.ffa_titan_azdo_agent_pool.id
  depends_on = [
    azuredevops_agent_pool.ffa_titan_azdo_agent_pool
  ]
}

# Grant AZDO Pipeline permission to AZDO agent queue
resource "azuredevops_pipeline_authorization" "ffa_titan_azdo_agent_queue_authorization" {
  project_id  = var.AZUREDEVOPSPROJECTUUID
  resource_id = azuredevops_agent_queue.ffa_titan_azdo_agent_queue.id
  type        = "queue"
  pipeline_id = var.AZUREDEVOPSPIPEID
  depends_on = [
    azuredevops_agent_queue.ffa_titan_azdo_agent_queue
  ]
}

please note: you need the terraform Azure DevOps provider and a Azure DevOps Personal Access Token of a service account or user account with full control permissions in your Azure DevOps organization.

Step 2: Create Azure Linux VM inside private subnet

This step creates the Self-Host Azure DevOps Agent VM in the private vnet of our FFA Titan instance.

# KEYVAULT COMPONENTS
# buildagent administrator password generator
resource "random_password" "ffa_titan_password" {
  length           = 16
  min_lower        = 4
  min_numeric      = 4
  min_upper        = 4
  min_special      = 4
  override_special = "!#$%_"
}

# buildagent administrator username
resource "azurerm_key_vault_secret" "ffa_titan_keyvault_secret_adobuildagent_admin_username" {
  name         = "ffatitan-adobuildagent-admin-username-${var.environment}"
  value        = "adminuser"
  key_vault_id = data.azurerm_key_vault.ds_ffa_titan_key_vault_acr.id

  lifecycle {
    ignore_changes = [value, version]
  }

  provider = azurerm.ntw

}

# buildagent administrator password
resource "azurerm_key_vault_secret" "ffa_titan_keyvault_secret_adobuildagent_admin_password" {
  name         = "ffatitan-adobuildagent-admin-password-${var.environment}"
  value        = random_password.ffa_titan_password.result
  key_vault_id = data.azurerm_key_vault.ds_ffa_titan_key_vault_acr.id

  lifecycle {
    ignore_changes = [value, version]
  }

  provider = azurerm.ntw

}


# VM COMPONENTS
resource "azurerm_network_interface" "ffa_titan_vnet_adobuildagent_nic" {
  name                = "adobuildagentvm--nic"
  location            = var.location
  resource_group_name = var.resourcegroup_name

  ip_configuration {
    name                          = "ip"
    subnet_id                     = azurerm_subnet.ffa_titan_ado_buildagent_subnet.id
    private_ip_address_allocation = "Dynamic"
  }
  depends_on = [
    azurerm_subnet.ffa_titan_ado_buildagent_subnet
  ]

}

resource "azurerm_virtual_machine" "ffa_titan_vm_linux_adobuildagent" {
  name                = "adobuildagentvm"
  location            = var.location
  resource_group_name = var.resourcegroup_name
  network_interface_ids = [
    azurerm_network_interface.ffa_titan_vnet_adobuildagent_nic.id
  ]
  vm_size                          = "Standard_A1_v2"
  delete_data_disks_on_termination = true
  delete_os_disk_on_termination    = true

  storage_os_disk {
    name              = "adobuildagentvm-osdisk"
    caching           = "ReadWrite"
    create_option     = "FromImage"
    managed_disk_type = "Standard_LRS"
    disk_size_gb = 128
  }

  storage_image_reference {
    publisher = "Canonical"
    offer     = "0001-com-ubuntu-server-jammy"
    sku       = "22_04-lts"
    version   = "latest"
  }

  os_profile {
    computer_name  = "adobuildagentvm"
    admin_username = azurerm_key_vault_secret.ffa_titan_keyvault_secret_adobuildagent_admin_username.value
    admin_password = random_password.ffa_titan_password.result
  }

  os_profile_linux_config {
    disable_password_authentication = false
  }
}

Step 3: Inject and run a shell script on the Azure Linux VM

This step injects and executes the shell script that installs all dependencies, required tooling, and the self-hosted Azure DevOps agent and registers it with the right Azure Devops Agent Pool in your environment..

resource "azurerm_virtual_machine_extension" "ffa_titan_vm_linux_adobuildagent_extension" {
  name                 = "devops-extension"
  virtual_machine_id   = azurerm_virtual_machine.ffa_titan_vm_linux_adobuildagent.id
  publisher            = "Microsoft.Azure.Extensions"
  type                 = "CustomScript"
  type_handler_version = "2.1"
  settings = <<SETTINGS
    {
        "script": "${base64encode(templatefile("${path.cwd}/${var.stack}/helper/03_adobuildagent_setup.sh", {
  azuredevopsuri           = "${var.AZUREDEVOPSURI}",
  azuredevopssvcpat        = "${var.AZUREDEVOPSSVCPAT}"
  azuredevopsagentpoolname = "${var.azuredevopsagentpoolname}",
  azuredevopsagentname     = "${var.azuredevopsagentname}",
  azuredevopsagentversion  = "${var.azuredevopsagentversion}"
}))}"
    }
SETTINGS
depends_on = [
  azurerm_virtual_machine.ffa_titan_vm_linux_adobuildagent,
  azurerm_virtual_network_dns_servers.ffa_titan_vnet_private_dns_resolver_as_dnsserver
]
}

Shellscript: Self-Hosted Azure DevOps Agent setup

This shell script is designed to automate the setup of a self-hosted Azure DevOps agent on a Linux VM inside a private Azure VNET. It installs the agent, registers it with Azure DevOps, and installs supporting tools like Azure CLI and Dockerall in a single execution.

#!/bin/sh

## INSTALL DEVOPS AGENT on VM

# create directory for devops agent binaries
sudo mkdir /myagent 
cd /myagent

# download & unzip & set permissions agent binaries and directory

#sudo wget https://vstsagentpackage.azureedge.net/agent/${azuredevopsagentversion}/vsts-agent-linux-x64-${azuredevopsagentversion}.tar.gz #need to make version of url dynamic through param
sudo wget https://download.agent.dev.azure.com/agent/${azuredevopsagentversion}/vsts-agent-linux-x64-${azuredevopsagentversion}.tar.gz #need to make version of url dynamic through param
sudo tar zxvf ./vsts-agent-linux-x64-${azuredevopsagentversion}.tar.gz
sudo chmod -R 777 /myagent

# install dependencies
sudo ./bin/installdependencies.sh

# config & connect vm to azure devops food for analytics
runuser -l adminuser -c "/myagent/config.sh --unattended  --url "${azuredevopsuri}" --auth pat --token "${azuredevopssvcpat}" --pool "${azuredevopsagentpoolname}" --agent "${azuredevopsagentname}" --acceptTeeEula --work ./_work --runAsService"


# install devops agent service
sudo ./svc.sh install adminuser

# start devops agent service
sudo ./svc.sh start

## INSTALL AZURE CLI
curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash

## INSTALL DOCKER

# Install dependencies
sudo apt-get update
sudo apt-get install ca-certificates curl gnupg

# Add Docker's official GPG key
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg

# Add docker's source to apt-list
echo \
  "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
  "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

# Install Docker Engine
sudo apt-get update && sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin -y

# Set docker permissions on socket for first run (required cause permissions for user adminuser do not take effect after first reboot)
sudo chmod 666 /var/run/docker.sock

# post install steps linux to prevent permissions denied on docker socket
sudo groupadd docker
sudo usermod -aG docker adminuser
runuser -l adminuser -c "newgrp docker" 

# exit with success
exit 0

please note: be aware that the url from which you can download the self-hosted Azure Devops Agent binary can change over time (or in other words; when Microsoft wants to do that). please check this beforehand.

Conclusion

By combining Terraform and a shell script, we’ve turned the manual, error-prone process of provisioning self-hosted Azure DevOps agents into a secure, automated, and repeatable workflow for all FFA Titan environments. Below you can see that the Self-Hosted Azure DevOps Agent has been successfully registered within the azure devops agent pool ' ffatitan-dem-dev-pool'

The agent is now ready for use within the Azure DevOps pipelines that have access to it. By using this self-hosted azure devops agent we gain:

• Full control over the build environment
• Secure agent registration with Azure DevOps
• Seamless integration with our private azure vnet
• Faster, more reliable CI/CD pipelines.

Automating this process with Terraform ensures that our platform remains ISO27001 compliant while gaiing full control of our own CI/CD-pipelines. This approach is part of our FFA Titan 2.0 philosophy: infrastructure should never be the bottleneck. It should be invisible, secure, and built for scale.

FAQ