Azure Monitor Alerts: Set Up Proactive Infrastructure Monitoring

Executive Snapshot

TL;DR

• Azure Monitor alerts are the first line of defence for infrastructure health — set them up before you need them, not after an outage.
• Three alert types matter most: metric alerts (low latency, real-time), log alerts (query-based, flexible), and activity log alerts (governance, change tracking).
• Action groups are reusable — one group can notify Slack, PagerDuty, email, and trigger a runbook simultaneously.
• Dynamic thresholds use ML-based baselines and significantly reduce alert fatigue compared to static thresholds on workloads with variable traffic.
• Infrastructure-as-code your alerts from day one; manual portal configuration does not scale and creates configuration drift.

Introduction

Your VM's CPU just hit 100% at 2 AM on a Sunday. Your database connection pool is exhausted. A critical Azure Key Vault access policy was silently deleted. Without proactive alerting, you find out from an angry end-user or, worse, a production outage post-mortem.

flowchart LR
  Res["Azure resources"] -->|metrics & logs| AM["Azure Monitor"]
  AM --> Rule{"Alert rule threshold"}
  Rule -->|breached| AG["Action group"]
  Rule -->|within range| NoOp["No action"]
  AG --> Notify["Email / SMS"]
  AG --> Automate["Webhook / Logic App"]

Azure Monitor is Microsoft's unified observability platform, and its alerting engine sits at the centre of every mature Azure operations practice. Yet many teams either under-configure alerts — relying on a handful of basic CPU thresholds — or overconfigure them, drowning in noise until engineers start ignoring everything.

This tutorial walks you through a production-ready azure monitor alerts setup: from understanding signal types and building your first metric alert, to structuring action groups, writing log query alerts, and codifying everything in Bicep. By the end, you'll have a repeatable, scalable alerting framework that catches real problems early — without waking anyone up for a false positive at 3 AM.

Assumed environment: Azure subscription with Contributor access, one or more VMs or App Services, and a Log Analytics workspace already provisioned.

Prerequisites

• Azure subscription with Contributor role (or Monitoring Contributor for alert-only work)
• Azure CLI ≥ 2.55 installed and authenticated (az login)
• An existing Log Analytics workspace (required for log alerts)
• Azure Monitor Diagnostic Settings enabled on target resources
• Basic familiarity with Azure Resource Manager concepts (resource groups, scopes)
• Optional but recommended: Bicep CLI ≥ 0.26 for IaC examples

1. Understand the Azure Monitor Alert Signal Types

Before writing a single alert rule, map your monitoring requirements to the correct signal type. Choosing the wrong type is the single biggest source of alert gaps and false positives.

Metric Alerts

• Data source: Azure Monitor Metrics (time-series telemetry, ~1-min granularity)
• Best for: CPU, memory, disk I/O, HTTP 5xx rates, response time thresholds
• Latency: Near real-time (1–5 minutes)
• Cost: Charged per monitored time series

Log Alerts

• Data source: Log Analytics workspace (KQL queries over ingested logs)
• Best for: Application errors, security events, custom business metrics, cross-resource correlation
• Latency: 1–15 minutes depending on ingestion and frequency
• Cost: Charged per query execution

Activity Log Alerts

• Data source: Azure Activity Log (control-plane operations)
• Best for: Resource deletions, policy changes, RBAC modifications, service health events
• Latency: Near real-time (~5 minutes)
• Cost: Free

Smart Detection (Application Insights)

• Data source: Application Insights telemetry
• Best for: Anomaly detection in application performance without manual thresholds
• Latency: ~24 hours for trained models

Rule of thumb: If the metric exists in the Metrics Explorer, use a metric alert. If you need to join tables, filter on log fields, or aggregate across resources with conditions, use a log alert.

2. Create an Action Group (Do This First)

Action groups define what happens when an alert fires. Creating them first means you can reuse them across dozens of alert rules.

Via Azure CLI

# Create a resource group for monitoring resources (best practice: isolate monitoring)
az group create \
  --name rg-monitoring-prod \
  --location eastus

# Create an action group with email + webhook (e.g., PagerDuty endpoint)
az monitor action-group create \
  --resource-group rg-monitoring-prod \
  --name ag-ops-critical \
  --short-name OPSCrit \
  --action email ops-lead ops-lead@contoso.com \
  --action webhook pagerduty https://events.pagerduty.com/integration/YOUR_KEY/enqueue

Adding an Azure Automation Runbook Receiver

# Add a runbook action to an existing action group
az monitor action-group update \
  --resource-group rg-monitoring-prod \
  --name ag-ops-critical \
  --add-action automationrunbook \
    --runbook-name "Restart-UnhealthyVM" \
    --automation-account-id "/subscriptions/SUB_ID/resourceGroups/rg-automation/providers/Microsoft.Automation/automationAccounts/aa-prod" \
    --webhook-resource-id "/subscriptions/SUB_ID/resourceGroups/rg-automation/providers/Microsoft.Automation/automationAccounts/aa-prod/webhooks/wh-restart-vm" \
    --is-global-runbook false \
    --use-common-alert-schema true

Pro tip: Always enable Use Common Alert Schema on action group receivers. It standardises the payload structure across all alert types and simplifies webhook/Logic App parsing downstream.

3. Create a Metric Alert Rule

Let's create a production-grade CPU alert for a VM using dynamic thresholds — a significant upgrade over static thresholds for workloads with daily traffic patterns.

Static Threshold Alert (CLI)

# Alert fires when CPU > 85% for 5 consecutive minutes
az monitor metrics alert create \
  --name "alert-vm-cpu-high-prod" \
  --resource-group rg-workloads-prod \
  --scopes "/subscriptions/SUB_ID/resourceGroups/rg-workloads-prod/providers/Microsoft.Compute/virtualMachines/vm-web-01" \
  --condition "avg Percentage CPU > 85" \
  --window-size 5m \
  --evaluation-frequency 1m \
  --severity 2 \
  --description "VM CPU exceeded 85% threshold" \
  --action "/subscriptions/SUB_ID/resourceGroups/rg-monitoring-prod/providers/Microsoft.Insights/actionGroups/ag-ops-critical" \
  --auto-mitigate true

Dynamic Threshold Alert (CLI)

# Dynamic threshold: alert on anomalous CPU using ML baseline
az monitor metrics alert create \
  --name "alert-vm-cpu-dynamic-prod" \
  --resource-group rg-workloads-prod \
  --scopes "/subscriptions/SUB_ID/resourceGroups/rg-workloads-prod/providers/Microsoft.Compute/virtualMachines/vm-web-01" \
  --condition "avg Percentage CPU dynamic high 2 of 4 since 2026-01-01T00:00:00.000Z" \
  --window-size 15m \
  --evaluation-frequency 5m \
  --severity 2 \
  --description "VM CPU anomaly detected via dynamic threshold" \
  --action "/subscriptions/SUB_ID/resourceGroups/rg-monitoring-prod/providers/Microsoft.Insights/actionGroups/ag-ops-critical"

The dynamic high 2 of 4 syntax means: alert when the metric is high (above the learned upper bound) in at least 2 of the last 4 evaluation windows. This dramatically reduces noise from transient spikes.

4. Create a Log Alert Rule (KQL-Based)

Log alerts give you full KQL expressiveness. This example detects VMs with repeated failed SSH/RDP login attempts — a real-world security use case.

KQL Query for Failed Logins

// Save this as a reference — used in the alert rule below
SecurityEvent
| where TimeGenerated >= ago(15m)
| where EventID == 4625  // Windows failed logon
| summarize FailedAttempts = count() by Computer, IpAddress = tostring(EventData)
| where FailedAttempts > 10
| project Computer, FailedAttempts, IpAddress

Create the Log Alert via CLI

# Create a log alert for brute-force login detection
az monitor scheduled-query create \
  --name "alert-log-failed-logins-prod" \
  --resource-group rg-monitoring-prod \
  --scopes "/subscriptions/SUB_ID/resourceGroups/rg-workloads-prod/providers/Microsoft.OperationalInsights/workspaces/law-prod" \
  --condition-query "SecurityEvent | where TimeGenerated >= ago(15m) | where EventID == 4625 | summarize FailedAttempts = count() by Computer | where FailedAttempts > 10" \
  --condition-time-aggregation "Count" \
  --condition-operator "GreaterThan" \
  --condition-threshold 0 \
  --evaluation-frequency "5m" \
  --window-duration "15m" \
  --severity 1 \
  --description "More than 10 failed logins from single host in 15 minutes" \
  --action-groups "/subscriptions/SUB_ID/resourceGroups/rg-monitoring-prod/providers/Microsoft.Insights/actionGroups/ag-ops-critical" \
  --auto-mitigate false \
  --check-workspace-alerts-storage-configured false

Note: Set --auto-mitigate false on security alerts. Auto-mitigation automatically resolves the alert when the condition clears — for security events, you want the alert to stay open until a human acknowledges it.

5. Create an Activity Log Alert (Governance)

Activity log alerts are free and essential for change-control monitoring. This example fires when anyone deletes a Network Security Group — a common misconfiguration root cause.

# Alert on NSG deletion across the entire subscription
az monitor activity-log alert create \
  --name "alert-actlog-nsg-delete" \
  --resource-group rg-monitoring-prod \
  --scope "/subscriptions/SUB_ID" \
  --condition \
    category=Administrative \
    operationName=Microsoft.Network/networkSecurityGroups/delete \
    status=Succeeded \
  --action-group "/subscriptions/SUB_ID/resourceGroups/rg-monitoring-prod/providers/Microsoft.Insights/actionGroups/ag-ops-critical" \
  --description "An NSG was successfully deleted in the subscription" \
  --enabled true

Extend this pattern to cover: Key Vault policy changes (Microsoft.KeyVault/vaults/write), VM deallocations, and storage account public access modifications.

6. Define Alert Rules in Bicep (IaC)

Manual portal or CLI configurations create drift. Use Bicep to version-control your alert rules alongside your infrastructure.

// alert-rules.bicep
// Deploy with: az deployment group create -g rg-monitoring-prod -f alert-rules.bicep

param location string = resourceGroup().location
param targetVmResourceId string
param actionGroupId string

resource cpuAlert 'Microsoft.Insights/metricAlerts@2018-03-01' = {
  name: 'alert-vm-cpu-high-prod'
  location: 'global'           // Metric alerts always deploy to 'global'
  properties: {
    description: 'VM CPU exceeded 85% for 5 minutes'
    severity: 2
    enabled: true
    scopes: [targetVmResourceId]
    evaluationFrequency: 'PT1M'
    windowSize: 'PT5M'
    criteria: {
      'odata.type': 'Microsoft.Azure.Monitor.SingleResourceMultipleMetricCriteria'
      allOf: [
        {
          name: 'HighCPU'
          metricName: 'Percentage CPU'
          metricNamespace: 'Microsoft.Compute/virtualMachines'
          operator: 'GreaterThan'
          threshold: 85
          timeAggregation: 'Average'
          criterionType: 'StaticThresholdCriterion'
        }
      ]
    }
    autoMitigate: true
    actions: [
      {
        actionGroupId: actionGroupId
        webHookProperties: {}
      }
    ]
  }
}

resource nsgDeleteAlert 'Microsoft.Insights/activityLogAlerts@2020-10-01' = {
  name: 'alert-actlog-nsg-delete'
  location: 'global'
  properties: {
    scopes: [subscription().id]
    enabled: true
    description: 'NSG deleted in subscription'
    condition: {
      allOf: [
        { field: 'category', equals: 'Administrative' }
        { field: 'operationName', equals: 'Microsoft.Network/networkSecurityGroups/delete' }
        { field: 'status', equals: 'Succeeded' }
      ]
    }
    actions: {
      actionGroups: [
        { actionGroupId: actionGroupId }
      ]
    }
  }
}

Deploy with parameters:

az deployment group create \
  --resource-group rg-monitoring-prod \
  --template-file alert-rules.bicep \
  --parameters \
    targetVmResourceId="/subscriptions/SUB_ID/resourceGroups/rg-workloads-prod/providers/Microsoft.Compute/virtualMachines/vm-web-01" \
    actionGroupId="/subscriptions/SUB_ID/resourceGroups/rg-monitoring-prod/providers/Microsoft.Insights/actionGroups/ag-ops-critical"

7. Configure Alert Suppression with Action Rules

Alert suppression (via Action Rules) prevents notification floods during planned maintenance without disabling the underlying alert rules.

# Suppress all alerts from rg-workloads-prod every Sunday 02:00-04:00 UTC
az monitor action-rule create \
  --resource-group rg-monitoring-prod \
  --name "suppress-maintenance-window-sunday" \
  --rule-type Suppression \
  --scope-type ResourceGroup \
  --scope "/subscriptions/SUB_ID/resourceGroups/rg-workloads-prod" \
  --suppression-recurrence-type Weekly \
  --suppression-recurrence 0 \
  --suppression-start-time "02:00:00" \
  --suppression-end-time "04:00:00" \
  --status Enabled \
  --description "Weekly Sunday maintenance window suppression"

8. Pitfalls to Avoid

1. Alerting on every metric at static thresholds
Starting with static 80% CPU thresholds on all VMs creates constant noise on bursty workloads. Use dynamic thresholds or establish a 2-week baseline before setting static values.

2. One action group per alert rule
This is a maintenance nightmare. Create role-based action groups (ag-ops-critical, ag-ops-warning, ag-dev-team) and reuse them. When an on-call number changes, update one action group, not 50 alert rules.

3. Forgetting to test action groups
The Azure portal has a Test Action Group button for a reason. Test every receiver — especially webhooks and runbooks — before they need to fire in anger at 2 AM.

4. Ignoring alert dimensions
Metric alerts support splitting by dimensions (e.g., per disk, per API endpoint). Without dimensions, a single alert rule can mask which specific resource is degraded.

# Split CPU alert by VM instance (useful for scale sets)
az monitor metrics alert create \
  --name "alert-vmss-cpu-per-instance" \
  --resource-group rg-workloads-prod \
  --scopes "/subscriptions/SUB_ID/resourceGroups/rg-workloads-prod/providers/Microsoft.Compute/virtualMachineScaleSets/vmss-web" \
  --condition "avg Percentage CPU > 85 where VMName includes *" \
  --window-size 5m \
  --evaluation-frequency 1m \
  --severity 2 \
  --description "VMSS per-instance CPU alert"

5. Not enabling Common Alert Schema
Without it, each alert type sends a different JSON payload structure, forcing you to write conditional parsing logic in every downstream integration.

6. Skipping alert documentation
Every alert rule should have a description that includes: what it means, likely causes, and a runbook URL. The person waking up at 3 AM might not be you.

Troubleshooting

Alert fired but no notification received

# Check action group delivery history in the portal, or query Activity Log
az monitor activity-log list \
  --resource-group rg-monitoring-prod \
  --start-time 2026-06-17T00:00:00Z \
  --end-time 2026-06-17T23:59:59Z \
  --query "[?contains(operationName.value, 'actionGroups')]" \
  --output table

Likely causes: Webhook endpoint returning non-2xx, email in spam, action rule suppression active, action group in disabled state.

Log alert query returns no results in testing but fires alerts

Cause: Time range mismatch. In Log Analytics query editor, the default time range overrides the ago() filter in your query. Test your alert query with the exact same time window as the alert's --window-duration.

# Validate the alert fired correctly and check result count
az monitor scheduled-query list \
  --resource-group rg-monitoring-prod \
  --output table

Dynamic threshold alert not firing as expected

Cause: Insufficient training data. Dynamic thresholds require at least 3 days of historical metric data to build a baseline. For new resources, use static thresholds for the first week, then switch.

"The resource is not supported for metric alerts" error

Cause: Not all Azure resources emit metrics to Azure Monitor Metrics. Check az monitor metrics list-definitions --resource RESOURCE_ID — if it returns empty, you must use log-based alerting via Diagnostic Settings instead.

# List available metric definitions for a resource
az monitor metrics list-definitions \
  --resource "/subscriptions/SUB_ID/resourceGroups/rg-workloads-prod/providers/Microsoft.Compute/virtualMachines/vm-web-01" \
  --output table

Key Takeaways

• Match signal to alert type: Metric alerts for real-time thresholds, log alerts for KQL-based detection, activity log alerts for free governance monitoring.
• Build action groups before alert rules — reusable groups keyed to role (ops-critical, ops-warning, dev-team) eliminate duplication and simplify on-call rotation updates.
• Dynamic thresholds reduce alert fatigue for workloads with variable patterns; commit to at least 3 days of baseline data before tuning sensitivity.
• IaC is non-negotiable at scale — Bicep or Terraform-managed alert rules prevent configuration drift, enable peer review, and survive subscription migrations.
• Suppression rules preserve signal quality — use action rules to mute expected noise during maintenance rather than disabling alert rules entirely.
• Always enable Common Alert Schema on action group receivers to future-proof your notification pipeline integrations.
• Document every alert rule in its description field with cause, context, and a runbook URL — operational knowledge belongs in the system, not in someone's head.

Next Steps

1. Extend to Application Insights: Instrument your applications with the App Insights SDK and create availability tests and failure anomaly alerts alongside your infrastructure alerts.
2. Build an Alert Dashboard: Use Azure Monitor Workbooks to create a live operations dashboard that surfaces active alerts, alert history trends, and MTTD/MTTR metrics.
3. Integrate with ITSM: Connect your action groups to ServiceNow or Jira via the ITSM Connector to automatically create incidents from high-severity alerts.
4. Implement Alert-as-Code in CI/CD: Add a bicep lint + what-if step to your pull request pipeline to validate alert rule changes before they reach production.
5. Review Azure Advisor recommendations: Azure Advisor surfaces recommended alert rules for resources that have none configured — use it as an audit tool monthly.

Related Articles

• [Azure Log Analytics KQL Queries: The Ops Playbook for 2026] — /azure/log-analytics-kql-queries-ops-playbook/
• [Azure Bicep vs Terraform: Choosing the Right IaC Tool for Azure] — /azure/bicep-vs-terraform-azure-iac-comparison/
• [Azure Cost Monitoring: Build a Real-Time Budget Alert System] — /azure/azure-cost-monitoring-budget-alerts-setup/

Have a question or a monitoring pattern worth sharing? Drop it in the comments — the IT Pro Insights team reads every one.