Monitoring & Logging

Azure Monitoring Architecture

Azure Monitor provides a comprehensive solution for collecting, analyzing, and acting on telemetry from cloud and on-premises environments. It enables full-stack observability across applications, infrastructure, and networks.

inputData SourcesMetrics & Logs
monitoringAzure MonitorCollect & Store
analyticsAnalyzeQuery & Visualize
notifications_activeRespondAlert & Automate

Azure Monitor Data Sources

Azure Monitor collects data from multiple tiers of your application architecture. Understanding data sources is critical for designing comprehensive monitoring solutions.

Data Types

speed Metrics

Numerical values collected at regular intervals describing system performance.

  • Platform metrics - Collected automatically from Azure resources
  • Custom metrics - Application-specific via SDK or API
  • Guest OS metrics - From inside VMs via agent
  • Stored in time-series database for 93 days

description Logs

Structured or unstructured event data with timestamps and properties.

  • Activity logs - Subscription-level events
  • Resource logs - Resource-level diagnostics
  • Application logs - Custom app telemetry
  • Stored in Log Analytics workspace (configurable retention)

Source Tiers

TierData SourcesCollection Method
ApplicationPerformance, requests, exceptions, tracesApplication Insights SDK, auto-instrumentation
Guest OSPerformance counters, event logs, syslogAzure Monitor Agent (AMA)
Azure ResourcesPlatform metrics, resource logsAutomatic + diagnostic settings
SubscriptionActivity log, Service HealthAutomatic
TenantEntra ID logs, audit eventsDiagnostic settings to Log Analytics
Custom SourcesAny REST client, on-premisesData Collector API, Azure Arc

Exam Notes

Azure Monitor Agent (AMA) is the unified agent replacing Log Analytics agent (MMA/OMS) and Diagnostics extension. Use Data Collection Rules (DCRs) to configure what data to collect and where to send it.

Log Analytics Workspaces

Log Analytics workspace is the primary destination for log data in Azure Monitor. It provides a unified data store for querying and analyzing logs using Kusto Query Language (KQL).

Workspace Design Considerations

Single Workspace

Centralized approach - simpler management and cross-resource queries.

  • Easier correlation across resources
  • Simplified access management
  • Lower administrative overhead
  • Best for most organizations

Multiple Workspaces

Distributed approach - when isolation or specific requirements exist.

  • Data sovereignty - Regional compliance
  • Access isolation - Separate security boundaries
  • Billing separation - Per-department costs
  • Scale limits - Very high data volumes

Access Control Models

Workspace-context

Access to all data in the workspace based on RBAC roles.

  • Log Analytics Reader
  • Log Analytics Contributor
  • Monitoring Reader/Contributor

Resource-context

Access to logs for specific Azure resources you have RBAC access to.

  • Uses existing resource RBAC
  • No workspace permissions needed
  • Query via resource's Logs blade

Table-level RBAC

Granular access to specific tables within a workspace.

  • Custom roles with table access
  • Restrict sensitive data tables
  • Combine with other models

Data Retention & Archiving

TierRetentionQuery LatencyCost
Analytics (Interactive)30 days free, up to 730 daysImmediateStandard ingestion + retention
Basic Logs8 daysImmediateReduced ingestion (limited queries)
ArchiveUp to 12 yearsSearch job requiredLow storage, pay per search

Exam Notes

Basic Logs are ideal for high-volume, low-value data (verbose debugging, traces). They have reduced cost but limited query capabilities - no alerts, no joins with Analytics tables.

Azure Workbooks & Insights

Azure Workbooks provide a flexible canvas for data analysis and creating rich visual reports. Azure Insights are pre-built monitoring experiences for specific services.

Workbooks

Data Sources

  • Logs - KQL queries against Log Analytics
  • Metrics - Azure Monitor metrics
  • Azure Resource Graph - Resource inventory
  • Azure Resource Manager - Management data
  • Custom endpoints - REST APIs, JSON

Visualization Types

  • Charts - Line, area, bar, scatter, pie
  • Grids - Tables with formatting
  • Tiles - Single value, sparklines
  • Maps - Geographic visualizations
  • Graphs - Topology, relationships

Key Features

  • Parameters - Interactive filters and dropdowns
  • Conditional visibility - Show/hide based on values
  • Groups - Organize sections, expand/collapse
  • Templates - Reusable starting points
  • Sharing - Pin to dashboards, export reports

Workbooks vs Dashboards

FeatureWorkbooksDashboards
Rich textYesNo
ParametersYesLimited
Flexible layoutYesGrid-based
Real-time refreshOn demandAuto

Azure Insights

web

Application Insights

APM for web applications

dns

VM Insights

VM performance & dependencies

deployed_code

Container Insights

AKS & container monitoring

lan

Network Insights

Network health & topology

database

SQL Insights

Azure SQL monitoring

storage

Storage Insights

Storage accounts health

encrypted

Key Vault Insights

Vault operations & access

backup

Backup Insights

Backup jobs & health

Azure Data Explorer

Azure Data Explorer (ADX) is a fast, fully managed data analytics service for real-time analysis on large volumes of data. It uses the same KQL as Log Analytics but is designed for high-scale, long-term analytics scenarios.

When to Use ADX

  • High volume - Petabytes of data, billions of records
  • Long retention - Years of historical data
  • Complex analytics - Time series, ML, anomaly detection
  • Multiple sources - IoT, logs, telemetry, business data
  • Custom applications - Embedded analytics

ADX vs Log Analytics

AspectLog AnalyticsADX
Primary useOps monitoringBig data analytics
ScaleWorkspace limitsPetabyte scale
RetentionUp to 12 yearsUnlimited
ManagementFully managedCluster management
Cost modelPer GB ingestedCompute + storage

Integration with Azure Monitor

Cross-query

Query Log Analytics from ADX using the adx() function, or query ADX from Log Analytics using adx cluster proxy.

Data Export

Configure continuous export from Log Analytics to ADX for long-term retention and advanced analytics.

ADX as External Table

Reference ADX tables from Log Analytics queries for hybrid analysis across both platforms.

Exam Notes

Consider Azure Data Explorer when Log Analytics retention (12 years max) or scale limits are insufficient, or when you need dedicated compute for complex analytics, time-series analysis, or ML workloads on operational data.

Alerting & Automation

Azure Monitor alerts proactively notify you of critical conditions and can trigger automated responses.

Alert Types

Metric Alerts

  • Based on metric thresholds
  • Static or dynamic thresholds
  • Near real-time (1 min frequency)
  • Multi-dimensional filtering

Log Alerts

  • Based on KQL query results
  • Number of results or metric measurement
  • Frequency: 5 min to 1 day
  • Complex conditions supported

Activity Log Alerts

  • Based on activity log events
  • Service health alerts
  • Resource health alerts
  • Administrative events

Action Groups

Notification Types

  • Email/SMS - Direct notifications
  • Push notification - Azure mobile app
  • Voice - Automated phone call

Action Types

  • Azure Functions - Custom code execution
  • Logic Apps - Workflow automation
  • Webhook - Call external services
  • Runbook - Azure Automation
  • ITSM - ServiceNow, etc.

Design Best Practices

Workspace Strategy

Start with single workspace per region. Split only for specific requirements like compliance or billing isolation.

Cost Optimization

Use Basic Logs for high-volume, low-value data. Configure appropriate retention and archive for long-term storage.

Use Azure Monitor Agent

Migrate from legacy agents to AMA. Use Data Collection Rules for flexible, centralized configuration.

Application Insights

Enable Application Insights for all web applications. Use workspace-based mode for unified log queries.

Alert Design

Use dynamic thresholds for baseline-aware alerting. Implement action groups with escalation paths.

Governance

Use Azure Policy to enforce diagnostic settings. Ensure all resources send logs to appropriate workspaces.

Quick Reference

ComponentPurposeKey Points
Azure MonitorUnified monitoring platformCollects metrics & logs, alerts, insights
Log AnalyticsLog storage & query engineKQL queries, workspace-based, retention policies
Azure Monitor AgentData collection agentReplaces MMA/OMS, uses Data Collection Rules
Application InsightsAPM solutionRequests, dependencies, exceptions, traces
WorkbooksInteractive reportsKQL + metrics + ARM + parameters
Azure Data ExplorerBig data analyticsPetabyte scale, long retention, ML capabilities
Action GroupsAlert responseNotifications + automation actions