Azure Defender provides a built-in policy to enable Defender Plan for OSS databases. We should replace the custom policy with this built-in.

Name: Configure Azure Defender for open-source relational databases to be enabled
Policy Definition ID: /providers/Microsoft.Authorization/policyDefinitions/44433aa3-7ec2-4002-93ea-65c65ff0310a

We need to document an optional step to turn on audit log streaming from Azure DevOps to Log Analytics Workspace.

The instructions are based on https://docs.microsoft.com/en-us/azure/devops/organizations/audit/auditing-streaming?view=azure-devops

Reference to Sentinel detection rules: https://github.com/Azure/Azure-Sentinel/tree/master/Detections/AzureDevOpsAuditing

Steps to reproduce:

1. Run either platform-connectivity-hub-azfw-ci or platform-connectivity-hub-nva-ci pipeline.

Error:

Hub Network will manage private dns zones, creating Azure Policy assignment to automatically create Private Endpoint DNS Zones
Deploying policy assignment using policy/custom/assignments/dns-private-endpoints.bicep
ERROR: An error occurred reading file. Could not find file '/home/vsts/work/1/s/policy/custom/assignments/dns-private-endpoints.bicep'.

##[error]Caller: 10 /home/vsts/work/_temp/azureclitaskscript1638139169085.sh, LineNo: 1, Command: az deployment mg create --location canadacentral --management-group-id pubsec --template-file dns-private-endpoints.bicep --parameters policyAssignmentManagementGroupId='pubsec' policyDefinitionManagementGroupId='pubsec' privateDNSZoneSubscriptionId='ed7f4eed-9010-4227-b115-2a5e37728f27' privateDNSZoneResourceGroupName='pubsec-dns-rg'
##[error]Script failed with exit code: 1

Related to #78

Improve the GitHub Action used to validate pull request so that it can:

• Validate JSON Schema for subscription deployments
• Run JSON Schema unit tests
• Compile all bicep templates

Reference to Azure DevOps PR validation pipeline:
https://github.com/Azure/CanadaPubSecALZ/blob/main/.pipelines/pull-request-check.yml

Automation Accounts have a new log category that needs to be added for Diagnostic Settings Policy.

Category groups are a collection of different logs to help you achieve different monitoring goals. These groups are defined dynamically and Microsoft may add or remove categories as the product evolves, resulting in new charges.

Instead of specifying each log category separately, use the allLogs to ensure new categories added to diagnostic settings are automatically included.

Option in Azure Portal:

In a secure environment where Azure Firewall or NVA is used to restrict and control egress traffic, it is important to secure AKS egress trafic as well.

https://docs.microsoft.com/en-us/azure/aks/egress-outboundtype

Adding code to default the AKS configurattion to outboundType: userDefinedRouting
updating the AKS Subnet configuration in Machine Learning LZ to use a UDR to support this routing and configuration requirement
Adding the Azure Policy that prevents the creation of a load balancer with a public IP
Adding Azure Firewall Policy rules to allow trafic to Microsoft endpoints and Ubuntu NTP and Security patches: https://docs.microsoft.com/en-us/azure/aks/limit-egress-traffic

The solution will provide the following:

• All Egress traffic from the Cluster and its pods will be routed through the network virtual applicance (Azure Firewall)
• There will be no default Loadbalancer provisionned with Public IP (as it is the case now), and therefore the Cluster will have zero public IPs.
• Block the possibility to create K8s service of type loadbalancer with public IP

Is your feature request related to a problem? Please describe.
We are unable to deploy resources from the portal due to the required tags having to exist for resources to be deployed, it also makes it difficult to follow some guides for deploying things like Sentinel Connectors.

Describe the solution you'd like
A variable that allows us to switch enforcement from deny deployments to audits. This way we can still require tags but don't have to block deployments through the portal. I like to manually configure some resources then sync them with Terraform after the fact since I'm not as comfortable with Azure as I am with other CSP's at the moment.

Describe alternatives you've considered
I can remove the required tags but it's still nice to be able some enforcement in pace

Is your feature request related to a problem? Please describe.
When testing the landing zone, it's inevitable the same deployment will be deployed multiple times. because keyvault is set with anti purging, there's no wait to create a vault with the same name as a deleted vault.

Describe the solution you'd like
use date and time of the creation in the uniquestring function to generate a hash that is dependent on a variable not just a fixed value like the resource group id.

As stated in title, make it clearer that the YAML variable files are organized by (deployment) environments.

Develop an onboarding guide for GitHub that includes settings up secrets and workflows.

We have two options:

• Build GitHub Actions and document their integration with Azure; OR
• Document instructions for using Azure DevOps Pipelines with Git repo in GitHub

Reference to Azure DevOps guide: https://github.com/Azure/CanadaPubSecALZ/blob/main/docs/onboarding/ado.md

Describe the bug
In landingzones/lz-healthcare/main.parameters-sample.json

"serviceHealthAlerts": {
      "value": {
        "resourceGroupName": "pubsec-service-health",
        "incidentTypes": [ "Incident", "Security" ],
        "regions": [ "Global", "Canada East", "Canada Central" ],
        "receivers": {
          "app": [ "[email protected]" ],
          "email": [ "[email protected]" ],
          "sms": [
              { "countryCode": "1", "phoneNumber": "5555555555" }
          ],
          "voice": [
              { "countryCode": "1", "phoneNumber": "5555555555" }
          ]
        }
      }
    }

This object missing

"actionGroupName": "ALZ action group",
    "actionGroupShortName": "alz-alert",
    "alertRuleName": "ALZ alert rule",
    "alertRuleDescription": "Alert rule for Azure Landing Zone"

To Reproduce
Steps to reproduce the behavior:
az deployment sub create --location CanadaCentral --template-file main.bicep --parameters main.parameters-sample.json

Expected behavior
Successful deployment

Error
The language expression property 'actionGroupName' doesn't exist, available properties are 'resourceGroupName, incidentTypes, regions, receivers'.'\",\r\n \"additionalInfo\": [\r\n {\r\n \"type\": \"TemplateViolation\",\r\n \"info\": {\r\n \"lineNumber\": 1,\r\n \"linePosition\": 7753,\r\n \"path\": \"\"\r\n }\r\n }\r\n ]\r\n }\r\n}"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}"}]}}

Describe the bug
When I open a new PR in my repo with pipelines configured according to your onboarding guide a job kicks off that insta fails due to missing files.

I'm okay with it failing my issue is it shouldn't be running my assumption was that these pipelines were only to run when merging to main not when a PR is created.

To Reproduce

1. Follow onboarding guide
2. Create a pipeline using the yml files in the repo
3. Create a PR
4. watch it fail

Expected behavior
I expect it not to kick off a pipeline job when a PR opens

Screenshots

Is your feature request related to a problem? Please describe.
Log query audit logs provide telemetry about log queries run in Azure Monitor. This includes information such as when a query was run, who ran it, what tool was used, the query text, and performance statistics describing the query's execution.

On Dec 1, Audit logs for Log Analytics Workspace became generally available.

Announcement: https://azure.microsoft.com/en-us/updates/general-availability-audit-logs-of-azure-monitor-log-queries/

Align Platform management group based on Reference Architecture described at https://docs.microsoft.com/en-us/azure/cloud-adoption-framework/ready/enterprise-scale/architecture#high-level-architecture

The change includes:

1. 3 child management groups under Platform: Identity, Connectivity, Management
2. Networking subscription will be moved to Connectivity MG
3. Log Analytics subscription will be moved to Management MG

Revised management group structure:

Instead of validating all the files in the Bicep files in the repo it should only validate bicep scripts that have been modified since the last PR

This scan takes 6 minutes to run in a PR that has zero changes to bicep scripts, although this PR does add a variable that was missing. However based on the fact that my PR missing the required variable passed this check I'm assuming the template validation doesn't check for variables so I'm guessing it doesn't need to run unless files being validated are changed.

Develop an onboarding guide for defining and assigning Azure Policies through the automation pipeline. The scenarios we need to document are:

1. How to create a new custom policy, a new policy set and policy set assignment
2. How to update an existing policy set with new policy definitions using custom and built in policies

Authoring guide will expand on https://github.com/Azure/CanadaPubSecALZ/tree/main/docs/policy

Leverage the point of view of first time user

Customer usage attribution associates usage from Azure resources in customer subscriptions created while deploying Azure Landing Zones for Canadian Public Sector. Forming these associations in internal Microsoft systems brings greater visibility to the Azure footprint deploying the landing zones.

This will align to

1. Planned implementation in Azure Landing Zone Reference Implementation (http://github.com/azure/enterprise-Scale/)

2. Existing work in Terraform modules for usage tracking
   https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs
   

We will implement this metric with enabled by default with a single flag to disable if customers do not want the information to be tracked. Tracking ID will be a83f6385-f514-415f-991b-2d9bd7aed658

This work will also update the documentation to highlight the telemetry that is being tracked. Recommended wording:

When you deploy Azure Landing Zones for Canadian Public Sector, Microsoft can identify the deployments of the ARM/Bicep templates with the deployed Azure resources. Microsoft can correlate these resources used to support the deployments. Microsoft collects this information to provide the best experiences with their products and to operate their business. The data is collected and governed by Microsoft's privacy policies, located at https://www.microsoft.com/trustcenter.

Reference: https://docs.microsoft.com/en-us/azure/marketplace/azure-partner-customer-usage-attribution

Is your feature request related to a problem? Please describe.
Create an azure policy to enable Azure Site Recovery diagnostic events to be sent to a log analytics workspace

Describe the solution you'd like
An Azure Policy is created to enable the following Site Recovery events to be to a log analytics workspace

• AzureSiteRecoveryJobs
• AzureSiteRecoveryEvents
• AzureSiteRecoveryReplicatedItems
• AzureSiteRecoveryReplicationStats
• AzureSiteRecoveryRecoveryPoints
• AzureSiteRecoveryReplicationDataUploadRate
• AzureSiteRecoveryProtectedDiskDataChurn

Describe alternatives you've considered
N/A

Additional context
N/A

Parameterization will allow for streamlined automation that allows for:

• Log Analytics Workspace (LAW) data retention to be configured through input parameters

• Azure Policy assignments that check for LAW retention period to be configured through input parameters and avoids multiple edits in *.parameters.json in https://github.com/Azure/CanadaPubSecALZ/tree/main/policy

Was wondering if anyone had looked at converting the cloud guardrails for cloud profile 3 into the custom policies in this repo.

I know some of the documentation for controls has been uploaded here
https://github.com/canada-ca/cloud-guardrails-azure

and here

https://github.com/canada-ca/cloud-guardrails

At Ignite, a few Azure services were rebranded. We need to update the names in docs, images and code to reflect the changes:

• Azure Security Center is now Microsoft Defender for Cloud
• Azure Sentinel is now Microsoft Sentinel

Is your feature request related to a problem? Please describe.
Although most products needed for the Analytics landing zone are available, there's no Analytics landing zone

Describe the solution you'd like
Create a new landing zone for data analytics

We use JSON Schemas to validate input parameters provided during an archetype deployment. The input parameters are organized into a JSON Parameters file and provided to the subscription-ci Azure DevOps Pipeline.

We chose to use JSONSchema to validate the inputs since ARM doesn't have a native method to validate object types. The inputs are organized into JSON object so that the parameters are contextually organized for better readability and configuration.

Today, the JSONSchema for archetypes are located in https://github.com/Azure/CanadaPubSecALZ/tree/main/schemas and versioned based on the changes to the archetype input parameters. When a version change occurs, multiple tasks needs to occur such as:

• Update Azure DevOps pipeline for subscription-ci
• Update Pull Request Pipeline for validation
• Update Archetype documentation to latest version

This increases the effort for making incremental changes and we need to improve the versioning scheme.

We are going to move to the following schema to help improve the authoring experience:

• latest
• v0.1.0
• v0.2.0
• vx.y.z

latest is the version that will be referenced in all of our pipelines and docs. The latest folder will have a readme.md that has the changelog and last updated date. The version folders are kept to pin the changes a specific version and can be used for diff between version to see the changes.

Describe the bug
When declaring the recovery vault using the Microsoft.RecoveryServices/vaults@2020-02-02 API version, the below non-blocking syntax warning appears in the VS code
The property "tier" is not allowed on objects of type "Sku". No other properties are allowed. If this is an inaccuracy in the documentation, please report it to the Bicep Team

To Reproduce
Steps to reproduce the behavior:

1. Open backup-recovery-vault.bicep file in vscode
2. On line 31, a squiggly line appears in the "tier" property associated with the SKU object
3. which indicates the following message
   The property "tier" is not allowed on objects of type "Sku". No other properties are allowed. If this is an inaccuracy in the documentation, please report it to the Bicep Team

Expected behavior
No error message

Screenshots

• Azure Bicep version v0.4.1008

The tags.bicep scripts should be able to handle no required tags.

This is the error message we get

Deploying Tags.bicep using create operation...
ERROR: {"status":"Failed","error":{"code":"DeploymentFailed","message":"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.","details":[{"code":"BadRequest","message":"{\r\n  \"error\": {\r\n    \"code\": \"InvalidCreatePolicySetDefinitionRequest\",\r\n    \"message\": \"The policy set definition 'audit-required-tags-on-resources' create request is invalid. At least one policy definition must be referenced.\"\r\n  }\r\n}"},{"code":"BadRequest","message":"{\r\n  \"error\": {\r\n    \"code\": \"InvalidCreatePolicySetDefinitionRequest\",\r\n    \"message\": \"The policy set definition 'required-tags-on-resource-group' create request is invalid. At least one policy definition must be referenced.\"\r\n  }\r\n}"},{"code":"BadRequest","message":"{\r\n  \"error\": {\r\n    \"code\": \"InvalidCreatePolicySetDefinitionRequest\",\r\n    \"message\": \"The policy set definition 'custom-tags-inherited-from-resource-group' create request is invalid. At least one policy definition must be referenced.\"\r\n  }\r\n}"}]}}
##[error]Caller: 10 /home/vsts/work/_temp/azureclitaskscript1637855831889.sh, LineNo: 1, Command: az deployment mg create --location canadacentral --management-group-id pubsec --template-file Tags.bicep --parameters Tags.parameters.json
##[error]Script failed with exit code: 1

This is what I have for the https://github.com/Azure/CanadaPubSecALZ/blob/main/policy/custom/definitions/policyset/Tags.parameters.json file

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "policyDefinitionManagementGroupId": {
            "value": "{{var-topLevelManagementGroupName}}"
        },
        "requiredResourceTags": {
            "value": [ ]
        }
    }
}

The Automation Account that is created by the landing zone scripts fails the Audit Diagnostic Setting policy from the PBMM Controls.

It either doesn't have Logging or Metrics setup I haven't dug too much into it yet:

Use the storage account to store saved queries (required for CMK encryption)

Reference: https://docs.microsoft.com/en-us/azure/azure-monitor/logs/private-storage#link-storage-accounts-to-your-log-analytics-workspace

From FedRamp Moderate

Provide a script for the creation of the Pipelines in Azure DevOps

The solution will provide an option for automatd provisionning of the pipelines

Is your feature request related to a problem? Please describe.
storing SQL DB connection string in key vault is unnecessary extra step that is not needed. The SQL user name and password are saved in AKV in the case of SQL authentication and in the case of AAD only authentication, there's no need to save anything.

Describe the solution you'd like
Remove the module that stores the connection string in AKV

Adding code to support Azure CNI network plugin for the AKS module

https://docs.microsoft.com/en-us/azure/aks/concepts-network

Azure CNI, provides the following capabilities (compared to Kubenet):

• Network policy (Azure) is native – Calico can be used as an add-on at Cluster creation
• Support for Windows Nodes
• Support for Virtual nodes
• Allows connecting to existing Azure | on-premises resources directly via IP addresses assigned to each pod

However, Azure CNI requires pre-designing and creating a VNET in advance as well as more IPs (than Kubenet)

Adding the option to use either:

• _Network Plugin: Kubenet or Azure CNI
• Network Policy: for Kubenet: Calico | for Azure CNI: azure (Azure Network Policy) or Calico
• https://docs.microsoft.com/en-us/azure/aks/use-network-policies

• Add Readme link for subs archetypes

A document to describe setting up Azure DevOps organization, projects and logging.

• Add table of contents
• Expand on Service Principal and RBAC requirements
• CLI commands for creating Service Principal & role assignment
• Explain security group and role assignment for Logging & Network Section
• Expand on the steps for service connection setup
• Expand on using an existing log analytics workspace

Pipelines in Azure DevOps should run when changes to https://github.com/Azure/CanadaPubSecALZ/tree/main/config/ are merged to main.

Is your feature request related to a problem? Please describe.
AKS deployment and maintenance is quite complicated compared to Azure App Service

Describe the solution you'd like
Support Azure App Service as a deployment target for AML in the ML landing zone

We need to improve the documentation for archetypes. Specifically:

• Folder structure used for organization archetype configurations (ARM parameters)
• Steps for deploying a new instance of an archetype
• Steps for creating a new archetype

Azure CIS 1.3.0

Add the option to :

• use a Service Principal for each Pipeline
• or use a master SP (as Currently configured)

Solution:

• breakdown the Azure Built-in Roles needed for each Pipeline
• provide the option to use a different SP for each pipeline with the appropriate RBAC role
• Maintain the option to use a master SP for all the pipelines

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. E.g. I'm always frustrated when [...]

Describe the solution you'd like
Configure and manage Azure AD authentication with Azure SQL

Is your feature request related to a problem? Please describe.
AKS deployment and maintenance is quite complicated compared to Azure Container Instances

Describe the solution you'd like
Support Azure Container Instance as a deployment target for AML in the ML landing zone

logAnalyticsRetentionInDays needs to be removed from the example ADO document

its not supported AND not used in the two other example files

Describe the bug
Running the roles-ci pipeline fails

Current working directory: /home/vsts/work/1/s/roles

Deploying la-vminsights-readonly.bicep using create operation...
ERROR: {"error":{"code":"AuthorizationFailed","message":"The client '71734fab-5326-4534-8b2c-c2e51a4fd2fa' with object id '71734fab-5326-4534-8b2c-c2e51a4fd2fa' does not have authorization to perform action 'Microsoft.Resources/deployments/validate/action' over scope '/providers/Microsoft.Management/managementGroups/Science-Program/providers/Microsoft.Resources/deployments/la-vminsights-readonly' or the scope is invalid. If access was recently granted, please refresh your credentials."}}
##[error]Caller: 5 /home/vsts/work/_temp/azureclitaskscript1635988781424.sh, LineNo: 1, Command: az deployment mg create --template-file la-vminsights-readonly.bicep --location canadacentral --management-group-id Science-Program --parameters assignableMgId=Science-Program
##[error]Script failed with exit code: 1
/usr/bin/az account clear
Finishing: Create Custom Roles - la-vminsights-readonly.bicep

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. E.g. I'm always frustrated when [...]

Describe the solution you'd like
Azure Active Directory only authentication - Azure SQL Database & Azure SQL Managed Instance | Microsoft Docs
https://docs.microsoft.com/en-us/azure/azure-sql/database/authentication-azure-ad-only-authentication?tabs=azure-cli

Not sure where to check but the path for the management group isn't correct.

Ive probably edited a config file wrong, but unsure where to look

Actual error code

2021-11-04T19:17:24.7955997Z Moving subscription c6fc362a-00c0-4c77-a09d-b916de32fd4a to management group ScPx-SPScPx-SP-LandingZonesScPx-SP-LandingZones-DevTest...
2021-11-04T19:17:27.9971221Z ERROR: {"error":{"code":"AuthorizationFailed","message":"The client '71734fab-5326-4534-8b2c-c2e51a4fd2fa' with object id '71734fab-5326-4534-8b2c-c2e51a4fd2fa' does not have authorization to perform action 'Microsoft.Resources/deployments/validate/action' over scope '/providers/Microsoft.Management/managementGroups/ScPx-SPScPx-SP-LandingZonesScPx-SP-LandingZones-DevTest/providers/Microsoft.Resources/deployments/move-subscription-c6fc362a-00c0-4c77-a09d-b916de32fd4a-canadace' or the scope is invalid. If access was recently granted, please refresh your credentials."}}
2021-11-04T19:17:28.0547009Z ##[error]Caller: 8 /home/vsts/work/_temp/azureclitaskscript1636053430754.sh, LineNo: 1, Command: az deployment mg create --location canadacentral --management-group-id ScPx-SPScPx-SP-LandingZonesScPx-SP-LandingZones-DevTest --template-file move-subscription.bicep --name ${deployName} --parameters managementGroupId='ScPx-SPScPx-SP-LandingZonesScPx-SP-LandingZones-DevTest' subscriptionId='c6fc362a-00c0-4c77-a09d-b916de32fd4a'
2021-11-04T19:17:28.0559707Z ##[error]Script failed with exit code: 1

Describe the solution you'd like
The solution will enable an option to deploy an Azure backup for VMs backup during the deployment of the generic subscription Archetype
Describe alternatives you've considered
N/A

Additional context

Automation uses Azure Bastion Basic SKU in the Hub Network deployment (in NVA and Azure Firewall deployment)

Azure Bastion Standard SKU is now GA'ed (Nov 2nd, 2021) and provides these 2 capabilities:

• Manually scale Bastion host Virtual Machine instances: Azure Bastion supports manual scaling of the Virtual Machine (VM) instances facilitating Bastion connectivity. You can configure 2-50 instances to manage the number of concurrent SSH and RDP sessions Azure Bastion can support.

• Azure Bastion admin panel: Azure Bastion supports enabling/disabling features accessed by the Bastion host.

Reference: https://azure.microsoft.com/en-us/updates/general-availability-bastion-standard-sku/

We are looking for feedback on this proposed change. We will target this change for December release (v0.6.0). Please share any alternative solutions via comments.

Landing Zones management groups are defined in this reference implementation as DevTest, QA and Prod to create alignment to various environments a customer might have. These are provided as examples and can be modified by each customer based on their preferred structure.

From Azure Portal:

We are considering to change the management groups from environment focused to data sensitivity focused.

The proposed target definition will enable us to customize Azure Policies (used for guardrails) based on data classification of subscriptions such as Unclassified, Protected A, Protected B, etc..

All subscriptions will be kept in these management groups and we would not add another level to break it out by environments.

Therefore, the change would be:

From current definition

pubsec
 |- Landing Zones
    |- DevTest
    |- QA
    |- Prod
 |- other management groups removed for simplicity   

To proposed target definition

pubsec
 |- Landing Zones
    |- Unclassified
    |- Protected A
    |- Protected B
 |- other management groups removed for simplicity   

The example region specified is causing issues https://github.com/Azure/CanadaPubSecALZ/blob/main/docs/onboarding/ado.md#step-31-update-commonyml-in-git-repository

When I try to deploy the G-Suite Workplace connector to canadacentral it fails because the function (I'm guessing linux functions aren't available in the region) can't run there.

Because i was unaware of that I have to either migrate our LogWorkspace to canadacentral or pay to egress data across regions.

Since functions is probably going to be a common pattern to ingest logs into sentinel the example should probably default to canadacentral.