The technology used in this tutorial is now obsolete.

For information on using Azure Data Factory mapping data flows to read and write CDM entity data, see this blog post, which describes the overall solution, with links to an article describing how CDM support uses inline datasets, and an article providing details of the source and sink properties.

For information on the new Spark CDM Connector for use in Azure Databricks and Synapse to read and write CDM entity data, see https://github.com/Azure/spark-cdm-connector

Tutorial and sample code for integrating Power BI dataflows and Azure Data Services using Common Data Model (CDM) folders in Azure Data Lake Storage Gen2. For more information on the scenario, see this blog post.

The tutorial walks through use of CDM folders in a modern data warehouse scenario. In it you will:

• Configure your Power BI account to save Power BI dataflows as CDM folders in ADLS Gen2;
• Create a Power BI dataflow by ingesting order data from the Wide World Importers sample database and save it as a CDM folder;
• Use an Azure Databricks notebook that prepares and cleanses the data in the CDM folder, and then writes the updated data to a new CDM folder in ADLS Gen2;
• Use Azure Machine Learning to train and publish a model using data from the CDM folder;
• Use an Azure Data Factory pipeline to load data from the CDM folder into staging tables in Azure SQL Data Warehouse and then invoke stored procedures that transform the data into a dimensional model;
• Use Azure Data Factory to orchestrate the overall process and monitor execution.

Each step leverages metadata contained in the CDM folder to make it easier and simpler to accomplish the task.

The provided samples include code, libraries, and Azure resource templates that you can use with CDM folders you create from your own data.

IMPORTANT: the sample code is provided as-is with no warranties and is intended for learning purposes only.

See the tutorial for pre-requisites and installation details.

The sample code and tutorials in this project are licensed under the MIT license. See the LICENSE file for more details.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.