Migrating from Azure Data Factory to Azure Synapse Integration


Did you think that it’s straightforward? I mean, did you think you can simply export the ARM template from Azure Data Factory and import it into Azure Synapse? Well, let me break the bad news for you! There is no provision to import an ARM template into Azure Synapse.

Okay! Okay! Hold on! That’s too fast-paced. Hence, let’s introduce the characters here. Firstly comes the Azure Data Factory. It is Microsoft Azure’s primary ETL/ELT tool. Next comes Azure Synapse Family comprising two members viz. dedicated SQL pool (Formerly MPP SQLDW) and Synapse workspace. The integration service of Synapse workspace is nothing but the ADF equivalent of the Synapse family.

You may wonder why does this topic deserves an article? Can’t we export the ARM template from  ADF and import it to Synapse? Well, the answer is NO. So, wherein lies the answer? It’s in two steps:

  1. Download ADF pipeline support files.
  2. Deploying them using PowerShell.

Download ADF support files

I won’t reinvent the wheel by repeating the steps that have been shown succinctly on this blog: Download Azure Data Factory support files.

Follow the steps mentioned and download the support files on your machine.

Deploying pipelines to Azure Synapse using PowerShell

Before pipeline deployment, we need to understand the way ADF is organized. The most fundamental entity of ADF is a Linked Service. Linked Services set up a connection between ADF and the datastores (sources and sink). On top of linked services, we have datasets. A dataset is a logical view on which data processing activities are built. Finally, the activities combine to create a pipeline. The below diagram is an excellent summary.

Hence, the deployment/migration of ADF happens in the following order:

  • Linked Service
  • Datasets
  • Pipelines

Now let’s go through the steps for migration of ADF to synapse:

1 – Establish a connection

Connect to your Azure tenant:


Make sure the current subscription content is correctly set  by running the following command:

$context = Get-AzContext

If not, run the following command.

$context = Select-AzSubscription -SubscriptionName "<subscription name>"

However, make sure that you have the Azure Synapse PowerShell module installed before you go further, using these commands:

Install-Module Az.Synapse
Import-Module Az.Synapse

2 – Import a linked service

Create linked services using the following command:

Set-AzSynapseLinkedService -WorkspaceName <synapse workspace name> -Name <name of linked service name> -DefinitionFile "<folder path to json file\json file name>.json" -DefaultProfile $context

3 – Import a dataset

Create dataset using the following command:

Set-AzSynapseDataSet -WorkspaceName <synapse workspace name> -Name <dataset name> -DefinitionFile "<folder path to json file\json file name>.json" -DefaultProfile $context

4 – Import a pipeline

Finally, import the pipeline using the following command:

Set-AzSynapsePipeline -WorkspaceName <synapse workspace name> -Name <pipeline name> -DefinitionFile "<folder path to json file\json file name>.json" -DefaultProfile $context


Reference: https://github.com/solliancenet/azure-synapse-analytics-ga-content-packs/blob/main/hands-on-labs/lab-03/README.md

P.C. Azure Synapse Analytics


I am a Data Scientist with 6+ years of experience.

Leave a Reply