What is a Data Hub: Concepts and Guidelines
The V’s of Big Data: Before Data Hub Life was simple when we restricted ourselves to spreadsheets and relational data stores. Data was structured, databases were...
Azure Logic App deployment process with Powershell
Prologue A software product goes through three broad phases viz. development, testing and deployment. The last phase can be approached in multiple ways. One of the ways...
Making ADF Web activity synchronous with Logic App
Prologue It is a common scenario in ETL jobs to send Email Notifications. With Azure Data Factory, such an example can be found in this article. Accordingly, we use web...
Log Analytics using STRING_SPLIT function
Motivating Log Analytics Being a Data Engineer comes with its own set of challenges and opportunities since 80 per cent time is spent on cleaning and munging the raw...
Move Files with Azure Data Factory- End to End
Pre-requisite In the first part of this series, we focused on moving a single file from one blob location to another using ADF. Here is the link for the first part: Move...
Move Files with Azure Data Factory- Part II
Prologue In the first part of this series i.e. Move Files with Azure Data Factory- Part I, we went through the approach and demonstration to move a single file from one...
Move Files with Azure Data Factory- Part I
Why Move Files? A typical ETL process entails extracting the data, transforming it and loading the same in the destination. Firstly, we extract files in a...
Azure Data Factory Webhook activity
Motivating Webhook Before getting into details of a webhook activity, we should analyse its need in ADF. Apart from being an ETL/ELT tool, Azure Data Factory(ADF) has...
Azure Machine Learning Designer
Prologue: The Azure Machine Learning family Data Science process entails data extraction, data preparation, modeling, training, testing and evaluation, deployment etc....
Motivating Data Science with Azure Machine Learning Studio
Curve Fitting Although Azure Machine Learning Studio is a great tool, it is basically a means to an end viz. Machine Learning. Before we dive into the new incarnation of...