README.md

Lab Overview

Azure Machine Learning designer (preview) gives you a cloud-based interactive, visual workspace that you can use to easily and quickly prep data, train and deploy machine learning models. It supports Azure Machine Learning compute, GPU or CPU. Machine Learning designer also supports publishing models as web services on Azure Kubernetes Service that can easily be consumed by other applications.

In this lab, we will be compare the performance of two binary classifiers: Two-Class Boosted Decision Tree and Two-Class Logistic Regression for predicting customer churn. The goal is to run an expensive marketing campaign for high risk customers; thus, the precision metric is going to be key in evaluating performance of these two algorithms. We will do all of this from the Azure Machine Learning designer without writing a single line of code.

Exercise 1: Create Training Pipeline

Task 1: Open Sample 5: Binary Classification – Customer Relationship Prediction

1. In Azure portal, open the available machine learning workspace.

2. Select Launch studio under the Try the new Azure Machine Learning studio message.

   

3. When you first launch the studio, you may need to set the directory and subscription. If so, you will see this screen:

   

   For the directory, select Udacity and for the subscription, select Azure Sponsorship. For the machine learning workspace, you may see multiple options listed. Select any of these (it doesn't matter which) and then click Get started.

4. From the studio, select Designer, Sample 5: Binary Classification – Customer Relationship Prediction.

   

Task 2: Setup Compute Target

1. In the settings panel on the right, select Select compute target.

   

2. In the Set up compute target editor, select the available compute, and then select Save.

Note: If you are facing difficulties in accessing pop-up windows or buttons in the user interface, please refer to the Help section in the lab environment.

Task 3: Delete Pipeline Modules

1. From the right-hand-side of the pipeline, select the Two-Class Boosted Decision Tree module and then select the Delete Icon.

   

2. From the right-hand-side of the pipeline, select the SMOTE module and then select the Delete Icon.

   

Task 4: Setup the Two-Class Logistic Regression Module

1. Select Machine Learning Algorithms section in the left navigation. Follow the steps outlined below:

   1. Select the Two-Class Logistic Regression prebuilt module

   2. Drag and drop the selected module on to the canvas

   3. Connect the Two-Class Logistic Regression module to the first input of the Train Model module

   4. Connect the first output of the Split Data module to the second input of the Train Model module

   

Exercise 2: Submit Training Pipeline

Task 1: Create Experiment and Submit Pipeline

1. Select Submit to open the Setup pipeline run editor.

   

   Please note that the button name in the UI is changed from Run to Submit.

2. In the Setup pipeline run editor, select Experiment, Create new and provide New experiment name: Churn-Predictor, and then select Submit.

   

3. Wait for pipeline run to complete. It will take around 45 minutes to complete the run.

4. While you wait for the model training to complete, you can learn more about the evaluation metrics for the classification algorithm used in this lab by selecting Metrics for classification models.

Exercise 3: Compare Model Performance

Task 1: Open Evaluation Results for Two-Class Boosted Decision Tree

1. From the left-hand-side of the pipeline, select Evaluate Model, Outputs, Visualize to open the Evaluate Model result visualization dialog for the Two-Class Boosted Decision Tree module.

   

Task 2: Evaluate Two-Class Boosted Decision Tree Performance

1. Scroll down to review model performance metrics for Two-Class Boosted Decision Tree. Observe that the Precision value is around 0.7.

   

Task 3: Open Evaluation Results for Two-Class Logistic Regression

1. From the right-hand-side of the pipeline, select Evaluate Model, Outputs, Visualize to open the Evaluate Model result visualization dialog for the Two-Class Logistic Regression module.

   

Task 4: Evaluate Two-Class Logistic Regression Performance

1. Scroll down to review model performance metrics for Two-Class Logistic Regression. Observe that the Precision value is around 0.3.

   

Task 5: Conclusion

1. Based on the primary performance metric, Precision, it shows that the Two-Class Boosted Decision Tree algorithm outperforms the Two-Class Logistic Regression algorithm.

Next Steps

Congratulations! You have trained and compared performance of two different classification machine learning models. You can continue to experiment in the environment but are free to close the lab environment tab and return to the Udacity portal to continue with the lesson.