Updated lab

UVADS · Oct 3, 2023 · 5dc87c4 · 5dc87c4
1 parent 1c97e28
commit 5dc87c4
Show file tree

Hide file tree

Showing 10 changed files with 81 additions and 72 deletions.
diff --git a/ddsbook/_book/Defining-Data-Science.pdf b/ddsbook/_book/Defining-Data-Science.pdf
diff --git a/ddsbook/_book/search.json b/ddsbook/_book/search.json
@@ -53,7 +53,7 @@
     "href": "value-lab-II.html",
     "title": "8  Labs Rubric - AI Fairness 360",
     "section": "",
-    "text": "Individual Assignment\nGeneral Descripition: This lab is designed for you to get exposure to AI fairness approaches in a no code environment on the website AI Fairness 360. You will be able to work through the various fairness methods at different stages of the pipeline and reflect on which methods seem to work the best on the given datasets.\nPreparatory Assignments - None\nWhy am I doing this? In order to give you exposure to and practice with the various methods being developed and deployed in the ML fairness space. After completing the lab you’ll have a better sense of how these tools are used, when they are used and how the work.\nWhat am I going to do? The AI Fairness 360 website has a demo module that includes three datasets. Work through the demo on all three datasets, trying all the methods provided, and answer the questions below.\nAnswer these questions:\n\nFor each protected class variable which evaluation methods showed bias to be present?\nNote how each method preform at removing bias.\nWas the accuracy of the model effected when using the various approaches, if so how?\nGiven the above what are some patterns you noticed, which methods seem to work the best, where in the data process are these methods located (pre/in/post).\n\nTips for success:\n\nTake careful notes as you go through each method\nHave fun\n\nHow will I know I have succeeded:\n\n\n\n\n\n\n\nSpecs Category\nSpecs Details\n\n\n\n\nFormatting\n\nSubmit Via Canvas\nText answers to questions\n\n\n\nText\n\nGoal: The questions are designed to be answer during or right after the lab period.\nBullett points are fine for the first three questions, paragraph from for the fourth\n\n\n\n\nAcknowledgements: Special thanks for Jess Taggart from UVA CTE for coaching us. This structure is pulled directory from Steifer & Palmer (2020)."
+    "text": "Individual Assignment\nGeneral Descripition: This lab is designed for you to get exposure to AI fairness approaches in a no code environment on the website AI Fairness 360. You will be able to work through the various fairness methods at different stages of the pipeline and reflect on which methods seem to work the best on the given datasets.\nPreparatory Assignments - None\nWhy am I doing this? In order to give you exposure to and practice with the various methods being developed and deployed in the ML fairness space. After completing the lab you’ll have a better sense of how these tools are used, when they are used and how the work.\nWhat am I going to do? The AI Fairness 360 website has a demo module that includes three datasets. Work through the demo on all three datasets, trying all the methods provided, and answer the questions below.\nAnswer these questions/Do the Following:\n\nYou will be using this demo to track changes in protected classes in the Compas (ProPublica recidivism) dataset. The first step is to document how bias is present in each of the protected classes in the datasets. There are two protected classes included in the dataset. AI Fairness 360 uses 5 metrics to determine whether bias is present on a pre-trained machine learning algorithm. In your own words, define each of these metrics as they relate to reporting bias. The site provides a brief explanation of each, but you may want to look at outside sources for more information.\nNext make note, in table format, of which bias metrics indicate bias for each protected class. Then, observe and record the effect of the Reweighting, Optimized Pre-Processing, and Reject Option Based Classification mitigation methods on all 5 of the bias metrics and overall accuracy for each protected class.\nNote – Adversarial Debiasing does not seem to run, so we are skipping that method\nGiven your observations, which mitigation method AND which bias metric(s) would you use to best eliminate/detect bias for both protected classes if you were the data scientist working with this dataset? Be explicit in what factor(s) influenced your decision.\nSimilar to the exercise presented by Dr. Mona Sloan on Tuesday, write a brief summary of recommendations you would give to decision makers that don’t have technical knowledge as it relates to creating policy around choosing what mitigation method and bias metric(s) should be used when working with sensitive data. (200-word min)\n\nTips for success:\n\nTake careful notes as you go through each method\nHave fun\n\nHow will I know I have succeeded:\n\n\n\n\n\n\n\nSpecs Category\nSpecs Details\n\n\n\n\nFormatting\n\nSubmit Via Canvas\nUpload a Document (word or pdf) that addresses the requirements |\n\n\n\nText\n\nGoal: The questions are designed to be answer during or right after the lab period.\nBulletts are fine for questions 1 and 3. Table for 2 and paragraph for 4. |\n\n\n\n\nAcknowledgements: Special thanks for Jess Taggart from UVA CTE for coaching us. This structure is pulled directory from Steifer & Palmer (2020)."
   },
   {
     "objectID": "value-lab-III.html",

diff --git a/ddsbook/_book/sitemap.xml b/ddsbook/_book/sitemap.xml
@@ -2,106 +2,106 @@
 <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
   <url>
     <loc>http://definingdatascience.com/index.html</loc>
-    <lastmod>2023-09-20T19:41:29.800Z</lastmod>
+    <lastmod>2023-09-28T16:57:46.498Z</lastmod>
   </url>
   <url>
     <loc>http://definingdatascience.com/intro.html</loc>
-    <lastmod>2023-09-20T19:41:29.802Z</lastmod>
+    <lastmod>2023-09-28T16:57:46.503Z</lastmod>
   </url>
   <url>
     <loc>http://definingdatascience.com/design.html</loc>
-    <lastmod>2023-09-20T19:41:29.804Z</lastmod>
+    <lastmod>2023-09-28T16:57:46.506Z</lastmod>
   </url>
   <url>
     <loc>http://definingdatascience.com/design-sds.html</loc>
-    <lastmod>2023-09-20T19:41:29.806Z</lastmod>
+    <lastmod>2023-09-28T16:57:46.509Z</lastmod>
   </url>
   <url>
     <loc>http://definingdatascience.com/design-lab.html</loc>
-    <lastmod>2023-09-20T19:41:29.808Z</lastmod>
+    <lastmod>2023-09-28T16:57:46.510Z</lastmod>
   </url>
   <url>
     <loc>http://definingdatascience.com/design-external.html</loc>
-    <lastmod>2023-09-20T19:41:29.810Z</lastmod>
+    <lastmod>2023-09-28T16:57:46.510Z</lastmod>
   </url>
   <url>
     <loc>http://definingdatascience.com/design-case-study.html</loc>
-    <lastmod>2023-09-20T19:41:29.812Z</lastmod>
+    <lastmod>2023-09-28T16:57:46.510Z</lastmod>
   </url>
   <url>
     <loc>http://definingdatascience.com/value.html</loc>
-    <lastmod>2023-09-20T19:41:29.814Z</lastmod>
+    <lastmod>2023-09-28T16:57:46.510Z</lastmod>
   </url>
   <url>
     <loc>http://definingdatascience.com/value-sds.html</loc>
-    <lastmod>2023-09-20T19:41:29.816Z</lastmod>
+    <lastmod>2023-09-28T16:57:46.522Z</lastmod>
   </url>
   <url>
     <loc>http://definingdatascience.com/value-lab.html</loc>
-    <lastmod>2023-09-20T19:41:29.819Z</lastmod>
+    <lastmod>2023-09-28T16:57:46.525Z</lastmod>
   </url>
   <url>
     <loc>http://definingdatascience.com/value-lab-II.html</loc>
-    <lastmod>2023-09-20T19:41:29.821Z</lastmod>
+    <lastmod>2023-09-28T16:57:46.527Z</lastmod>
   </url>
   <url>
     <loc>http://definingdatascience.com/value-lab-III.html</loc>
-    <lastmod>2023-09-20T19:41:29.823Z</lastmod>
+    <lastmod>2023-09-28T16:57:46.530Z</lastmod>
   </url>
   <url>
     <loc>http://definingdatascience.com/value-external.html</loc>
-    <lastmod>2023-09-20T19:41:29.825Z</lastmod>
+    <lastmod>2023-09-28T16:57:46.533Z</lastmod>
   </url>
   <url>
     <loc>http://definingdatascience.com/value-case-study.html</loc>
-    <lastmod>2023-09-20T19:41:29.828Z</lastmod>
+    <lastmod>2023-09-28T16:57:46.536Z</lastmod>
   </url>
   <url>
     <loc>http://definingdatascience.com/systems-sds.html</loc>
-    <lastmod>2023-09-20T19:41:29.830Z</lastmod>
+    <lastmod>2023-09-28T16:57:46.539Z</lastmod>
   </url>
   <url>
     <loc>http://definingdatascience.com/systems-lab.html</loc>
-    <lastmod>2023-09-20T19:41:29.832Z</lastmod>
+    <lastmod>2023-09-28T16:57:46.541Z</lastmod>
   </url>
   <url>
     <loc>http://definingdatascience.com/systems-external.html</loc>
-    <lastmod>2023-09-20T19:41:29.835Z</lastmod>
+    <lastmod>2023-09-28T16:57:46.543Z</lastmod>
   </url>
   <url>
     <loc>http://definingdatascience.com/systems-case-study.html</loc>
-    <lastmod>2023-09-20T19:41:29.837Z</lastmod>
+    <lastmod>2023-09-28T16:57:46.545Z</lastmod>
   </url>
   <url>
     <loc>http://definingdatascience.com/analytics-sds.html</loc>
-    <lastmod>2023-09-20T19:41:29.840Z</lastmod>
+    <lastmod>2023-09-28T16:57:46.548Z</lastmod>
   </url>
   <url>
     <loc>http://definingdatascience.com/analytics-lab.html</loc>
-    <lastmod>2023-09-20T19:41:29.842Z</lastmod>
+    <lastmod>2023-09-28T16:57:46.550Z</lastmod>
   </url>
   <url>
     <loc>http://definingdatascience.com/analytics-external.html</loc>
-    <lastmod>2023-09-20T19:41:29.844Z</lastmod>
+    <lastmod>2023-09-28T16:57:46.552Z</lastmod>
   </url>
   <url>
     <loc>http://definingdatascience.com/analytics-case-study.html</loc>
-    <lastmod>2023-09-20T19:41:29.846Z</lastmod>
+    <lastmod>2023-09-28T16:57:46.554Z</lastmod>
   </url>
   <url>
     <loc>http://definingdatascience.com/appendix-sources.html</loc>
-    <lastmod>2023-09-20T19:41:29.852Z</lastmod>
+    <lastmod>2023-09-28T16:57:46.559Z</lastmod>
   </url>
   <url>
     <loc>http://definingdatascience.com/BSDS-Course-Info.html</loc>
-    <lastmod>2023-09-20T19:41:29.855Z</lastmod>
+    <lastmod>2023-09-28T16:57:46.559Z</lastmod>
   </url>
   <url>
     <loc>http://definingdatascience.com/references.html</loc>
-    <lastmod>2023-09-20T19:41:29.857Z</lastmod>
+    <lastmod>2023-09-28T16:57:46.559Z</lastmod>
   </url>
   <url>
     <loc>http://definingdatascience.com/Defining-Data-Science.pdf</loc>
-    <lastmod>2023-09-20T19:41:28.585Z</lastmod>
+    <lastmod>2023-09-28T16:57:45.137Z</lastmod>
   </url>
 </urlset>
diff --git a/ddsbook/_book/value-lab-II.html b/ddsbook/_book/value-lab-II.html
@@ -294,16 +294,17 @@ <h1 class="title d-none d-lg-block"><span class="chapter-number">8</span>&nbsp;
 
 <p>DS 1001 - Spring 2023 - Professors Wright and Alonzi Due: End of lab period (or later that day) Submission format: Word doc or PDF summarizing your findings</p>
 <p>Individual Assignment</p>
-<p><strong>General Descripition:</strong> This lab is designed for you to get exposure to AI fairness approaches in a no code environment on the website <a href="https://aif360.mybluemix.net/">AI Fairness 360</a>. You will be able to work through the various fairness methods at different stages of the pipeline and reflect on which methods seem to work the best on the given datasets.</p>
+<p><strong>General Descripition:</strong> This lab is designed for you to get exposure to AI fairness approaches in a no code environment on the website <a href="https://aif360.res.ibm.com/">AI Fairness 360</a>. You will be able to work through the various fairness methods at different stages of the pipeline and reflect on which methods seem to work the best on the given datasets.</p>
 <p>Preparatory Assignments - None</p>
 <p><strong>Why am I doing this?</strong> In order to give you exposure to and practice with the various methods being developed and deployed in the ML fairness space. After completing the lab you’ll have a better sense of how these tools are used, when they are used and how the work.</p>
 <p><strong>What am I going to do?</strong> The AI Fairness 360 website has a demo module that includes three datasets. Work through the demo on all three datasets, trying all the methods provided, and answer the questions below.</p>
-<p><strong>Answer these questions:</strong></p>
+<p><strong>Answer these questions/Do the Following:</strong></p>
 <ol type="1">
-<li>For each protected class variable which evaluation methods showed bias to be present?</li>
-<li>Note how each method preform at removing bias.</li>
-<li>Was the accuracy of the model effected when using the various approaches, if so how?</li>
-<li>Given the above what are some patterns you noticed, which methods seem to work the best, where in the data process are these methods located (pre/in/post).</li>
+<li><p>You will be using this demo to track changes in protected classes in the Compas (ProPublica recidivism) dataset. The first step is to document how bias is present in each of the protected classes in the datasets. There are two protected classes included in the dataset. AI Fairness 360 uses 5 metrics to determine whether bias is present on a pre-trained machine learning algorithm. In your own words, define each of these metrics as they relate to reporting bias. The site provides a brief explanation of each, but you may want to look at outside sources for more information.</p></li>
+<li><p>Next make note, in table format, of which bias metrics indicate bias for each protected class. Then, observe and record the effect of the Reweighting, Optimized Pre-Processing, and Reject Option Based Classification mitigation methods on all 5 of the bias metrics and overall accuracy for each protected class.<br>
+Note – Adversarial Debiasing does not seem to run, so we are skipping that method</p></li>
+<li><p>Given your observations, which mitigation method AND which bias metric(s) would you use to best eliminate/detect bias for both protected classes if you were the data scientist working with this dataset? Be explicit in what factor(s) influenced your decision.</p></li>
+<li><p>Similar to the exercise presented by Dr.&nbsp;Mona Sloan on Tuesday, write a brief summary of recommendations you would give to decision makers that don’t have technical knowledge as it relates to creating policy around choosing what mitigation method and bias metric(s) should be used when working with sensitive data. (200-word min)</p></li>
 </ol>
 <p>Tips for success:</p>
 <ul>
@@ -327,14 +328,14 @@ <h1 class="title d-none d-lg-block"><span class="chapter-number">8</span>&nbsp;
 <td style="text-align: center;">Formatting</td>
 <td style="text-align: left;"><ul>
 <li>Submit Via Canvas</li>
-<li>Text answers to questions</li>
+<li>Upload a Document (word or pdf) that addresses the requirements |</li>
 </ul></td>
 </tr>
 <tr class="even">
 <td style="text-align: center;">Text</td>
 <td style="text-align: left;"><ul>
 <li>Goal: The questions are designed to be answer during or right after the lab period.</li>
-<li>Bullett points are fine for the first three questions, paragraph from for the fourth</li>
+<li>Bulletts are fine for questions 1 and 3. Table for 2 and paragraph for 4. |</li>
 </ul></td>
 </tr>
 </tbody>