Built site for gh-pages

.nojekyll

@@ -1 +1 @@
-02cb5d4d
+69547296

schedule/slides/21-nnets-intro.html

@@ -333,7 +333,7 @@
 <h2>21 Neural nets</h2>
 <p><span class="secondary">Stat 406</span></p>
 <p><span class="secondary">Geoff Pleiss, Trevor Campbell</span></p>
-<p>Last modified – 02 November 2023</p>
+<p>Last modified – 06 November 2024</p>
 <p><span class="math display">\[
 \DeclareMathOperator*{\argmin}{argmin}
 \DeclareMathOperator*{\argmax}{argmax}
@@ -413,8 +413,8 @@ <h2>High level overview</h2>
 </figure>
 </div>
 </section>
-<section id="recall-nonparametric-regression" class="slide level2">
-<h2>Recall nonparametric regression</h2>
+<section id="recall-basis-regression" class="slide level2">
+<h2>Recall basis regression</h2>
 <p>Suppose <span class="math inline">\(Y \in \mathbb{R}\)</span> and we are trying estimate the regression function <span class="math display">\[\Expect{Y\given X} = f_*(X)\]</span></p>
 <p>In Module 2, we discussed basis expansion,</p>
 <ol type="1">
@@ -423,12 +423,11 @@ <h2>Recall nonparametric regression</h2>
 <li><p>Estimate <span class="math inline">\(\beta_k\)</span> with least squares</p></li>
 </ol>
 </section>
-<section id="recall-nonparametric-regression-1" class="slide level2">
-<h2>Recall nonparametric regression</h2>
+<section id="recall-basis-regression-1" class="slide level2">
+<h2>Recall basis regression</h2>
 <p>The weaknesses of this approach are:</p>
 <ul>
 <li>The basis is fixed and independent of the data</li>
-<li>If <span class="math inline">\(p\)</span> is large, then nonparametrics doesn’t work well at all (recall the Curse of Dimensionality)</li>
 <li>If the basis doesn’t “agree” with <span class="math inline">\(f_*\)</span>, then <span class="math inline">\(K\)</span> will have to be large to capture the structure</li>
 <li>What if parts of <span class="math inline">\(f_*\)</span> have substantially different structure? Say <span class="math inline">\(f_*(x)\)</span> really wiggly for <span class="math inline">\(x \in [-1,3]\)</span> but smooth elsewhere</li>
 </ul>
@@ -520,6 +519,14 @@ <h2>What about classification (10 classes, 2 layers)</h2>
 </div>
 </div>
 </section>
+<section id="nonlinear-activation-functions" class="slide level2">
+<h2>(Nonlinear) activation functions</h2>
+
+<img data-src="21-nnets-intro_files/figure-revealjs/sigmoid-1.svg" class="quarto-figure quarto-figure-center r-stretch"></section>
+<section id="effect-of-depth" class="slide level2">
+<h2>Effect of depth</h2>
+
+<img data-src="gfx/nn_depth_features.png" class="quarto-figure quarto-figure-center r-stretch"></section>
 <section id="two-observations" class="slide level2">
 <h2>Two observations</h2>
 <ol type="1">
@@ -559,8 +566,7 @@ <h2>Two observations</h2>
 <ul>
 <li>ReLU is the current fashion (used to be tanh or logistic)</li>
 </ul>
-
-<img data-src="21-nnets-intro_files/figure-revealjs/sigmoid-1.svg" class="quarto-figure quarto-figure-center r-stretch"></section>
+</section>
 <section id="next-time" class="title-slide slide level1 center">
 <h1>Next time…</h1>
 <p>How do we estimate these monsters?</p>