Unit 8, 9, 10

PCP.ipynb

@@ -164,7 +164,7 @@
     "<tr text-align=\"left\" style=\"border:1px solid #C8C8C8\">\n",
     "    <td style=\"border:none; text-align:left\"><strong>10</strong></td>\n",
     "    <td style=\"border:none; text-align:left\"><a href=\"PCP_module.html\">Python Modules and Packages</a></td> \n",
-    "    <td style=\"border:none; text-align:left\">Python modules; Python packages; libpcp; documentation; docstring</td>      \n",
+    "    <td style=\"border:none; text-align:left\">Python modules; Python packages; libpcp; documentation; docstring; sample solutions to exercises</td>      \n",
     "    <td style=\"border:none; text-align:left\"><a href=\"PCP_module.html\">[html]</a></td>      \n",
     "    <td style=\"border:none; text-align:left\"><a href=\"PCP_module.ipynb\">[ipynb]</a></td>\n",
     "</tr>    \n",

PCP_dft.ipynb

@@ -15,9 +15,8 @@
    "source": [
     "# Unit 9: Discrete Fourier Transform (DFT)\n",
     "\n",
-    "The Fourier transform is one of the most important tools for a wide range of applications in engineering and computer science. In this notebook, we have a look at a discrete variant of the Fourier transform known as <strong>Discrete Fourier Transform</strong> (DFT). Furthermore, we introduce and implement the <strong>fast Fourier transform</strong> (FFT), which is an efficient algorithm for computing the DFT. This notebook closely follows the [FMP Notebook on the Discrete Fourier Transform](https://www.audiolabs-erlangen.de/resources/MIR/FMP/C2/C2_ComplexNumbers.html).\n",
-    "\n",
     "<ul> \n",
+    "    <li><a href='#learn'>Overview and Learning Objectives</a></li>    \n",
     "    <li><a href='#inner'>Inner Product</a></li>\n",
     "    <li><a href='#dft'>Definition of DFT</a></li>\n",
     "    <li><a href='#dftmatrix'>DFT Matrix</a></li>\n",
@@ -26,8 +25,21 @@
     "    <li><a href='#exercise_missing_time'>Exercise 2: Missing Time Localization</a></li>    \n",
     "    <li><a href='#exercise_chirp'>Exercise 3: Chirp Signal</a></li>   \n",
     "    <li><a href='#exercise_inverse'>Exercise 4: Inverse DFT</a></li>    \n",
-    "</ul>\n",
-    "\n"
+    "</ul>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "<a id='learn'></a> \n",
+    "<div class=\"alert alert-block alert-warning\">\n",
+    "<h2>Overview and Learning Objectives</h2>\n",
+    "\n",
+    "    \n",
+    "The <strong>Fourier transform</strong> is one of the most important tools for a wide range of engineering and computer science applications. The general idea of <strong>Fourier analysis</strong> is to decompose a given signal into a weighted superposition of sinusoidal functions. Since these functions possess an explicit physical meaning regarding their frequencies, the decomposition is typically more accessible for subsequent processing steps than the original signal. Assuming that you are familiar with the Fourier transform and its applications in signal processing, we review in this unit the discrete variant of the Fourier transform known as <strong>Discrete Fourier Transform</strong> (DFT). We define the inner product that allows for comparing two vectors (e.g., discrete-time signals of finite length). The DFT can be thought of as comparing a given signal of finite length with a specific set of exponential signals (a complex variant of sinusoidal signals), each comparison yielding a complex-valued Fourier coefficient. Then, using suitable visualizations, we show how you can interpret the amplitudes and phases of these coefficients. Recall that one can express the DFT as a complex-valued square matrix. We show how separately plotting the real and imaginary parts leads to beautiful and insightful images. Applying a DFT boils down to computing matrix&ndash;vector product, which we implement via the standard NumPy function <code>np.dot</code>. Since the number of operations for computing a DFT via a simple matrix&ndash;vector product is quadratic in the input length, the runtime of this approach becomes problematic with increasing length. This issue is exactly where the fast Fourier transform (FFT) comes into the game. We present this famous divide-and-conquer algorithm and provide a Python implementation. Furthermore, we compare the runtime behavior between the FFT implementation and the naive DFT implementation. We will further deepen your understanding of the Fourier transform by considering further examples and visualization in the exercises. In <a href='#exercise_freq_index'>Exercise 1</a>, you will learn how to interpret and plot frequency indices in a physically meaningful way. In <a href='#exercise_missing_time'>Exercise 2</a>, we discuss the issue of loosing time information when applying the Fourier transform, which is the main motivation for the <a href='https://www.audiolabs-erlangen.de/resources/MIR/FMP/C2/C2_STFT-Basic.html'>short-time Fourier transform</a>. In <a href='#exercise_chirp'>Exercise 3</a>, you will apply the DFT to a <strong>chirp signal</strong>, which yields another illustrative example of the DFT's properties. Finally, in <a href='#exercise_inverse'>Exercise 4</a>, we will invite you to explore the relationship between the DFT and its inverse. Again, an overarching of this unit is to apply and deepen your Python programming skills within the context of a central topic for signal processing. \n",
+    "    \n",
+    "</div>"
    ]
   },
   {
@@ -37,13 +49,19 @@
     "<a id='inner'></a> \n",
     "## Inner Product\n",
     "\n",
-    "In this notebook, we consider [discrete-time (DT) signals](PCP_Signal.html) of finite length $N\\in\\mathbb{N}$, which we represent as vector $x=(x(0),x(1),...,x(N-1))^\\top\\in\\mathbb{R}^N$ with samples $x(n)\\in\\mathbb{R}^N$ for $n\\in[0:N-1]$. Note that we start indexing with the index $0$. A general concept for comparing two vectors (or signals) is the **inner product**. Given two vectors $x, y \\in \\mathbb{R}^N$, the inner product between $x$ and $y$ is defined as follows:\n",
+    "In this notebook, we consider [discrete-time (DT) signals](PCP_Signal.html) of finite length $N\\in\\mathbb{N}$, which we represent as vector \n",
+    "\n",
+    "$$\n",
+    "x=(x(0),x(1),...,x(N-1))^\\top\\in\\mathbb{R}^N\n",
+    "$$ \n",
+    "\n",
+    "with samples $x(n)\\in\\mathbb{R}^N$ for $n\\in[0:N-1]$. Note that $\\top$ indicates the transpose of a vector, thus converting a row vector into a column vector. Furthermore, note that we start indexing with the index $0$ (thus adapting our mathematical notation to Python conventions). A general concept for comparing two vectors (or signals) is the **inner product**. Given two vectors $x, y \\in \\mathbb{R}^N$, the inner product between $x$ and $y$ is defined as follows:\n",
     "\n",
     "$$ \n",
     "\\langle x | y \\rangle := \\sum_{n=0}^{N-1} x(n) y(n).\n",
     "$$\n",
     "\n",
-    "The absolute value of the inner product may be interpreted as a measure of similarity between $x$ and $y$: if $x$ and $y$ point to the same direction (i.e., $x$ and $y$ are similar), the inner product $|\\langle x | y \\rangle|$ is large. If $x$ and $y$ are orthogonal (i.e., $x$ and $y$ are dissimilar), the inner product $|\\langle x | y \\rangle|$ is zero.\n",
+    "The absolute value of the inner product may be interpreted as a measure of similarity between $x$ and $y$. If $x$ and $y$  are similar (i.e., if they point to more or less the same direction), the inner product $|\\langle x | y \\rangle|$ is large. If $x$ and $y$ are dissimilar (i.e., if $x$ and $y$ are more or less orthogonal to each other), the inner product $|\\langle x | y \\rangle|$ is close to zero.\n",
     "\n",
     "One can extend this concept to **complex-valued** vectors $x,y\\in\\mathrm{C}^N$, where the inner product is defined as \n",
     "\n",
@@ -154,7 +172,7 @@
     "* The signal $x_1$ is similar to itself, leading to a large value of $\\langle x_1 | x_1 \\rangle=40.0$.\n",
     "* The overall course of the signal $x_1$ strongly correlates with the sinusoid $x_2$, which is reflected by a relatively large value of $\\langle x_1 | x_2 \\rangle=29.9$.\n",
     "* There are some finer oscillations of $x_1$ that are captured by $x_3$, leading to a still noticeable value of  $\\langle x_1 | x_3 \\rangle=14.7$. \n",
-    "* The two sinusoids $x_2$ and $x_3$ are not correlated at all, which is revealed by the value of $\\langle x_2 | x_3 \\rangle=0$. \n",
+    "* The two sinusoids $x_2$ and $x_3$ are more or less uncorrelated, which is revealed by the value of $\\langle x_2 | x_3 \\rangle\\approx 0$. \n",
     "\n",
     "In other words, the above comparison reveals that the signal $x_1$ has a strong signal component of $2~\\mathrm {Hz}$ (frequency of $x_2$) and $6~\\mathrm {Hz}$ (frequency of $x_3$). Measuring correlations between an arbitrary signal and sinusoids of different frequencies is exactly the idea of performing a Fourier (or spectral) analysis. "
    ]
@@ -180,7 +198,7 @@
     "\n",
     "This vector can be regarded as a [sampled version](PCP_signal.html) of the [exponential function](PCP_exp.html) of frequency $k/N$. Using inner products, the DFT can be expressed as\n",
     "\n",
-    "$$ X(k) = \\sum_{n=0}^{N-1} x(n) \\overline{\\mathbf{e}_k} = \\langle x | \\mathbf{e}_k \\rangle,$$\n",
+    "$$ X(k) = \\sum_{n=0}^{N-1} x(n) \\overline{\\mathbf{e}_k}(n) = \\langle x | \\mathbf{e}_k \\rangle,$$\n",
     "\n",
     "thus measuring the similarity between the signal $x$ and the sampled exponential functions $\\mathbf{e}_k$. The absolute value $|X(k)|$ indicates the degree of similarity between the signal $x$ and $\\mathbf{e}_k$. In the case that $x\\in \\mathbb{R}^N$ is a real-valued vector (which is typically the case for audio signals), we obtain:\n",
     "\n",
@@ -698,11 +716,11 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Runtime (ms) for N =  256 : DFT      12.14,  FFT 0.01498,  FFT_np 0.00001375\n",
-      "Runtime (ms) for N =  512 : DFT      33.29,  FFT 0.02594,  FFT_np 0.00001610\n",
-      "Runtime (ms) for N = 1024 : DFT     118.72,  FFT 0.04788,  FFT_np 0.00002885\n",
-      "Runtime (ms) for N = 2048 : DFT     421.35,  FFT 0.09456,  FFT_np 0.00004455\n",
-      "Runtime (ms) for N = 4096 : DFT    1543.14,  FFT 0.18793,  FFT_np 0.00010600\n"
+      "Runtime (ms) for N =  256 : DFT      23.99,  FFT 0.01239,  FFT_np 0.00001220\n",
+      "Runtime (ms) for N =  512 : DFT      31.06,  FFT 0.02377,  FFT_np 0.00001618\n",
+      "Runtime (ms) for N = 1024 : DFT     105.50,  FFT 0.04329,  FFT_np 0.00002411\n",
+      "Runtime (ms) for N = 2048 : DFT     381.55,  FFT 0.08357,  FFT_np 0.00004250\n",
+      "Runtime (ms) for N = 4096 : DFT    1441.13,  FFT 0.15967,  FFT_np 0.00008037\n"
      ]
     }
    ],

PCP_module.ipynb

@@ -15,16 +15,28 @@
    "source": [
     "# Unit 10: Python Modules and Packages\n",
     "\n",
-    "This notebook gives a short introduction to Python packages and Python modules. Furthermore, we explain the structure of <code>libpcp</code>, the Python package that accompanies the PCP notebooks.\n",
-    "\n",
     "<ul>\n",
+    "    <li><a href='#learn'>Overview and Learning Objectives</a></li>    \n",
     "    <li><a href='#python_modules'>Python Modules</a></li>\n",
     "    <li><a href='#python_packages'>Python Packages</a></li>\n",
     "    <li><a href='#lib_PCP'>The Package <code>libpcp</code> </a></li>\n",
     "    <li><a href='#doc_functions'>Documentation of Functions</a></li>    \n",
     "</ul>    "
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "<a id='learn'></a> \n",
+    "<div class=\"alert alert-block alert-warning\">\n",
+    "<h2>Overview and Learning Objectives</h2>\n",
+    "\n",
+    "This final unit of the PCP notebooks serves several purposes. First, we give a general introduction to  <a href='#python_modules'>Python modules</a> and <a href='#python_packages'>Python packages</a>, which are fundamental concepts for organizing and making Python code available. Second, we introduce the Python package <code>libpcp</code> (that accompany the PCP notebooks) and use this package as a concrete example for illustrating the Python concepts. At the same time, this unit (together with <a href='https://www.audiolabs-erlangen.de/PCP/PCP_getstarted.html'>Unit 1</a>) also documents the technical backbone underlying the PCP notebooks. Last but not least, we will also uncover in this unit the secret of where one can find the sample solutions for all exercises. In summary, we hope that the PCP notebooks help students naturally transition from learning about Python programming and signal processing to beginning independent research following good scientific practices. Another main motivation of the notebooks is to indirectly guide students to employ open-source tools for software development and reproducible research. \n",
+    "\n",
+    "</div>"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},