310 Updated PDF to follow Edward Tufte style

_quarto.yml

@@ -49,7 +49,7 @@ book:
 
   page-navigation: true
   title: "Machine Learning Systems"
-  subtitle: "with tinyML"
+#  subtitle: "with tinyML"
   abstract: "{{< var title.long >}} offers readers an entry point to understand machine learning (ML) systems by grounding concepts in applied ML. As the demand for efficient and scalable ML solutions grows, the ability to construct robust ML pipelines becomes increasingly crucial. This book aims to demystify the process of developing complete ML systems suitable for deployment, spanning key phases like data collection, model design, optimization, acceleration, security hardening, and integration, all from a systems perspective. The text covers a wide range of concepts relevant to general ML engineering across industries and applications, using TinyML as a pedagogical tool due to its global accessibility. Readers will learn basic principles around designing ML model architectures, hardware-aware training strategies, performant inference optimization, and benchmarking methodologies. The book also explores crucial systems considerations in areas like reliability, privacy, responsible AI, and solution validation. Enjoy reading it!"
 
   repo-url: https://github.com/harvard-edge/cs249r_book
@@ -185,11 +185,31 @@ comments:
   giscus:
     repo: harvard-edge/cs249r_book
 
+crossref:
+  appendix-title: "Appendix"
+  appendix-delim: ":"
+
+  custom:
+    - kind: float
+      reference-prefix: Lab
+      key: labq
+      latex-env: lab
+
+    - kind: float
+      reference-prefix: Exercise
+      key: exr
+      latex-env: exr
+
+    - kind: float
+      reference-prefix: Video
+      key: vid
+      latex-env: vid
+
+editor:
+  render-on-save: true
+
 format:
   html:
-    reference-location: margin
-    citation-location: margin
-
     theme: 
       light:
         - spacelab
@@ -199,8 +219,10 @@ format:
         - darkly
         - style.scss
         - style-dark.scss
-    mainfont: Nunito
-    fontsize: 1rem
+    reference-location: margin
+    citation-location: margin
+    citeproc: true
+    sidenote: true  # Enable sidenotes for Tufte style
     linkcolor: "#A51C30"
     urlcolor: "#A51C30"
     highlight-style: github
@@ -215,7 +237,7 @@ format:
     fig-height: 6
     number-depth: 3
     toc: true
-    toc-depth: 4	
+    toc-depth: 4
     include-in-header:
       text: | 
         <script src="/scripts/welcome/confetti.js" defer></script>
@@ -224,123 +246,131 @@ format:
         <script src="/scripts/ai_menu/dist/142.bundle.js" defer></script>
         <script src="/scripts/ai_menu/dist/384.bundle.js" defer></script>
         <script src="/scripts/ai_menu/dist/761.bundle.js" defer></script>
-
-    citeproc: true
-
+
   pdf:
+    reference-location: margin
+    citation-location: block
+    title-block-style: none
+    indent: 0px
     documentclass: scrbook
-    classoption: [abstract]
+    classoption: [abstract,openleft,sfsidenotes]
     keep-tex: true
     toc: true  # Table of Contents
     toc-depth: 3  # Depth of headings to include in TOC
     number-sections: true
     latex-engine: xelatex  # or pdflatex, lualatex, etc.
-    geometry: margin=1in
     fontsize: 10pt
-    papersize: letter  # or a4, etc.
+    papersize: "7.5in x 9.25in"  # Set to specific textbook size
+    geometry: 
+      - top=1in
+      - bottom=1in
+      - inner=1in
+      - outer=2.25in
+      - marginparwidth=1.5in
     fig_caption: true
     link-citations: true
     citation_package: natbib  # or biblatex
-    fig-cap-location: bottom
-    tbl-cap-location: top
+    fig-cap-location: margin
+    tbl-cap-location: margin
+    hyperrefoptions:
+      - linktoc=all
+      - pdfwindowui
+      - pdfpagemode=FullScreen
 
     include-in-header:
       text: |
+        \usepackage{geometry}
+        \geometry{
+          paperwidth=7.5in,
+          paperheight=9.25in,
+          top=1in,
+          bottom=1in,
+          inner=1in,
+          outer=2.25in,
+          marginparwidth=1.5in,
+        }
+        \usepackage{imakeidx}
+        \usepackage[acronym]{glossaries}
         \usepackage{fancyhdr}
         \usepackage{graphicx}
         \usepackage{mathptmx}
         \usepackage{fontspec}
         \usepackage{underscore}
         \usepackage[english]{babel}
         \usepackage{etoolbox}
-        \usepackage{fontspec}
         \usepackage{newpxtext} % Palatino-like font
-        \usepackage{hyperref} % For hyperlinks
+        \usepackage{hyperref}
         \usepackage{xcolor}
-        \usepackage[format=plain,
-            labelfont={bf,it},
-            textfont=it, labelsep=space]{caption}
-
+        \usepackage[format=plain, labelfont={bf,it}, textfont=it, labelsep=space]{caption}
         \definecolor{crimson}{RGB}{165, 28, 48}
-
         \hypersetup{
-          colorlinks=true, % Enable colored links
-          linkcolor=crimson, % Color of internal links
-          citecolor=crimson, % Color of citations
-          urlcolor=crimson % Color of URLs
+          linkcolor=crimson,
+          citecolor=crimson,
+          urlcolor=crimson
         }
-
+        \usepackage{marginfix} % Fixes the issue of margin notes being cut off
+        \usepackage{marginnote}
+
         \patchcmd{\chapter}{\thispagestyle{plain}}{\thispagestyle{fancy}}{}{}
 
-        %\newfontfamily\tocfont{Times New Roman}
+        % Disable the default title page
+        \renewcommand{\maketitle}{}
 
         \let\endtitlepage\relax
 
         \AtBeginDocument{
+          \newgeometry{top=1in,bottom=1in,inner=1in,outer=1in} % Geometry for title page
           \begin{titlepage}
           \centering
           \vspace{-3em}
           \includegraphics[width=\textwidth]{cover-image-white.png} % Adjust the size and path to your image
 
-          {{\Huge\bfseries Machine Learning Systems}\\[1em] \Large with TinyML\par}
+          {{\Huge\bfseries Machine Learning Systems}\\[1em] \par}
 
           \vspace*{\fill}
             {\large Written, edited and curated by \\[.2cm] Prof. Vijay Janapa Reddi \\[.2cm] Harvard University \\[1em] \normalsize {\itshape With special thanks to the community for their contributions and support.} \\[1em] \pagebreak \vfill \scriptsize Last Modified: \today\par \vfill}
           \vspace*{\fill}
 
           \end{titlepage}
+          \restoregeometry % Restore original geometry
 
           %\addtocontents{toc}{\tocfont}
         }
 
         \let\endtitlepage\relax
 
         \pagestyle{fancy}
-        \fancyhf{} % Clear all header and footer fields
-        \fancyhead[LE,RO]{\thepage} % Page number on the left on even pages, right on odd pages
-        \fancyhead[RE,LO]{\nouppercase{\leftmark}} % Chapter name on both sides
+        \fancyhf{}
+        \fancyhead[LE]{\color{crimson}\nouppercase{\rightmark}}
+        \fancyhead[RO]{\color{crimson}\thepage}
+        \fancyhead[LO]{\color{crimson}\MakeUppercase{\leftmark}}
+        \fancyhead[RE]{\color{crimson}\thepage}
         \renewcommand{\headrulewidth}{0.4pt}
         \renewcommand{\footrulewidth}{0pt}
-
         \fancypagestyle{plain}{%
-          \fancyhf{} % clear all header and footer fields
-          \fancyhead[LE,RO]{\thepage} % Page number
+          \fancyhf{}
+          \fancyhead[LE,RO]{\color{crimson}\thepage}
           \renewcommand{\headrulewidth}{0.4pt}
           \renewcommand{\footrulewidth}{0pt}
         }
-
         \addtokomafont{disposition}{\rmfamily\color{crimson}}
         \addtokomafont{chapter}{\color{crimson}}
         \addtokomafont{section}{\color{crimson}}
-
-        % Define the abstract environment
+        \addtokomafont{subsection}{\color{crimson}}
         \newenvironment{abstract}{%
             \chapter*{\abstractname}%
             \addcontentsline{toc}{chapter}{\abstractname}%
             \small
         }{%
             \clearpage
         }
-
-crossref:
-  appendix-title: "Appendix"
-  appendix-delim: ":"
-
-  custom:
-    - kind: float
-      reference-prefix: Lab
-      key: labq
-      latex-env: lab
 
-    - kind: float
-      reference-prefix: Exercise
-      key: exr
-      latex-env: exr
+        \makeindex
+        \makeglossaries
 
-    - kind: float
-      reference-prefix: Video
-      key: vid
-      latex-env: vid
-
-editor:
-  render-on-save: true
+    include-after-body:
+      text: |
+        \clearpage
+        \printglossary[type=main, title=Glossary, toctitle=Glossary]
+        \clearpage
+        \printindex

contents/ai_for_good/ai_for_good.qmd

@@ -290,3 +290,6 @@ In addition to exercises, we offer a series of hands-on labs allowing students t
 
 * _Coming soon._
 :::
+
+
+

contents/data_engineering/data_engineering.qmd

@@ -541,3 +541,6 @@ In addition to exercises, we offer a series of hands-on labs allowing students t
 
 * _Coming soon._
 :::
+
+
+

contents/frameworks/frameworks.qmd

@@ -795,3 +795,6 @@ In addition to exercises, we offer a series of hands-on labs allowing students t
 
 * _Coming soon._
 :::
+
+
+

contents/introduction/introduction.qmd

@@ -12,7 +12,7 @@ In the early 1990s, [Mark Weiser](https://en.wikipedia.org/wiki/Mark_Weiser), a
 
 ![Ubiqutous computing.](images/png/21st_computer.png){#fig-ubiqutous width=50%}
 
-In the vision of ubiquitous computing[@weiser1991computer], the integration of processors into everyday objects is just one aspect of a larger paradigm shift. The true essence of this vision lies in creating an intelligent environment that can anticipate our needs and act on our behalf, enhancing our experiences without requiring explicit commands. To achieve this level of pervasive intelligence, it is crucial to develop and deploy machine learning systems that span the entire ecosystem, from the cloud to the edge and even to the tiniest IoT devices.
+In the vision of ubiquitous computing [@weiser1991computer], the integration of processors into everyday objects is just one aspect of a larger paradigm shift. The true essence of this vision lies in creating an intelligent environment that can anticipate our needs and act on our behalf, enhancing our experiences without requiring explicit commands. To achieve this level of pervasive intelligence, it is crucial to develop and deploy machine learning systems that span the entire ecosystem, from the cloud to the edge and even to the tiniest IoT devices.
 
 By distributing machine learning capabilities across the computing continuum, we can harness the strengths of each layer while mitigating their limitations. The cloud, with its vast computational resources and storage capacity, is ideal for training complex models on large datasets and performing resource-intensive tasks. Edge devices, such as gateways and smartphones, can process data locally, enabling faster response times, improved privacy, and reduced bandwidth requirements. Finally, the tiniest IoT devices, equipped with machine learning capabilities, can make quick decisions based on sensor data, enabling highly responsive and efficient systems.
 

contents/ml_systems/ml_systems.bib

@@ -10,7 +10,7 @@ @misc{armcomfuture
 
 @article{lin2023tiny,
  author = {Lin, Ji and Zhu, Ligeng and Chen, Wei-Ming and Wang, Wei-Chen and Han, Song},
- title = {Tiny Machine Learning: {Progress} and Futures {[Feature]}},
+ title = {Tiny Machine Learning: Progress and Futures Feature},
  journal = {IEEE Circuits Syst. Mag.},
  volume = {23},
  number = {3},

contents/ml_systems/ml_systems.qmd

@@ -28,7 +28,7 @@ As this chapter progresses, we will explore embedded systems' complex and fascin
 
 :::
 
-## Machine Learning Systems
+## Introductions
 
 ML is rapidly evolving, with new paradigms reshaping how models are developed, trained, and deployed. One such paradigm is embedded machine learning, which is experiencing significant innovation driven by the proliferation of smart sensors, edge devices, and microcontrollers. Embedded machine learning refers to the integration of machine learning algorithms into the hardware of a device, enabling real-time data processing and analysis without relying on cloud connectivity. This chapter explores the landscape of embedded machine learning, covering the key approaches of Cloud ML, Edge ML, and TinyML (@fig-cloud-edge-tinyml-comparison).
 
@@ -38,7 +38,18 @@ ML began with Cloud ML, where powerful servers in the cloud were used to train a
 
 @fig-vMLsizes shows the key differences between Cloud ML, Edge ML, and TinyML in terms of hardware, latency, connectivity, power requirements, and model complexity. This significant disparity in available resources poses challenges when attempting to deploy deep learning models on microcontrollers, as these models often require substantial memory and storage. For instance, widely used deep learning models such as ResNet-50 exceed the resource limits of microcontrollers by a factor of around 100, while more efficient models like MobileNet-V2 still surpass these constraints by a factor of approximately 20. Even when quantized to use 8-bit integers (int8) for reduced memory usage, MobileNetV2 requires more than 5 times the memory typically available on a microcontroller, making it difficult to fit the model on these tiny devices.
 
-![From cloud GPUs to microcontrollers: Navigating the memory and storage landscape across computing devices. Credit: @lin2023tiny.](./images/jpg/cloud_mobile_tiny_sizes.jpg){#fig-vMLsizes}
+![From cloud GPUs to microcontrollers: Navigating the memory and storage landscape across computing devices. Credit: [@lin2023tiny]](./images/jpg/cloud_mobile_tiny_sizes.jpg){#fig-vMLsizes}
+
+<!-- ```{r}
+#| label: fig-vMLsizes
+#| fig-cap: "From cloud GPUs to microcontrollers: Navigating the memory and storage landscape across computing devices. Credit: @lin2023tiny."
+#| fig-cap-location: margin
+#| echo: false
+#| warning: FALSE
+#| message: FALSE 
+
+knitr::include_graphics("./images/jpg/cloud_mobile_tiny_sizes.jpg")
+``` -->
 
 ## Cloud ML
 

contents/ops/ops.qmd

@@ -975,3 +975,4 @@ In addition to exercises, we also offer a series of hands-on labs that allow stu
 
 * _Coming soon._
 :::
+

contents/optimizations/optimizations.qmd

@@ -1009,3 +1009,4 @@ In addition to exercises, we also offer a series of hands-on labs that allow stu
 
 * *Coming soon.*
 :::
+

contents/privacy_security/privacy_security.qmd

@@ -1150,3 +1150,4 @@ In addition to exercises, we offer a series of hands-on labs allowing students t
 
 * _Coming soon._
 :::
+

contents/responsible_ai/responsible_ai.qmd

@@ -502,3 +502,5 @@ In addition to exercises, we offer a series of hands-on labs allowing students t
 
 * _Coming soon._
 :::
+
+

contents/robust_ai/robust_ai.qmd

@@ -1067,3 +1067,4 @@ In addition to exercises, we offer a series of hands-on labs allowing students t
 
 * _Coming soon._
 :::
+

contents/sustainable_ai/sustainable_ai.qmd

@@ -708,3 +708,4 @@ In addition to exercises, we offer hands-on labs that allow students to gain pra
 
 * _Coming soon._
 :::
+

contents/training/training.qmd

@@ -1167,3 +1167,5 @@ In addition to exercises, we offer a series of hands-on labs allowing students t
 
 * _Coming soon._
 :::
+
+