2_1background.tex

\chapter{Background}
Background here



Efforts :
Breaking the Curse of Dimensionality with Convex Neural Networks

Notes : - 
We consider neural networks with a single hidden layer and non-decreasing positively homogeneous activation functions like the rectified linear units.

By letting the number of
hidden units grow unbounded and using classical non-Euclidean regularization tools on
the output weights, they lead to a convex optimization problem


\textbf{H: Non-decreasing activation function was a very important choice bcz of it favours to preserve convexity. [non-negative weighted sums of a convex function are convex]}





\section{Why Neural networks}

\section{Seperation Theorem}

\section{Optimization of Neural networks - Convex or Non-Convex}

\subsection{Depth}

\subsection{Width}

\subsection{Initialization}
Why initialization matters in NN w.r.t  Non-convex problem. 
\subsection{Objective function}
\subsection{Most Popular: Pre-training - Fine-tuning Vs Transfer learning}
What is Pretraining?
What is Fine-tuning?
What is Transfer learning?
what is convex and when it becomes non-convex ?
Pretraining Populairty
-Finetuning, Transferlearning , 
When to use which methods multiple cases
Proof of convexity in pretraining
Follow the gradient
Roots of pretrainign from Boosting