\documentclass{article}[12pt]

% useful packages
\usepackage{titlesec}
\usepackage{fullpage}
\usepackage{amsmath,amssymb,amsthm,amsfonts}
\usepackage{graphicx}
\usepackage{enumerate}
\usepackage{algorithm,algorithmic}
\usepackage{xcolor}
\usepackage{bbm}
\usepackage{url,hyperref}
\usepackage{dirtree}

% theorem type environments
\newtheorem{thm}{Theorem}
\newtheorem{prop}{Proposition}
\newtheorem{lemma}{Lemma}
\newtheorem{cor}{Corollary}
\newtheorem{defn}{Definition}
\newtheorem{assump}{Assumption}
\newtheorem{example}{Example}
\newtheorem{conjecture}{Conjecture}

% frequently used symbols
\newcommand{\bE}{\mathbb{E}}
\newcommand{\bP}{\mathbb{P}}
\newcommand{\bQ}{\mathbb{Q}}
\newcommand{\bR}{\mathbb{R}}
\newcommand{\bS}{\mathbb{S}}
\newcommand{\bN}{\mathbb{N}}
\newcommand{\bZ}{\mathbb{Z}}
\newcommand{\sC}{{\mathcal C}} 
\newcommand{\sD}{{\mathcal D}} 
\newcommand{\sE}{{\mathcal E}} 
\newcommand{\sF}{{\mathcal F}} 
\newcommand{\sL}{{\mathcal L}} 
\newcommand{\sH}{{\mathcal H}} 
\newcommand{\sN}{{\mathcal N}} 
\newcommand{\sO}{{\mathcal O}} 
\newcommand{\sP}{{\mathcal P}} 
\newcommand{\sR}{{\mathcal R}} 
\newcommand{\sS}{{\mathcal S}}
\newcommand{\sU}{{\mathcal U}} 
\newcommand{\sX}{{\mathcal X}} 
\newcommand{\sY}{{\mathcal Y}} 
\newcommand{\sZ}{{\mathcal Z}}
\newcommand{\dLdO}{{\partial L / \partial O}}

% operators
\newcommand{\sign}{\mathop{\mathrm{sign}}}
\newcommand{\supp}{\mathop{\mathrm{supp}}} % support
\newcommand{\argmin}{\operatornamewithlimits{arg\ min}}
\newcommand{\argmax}{\operatornamewithlimits{arg\ max}}
\newcommand{\dist}{\operatorname{dist}}
\newcommand{\tr}{\text{tr}}
\newcommand{\st}{\operatorname{s.t.}}
\newcommand{\cut}{\setminus}
\newcommand{\ind}[1]{\mathbbm{1}\left\{#1\right\}} 
\newcommand{\given}{\ | \ }

% grouping operators
\newcommand{\brac}[1]{\left[#1\right]}
\newcommand{\set}[1]{\left\{#1\right\}}
\newcommand{\abs}[1]{\left\lvert #1 \right\rvert}
\newcommand{\paren}[1]{\left(#1\right)}
\newcommand{\norm}[1]{\left\|#1\right\|}
\newcommand{\ip}[2]{\left\langle #1,#2 \right\rangle}

% header command
\newcommand{\homework}[3]{
    \pagestyle{myheadings}
    \thispagestyle{plain}
    \newpage
    \setcounter{page}{1}
    \setlength{\headsep}{10mm}
    \noindent
    \begin{center}
    \framebox{
        \vbox{\vspace{2mm}
            \hbox to 6.28in { {\bf EE 519: Deep Learning Theory \& Fundamentals
            \hfill Spring \the\year} }
        \vspace{4mm}
        \hbox to 6.28in { {\Large \hfill Homework #1 \hfill} }
        \vspace{2mm}
        \hbox to 6.28in { \Large \hfill Due: #2, 11:59PM PT \hfill }
        \vspace{2mm}
        \hbox to 6.28in { {\it Student Name: #3} \hfill {\it Instructor Name: John Lipor}}
        \vspace{2mm}}
   }
   \end{center}
   \markboth{Homework #1}{Homework #1}
   \vspace*{4mm}
}

% For problem titles
\titleformat{\section}{\normalfont\bf}{Problem \thesection}{1em}{}

\begin{document}

% PUT YOUR NAME BELOW WHERE I HAVE \X
\homework{2}{May 12, \the\year}{\X}
% PUT YOUR NAME ABOVE WHERE I HAVE \X

\section{\normalfont{\texttt{conv.py} (8 pts)}}

For the first problem of this assignment, you will complete a convolutional layer for your MyTorch package. Your task is to complete \texttt{conv.py} as detailed by the documentation in the corresponding file and the lecture notes. In particular, you will implement the forward and backward passes of a two-dimensional convolutional layer. For simplicity, we will ignore the bias term in this layer.

\textbf{Turn in} the output of the notebook \texttt{Conv2DTester.ipynb}.

\section{\normalfont{CNNs on CIFAR-10 (3 pts each)}}

The textbook focuses heavily on the Fashion-MNIST dataset since it is easy to quickly train a somewhat-accurate network on this dataset. However, this dataset may be too easy to develop an understanding of what works well and what doesn't in a CNN architecture. In this problem, we'll consider the more difficult (though still manageable) CIFAR-10 dataset, which is of similar size to Fashion-MNIST but has three input channels and is considered a much more challenging dataset. This
dataset can be accessed through \texttt{torchvision.datasets} in a manner similar to Fashion-MNIST.

Your task will be to train two CNNs on this dataset---one that utilizes a linear output layer and a second that uses global average pooling. You may make use of any available literature or blogs to gain insight into effective network architectures for this dataset. Your goal is to achieve the best accuracy you can while allowing time for the rest of the problems, which will utilize your networks. You \textbf{must} try both dropout and batch normalization for each architecture and report their impact on performance.

\begin{enumerate}[(a)]
    \item Implement a CNN in the vein of LeNet, AlexNet, or VGG that uses one or more convolutional layers, followed by one or more dense layers. Turn in:
        \begin{enumerate}
            \item a description of your final network architecture
            \item how you arrived at this architecture
            \item the impact of batch normalization
            \item the impact of dropout
            \item a plot of training and validation loss
            \item your final validation accuracy.
        \end{enumerate}
    \item Implement a CNN in the vein of NiN, ResNet, etc. that integrates nonlinearities within the convolutional portions and utilizes global average pooling. Turn in the same items as for part (a).
\end{enumerate}

\clearpage
\section{\normalfont{Feature Maps and Filters (3 pts each)}}

As discussed in class, interpreting CNNs is largely performed by visualizing their different elements. In this problem, your task is to visualize both the intermediate feature maps (outputs of convolutional layers) as well as some of the learned filters for each of the two networks trained in Problem 2. For each network, turn in:
\begin{enumerate}[(a)]
    \item images of subplots of feature maps for multiple convolutional layers. Each image should contain a number of subplots corresponding to the number of output channels for a given convolutional layer, and you should create one image per convolutional layer for no more than three layers.
    \item images of subplots of the learned filters for multiple convolutional layers, following the above specifications. Note whether your network ever learns filters similar to the well-known Gabor filters.
\end{enumerate}

\section{\normalfont{Class Activation Maps (5 pts)}}

In this problem, you will integrate class activation maps (CAMs) as defined in the paper \href{https://arxiv.org/pdf/1512.04150.pdf}{Learning Deep Features for Discriminative Localization}. You may use any code you find online, including the authors' own GitHub repository. The article \href{https://debuggercafe.com/pytorch-class-activation-map-using-custom-trained-model/}{here} may also be helpful.

\textbf{Turn in} some sample CAM images from each architecture developed in Problem 2. Be sure to display images that are correctly classified as well as incorrectly classified, as well as a brief interpretation of what the CAMs show.

\section{\normalfont{Guided Backpropagation (5 pts)}}

In this problem, you will produce saliency maps as defined in the paper \href{https://arxiv.org/pdf/1412.6806.pdf}{Striving for Simplicity: The All Convolutional Net}. You may use any code you find online. The article \href{https://www.coderskitchen.com/guided-backpropagation-with-pytorch-and-tensorflow/}{here} may also be helpful.

\textbf{Turn in} some sample saliency map images from each architecture developed in Problem 2. Be sure to display images that are correctly classified as well as incorrectly classified, as well as a brief interpretation of what the saliency maps show.

\section{\normalfont{Reflection on Datasets (5 pts)}}

Read Ch. 8 of the textbook \textit{Patterns, Predictions, and Actions} available \href{https://arxiv.org/pdf/2102.05242.pdf}{here}, which discusses datasets and their impact on machine learning. In the \texttt{interesting-reading} channel on Slack, state (1) one thing you learned from a technical perspective regarding datasets (i.e., from the material up to the section \textit{Harms associated with data}), (2) one thing you learned about the harms associated with data and how we
might overcome these harms, and (3) one question you have after reading the chapter.

\end{document}
