1) Neural Networks and Representation Learning

Neural Network:

• A neural Network is a type of Machine Learning Model inspired by the human brain
• Deep Learning uses a neural network for learning the model
• There are so many types of neural networks. Some of them are:
  • a) ANN
  • b) CNN
  • c) RNN

Representation Learning:

• The process of extracting important features from the raw data by the neural network is called as Representation Learning
• Representation Learning is also called as Feature Learning

2) Convolutional Operation, Filters in CNN

• Convolutional Operation is the mathematical operation used in CNN to extract the important features from the raw data

Single channel Convolution vs Multi-Channel Convolution

Single Channel Convolution	Multi-Channel Convolution
It uses a single channel to process the input in the CNN	It uses multiple channels to process the input in the CNN
Input Data: 2D Matrix	Input Data: 3D Tensor
Kernel: 2D Filter	Kernel: 3D Filter
Output: 2D Feature Map	Output: 3D Feature Map
It uses a few Parameters	It uses more parameters
Low Cost	High Cost

Filters in CNN:

• Filters are also called as Kernels
• It is a 2D matrix or 3D Tensor that slides over the input
• It automatically detects important features like:
  • Height
  • Width
  • Edges
• 32 Filters -> 32 Feature Maps
• 64 Filters -> 64 Feature Maps

3) Strides and Padding in CNN

Stride:

• The process of moving the kernel horizontally and vertically at each step during a convolution operation is called the stride
• The size of the feature map decreases because of the stride
• Loss of Information
• (5 X 5) X (3 X 3) -> (2 X 2)

Padding:

• The process of adding the pixels around the border of the input image is called as Padding
• The size of the feature map is equal to the input size
• prevent loss of information
• (5 X 5) X (3 X 3) -> (5 X 5)

4) Weight Sharing and Uses of CNN

Weight Sharing:

• The process of using the same filters across different parts of the input image is called as Weight Sharing
• Uses of Weight Sharing:
  • Few Parameters
  • Trains the model quickly
  • Reduce Overfitting
  • Translation-invariant

Uses of CNN:

• They Extract Invariant Features
• They Extract Hierarchical Features

• CNNs are faster and efficient when compared to other neural networks

5) BPTT Algorithm in RNN

• BPTT stands for BackPropagation Through Time
• It is used to update the weights to reduce the errors

steps:

6) GRU vs LSTM in RNN

GRU	LSTM
GRU stands for Gated Recurrent Unit	LSTM stands for Long Short-Term Memory
GRU is an Advanced type of RNN	LSTM is an Advanced type of RNN
GRU is used to solve the Vanishing Gradient Problem	LSTM is used to solve the Vanishing Gradient Problem
There are mainly two types of gates used in GRU. They are: Update Gate Reset Gate	There are mainly three types of gates used in LSTM. They are: Forget Gate Input Gate Output Gate
There is no Cell State	There is a Cell State in LSTM
It uses a few Parameters	It uses more parameters
Low Cost	High Cost

7) PyTorch vs TensorFlow