HKU DASC7606 Review

Motivation and Machine Learning
- The story behind Deep Learning (the imitation game)
- Supervised Learning, Classification & Regression Problems
- Linear Regression & Gradient Descent (Loss Function)
- Logistic Regression & Classification (Sigmoid)
- Performance Measures of Prediction
- Multi-classification (Softmax)
Neural Network Basics
- The Perceptron and Deep Neural Networks
- Power of NN: Computing Logic Functions and Arbitrary Functions
- Backpropagation
- Techniques to Improve Training
  - Learning Rate: Adam
  - Activation Functions: Sigmoid, Tanh, ReLU, ReLU-like, Softmax
  - Overfitting: Weight Regularisation, Dropout, Early Stopping
  - Mini Batches: Mini-batch SGD
  - Learning Curve: Bias and Variance
Computer Vision - CNN & ResNet
- CNN
- ImageNet: AlexNet 2012, ZFNet 2013, VGNet 2014, GoogLeNet 2014, ResNet 2015, ReduNet 2022
- Visualizing and Understanding CNN
- Object Localization
- Object Detection
  - Sliding Windows
  - R-CNN, Fast R-CNN, Faster R-CNN, YOLO
- Semantic Segmentation
  - Mask R-CNN
  - U-Net
- DeepDream and Image Style Transfer
- Adversatial Images
Deep Generative Models
- Autoencoders (with flu example and short diversion on network distillation)
- VAE: Variational Autoencoder
- GAN: Generative Adversarial Network
- Extensions of GAN: VAEGAN, C-GAN, Pix2Pix, CycleGAN
Natural Language Processing - Transformer & ChatGPT
- RNN and LSTM
- Language Model: Next word prediction
- Word Embeddings
  - Term-context Matrix, Co-occurrence Matrix
  - One Hot Vector, BOW, TF-IDF, Word2Vec(CBOW, Skip-gram), GloVe
  - Evaluation of Word Vectors
  - ELMo
- Machine Translation, Seq2Seq and Facebook's CNN
- Transformer - Attention
- BERT
- LLM, GPT, ChatGPT
- LLM and Finetuning (LORA), Text-to-image (Clip, Stable Diffusion)
Deep Reinforcement Learning
- Reinforcement Learning, Markov Decision Process, Q-Learning
- DQN and its variants
- Policy Learning, AlphaGo, AlphaGo Zero
- New Developments in Deep RL
Deep Issues in AI

感谢您的支持，我会继续努力的!