CCO - 726 - Introduction to Neural Networks

Total of Credits: 8
Hours for Theoretical Classes: 60
Hours for Exercises or Seminars: 60

Objective

Apresentar fundamentação sobre os principais modelos de Redes Neurais Artificiais existentes na literatura, e suas formalizações matemáticas, bem como seus respectivos modelos de aprendizado. Além disso, apresentar aplicações práticas dos modelos estudados. Ao final da disciplina, o estudante deve:

  • conhecer os conceitos fundamentais dos modelos de Redes Neurais Artificiais apresentados, tendo a capacidade de aplica-los em problemas reais;
  • ser capaz de explicar o funcionamento e aplicar os principais modelos de Redes Neurais Artificiais da Literatura;
  • ter conhecimento teórico para o desenvolvimento de métodos utilizando as Redes Neurais Artificias Vistas em aula.

(To present a reasoning about the main models of Artificial Neural Networks from the literature and their mathematical formalizations, as well as their respective learning models. In addition, it will be presented practical applications of the studied models. At the end of the course, the student should:

  • know the fundamental concepts of the Artificial Neural Network models presented, having the ability to apply them to real problems;
  • be able to explain the operation and apply the main models of Artificial Neural Networks in Literature;
  • have theoretical knowledge for the development of methods using the Artificial Neural Networks seen during the classes.)

Catalog Description

  1. Introduction
    1. What is an Artificial Neural Network
    2. Applications and Inspiration
    3. Types of learning and examples
    4. Benefits of the Artificial Neural Networks
    5. Model of Artificial Neuron
    6. Activation Functions
    7. Artificial Neural Network Architectures
    8. Knowledge Representation
    9. History
  2. Learning
    1. Learning Definition
    2. Learning by Error Correction
    3. Memory based Learning
    4. Hebbian Learning
    5. Competitive Learning
    6. Boltzmann Learning
  3. The Perceptron
    1. The Perceptron
    2. Perceptron Convergence Theory
    3. Exemplifying the Perceptron
    4. Practice: Perceptron
  4. The Multi-Layer Perceptron
    1. The Multi-Layer Perceptron (MLP)
    2. Training Modes
    3. The Backpropagation algorithm
    4. Activation Functions
    5. The XOR Problem
    6. Heuristics to Improve Performance
    7. Practice: MLP
  5. Radial Basis Function
    1. Introduction
    2. The Cover Theorem
    3. The XOR Problem
    4. The Interpolation Problem
    5. Radial Basis Function (RBF)
    6. K-means Clustering
    7. Summary of Training for an RBF
    8. Practice: RBF
  6. Self-Organizing Maps
    1. Introduction
    2. Models for Mapping and Characteristics
    3. Self-Organizing Maps (SOM)
    4. Competitive Process
    5. Cooperation Process
    6. Adaptation Process
    7. The SOM Algorithm
    8. Kohonen Maps Properties
    9. Practice: Kohonen Maps
  7. The Hopfield Model
    1. Introduction to the Hopfield Model
    2. Architecture
    3. Operation Phases
    4. Exemplification
    5. Practice: the Hopfield Model
  8. Restricted Boltzmann Machines
    1. Introduction to Statistical Mechanics
    2. Restricted Boltzmann Machine
    3. Inference
    4. Free Energy
    5. The Contrastive Divergence Algorithm
    6. Persistent Contrastive Divergence
    7. Exemplification
    8. Practice: Restricted Boltzmann Machine
  9. Autoencoders
    1. Introduction
    2. Loss Function
    3. Hidden Layers Under and Overcomplete
    4. Denoising Autoencoders
    5. Contractive Autoencoder
    6. Practice: Autoencoder
  10. Introduction to Deep Learning
    1. Introduction and Motivation
    2. Theoretical Justification
    3. Examples of Success
    4. Difficulty in Training Deep Neural Networks
    5. Pre-training and Fine Tuning
    6. Stacked Autoencoders and Stacked RBMs
    7. Deep Autoencoders
    8. Convolutional Neural Networks
    9. Practice

Main Bibliography

  1. Simon Haykin, Neural Networks and Learning Machines (3rd Edition) Prentice Hall • Cloth, 936 pp ©2009 ISBN: 978-0131471399.
  2. Simon Haykin, Neural Networks: A Comprehensive Foundation (3rd Edition) Prentice-Hall, Inc. Upper Saddle River, NJ, USA ©2007 ISBN:0131471392.
  3. Braga, Antônio; Ludermir, Teresa; Carvalho, André Redes Neurais Artificiais: Teoria e Aplicações. LTC, 2000. 262p.

Complementary Bibliography

  1. Hassoun, M. H. (1995). Fundamentals of artificial neural networks. MIT Press, Cambridge : MA.
  2. Zurada, J.M. (1992) Introduction to Artificial Neural Systems, Publisher : West Pub. Co, US.
  3. Hertz, J., Krogh, A., and Palmer, R.G. (1991). Introduction to the theory of neural computation. AddisonWesley Publishing Company, Redwood City, CA.
  4. Aleksander, I. e Morton, H. (1995). An Introduction to Neural Computing, 2 Ed., International Thomson Editions.