Artificial Intelligence : Notes
  • Supervised Learning
    • Trees
      • AdaBoost
      • ID3
      • Random Forests
    • Convolutional Neural Networks
    • DNN for Classification
    • K-Nearest Neighbors
    • LDA
    • Logistic Regression
    • Perceptron
      • Mathematical framework
    • QDA
    • SVM
  • Unsupervised Learning
    • DBSCAN
    • Deep Autoencoder
    • Generative Adversarial Networks (GAN)
    • K-Means Clustering
    • Linear Regression
    • Principal Component Analysis (PCA)
    • Restricted Boltzmann Machines (RBM)
  • Reinforcement Learning
    • Markov Decision Process
    • Q-Learning
    • Deep Q-Learning
  • Ensemble Strategies
    • Ensemble Learning
    • Fine-tuning and resampling
  • Other Techniques
    • Expectation-Maximization
    • Recurrent Neural Networks

Perceptron

A perceptron is a basic unit of a neural network and serves as a simple model of a biological neuron. It takes multiple binary inputs, applies weights to these inputs, sums them up, and passes the result through an activation function to produce an output. The perceptron is the fundamental building block for more complex neural network architectures. It was introduced by Frank Rosenblatt in the late 1950s and forms the basis of single-layer neural networks.

Mathematical framework

A perceptron is an artificial neuron with a Heaviside activation function.

perceptron.png | center | 500image source : https://towardsdatascience.com/perceptrons-the-first-neural-network-model-8b3ee4513757

A perceptron is just an indicator function that returns 0 or 1 depending on which side of an hyperplane the data is on.

Nowadays, activation functions are usually RELU or sigmoid.

Prev
Logistic Regression
Next
QDA