Perceptron, also known as single layer perceptrion, is a type of machine learning model that can be tracked back to the 1950s and 1960s. It is a simple algorithm for binary classification just like logistic regression. The reason we discuss perceptrons is that their core structural components, specifically linear transformations $\mathbf{w} \cdot \mathbf{x}+b$ and activation functions $\sigma$, are the fundamental compoent in modern neural networks.

Perceptron ● Origin

The original perceptron was developed for classification tasks. An perceptron with 3 inputs and 3 weights is visualized below.

Left Side (Expanded View):

• Inputs $x_1,x_2,x_3$: These represent the features or input variables. In the Iris dataset, these could be the petal and sepal lengths and widths.

• Weights $w_1, w_2, w_3$: These are the parameters that the perceptron learns during the training process. Each feature is multiplied by its corresponding weight.

• Bias $b$: This is an additional parameter that helps the perceptron adjust the decision boundary.

• Sum $Σ$: The perceptron calculates a weighted sum of the inputs and the bias:

  $$ z = w_1 \cdot x_1 + w_2 \cdot x_2 + w_3 \cdot x_3 + b $$

• Activation (Step Function $\sigma$): This is a threshold function that decides whether the perceptron outputs a "1" or a "0" based on the value of $z$. For example, if $z$ is greater than or equal to a threshold, the output is 1; otherwise, it is 0.

• Output $\hat{y}$: This is the final prediction. In the case of the Iris dataset, this could represent whether the flower belongs to one of two categories (e.g., Setosa or Versicolor).

Right Side (Simplified View):

• This view simplifies the operation of the perceptron by showing how the weights $w$ connect the inputs directly to the output. It abstracts away the detailed summation and step function shown in the expanded view.

Application to Iris Dataset:

The perceptron can be used to classify the flowers in the Iris dataset based on their features (e.g., sepal length, sepal width, petal length, and petal width). Since the original perceptron handles binary classification, you would typically use it to classify between two classes, such as "Setosa" vs. "Versicolor" or "Versicolor" vs. "Virginica."

In the case of the Iris dataset:

• Input: The four features of each flower (sepal length, sepal width, petal length, and petal width).
• Output: The predicted class (e.g., "Setosa" or "Versicolor").

A perceptron would learn the weights and bias during training by minimizing the classification error. Once trained, the perceptron can classify new flowers based on their features.

The full math equation of the above diagram is given as follows: