In this section, we will explore how to view neural networks as operating modes of logic gates. This exploration is crucial for a deeper understanding of how neural networks make final decision classifications based on the presence or absence of various features, as well as the extent of these features.

Basic Assumptions and Model Significance: Firstly, we assume that the inputs and outputs of the neural network are limited to 0 or 1. This assumption is vital for our model. Without this condition, the logic gate model loses its fundamental meaning and practical value.

Neural Network and Logical Functions: To prove that neural networks can support any logical function, the key is to demonstrate that neural networks can implement basic logic gates - NOT and OR gates. Once these two gates are implemented, we can build AND gates and other complex logic modules, such as multiplexers (MUX) and demultiplexers (DEMUX). Additionally, we must establish a bypass for the Boolean number to pass through without being altered.

Logic Gates via Neural Networks

This proof is actually quite straightforward, mainly through constructing specific neurons and verifying whether there are weights ($w$) and biases ($b$) that satisfy the conditions.

OR Gate

Let's first consider the following neuron model $w_1=100, w_2=100, b=-50$:

$$ y = \sigma(100(x_1 + x_2 - 0.5))=\sigma(100x_1 + 100x_2 - 50) $$

When the inputs $x_1,x_2=0,0$, the value inside the activation function $\sigma$ is about -50, thus $\sigma(50) \approx 1$. When the inputs are $x_1, x_2 = 0, 1$ or $x_1, x_2 = 1, 0$, the value inside $\sigma$ is about 50, hence $\sigma(50) \approx 1$. If $x_1, x_2 = 1, 1$, the value in $\sigma$ is 150, i.e., $\sigma(-50) \approx 0$. The results of this truth table are consistent with the operation of an OR gate.

NOT Gate

Next, consider another neuron $w_1=-100, b=50$,

$$ y = \sigma(-100(x_1-0.5))=\sigma(-100x_1+50) $$

When $x_1=0$, the value inside $\sigma$ is 50, i.e., $\sigma(50) \approx 1$; when $x_1 = 1$, the value inside $\sigma$ is -50, i.e., $\sigma(-50) \approx 0$. These results match the operation of a NOT gate.

Bypass Gate

Finally, consider another neuron $w_1=100, b=-50$,

$$ y = \sigma(100(x_1-0.5))=\sigma(100x_1-50) $$

When $x_1=0$, the value inside $\sigma$ is -50, i.e., $\sigma(-50) \approx 0$; when $x_1 = 1$, the value inside $\sigma$ is 50, i.e., $\sigma(50) \approx 1$. These results directly correspond to the operation $y=x$.