[Python] Neural Network model in PyTorch prediction problem for anomaly detection in a...

I'm working on my final studies project, which is detecting anomalies in an irrigation system. We simulated the irrigation network on a limited environment as you can see in the photo below.the model we created

There is a PCB where an ESP32 microcontroller reads the values from the pressure sensor. In our model, 4 valves give us 2^4 = 16 combinations. My idea is to make it simple and create a simple neural network model using Pytorch that gets the state of valves as input (0 is off and 1 is on ex:0110) and one output, which is the pressure value. So that means our system will be predicting pressure from the valve combinations. The data set is created by getting certain valve combinations(using fractional factorial design), but I added other combinations like 1000, 0100, 0010, 00001, 1110, and 0111, and then it is augmented to get a large data set. here is the model :

import torch
import torch.nn as nn

class PressureNeuralNet(nn.Module):
def __init__(self, n_valves, hidden_size=32, dropout_rate=0.1):
super().__init__()
self.network = nn.Sequential(
nn.Linear(n_valves, hidden_size),
nn.ReLU(),
nn.Dropout(dropout_rate),

nn.Linear(hidden_size, hidden_size),
nn.ReLU(),
nn.Linear(hidden_size, 1)
)

def forward(self, x):
return self.network(x)


learning process:

nn_model = PressureNeuralNet(n_valves)

# Define loss and optimizers
criterion = nn.MSELoss()
nn_optimizer = optim.Adam(nn_model.parameters(), lr=0.01, weight_decay=1e-5)
nn_scheduler = optim.lr_scheduler.ReduceLROnPlateau(nn_optimizer, mode='min', patience=5, factor=0.5, verbose=True)



print("\nTraining Neural Network...")
nn_train_losses, nn_test_losses = train_model(
nn_model, train_loader, test_loader, criterion, nn_optimizer, nn_scheduler
)


train model function :

def train_model(model, train_loader, test_loader, criterion, optimizer, scheduler=None, epochs=1000):
train_losses = []
test_losses = []
best_test_loss = float('inf')

for epoch in range(epochs):
model.train()
running_loss = 0.0
for valve_states, pressures in train_loader:
optimizer.zero_grad()
outputs = model(valve_states).squeeze()
loss = criterion(outputs, pressures)
loss.backward()
optimizer.step()
running_loss += loss.item()

avg_train_loss = running_loss / len(train_loader)
train_losses.append(avg_train_loss)

model.eval()
test_loss = 0.0
with torch.no_grad():
for valve_states, pressures in test_loader:
outputs = model(valve_states).squeeze()
loss = criterion(outputs, pressures)
test_loss += loss.item()

avg_test_loss = test_loss / len(test_loader)
test_losses.append(avg_test_loss)

if avg_test_loss < best_test_loss:
best_test_loss = avg_test_loss
torch.save(model.state_dict(), 'best_model.pth')
print(f"Saved best model with Test Loss: {best_test_loss:.4f}")

if scheduler:
scheduler.step(avg_test_loss)

if (epoch + 1) % 10 == 0:
print(f"Epoch [{epoch + 1}/{epochs}], Train Loss: {avg_train_loss:.4f}, Test Loss: {avg_test_loss:.4f}")
if scheduler:
print(f"Current Learning Rate: {optimizer.param_groups[0]['lr']:.6f}")

return train_losses, test_losses


The MSE value is: 0.0004 That meas the model predicts very well. The problem now is that when predicting a combination that it didn't learn from the data set (1101 read value: 0.53 predicted: 0.10), due to the sampling method that we used (fractional factorial design). (If you ask why we used this method is to get certain samples that have influence on the output and to prevent water and energy consumption.) But the other combinations are well predicted. there might be a problem because of the values of the single valves combinations they all heve the same value: 0.5 but the combination 0100 has a pressure value : 0.45 . What I'm asking is whether my method is wrong or if there is a problem in my model. I'm not an AI expert, I'm just trying to get my bachelor's degree. Sorry if I didn't pose my question very well.

Continue reading...