[Python] How to handle multi-class classification where subclasses across different...

I have a malicious traffic feature dataset with 10 major categories label, and I know there are 207 fine-grained subclasses, each belonging to one of those 10 superclasses, and I don't have the subclasses label in dataset. It seems to be a simple classification problem of machine learning.

However I've discovered that Subclasses under the same superclass are often very different from each other and subclasses from different superclasses can be very similar, this cause low score in usual method to solve the classification problem.

After several experiments I found the label is extremly imbalanced, I found about 26 subclasses which is important. I tried to use KMeans to find these subclasses and then train the model with these subclasses label and found some subclasses with different superclasses label is quiet similar to each other, the confusion matrix is not good.

Is there any methods or idea to solve the problem? Training a classifier with superclass → subclass hierarchy performs poorly and Using coarse labels as intermediate supervision hurts accuracy.

confusion_matrix

cluster_feature

Continue reading...