[Python] Need Assistance with Fine-tuning my Content Moderation Neural Network with RoBERTa....

I can't quite figure out how to fine-tune my neural network with jigsaw and make it a multi-classifier rather than a binary-classifier (multiple types of output rather than just positive and negative) and need assistance with doing so.

(With PyTorch).

Here is the source code:

from transformers import AutoTokenizer
from transformers import AutoModelForSequenceClassification
from scipy.special import softmax
import nltk
from nltk.tokenize import word_tokenize

nltk.download('punkt')

example = "Dude, that's amazing!"

tokens = word_tokenize(example, preserve_line=True)

MODEL = "cardiffnlp/twitter-roberta-base-offensive"
tokenizer = AutoTokenizer.from_pretrained(MODEL)
model = AutoModelForSequenceClassification.from_pretrained(MODEL)

def evaluate_text(text):
encoded_text = tokenizer(text, return_tensors='pt')
output = model(**encoded_text)
scores = output[0][0].detach().numpy()
scores = softmax(scores)

neg_value = float(scores[1])
pos_value = float(scores[0])

print("Negative value:", neg_value)
print("Positive value:", pos_value)

evaluate_text(example)


Well, I really just need assistance rather than just resolving an issue.

Continue reading...