[Python] High-Accuracy Intent Classification with Small Models for AWS Lambda Deployment

I am developing a FastAPI service for intent classification to be deployed on AWS Lambda. The primary objective is to build a model with high accuracy (≥95%) on both training and testing datasets. This service will be part of a conversational AI system that classifies user queries into specific intents: "Greeting", "Positive Feedback", "Negative Feedback", "Help Request", and "Fallback".

A key technical constraint is the 50 MB limit for standard Lambda deployment packages.

Approaches I have tried:

1. Sentence Transformers + Logistic Regression:

• 
  Steps Taken:
  Used the SentenceTransformer library with the pre-trained model "paraphrase-multilingual-MiniLM-L12-v2" to generate embeddings, and trained a scikit-learn LogisticRegression classifier.
  
  
• 
  Code Snippet:
  
  embedder = SentenceTransformer("paraphrase-multilingual-MiniLM-L12-v2")
  input_embeddings = embedder.encode(texts)
  clf = LogisticRegression().fit(input_embeddings, labels)
• 
  Result:
  The model performs well, but the transformer model alone is over 100 MB, exceeding Lambda limits.
  

2. Custom-Trained spaCy Model:

• 
  Steps Taken:
  Trained a spaCy textcat pipeline based on the en_core_web_md model (~31 MB), using ~5,000 labeled examples.
  
  
• 
  Configuration:
  Used spacy train with a custom config.cfg, freezing components like tok2vec and only training textcat.
  
  
• 
  Result:
  The trained model + dependencies like scipy/numpy caused the deployment to exceed 50 MB.
  

3. Fine-Tuned DistilBERT:

• 
  Steps Taken:
  Fine-tuned DistilBERT using HuggingFace transformers and the Trainer API on a custom dataset of ~500 samples across 5 intents.
  
  
• 
  Result:
  Achieved over 95% accuracy, but the model size (~250 MB) makes it unsuitable for standard Lambda.
  

Question:


I am aware of the Lambda container image option and have explored it, but I'm specifically interested in whether there's any novel, lightweight, or efficient method to meet these goals within the 50 MB package limit.

Has anyone successfully deployed a high-accuracy NLP classifier under these constraints?

Continue reading...