[Python] How to improve FAISS search accuracy for long text using sentence-transformers?

I'm using sentence-transformers and FAISS to build a semantic search system for legal texts. It works fine with short clauses, but for longer paragraphs (300+ words), the retrieval becomes inaccurate or off-topic.

Here’s my current code:

from sentence_transformers import SentenceTransformer
import faiss
import numpy as np

model = SentenceTransformer("all-MiniLM-L6-v2")

docs = [
"Clause A: Every person has the right to legal representation.",
"Clause B: Appeals must be filed within 30 days of the ruling.",
"Clause C: Procedural delays should be reported in writing."
]

doc_embeddings = model.encode(docs, convert_to_numpy=True)
faiss.normalize_L2(doc_embeddings)
index = faiss.IndexFlatIP(doc_embeddings.shape[1])
index.add(doc_embeddings)

query = "What is the process for filing an appeal?"
query_embedding = model.encode([query], convert_to_numpy=True)
faiss.normalize_L2(query_embedding)

D, I = index.search(query_embedding, k=1)
print("Match:", docs[I[0][0]])


Problem:

Expected: Clause B (about appeals).
Actual: Often returns unrelated clauses (A or C), especially when docs are long (300+ words). No error, just poor match quality.

Question:
How can I improve FAISS search for long legal paragraphs without losing context? Is there a better way to encode long inputs for semantic similarity?

Continue reading...