[Python] Llama Index Vector Store: filter a list of documents with a NOT feature

I have a vector store of documents, each document is a json document with features. I'd like to filter the documents according to some criteria. The problem is that some of the documents contain a NOT criteria, so they should be retrieved if the filter does not contain this feature. Here's a MOE:

from llama_index.core.schema import TextNode
from llama_index.core import VectorStoreIndex
from llama_index.core.vector_stores import MetadataFilter, MetadataFilters, FilterOperator

node_1 = TextNode(text="Plans a and b",id_="1",metadata={"plan": ["a", "b", "c"]})
node_2 = TextNode(text="Plans a",id_="2",metadata={"plan": ["a"]})
node_3 = TextNode(text="Plans not a",id_="3",metadata={"plan": [{"not": ["a"]}]})

doc_index = VectorStoreIndex([node_1, node_2, node_3])

filter_plan_a = MetadataFilter(key="plan", operator=FilterOperator.CONTAINS, value="a")
filter_plan_b = MetadataFilter(key="plan", operator=FilterOperator.CONTAINS, value="b")

for filter_plan in [filter_plan_a, filter_plan_b]:
retriever = doc_index.as_retriever(
filters=MetadataFilters(filters=[filter_plan]), similarity_top_k=3
)

print(f"Results for filter {filter_plan}")
print("-" * 20)
for node_with_score in retriever.retrieve("plan"):
print(node_with_score.node.text)
print(node_with_score.node.metadata)
print()



With filter_plan_b, document id 3 should be retrieved, because a plan "b" is compatible with a not "a" condition.

How can I implement this? If all possible values of plans are known, this could be solved by replacing all not features with the value of all the other plans, but what can I do if the values are not known apriori?

Continue reading...