[Python] Redis index with multiple fields

I am using AWS MemoryDB 7.1 (Redis with RedisSearch enabled). I want to store items in MemoryDB represented as a vector, as well as JSON. The vector will be used to run user-item similarity searches, while the JSON will contain metadata about each item. When retrieving items, I want to be able to run a vector similarity search on the indexed vectors (against a user vector), as well as use some of the JSON fields for filtering the search. I want MemoryDB itself to handle the filtering (I don't want to filter post-search manually).

This is how I am creating my index in Python:

schema = [
VectorField(
"vector",
"HNSW",
{
"TYPE": "FLOAT32",
"DIM": 32,
"DISTANCE_METRIC": "COSINE"
}
),
TextField("metadata"),
TagField("$.metadata.tenant", as_name="tenant")
]

definition = IndexDefinition(
prefix=["item:"],
index_type=IndexType.HASH
)

redis_client.ft('my_index').create_index(schema, definition=definition)


The intention is to create an index called 'my_index' that indexes the 'vector' field, as well as the 'tenant' field that is a nested field instead the JSON metadata field.

I want to then be able to run a vector similarity search while filtering out items whose tenant field is a certain value. The code that does this is as follows:

ef_runtime = 512
offset = page_number * page_size
total_k = offset + page_size
tenant='abc'

query_string = f'tenant:{{{tenant}}}=>[KNN {total_k} @vector $query_vec AS vector_score]=>{{$EF_RUNTIME: {ef_runtime}}}'
q = (Query(query_string)
.dialect(2)
.sort_by('vector_score', asc=True)
.return_fields('vector_score', 'metadata')
.paging(offset, page_size))

params_dict = { "query_vec": user_vector_bytes }

results = redis_client.ft(self.index_name).search(q, params_dict)



This is not currently working. The vector search works but the filtering is not being applied. Is this even possible in MemoryDB? Eventually I want to be able to filter by numerical fields as well that are found in the JSON metadata.

Continue reading...