[Python] How to create custom columns when creating embeddings using llamaindex in postgres...

I'm building an agent for semantic query using llama index and Postgres(using pgvector extension) as vector store. I'm using llama index to create vector embeddings and vector search indexes with VectorStoreIndex.from_documents(). I've defined PGVectorStore as follows

self.vector_store = PGVectorStore.from_params(
database=request.tenant,
host=host,
password=password,
port=port,
user=user,
table_name="vectorstore",
embed_dim=1536,
hnsw_kwargs={
"hnsw_m": 16,
"hnsw_ef_construction": 64,
"hnsw_ef_search": 40,
"hnsw_dist_method": "vector_cosine_ops",
},
)


Using this llama index creates vector embeddings and indexes in data_vectorstore table. Currently I'm storing some of the columns that I want this table to have as metadata (in metadata column) like external_id. I want to create custom columns when creating embeddings. Is this possible using llama index?

Is there a way to do it? and How?

I tried adding those ids to the metadata but the I strictly require the schema to have those columns in the table and not in metadata json.

Continue reading...