[Python] Azure Prompt Flow with RankZephyr: How to handle Python 3.9 and 3.10 dependency...

I am developing a AI Assistant on Prompt Flow starting froma Multi-Round Q&A on Your Data prompt flow.

What I Did

Changed the Index Lookup:

I replaced the built-in index lookup with LangChain’s AzureSearch() class:

docs = vectorestore.similarity_search(
query=query,
k=TOP_K,
search_type="hybrid",
filters = filter_query,

)


I also added the required packages in requirements.txt.

Integrated RankZephyr for Reranking:

I want to pass the retrieved documents to RankZephyr for reranking. Here’s the function I implemented:

from langchain_community.document_compressors.rankllm_rerank import RankLLMRerank
from langchain.schema import Document
from promptflow import tool
from langchain.schema import Document
from typing import List, Dict
TOP_n = 5

@tool
def rerankZephyr(input1: str, input2: List[Document]) -> List[Document]:
compressor = RankLLMRerank(top_n=TOP_n, model="zephyr")
compression_retriever = ContextualCompressionRetriever(
base_compressor=compressor, base_retriever=input2
)

# Test con una query
query = input1
retrieved_docs = compression_retriever.get_relevant_documents(query)

# Stampa i risultati con gli score di ranking
#for doc in retrieved_docs:
# print(f"Score: {doc.metadata.get('score', 'N/A')} | Document: {doc.page_content}")


return retrieve_docs```

The Problem:
Azure Prompt Flow runs on Python 3.9, but RankZephyr requires Python 3.10+.
This version mismatch causes dependency conflicts, preventing me from using RankZephyr inside Prompt Flow.

Question:
What is the best way to integrate RankZephyr (Python 3.10) with Azure Prompt Flow (Python 3.9)?

1. Upgrading Python in Azure Prompt Flow adding Python>=3.10 in requirements.txt;
2. Using a separate Python 3.10 environment, but I am not sure how to integrate it with Prompt Flow.
3. Exploring Docker containers; thinking of running RankZephyr in a separate container, but unsure how to pass the data between Prompt Flow and the reranker efficiently.

Continue reading...