I'm actually working in a project where I have a Sharepoint with a bunch of PDF files stored in it. I need to store these files in Azure AI Search to perform semantic search. When I perform retrieval over my Index, I need to recover not only the relevant information but also in which page this information is stored. The Azure AI Search SplitSkill says that it can split documents by pages, but it actually splits them by chunks and I cannot retrieve the page number from there. So for this, I created a simple Custom Skill that gets the PDF binary, split it by pages using PyMuPDF and put it into an Azure Function, so I can create a WebApiSkill to split the PDF into pages and retrieve both, the page content (to be later further chunked and embedded) and the page number. The code for the split is simple: import fitz import json def split_pdf_by_pages(pdf_bytes): pdf_document = fitz.open(stream=pdf_bytes, filetype="pdf") pages = [] for page_num in range(len(pdf_document)): page = pdf_document.load_page(page_num) text = page.get_text("text") pages.append({ "page_number": page_num + 1, "content": text }) return json.dumps(pages) I still can't solve this problem, because seems that when I read information from Sharepoint, the objects that reach to my CustomSkill are no binaries, so my code fails. Do anyone knows how to solve this? Any other way to actually split the PDFs in the Sharepoint by actual pages and retrieve the page number? Continue reading...
