I'm using the OpenAi text-embedding-3-small model to create embeddings for each product category in a file. In total it's about 6000 product categories and they look like this: Vehicles & Parts > Vehicle Parts & Accessories > Vehicle Safety & Security > Off-Road & All-Terrain Vehicle Protective Gear Vehicles & Parts > Vehicle Parts & Accessories > Vehicle Safety & Security > Off-Road & All-Terrain Vehicle Protective Gear > ATV & UTV Bar Pads Vehicles & Parts > Vehicle Parts & Accessories > Vehicle Safety & Security > Vehicle Alarms & Locks Vehicles & Parts > Vehicle Parts & Accessories > Vehicle Safety & Security > Vehicle Alarms & Locks > Automotive Alarm Accessories Vehicles & Parts > Vehicle Parts & Accessories > Vehicle Safety & Security > Vehicle Alarms & Locks > Automotive Alarm Systems Vehicles & Parts > Vehicle Parts & Accessories > Vehicle Safety & Security > Vehicle Alarms & Locks > Motorcycle Alarms & Locks For each line in that file, I'm using the following code to generate an embedding: from openai import OpenAI client = OpenAI() response = client.embeddings.create( input="Vehicles & Parts > Vehicle Parts & Accessories > Vehicle Safety & Security > Vehicle Alarms & Locks", model="text-embedding-3-small", encoding_format="float" ) I'm storing the embeddings in a vector database (Cosmos DB for MongoDB). In many cases the vector similarity search work very well, but sometimes I'm getting bad results. For example, when I search for "Pinus Sylvestris" which is the name of a plant, I'm getting the wrong product category suggested. My question: Is it OK, to pass the product category in that hierarchical representation (with > character) into the model? Is there a way, how I can tell the model, that this is a product category for an e-commerce website, so that it understands the input better? Continue reading...
