I am trying to do a LLM project. So created an virtual env using venv module and started downloading the dependencies one after the other. I am have downloaded one package called langchain-chroma which installed chromadb. Then I tried installing transformers package. But it is complaining that tokenizers package dependency for transformers must be at least 0.21 or above but chromadb need version less than 0.20.3. How to solve this problem of dependency? I tried to look into documentation to get the package of transformers which was supported by tokenizers-0.20.3. But unfortunately I couldn't find much info. Continue reading...
