I am using AWS Glue where I'm trying to use psycopg2 in a pyspark script. Since glue does not support psycopg2 in its execution environment, I am passing it in --additional-python-moodules. Which is a way of installing additional Python modules in aws glue. After following the steps mentioned in the docs. I am getting an error while running the job which says ModuleNotFoundError: No module named 'psycopg2._psycopg' Here are the job parameters are being passed during the execution. Solutions I have already tried: Using psycopg2-binary Passing a zip/whl file in --additional-python-modules Changing glue version to 2.0 and 3.0 Continue reading...
