[Python] "cannot import name 'DEFAULT_CIPHERS' from 'urllib3.util.ssl_'" on AWS Lambda using a...

What I want to achieve


To scrape an website using AWS Lambda and save the data on S3.

The issues I'm having


When I execute Lambda, the following error message appears.

{ "errorMessage": "Unable to import module 'lambda_function': cannot import name 'DEFAULT_CIPHERS' from 'urllib3.util.ssl_' (/opt/python/urllib3/util/ssl_.py)", "errorType": "Runtime.ImportModuleError", "requestId": "fb66bea9-cbad-4bd3-bd4d-6125454e21be", "stackTrace": [] }
​

Code


The minimum Lambda code is as follows.

import requests
import boto3

def lambda_handler(event, context):
s3 = boto3.client('s3')
upload_res = s3.put_object(Bucket='horserace-dx', Key='/raw/a.html', Body='testtext')

return event


An layer was added to the Lambda. Files were save in python folder using the commands below , frozen in a zip file, then uploaded to AWS Lambda as a layer.

!pip install requests -t ./python --no-user
!pip install pandas -t ./python --no-user
!pip install beautifulsoup4 -t ./python --no-user

• The bucket horserace-dx exists
• The folder raw exists
• The role of the Lambda is properly set. It can read from and write to S3
• The runtime of the Lambda is Python 3.9. The python version of the local computer is 3.9.13.

What I did so far


I google "cannot import name 'DEFAULT_CIPHERS' from 'urllib3.util.ssl_'" and found some suggestions. I made the layer with the following code and tried again in vain.

!pip install requests -t ./python --no-user
!pip install pandas -t ./python --no-user
!pip install beautifulsoup4 -t ./python --no-user
!pip install urllib3==1.26.15 -t ./python --no-user


So what should I do to achieve what I want to achieve? Any suggestions would be greatly appreciated.

Continue reading...