[Python] How to resolve this error if using spark in docker containers?

(https://i.sstatic.net/cWOd3lxg.png)

I am using docker container for spark and facing this above error in the image

spark-master:
image: bitnami/spark:3.5.5
container_name: spark-master
environment:
- JAVA_HOME=/opt/bitnami/java
- SPARK_MODE=master
- SPARK_RPC_AUTHENTICATION_ENABLED=no
- SPARK_RPC_ENCRYPTION_ENABLED=no
- SPARK_LOCAL_STORAGE_ENCRYPTION_ENABLED=no
- SPARK_SSL_ENABLED=no
ports:
- "8081:8080" # Spark Web UI
- "7077:7077" # Spark Master Port
networks:
- spark-network

spark-worker-1:
image: bitnami/spark:3.5.5
container_name: spark-worker-1
environment:
- SPARK_MODE=worker
- SPARK_MASTER_URL=spark://spark-master:7077
- SPARK_WORKER_MEMORY=20G
- SPARK_WORKER_CORES=4
depends_on:
- spark-master
networks:
- spark-network


Haven't tried anything

Continue reading...