[Python] I'm getting an "ERROR (spawn error)" when I try to start my celery/supervisor instance

I've gone through how to use celery on my django production server using supervisor.

However when I try to start supervisor with sudo supervisorctl start app-celery - it returns:

app-celery: ERROR (spawn error)


Here is my config /etc/supervisor/conf.d/app-celery.conf:

[program:app-celery]
command=/home/zorgan/app/env/bin/celery worker -A draft1 --loglevel=INFO
directory=/home/zorgan/app/draft1

numprocs=1
stdout_logfile=/var/log/supervisor/celery.log
stderr_logfile=/var/log/supervisor/celery.log
autostart=true
autorestart=true
startsecs=10

; Need to wait for currently executing tasks to finish at shutdown.
; Increase this if you have very long running tasks.
stopwaitsecs = 600

stopasgroup=true

; Set Celery priority higher than default (999)
; so, if rabbitmq is supervised, it will start first.
priority=1000


I've followed the tutorial word for word - I don't know why it is not working. I've checked that my path to celery is /home/zorgan/app/env/bin/celery, and my celery.py and tasks.py is in /home/zorgan/app/draft1. As well as the init file in /home/zorgan/app/draft1 being changed to:

from __future__ import absolute_import, unicode_literals

#This will make sure the app is always imported when
#Django starts so that shared_task will use this app.
from .celery import app as celery_app

__all__ = ['celery_app']


Here's my celery.py:

import os
from celery import Celery
from celery.schedules import crontab
from .settings import CELERY_BROKER_URL #CELERY_BROKER_URL = 'amqp://174.138.62.649' (changed the number for privacy reasons)

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'draft1.settings')

app = Celery("draft1", broker=CELERY_BROKER_URL)
app.config_from_object('django.conf:settings', namespace='CELERY')
app.autodiscover_tasks()


Are there any red flags here? Because I'm also getting this error in my celery.log file:

File "/home/zorgan/app/env/lib/python3.5/site-packages/kombu/utils/imports.py", line 56, in symbol_by_name
module = imp(module_name, package=package, **kwargs)
File "/home/zorgan/app/env/lib/python3.5/site-packages/celery/utils/imports.py", line 101, in import_from_cwd
return imp(module, package=package)
File "/home/zorgan/app/env/lib/python3.5/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 986, in _gcd_import
File "<frozen importlib._bootstrap>", line 969, in _find_and_load
File "<frozen importlib._bootstrap>", line 956, in _find_and_load_unlocked
ImportError: No module named 'draft1'


but I'm not sure if that's related. Here a snippet of my supervisord.log:

2018-04-25 03:15:07,665 INFO spawned: 'app-celery' with pid 24296
2018-04-25 03:15:08,050 INFO exited: app-celery (exit status 1; not expected)
2018-04-25 03:15:09,052 INFO gave up: app-celery entered FATAL state, too many start retries too quickly
2018-04-25 03:15:09,815 INFO spawned: 'app-celery' with pid 24302
2018-04-25 03:15:10,221 INFO exited: app-celery (exit status 1; not expected)
2018-04-25 03:15:11,231 INFO spawned: 'app-celery' with pid 24309
2018-04-25 03:15:11,646 INFO exited: app-celery (exit status 1; not expected)
2018-04-25 03:15:13,650 INFO spawned: 'app-celery' with pid 24313
2018-04-25 03:15:14,068 INFO exited: app-celery (exit status 1; not expected)
2018-04-25 03:15:17,074 INFO spawned: 'app-celery' with pid 24317
2018-04-25 03:15:17,505 INFO exited: app-celery (exit status 1; not expected)


Any idea what the problem is?

EDIT:

project tree:

home / zorgan / app / draft1
... ...
manage.py celery.py
env tasks.py

Continue reading...