如何使用Supervisor启动Celery时避免SECRET_KEY错误？

Question

我有一个运行在Debian 7.8上的Django 1.62应用程序，Nginx 1.2.1作为我的代理服务器，Gunicorn 19.1.1作为我的应用程序服务器。我已经安装了Celery 3.1.7和RabbitMQ 2.8.4来处理异步任务。我试图使用Supervisor 3.0a8来管理我的各种应用程序，尤其是Celery。问题是，当我尝试开始通过主管芹菜，我得到这个错误：（我显示在底部的整个堆栈跟踪）

ImproperlyConfigured: The SECRET_KEY setting must not be empty. 

我所有的配置文件都保存在一个“的conf”目录只是坐在下面的我‘的Myproj’项目目录是这样的：

conf 
├── celeryconfig.py 
├── celeryconfig.pyc 
├── celery.py 
├── __init__.py 
├── middleware.py 
├── settings 
│   ├── base.py 
│   ├── dev.py 
│   ├── __init__.py 
│   ├── prod.py 
├── urls.py 
├── wsgi.py 

我生产的Django的设置都保存在它继承我base.py基地设置prod.py设置文件。我将密钥放在我的虚拟环境的postactivate文件中，然后将其读入我的生产设置中，如下所示。我通过Python解释器验证了密钥存在于我的生产环境中。

# conf/settings/prod.py 
from conf.settings.base import * 
... 
# get_env_variable is defined in base.py 
SECRET_KEY = get_env_variable("SECRET_KEY") 

这是get_env_variable函数读取密钥：

# conf/settings/base.py 
def get_env_variable(var_name): 
    try: 
     return os.environ[var_name] 
    except KeyError: 
     error_msg = "Set the %s environment variable" % var_name 
     raise ImproperlyConfigured(error_msg) 

这是我的上司的配置文件。它是基于样本文件显示在Celery documentation：

# /etc/supervisor/conf.d/myproj.conf 
[program:myproj] 
command = /www/myproj/bin/start-gunicorn 
user = root 
stdout_logfile = /var/log/gunicorn/supervisor.log 
redirect_stderr = true 

[program:celery] 
directory=/www/myproj 
command=/home/myproj/venv/myproj/bin/celery worker --app=conf -l debug 
user=nobody 
numprocs=1 
stdout_logfile=/var/log/celery/celery.log 
stderr_logfile=/var/log/celery/celery.log 
autostart=true 
autorestart=true 
startsecs=10 
stopwaitsecs=600 
killasgroup=true 
priority=998 

下面是如何使用监事加载新的芹菜配置：

sudo service supervisor stop 
sudo service supervisor start 

这是我的芹菜应用程序文件：

# conf/celery.py 
from __future__ import absolute_import 
import os 
from celery import Celery 
from django.conf import settings 
from conf import celeryconfig 

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'conf.settings') 
app = Celery('conf') 
app.config_from_object(celeryconfig) 
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS) 

这是我的芹菜配置文件：

# conf/celeryconfig.py 
BROKER_URL = 'amqp://guest@localhost:5672//' 
CELERY_RESULT_BACKEND = 'amqp' 
CELERY_ACCEPT_CONTENT = ['json', ] 
CELERY_TASK_SERIALIZER = 'json' 
CELERY_RESULT_SERIALIZER = 'json' 
CELERY_TASK_RESULT_EXPIRES = 3600 
CELERY_SEND_TASK_ERROR_EMAILS = True 

每芹菜文件，我已经修改了__init__.py：

# conf/__init__.py 
from __future__ import absolute_import 
from .celery import app as celery_app 

我可以用手使用以下命令启动芹菜，它只是罚款启动：

workon myproj # Activate project's virtual environment 
celery worker -A conf -l info 

然而，当我尝试通过主管启动它，我得到了我所描述的错误。考虑到可能Supervisor无法访问Django密钥，因为它是一个环境变量，我试图在我的prod.py设置文件中对密钥进行硬编码（而不是通过get_env_variable函数读取它），但是这并没有修复问题。

我尝试将所有设置合并到包含实际密钥的prod.py设置文件中，但没有帮助。

我也尝试添加该环境参数到超级配置文件，如下所示但这并没有解决问题：

# /etc/supervisor/conf.d/myproj.conf 
... 
[program:celery] 
environment=SECRET_KEY="(my secret key)" 
directory=/www/myproj 
command=/home/myproj/venv/myproj/bin/celery worker --app=conf -l debug 
user=nobody 
... 

我尝试了“用户”设置为用户的名字，我启动网站，但这也没有帮助。

# /etc/supervisor/conf.d/myproj.conf 
... 
[program:celery] 
environment=SECRET_KEY="(my secret key)" 
directory=/www/myproj 
command=/home/myproj/venv/myproj/bin/celery worker --app=conf -l debug 
user=myproj 
... 

我读到了一个名为django-supervisor库，应该缓解Django和主管之间的集成，但我得到了同样的错误，如果我使用该库。

最后我读here on SO如果你的密钥包含“％”符号，Supervisor不会喜欢它。我注意到我的密钥确实包含一个“％”符号，所以我像这个“%%”一样逃脱了它，但是这并没有解决问题。

任何人都可以看到我做错了什么？我在这里看到了来自不同情况下遇到同样错误的用户的其他问题，但我试图实施讨论过的各种解决方案，但他们都没有解决问题。

非常感谢您的想法。对于长期问题抱歉，但这个问题有很多移动部分。

下面是完整的堆栈跟踪：

Traceback (most recent call last): 
    File "/home/myproj/venv/myproj/bin/celery", line 11, in <module> 
    sys.exit(main()) 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/celery/__main__.py", line 30, in main 
    main() 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/celery/bin/celery.py", line 81, in main 
    cmd.execute_from_commandline(argv) 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/celery/bin/celery.py", line 769, in execute_from_commandline 
    super(CeleryCommand, self).execute_from_commandline(argv))) 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/celery/bin/base.py", line 307, in execute_from_commandline 
    return self.handle_argv(self.prog_name, argv[1:]) 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/celery/bin/celery.py", line 761, in handle_argv 
    return self.execute(command, argv) 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/celery/bin/celery.py", line 693, in execute 
    ).run_from_argv(self.prog_name, argv[1:], command=argv[0]) 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/celery/bin/worker.py", line 179, in run_from_argv 
    return self(*args, **options) 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/celery/bin/base.py", line 270, in __call__ 
    ret = self.run(*args, **kwargs) 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/celery/bin/worker.py", line 212, in run 
    state_db=self.node_format(state_db, hostname), **kwargs 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/celery/worker/__init__.py", line 95, in __init__ 
    self.app.loader.init_worker() 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/celery/loaders/base.py", line 128, in init_worker 
    self.import_default_modules() 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/celery/loaders/base.py", line 116, in import_default_modules 
    signals.import_modules.send(sender=self.app) 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/celery/utils/dispatch/signal.py", line 166, in send 
    response = receiver(signal=self, sender=sender, **named) 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/celery/fixups/django.py", line 69, in on_import_modules 
    self.worker_fixup.validate_models() 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/kombu/utils/__init__.py", line 322, in __get__ 
    value = obj.__dict__[self.__name__] = self.__get(obj) 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/celery/fixups/django.py", line 64, in worker_fixup 
    self._worker_fixup = DjangoWorkerFixup(self.app) 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/celery/fixups/django.py", line 99, in __init__ 
    self._cache = import_module('django.core.cache') 
    File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module 
    __import__(name) 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/django/core/cache/__init__.py", line 69, in <module> 
    if DEFAULT_CACHE_ALIAS not in settings.CACHES: 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/django/conf/__init__.py", line 54, in __getattr__ 
    self._setup(name) 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/django/conf/__init__.py", line 49, in _setup 
    self._wrapped = Settings(settings_module) 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/django/conf/__init__.py", line 151, in __init__ 
    raise ImproperlyConfigured("The SECRET_KEY setting must not be empty.") 
django.core.exceptions.ImproperlyConfigured: The SECRET_KEY setting must not be empty. 

Answer 1

在你的芹菜配置，你需要指定要使用的设置。目前你有prod.py您使用为您的生产代码，以便您celery.py应该

# conf/celery.py 
from __future__ import absolute_import 
import os 
from celery import Celery 
from django.conf import settings 
from conf import celeryconfig 

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'conf.settings.prod') 
app = Celery('conf') 
app.config_from_object(celeryconfig) 
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS) 

因此芹菜会知道你想使用（如果你在设置文件夹中有多个设置）的主管