仅限Scrapyd - 无模块名称django_project.settings

Question

此错误仅发生在命令scrapyd-deploy上。

Packing version r240-master 
Deploying to project "foo_bot" in http://localhost:6802/addversion.json 
Server response (200): 

{ 
    "status": "error", 
    "message": "Traceback (most recent call last): 
     File \"/usr/lib/python2.7/runpy.py\", line 174, in _run_module_as_main 
     \"__main__\", fname, loader, pkg_name) 
     File \"/usr/lib/python2.7/runpy.py\", line 72, in _run_code 
     exec code in run_globals 
     File \"/usr/local/lib/python2.7/dist-packages/scrapyd/runner.py\", line 40, in <module> 
     main() 
     File \"/usr/local/lib/python2.7/dist-packages/scrapyd/runner.py\", line 37, in main 
     execute() 
     File \"/usr/local/lib/python2.7/dist-packages/scrapy/cmdline.py\", line 109, in execute 
     settings = get_project_settings() 
     File \"/usr/local/lib/python2.7/dist-packages/scrapy/utils/project.py\", line 68, in get_project_settings 
     settings.setmodule(settings_module_path, priority='project') 
     File \"/usr/local/lib/python2.7/dist-packages/scrapy/settings/__init__.py\", line 292, in setmodule 
     module = import_module(module) 
     File \"/usr/lib/python2.7/importlib/__init__.py\", line 37, in import_module 
     __import__(name) 
     File \"build/bdist.linux-x86_64/egg/foo_bot/settings.py\", line 21, in <module> 
     File \"/usr/local/lib/python2.7/dist-packages/django/__init__.py\", line 22, in setup 
     configure_logging(settings.LOGGING_CONFIG, settings.LOGGING) 
     File \"/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py\", line 56, in __getattr__ 
     self._setup(name) 
     File \"/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py\", line 41, in _setup 
     self._wrapped = Settings(settings_module) 
     File \"/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py\", line 110, in __init__ 
     mod = importlib.import_module(self.SETTINGS_MODULE) 
     File \"/usr/lib/python2.7/importlib/__init__.py\", line 37, in import_module 
     __import__(name) 
     ImportError: No module named foo_project.settings 
    ", 
    "node_name": "ubuntu" 
} 

这个项目的结构如下：

/home/dev/workspace/fufu 
|--dev_venv/bin/activate 
| 
|--foo_bot 
| |--scrapy.cfg 
| | 
| |--foo_bot 
|  |--settings.py 
|  |--settings_dev.py 
|  |--settings_deploy.py 
|  |--items.py and blablabla 
|  |--spiders 
|  |--blablablaspider.py 
| 
|--foo_project 
    |--manage.py 
    | 
    |--app 
    | |--models.py and blablabla 
    | 
    |--foo_project 
     |--settings.py 
     |--settings_dev.py 
     |--settings_deploy.py 
     |--wsgi.py and blablabla 

scrapy.cfg

[settings] 
default = foo_bot.settings 

[deploy:local] 
url = http://localhost:6801/ 
project = foo_bot 

[deploy:development] 
url = http://localhost:6802/ 
project = foo_bot 
version = GIT 

[deploy:deployment] 
url = http://localhost:6800/ 
project = foo_bot 
version = GIT 

foo_bot/foo_bot/settings.py

import os 
import sys 
import django 

########## PATH CONFIGURATION 
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) 
FALSE_BASE_DIR = os.path.dirname(BASE_DIR) 
#WORKSPACE_DIR = os.path.dirname(FALSE_BASE_DIR) // fail with just scrapy crawl bar 
WORKSPACE_DIR = '/'.join([os.path.expanduser('~'), 'workspace']) 
BOT_NAME = 'foo_bot' 
########## END PATH CONFIGURATION 

########## DJANGO CONFIGURATION 
DJANGO_PROJECT_PATH = '/'.join([FALSE_BASE_DIR, 'foo_project']) 
DJANGO_SETTINGS_MODULE = 'foo_project.settings' 
sys.path.append(DJANGO_PROJECT_PATH) 
sys.path.insert(0, DJANGO_PROJECT_PATH) 
os.environ['DJANGO_SETTINGS_MODULE'] = DJANGO_SETTINGS_MODULE 
django.setup() 
########## END DJANGO CONFIGURATION 

#blablabla settings 

try: 
    from settings_dev import * 
except ImportError: 
    pass 

foo_project/foo_project/settings.py

import os 

########## PATH CONFIGURATION 
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) 
FALSE_BASE_DIR = os.path.dirname(BASE_DIR) 
WORKSPACE_DIR = os.path.dirname(FALSE_BASE_DIR) 
PROJECT_NAME = 'foo_project' 
########## END PATH CONFIGURATION 

foo_project/foo_project/wsgi.py

而且我用virtualenv在activate脚本的末尾如下设置环境变量PYTHONPATH和DJANGO_SETTINGS_MODULE：

export PYTHONPATH="/home/dev/workspace/fufu/foo_project" 
export DJANGO_SETTINGS_MODULE="foo_project.settings" 

我在fufu/foo_bot运行scrapy crawl bar，它成功运行（并通过djangoitem保存到数据库）。

我运行django-admin.py在fufu/foo_project，它运行成功，没有任何警告。

我运行python manage.py在fufu/foo_project，它运行成功，没有任何警告。我运行scrapyd-deploy development -p foo_bot与用户dev和virutalenv dev_venv。

它返回错误，如上所述。

的scrapyd被称为在/home/dev/workspace/与用户dev和端口6802

不过，我还有一个用户deploy一个项目副本在/home/deploy/workspace/fufu/只有在from settings_dev到from settings_deploy和那些绝对路径不同。

我运行scrapyd-deploy deployment -p foo_bot用户deploy和virutalenv deploy_venv。

它运行完美。

的scrapyd被称为在/home/deploy/workspace/与用户deploy和端口6800。

我该如何解决这个问题？我已经看到所有的线程，我不知道为什么dev用户失败，但deploy用户的工作。

Answer 1

好的，我明白了。 这是因为os.path.dirname()会让我空路径。

foo_bot/foo_bot/settings.py

########## PATH CONFIGURATION 
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) 
#FALSE_BASE_DIR = os.path.dirname(BASE_DIR) 
#WORKSPACE_DIR = os.path.dirname(FALSE_BASE_DIR) // fail with just scrapy crawl bar 
WORKSPACE_DIR = '/'.join([os.path.expanduser('~'), 'workspace']) 
FALSE_BASE_DIR = '/'.join([WORKSPACE_DIR, 'fufu']) 
BOT_NAME = 'foo_bot' 
########## END PATH CONFIGURATION