2013-09-23 32 views
0

TL延误; DR:芹菜在魔豆经纪人:在任务执行

我通过beanstalkd券商排队短期和简单的任务,celeryd,使用(的myNotifyTask()例如myNotifyTask.delay()代替)task.delay。尽管事实上延迟值应该是立即执行的,但任务需要大约一个小时才能执行(当它们只需要几秒钟)时。

从我的观察,似乎任务确实收到beanstalkd,但在很长一段时间内保持在ready状态。这发生尽管设置CELERYD_CONCURRENCY = 8。在查看beanstalkd的日志时,我收到有关read(): Connection reset by peer的错误,但这些任务最终会执行。

任何想法,为什么会发生这种情况?

详情如下。


使用豆秆版本1.4.6,芹菜3.0.20。

魔豆日志条目是这样的:

/usr/bin/beanstalkd: prot.c:709 in check_err: read(): Connection reset by peer 

当尝试使用celery诊断问题:

> celery -b "beanstalk://beanstalk_server:11300" status 
Error: No nodes replied within time constraint. 

当通过telnet连接到beanstalkd,我看到current-jobs-ready: 343,这表明就业卡在ready状态(而不是delayed)。以下是完整的输出:

> telnet localhost 11300 
stats 
OK 850 
--- 
current-jobs-urgent: 343 
current-jobs-ready: 343 
current-jobs-reserved: 0 
current-jobs-delayed: 0 
current-jobs-buried: 0 
cmd-put: 2484 
cmd-peek: 0 
cmd-peek-ready: 7 
cmd-peek-delayed: 1 
cmd-peek-buried: 1 
cmd-reserve: 0 
cmd-reserve-with-timeout: 52941 
cmd-delete: 2141 
cmd-release: 0 
cmd-use: 2485 
cmd-watch: 42 
cmd-ignore: 40 
cmd-bury: 0 
cmd-kick: 0 
cmd-touch: 0 
cmd-stats: 497655 
cmd-stats-job: 2141 
cmd-stats-tube: 3 
cmd-list-tubes: 2 
cmd-list-tube-used: 1 
cmd-list-tubes-watched: 52954 
cmd-pause-tube: 0 
job-timeouts: 0 
total-jobs: 2484 
max-job-size: 65535 
current-tubes: 3 
current-connections: 6 
current-producers: 2 
current-workers: 2 
current-waiting: 1 
total-connections: 502958 
pid: 989 
version: 1.4.6 
rusage-utime: 45.778861 
rusage-stime: 56.595537 
uptime: 2489047 
binlog-oldest-index: 0 
binlog-current-index: 0 
binlog-max-size: 10485760 

后不久:

stats-tube celery 
OK 257 
--- 
name: celery 
current-jobs-urgent: 348 
current-jobs-ready: 348 
current-jobs-reserved: 0 
current-jobs-delayed: 0 
current-jobs-buried: 0 
total-jobs: 2739 
current-using: 3 
current-watching: 1 
current-waiting: 0 
cmd-pause-tube: 0 
pause: 0 
pause-time-left: 0 

回答

0

原来,问题是,一个芹菜任务有一个很长的超时,导致其工人要等待很长的时间。即使开启了并发性,超时时间也很长,并且任务不断堆积在beanstalk中(没有芹菜消耗它们,因为所有的工作人员最终都很忙)。