2013-10-09 52 views
1

我遇到了一些行为,试图在芹菜任务期间保存对我的模型的更改,但更改没有提交。我有一个保存用户上传记录的模型,一旦上传文件,就会运行一个芹菜任务来处理csv并将过程的结果保存到数据库中,例如,进程状态,及时处理,处理的记录数等Django芹菜任务处理csv - 模型不保存

在models.py,我有以下几种方法:

def import_records_data(self): 
    total_records = 0 
    with self.filepath.file as csvfile: 
     reader = csv.reader(csvfile, delimiter=',') 
     next(reader, None) # skip header 
     for row in reader: 
      # process record 
      total_records += 1 
    return total_records 

def process(self): 
    self.date_start_processing = datetime.datetime.utcnow().replace(tzinfo=utc) 
    try: 
     # process upload data, 
     records_processed = self.import_records_data() 
    except Exception, e: 
     self._mark_failed(unicode(e)) 
    else: 
     self._mark_processed(num_records=records_processed) 

def _mark_processed(self, num_records, description=None): 
    self.status = self.PROCESSED 
    self.date_end_processing = datetime.datetime.utcnow().replace(tzinfo=utc) 
    self.num_records = num_records 
    self.processing_description = description 
    self.save() 

def _mark_failed(self, description): 
    self.status = self.FAILED 
    self.processing_description = description 
    self.save() 

def was_processing_successful(self): 
    return self.status == self.PROCESSED 

当_mark_processed或_mark_failed被调用时,所做的更改不会保存到数据库时调用self.save()。这种方法从tasks.py调用:

@task(name='csv-process-upload') 
def process_upload(upload_id): 
    upload = Upload.objects.get(id=upload_id) 
    upload.process() 

    if upload.was_processing_successful(): 
     message_user(
      upload.user, 
      "Your upload '%s' was processed successfully, %s records processed" % (
       upload.filename, 
       upload.num_records)) 
    else: 
     message_user(
      upload.user, 
      "Your upload '%s' could not be processed, error message: %s" % (
       upload.filename, 
       upload.processing_description,)) 

什么可以防止模型保存?当我在shell中调试_mark_processed并键入self.save()时,更改将反映在数据库中。

+0

你如何触发芹菜任务?如果它在Django视图中触发,则初始模型保存/创建可以保存在事务中,并且在任务运行时对celery不可见。 –

+0

是的,它是通过视图触发的。那我该怎么办?谢谢。 – mrkre

回答

1

尝试使用django celery transactions调用您的任务以确保您的视图在触发任务执行之前完成事务。

+0

经仔细检查,似乎以前的芹菜任务没有完成,我不得不手动清除它们。之后,该任务能够成功运行,而无需依赖django-celery-transactions。 – mrkre

+0

听起来不错。查看芹菜花(以及'--events'),以获得一个漂亮的网络管理员,以监控您的芹菜工和任务队列。 –

+0

感谢您的提示。应该看看它。 – mrkre