2017-07-12 45 views
5

我发现自己处于手动触发DAG运行(通过airflow trigger_dag datablocks_dag)运行的情况,并且Dag Run在界面中显示,但它会永远保持“运行”状态,而无需执行任何操作。气流DAG运行已触发,但从未执行?

当我在UI检查该DAG运行,我看到以下内容:

enter image description here

我有start_date设置为datetime(2016, 1, 1),并schedule_interval设置为@once我的从阅读文档的理解是,现在start_date <现在,DAG将被触发。 @once确保它只发生一次。

我的日志文件说:

[2017-07-11 21:32:05,359] {jobs.py:343} DagFileProcessor0 INFO - Started process (PID=21217) to work on /home/alex/Desktop/datablocks/tests/.airflow/dags/datablocks_dag.py 
[2017-07-11 21:32:05,359] {jobs.py:534} DagFileProcessor0 ERROR - Cannot use more than 1 thread when using sqlite. Setting max_threads to 1 
[2017-07-11 21:32:05,365] {jobs.py:1525} DagFileProcessor0 INFO - Processing file /home/alex/Desktop/datablocks/tests/.airflow/dags/datablocks_dag.py for tasks to queue 
[2017-07-11 21:32:05,365] {models.py:176} DagFileProcessor0 INFO - Filling up the DagBag from /home/alex/Desktop/datablocks/tests/.airflow/dags/datablocks_dag.py 
[2017-07-11 21:32:05,703] {models.py:2048} DagFileProcessor0 WARNING - schedule_interval is used for <Task(BashOperator): foo>, though it has been deprecated as a task parameter, you need to specify it as a DAG parameter instead 
[2017-07-11 21:32:05,703] {models.py:2048} DagFileProcessor0 WARNING - schedule_interval is used for <Task(BashOperator): foo2>, though it has been deprecated as a task parameter, you need to specify it as a DAG parameter instead 
[2017-07-11 21:32:05,704] {jobs.py:1539} DagFileProcessor0 INFO - DAG(s) dict_keys(['example_branch_dop_operator_v3', 'latest_only', 'tutorial', 'example_http_operator', 'example_python_operator', 'example_bash_operator', 'example_branch_operator', 'example_trigger_target_dag', 'example_short_circuit_operator', 'example_passing_params_via_test_command', 'test_utils', 'example_subdag_operator', 'example_subdag_operator.section-1', 'example_subdag_operator.section-2', 'example_skip_dag', 'example_xcom', 'example_trigger_controller_dag', 'latest_only_with_trigger', 'datablocks_dag']) retrieved from /home/alex/Desktop/datablocks/tests/.airflow/dags/datablocks_dag.py 
[2017-07-11 21:32:07,083] {models.py:3529} DagFileProcessor0 INFO - Creating ORM DAG for datablocks_dag 
[2017-07-11 21:32:07,234] {models.py:331} DagFileProcessor0 INFO - Finding 'running' jobs without a recent heartbeat 
[2017-07-11 21:32:07,234] {models.py:337} DagFileProcessor0 INFO - Failing jobs without heartbeat after 2017-07-11 21:27:07.234388 
[2017-07-11 21:32:07,240] {jobs.py:351} DagFileProcessor0 INFO - Processing /home/alex/Desktop/datablocks/tests/.airflow/dags/datablocks_dag.py took 1.881 seconds 

可能会造成什么问题?

我误解start_date是如何运作的?

或者令人担忧的schedule_intervalWARNING行在日志文件中可能是问题的根源?

回答

3

问题是,dag已暂停。

在您提供的截图中,在左上角,将其翻转为On,应该这样做。

这是一个常见的“陷阱”,当开始与气流。

+0

嗯。我正在运行'airflow unpause datablocks_dag',接着是'airflow trigger_dag datablocks_dag'。我是否以错误的顺序运行它们,也许? –

+0

理论上应该很好。但是,在用户界面中,我们可以看到DAG仍然处于暂停状态。不知道为什么...手动解除它应该工作。 – jhnclvr

+1

我已提前完成并调用了调用的顺序,现在按预期工作。 –